Organized by Rybolos - Current server time: Jan. 21, 2021, 3:08 p.m. UTC


Private Test
Feb. 23, 2020, 9 p.m. UTC


Private Test
Feb. 23, 2020, 9 p.m. UTC


Competition Ends
Feb. 25, 2020, 11:59 p.m. UTC


We believe that multi-level language structures need to be labelled together, otherwise errors in one tag level will lead to errors in the following. Existing pipelines “tokenization - morphology - lemmatization - syntax” accumulate errors at each stage.
During the competition, participants aim to build systems that define:

  • Morphological characteristics of the word (part-of-speech and full tags).
  • Lemma of the word.
  • Syntactic relations (dependencies).

GramEval Results 🏆.

We offer the participants to try to build systems that implement complete morphological and syntactic markup with lemmatization within the framework of Universal Dependencies.



GramEval-2020: Evaluation

The cumulitive evaluation consists of:

  • POS tagging accuracy.
  • Morphological features accuracy.
  • LAS accuracy (syntax).
  • Lemmatization accuracy.

The testing procedure will include tests on “golden” texts in many genres and from different sources in Russian. We welcome systems that steadily process all the variety of texts in the Russian language (including texts that differ in style, scope and genre, region, time of creation).

GramEval-2020: Rules

Important Dates

  • February 1, 2020 - the release of gold and additional "dirty" training data obtained using automatic marking
  • February 24, 2020 - final submission
  • March 5, 2020 - announcement of the results

We are open to questions about data, metrics, and testing procedures.

Public Test

Start: Feb. 1, 2020, midnight

Private Test

Start: Feb. 23, 2020, 9 p.m.

Competition Ends

Feb. 25, 2020, 11:59 p.m.

You must be logged in to participate in competitions.

Sign In
# Username Score
1 qbic 0.91609
2 AlexeySorokin 0.90762
3 Shnurre 0.90411