Competition

1st “Monolingual Word Sense Alignment” Shared Task

Call for Participation

The ELEXIS project is organizing a shared task on the task of monolingual word sense alignment across dictionaries as part of the GLOBALEX 2020 – Linked Lexicography workshop at the 12th Language Resources and Evaluation Conference (LREC 2020) taking place on Tuesday, May 12 2020 in Marseille (France).

Monolingual word sense alignment is a challenging task of finding matching senses between two dictionary entries and will play a crucial role in the development of new lexical resources. In addition, this task presents a challenging combination of NLP, semantic textual similarity and reasoning in order to find the best alignment across a group of senses.

Description of Task

The task of monolingual word sense alignment is presented as a task of predicting the relationship between two senses in one of five categories: “exact”, “broader”, “narrower”, “related” or “none”. For each sense pair the following information will be provided:

• The lemma shared between the two entries
• The part of speech of the entries*
• The sense text (including definition) of the sense of the first entry
• The sense text (including definition) of the sense of the second entry
• (Training Data) The label of the relation (“exact”, “broader”, “narrower”, “related” or “none”)

For each pair of entry all mappings between senses will be provided, as such we expect the best systems to consider the mapping of an entry as a block.

Training data will be available for monolingual dictionaries in the following languages:

• Basque‡
• Bulgarian‡
• Danish
• Dutch
• English †
• Estonian
• German
• Hungarian*
• Irish
• Italian
• Portuguese‡
• Serbian
• Slovenian
• Spanish‡
• Russian

*For Hungarian part-of-speech information is not provided
† There are two English evaluation tasks with different dictionaries
‡ Data for these languages will be released in February

Participants may participate in any or all of the above languages. The test data will consist of a group of entries with the label of the relation missing, participants should submit the result in the same form of the training data, that is the test data with the predicted label.

Example data

Publication of Results

Participants will submit a system paper that should include a description of the system, the way the data has been processed, the applied algorithms, the obtained results, as well as the conclusions and ideas for future improvements. The papers will be peer reviewed prior to publication to confirm that all aspects are well covered.

The workshop will accept also regular papers from participants who are not participating in the shared task but still have worked in the topic of translation inference and want to publish novel results or ideas, maybe with different datasets and experimental basis as the ones proposed in this shared task. Such papers will be peer reviewed on the basis of their scientific quality.

All the accepted papers will be published as part of the Globalex workshop proceedings and presented during the workshop.

Important Dates

6/12/2019 – Call for participation
12/12/2019 – Technical description of the evaluation process and data provided by organisers
13/03/2020 01/04/2020 – Submission of results by participants / submission of regular papers
03/04/2020 – Evaluation results communicated by organisers / notification of regular papers
24/04/2020 – Submission of system description papers
12/05/2020 – Workshop day

Organizers

John P. McCrae - Data Science Institute, National University of Ireland Galway Sina Ahmadi - Data Science Institute, National University of Ireland Galway

Review Committee

To be announced