Linking mathematical symbols to their descriptions
Previous
EvaluationCurrent
Post-EvaluationEnd
Competition EndsWelcome to SymLink Challenge!
Mathematical symbols and descriptions appear in various forms across document section boundaries without explicit markup. Linking mathematical symbols and their descriptions has been conducted for a decade. However, the current state of research is distant toward real application. We believe that this competition will facilitate studies in understanding scientific text. As a result, we expect that it can increase the accessibility of scientific documents to less experienced and visually impaired readers while making their consumption more efficient to all.
With this competition, we encourage researcher to extract symbols and descriptions in scientific documents. The data contains documents from 5 domains including: math, physics, biology, computer science, and economy.
Get started
- Register here to get access
- Download the data after approval
- Check our Google Groups and FAQ
- Submit your results
- Win the Challenge
Important dates
- 2022 tasks announced: August 6, 2021
- Training data ready: September 3, 2021
- Evaluation data ready: December 3, 2021
- Evaluation start: January 10, 2022
- Evaluation end: by January 31, 2022
- Paper submissions due: roughly February 23, 2022
- Notification to authors: March 31, 2022
- Camera ready: TBD
- SemEval workshop: Summer 2022
Contacts
Please send inquiries to semeval.2022.task.12.symlink@gmail.com
For more updates, join our Google group: semeval-2022-task-12-symlink@googlegroups.com
Sponsors
Submissions
The participants should submit the output in JSON format with the same fields as the provided data.
Evaluation
The submitted results will evaluated separately for Named Entity Recognition (NER) task and Relation Extraction (RE) task. For NER, we use the entity-based strict/exact/partial/type from SemEval 2013 Task 9.1. The description NER and formula NER will be compute separately. For RE, we use standard precision, recall, F-score metrics. Relations output by participating system are correct if the prediction label matches in the gold standard.
Terms and Conditions
The dataset provided for this competition is licensed under CC BY-NC-SA 4.0 international license, and the evaluation script and the baseline are licensed under MIT license. By accepting the terms and conditions you agree that:
- Organizers have the right to publicly release the team name, the affiliation of the teams, and the scores (including all metrics computed in the evaluation script) in the upcoming publications.
- Organizers have the right to exclude the results of the teams that do not comply with the fair competition rules (e.g., by deceptive or erroneous results)
- You will not redistribute the dataset (including the training/development/test data)
- You will provide enough description of the details of the model used to make predictions on the test set (including but not limited to model architecture, training settings, word embedding types, etc)
Thien Huu Nguyen, University of Oregon, USA
Viet Dac Lai, University of Oregon, USA
Amir Pouran Ben Veyseh, University of Oregon, USA
Franck Dernoncourt, Adobe Research, USA
Practice
Start: Aug. 1, 2021, midnight
Evaluation
Start: Jan. 10, 2022, midnight
Post-Evaluation
Start: Jan. 31, 2022, midnight
Competition Ends
Never
You must be logged in to participate in competitions.
Sign In