Arabic has a widely varying collection of dialects. Many of these dialects remain under-studied due to the rarity of resources. The goal of the shared task is to alleviate this bottleneck in the context of fine-grained Arabic dialect identification. Dialect identification is the task of automatically detecting the source variety of a given text or speech segment. Previous work on Arabic dialect identification has focused on coarse-grained regional varieties such as Gulf or Levantine (e.g., Zaidan and Callison-Burch, 2013; Elfardy and Diab, 2013; Elaraby and Abdul-Mageed, 2018) or country-level varieties (e.g., Bouamor et al., 2018; Zhang and Abdul-Mageed, 2019) such as the MADAR shared task in WANLP 2019 (Bouamor, Hassan, and Habash, 2019). The MADAR shared task also involved city-level classification on human translated data. (To receive access to the data, teams intending to participate are invited to fill in the form on the official website of NADI shared task. ).
The Nuanced Arabic Dialect Identification (NADI) shared task targets province-level dialects, and as such will be the first to focus on naturally-occurring fine-grained dialect at the sub-country level. The data covers a total of 100 provinces from all 21 Arab countries and comes from the Twitter domain. Evaluation and task set up follow the MADAR 2019 shared task. The subtasks involved include:
Participants will also be provided with an additional 10M unlabeled tweets that can be used in developing their systems for either or both of the tasks.
The evaluation metrics will include precision/recall/f-score/accuracy. Macro Averaged F-score will be the official metric.
Participating teams will be provided with a common training data set and a common development set. No external manually labelled data sets are allowed. A blind test data set will be used to evaluate the output of the participating teams. All teams are required to report on the development and test set in their writeups.
Please visit the official website of the NADI shared task for more information.
For any questions related to this task, please contact the organizers directly using the following email address: email@example.com
Metrics: The evaluation metrics will include precision/recall/f-score/accuracy. Macro Averaged F-score will be the official metric.
To receive access to the data, teams intending to participate are invited to fill in the form on the official website.
Copyright (c) 2020 The University of British Columbia, Canada; Carnegie Mellon University Qatar; New York University Abu Dhabi. All rights reserved.
Start: Jan. 30, 2020, midnight
Description: Development phase: Develop your models and submit prediction labels on the DEV set of subtask 1. Note: The name of your submission should be 'teamname_subtask1_dev_numberOFsubmission.zip' that includes a text file of your prediction (e.g., A submission 'UBC_subtask1_dev_1.zip' that is the zip file of my first prediction for subtask 1 on DEV set containing 'UBC_subtask1_dev_1.txt' file.)
Start: March 31, 2020, noon
Description: Test phase: Submit your prediction labels on the TEST set of subtask 1. Each team is allowed a maximum of 3 submissions. Note: The name of your submission should be 'teamname_subtask1_test_numberOFsubmission.zip' that includes a text file of your predictions (e.g., A submission 'UBC_subtask1_test_1.zip' that is the zip file of my prediction for subtask 1 on TEST set containing the 'UBC_subtask1_test_1.txt' file.)
Start: June 6, 2020, noon
Description: Post-Evaluation: Submit your prediction on the TEST set of subtask 1 after the competition deadline. The name of your submission should be 'teamname_subtask1_test_numberOFsubmission.zip' that includes a text file of your predictions (e.g., A submission 'UBC_subtask1_test_1.zip' that is the zip file of my prediction for subtask 1 on TEST set containing the 'ubc_subtask1_test_1.txt' file.)
You must be logged in to participate in competitions.Sign In