SemEval-2021 Task 12 - Learning with Disagreements

Organized by alexandraG - Current server time: July 25, 2024, 12:19 a.m. UTC


Evaluation Phase
Jan. 10, 2021, midnight UTC


Feb. 16, 2021, noon UTC


Competition Ends

Practice Phase

Start: July 31, 2020, midnight

Description: This is the practice phase of the competition. In this phase, you are provided with datasets for tasks, starting with two tasks - PDIS and CIFAR10-IC. You are expected to craft novel approaches for training the models using the crowd labels. You may use the base models in as a starting point. Please Note: The AURC-8 Column actually shows the Humour results.

Evaluation Phase

Start: Jan. 10, 2021, midnight

Description: In this phase of the competition, the test data is released. Participants are expected to use the models trained in the practice phase to make predictions on the test data and make submissions. A total of 5 submissions are permitted. The submissions are then tested using the test dataset. Please Note: The AURC-8 Column actually shows the Humour results.


Start: Feb. 16, 2021, noon

Description: The official competition ends in the evaluation phase. However, the Post-competition phase allows you to continue to refine and test your models.

Competition Ends


You must be logged in to participate in competitions.

Sign In
# Username Score
1 oseibrefo 0.5421
2 alexandraG 0.1199
3 Grp4LeaDis 0.0208