Constraint@AAAI2021 - COVID19 Fake News Detection in English

Organized by parthpatwa - Current server time: Jan. 18, 2021, 8:38 a.m. UTC

Previous

Test Phase
Dec. 2, 2020, noon UTC

Current

Post Test Phase
Dec. 11, 2020, midnight UTC

End

Competition Ends
Never

COSTRAINT- First Workshop on ​Combating On​line Ho​st​ile Posts in ​Regional L​anguages dur​ing Emerge​ncy Si​tuation

Collocated with AAAI 2021

website https://constraint-shared-task-2021.github.io/

Bibtex

If you are a participant or a researcher using our dataset or find this work useful, please cite the following papers. 

Overview: 

@inproceedings{patwa2021overview,
title={Overview of CONSTRAINT 2021 Shared Tasks: Detecting English COVID-19 Fake News and Hindi Hostile Posts },
author={Parth Patwa and Mohit Bhardwaj and Vineeth Guptha and Gitanjali Kumari and Shivam Sharma and Srinivas PYKL and Amitava Das and Asif Ekbal and Shad Akhtar and Tanmoy Chakraborty},
booktitle = {Proceedings of the First Workshop on Combating Online Hostile Posts in Regional Languages during Emergency Situation ({CONSTRAINT})},
year = {2021},
publisher = {Springer},
}

English   (https://arxiv.org/ftp/arxiv/papers/2011/2011.03327.pdf):  

@misc{patwa2020fighting,
title={Fighting an Infodemic: COVID-19 Fake News Dataset}, 
author={Parth Patwa and Shivam Sharma and Srinivas PYKL and Vineeth Guptha and Gitanjali Kumari and Md Shad Akhtar and Asif Ekbal and Amitava Das and Tanmoy Chakraborty},
year={2020},
eprint={2011.03327},
archivePrefix={arXiv},
primaryClass={cs.CL}
}

Hindi (https://arxiv.org/pdf/2011.03588.pdf): 

@misc{bhardwaj2020hostility,
title={Hostility Detection Dataset in Hindi}, 
author={Mohit Bhardwaj and Md Shad Akhtar and Asif Ekbal and Amitava Das and Tanmoy Chakraborty},
year={2020},
eprint={2011.03588},
archivePrefix={arXiv},
primaryClass={cs.CL}
}

 

Overview

Please note that the codalab scoring is not working. however, gold labels for the test set have been released. 

COVID19 Fake News Detection in English - This subtask focuses on the detection of COVID19-related fake news in English. The sources of data are various social-media platforms such as Twitter, Facebook, Instagram, etc. Given a social media post, the objective of the shared task is to classify it into either fake or real news. For example, the following two posts belong to fake and real categories, respectively. 

If you take Crocin thrice a day you are safe. Fake
Wearing mask can protect you from the virus. Real

 

For the Hindi task: https://competitions.codalab.org/competitions/26654

Evaluation

Official Competition Metric: We will use F1 averaged across the classes (fake , real). The final ranking would be based on the weighted average F1 score.
However will release recall, precision, accuracy also.

Each participating team will initially have access to the training data and validation data only. Later, the unlabeled test data will be released.

Terms & Conditions

By submitting results to this competition, you consent to the public release of your scores at the constraint workshop and in the associated proceedings, at the task organizers' discretion. Scores may include but are not limited to, automatic and manual quantitative judgments, qualitative judgments, and such other metrics as the task organizers see fit. You accept that the ultimate decision of metric choice and score value is that of the task organizers.

You further agree that the task organizers are under no obligation to release scores and that scores may be withheld if it is the task organizers' judgment that the submission was incomplete, erroneous, deceptive, or violated the letter or spirit of the competition's rules. Inclusion of a submission's scores is not an endorsement of a team or individual's submission, system, or science.

You further agree that your system may be named according to the team name provided at the time of submission, or to a suitable shorthand as determined by the task organizers.

By downloading the data or by accessing it any manner, You agree not to redistribute the data except for the purpose of non-commercial and academic-research. The data must not be used for providing surveillance, analyses or research that isolates a group of individuals or any single individual for any unlawful or discriminatory purpose.

For any queries contact us on:

 tanmoy@iiitd.ac.in 

parthprasad.p17@iiits.in

Important Dates:

  • October 1, 2020: Release of the training set
  • December 1, 2020: Release of the test set
  • December 10, 2020: Deadline for submitting the final results
  • December 12, 2020: Announcement of the results
  • December 25, 2020: System paper submission deadline (All teams are invited to submit a paper)
Dr. Tanmoy Chakraborty
Indraprastha Institute of Information Technology Delhi, India

Md. Shad Akhtar
Indraprastha Institute of Information Technology Delhi, India

Dr. Asif Ekbal
IIT Patna, India

Dr. Amitava Das
Wipro Research, Bangalore, India

Parth Patwa
Indian Institute of Information Technology Sri City, India

Mohit Bhardwaj
Indraprastha Institute of Information Technology Delhi, India

Srinivas PYKL
Indian Institute of Information Technology Sri City, India

Shivam Sharma
Wipro Research, Bangalore, India

Vineeth Guptha
Wipro Research, Bangalore, India

Gitanjali Kumari
IIT Patna, India

Submit the data in a .zip file. The zip file should have a single file named 'answer.txt'. The 'answer.txt' should have 2 columns named id and label, where id is the unique id of the tweet. Ensure that there is no empty blank line at the end of the file. 

example:

id,label

1,fake

2,real

 

Important Details:

1) For test phase, BE SURE to predict on the test set only. 

2) You are allowed ONLY 5 submissions throughout the test phase. The best one will be selected for leaderboard. 

3) Daily submissions are limited to 1.

4) Leaderboard will be hidden during the test phase and will be made public afterwards. 

5) The text file you are submitting should be named as "answer.txt"

6) The submission file should be .zip file

7)  The precision, recall and F-score shown in output score file are not correct and should not be used to judge the system.

In case codalab shows that you have exhausted your submissions:

➔ Anytime during the test phase, send us a single email, having 5 CSVs with the following naming
conventions:
o team_name_1.csv
o team_name_2.csv
o team_name_3.csv
o team_name_4.csv
o team_name_5.csv
➔ The email should be sent to: constraintsharedtask@gmail.com
➔ Email subject should be [English Fake News Task Submission – Team Name]
➔ Only one member of the team should email us.
➔ In the body of the email add 2-3 lines describing method of each submission.
➔ Only the best submission out of your five submissions will be reflected on the leader board.

In this case ONLY the email submissions will be considered. 

 

Validation Phase

Start: Oct. 1, 2020, midnight

Test Phase

Start: Dec. 2, 2020, noon

Post Test Phase

Start: Dec. 11, 2020, midnight

Competition Ends

Never

You must be logged in to participate in competitions.

Sign In