Constraint@AAAI2021 - Hostile Post Detection in Hindi
Previous
Test PhaseCurrent
Post Test PhaseEnd
Competition EndsCOSTRAINT- First Workshop on Combating Online Hostile Posts in Regional Languages during Emergency Situation
Collocated with AAAI 2021
website https://constraint-shared-task-2021.github.io/
Bibtex
If you are a participant or a researcher using our dataset or find this work useful, please cite the following papers.
Overview:
@inproceedings{patwa2021overview,
title={Overview of CONSTRAINT 2021 Shared Tasks: Detecting English COVID-19 Fake News and Hindi Hostile Posts },
author={Parth Patwa and Mohit Bhardwaj and Vineeth Guptha and Gitanjali Kumari and Shivam Sharma and Srinivas PYKL and Amitava Das and Asif Ekbal and Shad Akhtar and Tanmoy Chakraborty},
booktitle = {Proceedings of the First Workshop on Combating Online Hostile Posts in Regional Languages during Emergency Situation ({CONSTRAINT})},
year = {2021},
publisher = {Springer},
}
@misc{patwa2020fighting,
title={Fighting an Infodemic: COVID-19 Fake News Dataset},
author={Parth Patwa and Shivam Sharma and Srinivas PYKL and Vineeth Guptha and Gitanjali Kumari and Md Shad Akhtar and Asif Ekbal and Amitava Das and Tanmoy Chakraborty},
year={2020},
eprint={2011.03327},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
@misc{bhardwaj2020hostility,
title={Hostility Detection Dataset in Hindi},
author={Mohit Bhardwaj and Md Shad Akhtar and Asif Ekbal and Amitava Das and Tanmoy Chakraborty},
year={2020},
eprint={2011.03588},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
Overview
Please note that the codalab scoring is not working. however, gold labels for the test set have been released.
Hostile Post Detection in Hindi -This subtask focuses on a variety of hostile posts in Hindi Devanagari script collected from Twitter and Facebook. The set of valid categories are fake news, hate speech, offensive, defamation, and non-hostile posts. It is a multi-label multi-class classification problem where each post can belong to one or more of these hostile classes. However, the non-hostile posts cannot be grouped with any other class.
Definitions of the class labels:
- Fake News: A claim or information that is verified to be not true.
- Hate Speech: A post targeting a specific group of people based on their ethnicity, religious beliefs, geographical belonging, race, etc., with malicious intentions of spreading hate or encouraging violence.
- Offensive: A post containing profanity, impolite, rude, or vulgar language to insult a targeted individual or group.
- Defamation: A mis-information regarding an individual or group.
- Non-hostile: A post without any hostility.
Examples
ये देखो इस्लाम क्या क्या सिखाता है जिहाद से लेकर आतंकवादी और दंगों से लेकर चोरी बुर्खे की आड़ में चद्दर चुराती महिलाएं {hate}
मोहतरमा JNU की 43 साल की छात्रा हैं , और कमाल की बात है कि उनकी बेटी मोना भी JNU में ही पड़ती है {Fake, Defamation}
जब इन दलितों को (सभी नहीं) हिन्दू धर्म और हिन्दू देवी देवताओं से इतनी नफरत भारी हुई है तो धूर्त कहीं के अपना नाम हिन्दुओं के जैसे ही क्यों रखते हैं। किसने रोका है कुछ भी बन से, बन जाओ मुस्लिम, ईसाई और जो मन करे। इस धूर्त की हिम्मत नहीं कि किसी दूसरे धर्म के बारे ऐसा बोल दे । {Hate, Offensive}
डॉक्टर कफ़ील ख़ान को हाईकोर्ट से मिली ज़मानत https://t.co/DH5WE370XT {Non-hostile}
For english task: https://competitions.codalab.org/competitions/26655
Evaluation
Official Competition Metric:The evaluation of this subtask will be two-dimensional as follows:
- Coarse-grained evaluation: It is a binary evaluation of hostile vs non-hostile posts. We are using weighted F1-score as evaluation metric for coarse grained hostility prediction.
- Fine-grained evaluation: Weighted F1-score of all four hostile-dimensions will be used to evaluate submissions for fine-grained prediction.
Additionally, we will also release the non-weighted F1 score of each hostile dimension as well.
Terms & Conditions
By submitting results to this competition, you consent to the public release of your scores at the constraint workshop and in the associated proceedings, at the task organizers' discretion. Scores may include but are not limited to, automatic and manual quantitative judgments, qualitative judgments, and such other metrics as the task organizers see fit. You accept that the ultimate decision of metric choice and score value is that of the task organizers.
You further agree that the task organizers are under no obligation to release scores and that scores may be withheld if it is the task organizers' judgment that the submission was incomplete, erroneous, deceptive, or violated the letter or spirit of the competition's rules. Inclusion of a submission's scores is not an endorsement of a team or individual's submission, system, or science.
You further agree that your system may be named according to the team name provided at the time of submission, or to a suitable shorthand as determined by the task organizers.
By downloading the data or by accessing it any manner, You agree not to redistribute the data except for the purpose of non-commercial and academic-research. The data must not be used for providing surveillance, analyses or research that isolates a group of individuals or any single individual for any unlawful or discriminatory purpose.
Important Dates:
- October 1, 2020: Release of the training set
- December 1, 2020: Release of the test set
- December 10, 2020: Deadline for submitting the final results
- December 12, 2020: Announcement of the results
- December 25, 2020: System paper submission deadline (All teams are invited to submit a paper)
Train: https://docs.google.com/spreadsheets/d/1xPHRnhcn-t_aCPhEiR_7-zNq_uxfHaxuWHvpHNdH6A0/edit?usp=sharing
Validation: https://docs.google.com/spreadsheets/d/1C-kuuykkvXTvC-mayqMjVtxFD-eBnFFFBNYHl1JI0dY/edit?usp=sharing
Test: https://docs.google.com/spreadsheets/d/1kOx_ylB2sR53jVdcEFfLfiLoKUEHqK0EXbHZRiJUTKY/edit?usp=sharing
Dr. Tanmoy Chakraborty Indraprastha Institute of Information Technology Delhi, India
Md. Shad Akhtar
Indraprastha Institute of Information Technology Delhi, India
Dr. Asif Ekbal
IIT Patna, India
Dr. Amitava Das
Wipro Research, Bangalore, India
Parth Patwa
Indian Institute of Information Technology Sri City, India
Mohit Bhardwaj
Indraprastha Institute of Information Technology Delhi, India
Srinivas PYKL
Indian Institute of Information Technology Sri City, India
Shivam Sharma
Wipro Research, Bangalore, India
Vineeth Guptha
Wipro Research, Bangalore, India
Gitanjali Kumari
IIT Patna, India
Validation Phase
Start: Oct. 1, 2020, midnight
Description: Please check the submission details for leaderboard
Test Phase
Start: Dec. 1, 2020, midnight
Description: Please check the submission details for leaderboard
Post Test Phase
Start: Dec. 11, 2020, midnight
Description: Please check the submission details for leaderboard
Competition Ends
Never
You must be logged in to participate in competitions.
Sign In