Constraint@AAAI2021 - Hostile Post Detection in Hindi

Organized by parthpatwa - Current server time: Oct. 23, 2020, 10:03 p.m. UTC

Current

Validation Phase
Oct. 1, 2020, midnight UTC

Next

Test Phase
Dec. 1, 2020, midnight UTC

End

Competition Ends
Never

COSTRAINT- First Workshop on ​Combating On​line Ho​st​ile Posts in ​Regional L​anguages dur​ing Emerge​ncy Si​tuation

Collocated with AAAI 2021

website https://constraint-shared-task-2021.github.io/

Overview

Hostile Post Detection in Hindi -This subtask focuses on a variety of hostile posts in Hindi Devanagari script collected from Twitter and Facebook. The set of valid categories are fake news, hate speech, offensive, defamation, and non-hostile posts. It is a multi-label multi-class classification problem where each post can belong to one or more of these hostile classes. However, the non-hostile posts cannot be grouped with any other class.
Definitions of the class labels:

  • Fake News: A claim or information that is verified to be not true.
  • Hate Speech: A post targeting a specific group of people based on their ethnicity, religious beliefs, geographical belonging, race, etc., with malicious intentions of spreading hate or encouraging violence.
  • Offensive: A post containing profanity, impolite, rude, or vulgar language to insult a targeted individual or group.
  • Defamation: A mis-information regarding an individual or group.
  • Non-hostile: A post without any hostility.


Examples

ये देखो इस्लाम क्या क्या सिखाता है जिहाद से लेकर आतंकवादी और दंगों से लेकर चोरी बुर्खे की आड़ में चद्दर चुराती महिलाएं {hate}
मोहतरमा JNU की 43 साल की छात्रा हैं , और कमाल की बात है कि उनकी बेटी मोना भी JNU में ही पड़ती है {Fake, Defamation}
जब इन दलितों को (सभी नहीं) हिन्दू धर्म और हिन्दू देवी देवताओं से इतनी नफरत भारी हुई है तो धूर्त कहीं के अपना नाम हिन्दुओं के जैसे ही क्यों रखते हैं। किसने रोका है कुछ भी बन से, बन जाओ मुस्लिम, ईसाई और जो मन करे। इस धूर्त की हिम्मत नहीं कि किसी दूसरे धर्म के बारे ऐसा बोल दे । {Hate, Offensive}
डॉक्टर कफ़ील ख़ान को हाईकोर्ट से मिली ज़मानत https://t.co/DH5WE370XT {Non-hostile}

 

For english task: https://competitions.codalab.org/competitions/26655

Evaluation

Official Competition Metric: We will use F1 averaged across the classes (fake , real). The final ranking would be based on the weighted average F1 score.
However will release recall, precision, accuracy also.

Each participating team will initially have access to the training data and validation data only. Later, the unlabeled test data will be released.

The evaluation of this subtask will be two-dimensional as follows:

  1. Coarse-grained evaluation: It is a binary evaluation of hostile vs non-hostile posts.
  2. Fine-grained evaluation: It is a fine-grained evaluation of the hostile classes.

Terms & Conditions

By submitting results to this competition, you consent to the public release of your scores at the constraint workshop and in the associated proceedings, at the task organizers' discretion. Scores may include but are not limited to, automatic and manual quantitative judgments, qualitative judgments, and such other metrics as the task organizers see fit. You accept that the ultimate decision of metric choice and score value is that of the task organizers.

You further agree that the task organizers are under no obligation to release scores and that scores may be withheld if it is the task organizers' judgment that the submission was incomplete, erroneous, deceptive, or violated the letter or spirit of the competition's rules. Inclusion of a submission's scores is not an endorsement of a team or individual's submission, system, or science.

You further agree that your system may be named according to the team name provided at the time of submission, or to a suitable shorthand as determined by the task organizers.

By downloading the data or by accessing it any manner, You agree not to redistribute the data except for the purpose of non-commercial and academic-research. The data must not be used for providing surveillance, analyses or research that isolates a group of individuals or any single individual for any unlawful or discriminatory purpose.

For any queries contact us on:

 tanmoy@iiitd.ac.in 

parthprasad.p17@iiits.in

Important Dates:

  • October 1, 2020: Release of the training set
  • December 1, 2020: Release of the test set
  • December 10, 2020: Deadline for submitting the final results
  • December 12, 2020: Announcement of the results
  • December 30, 2020: System paper submission deadline (All teams are invited to submit a paper)
Dr. Tanmoy Chakraborty
Indraprastha Institute of Information Technology Delhi, India

Md. Shad Akhtar
Indraprastha Institute of Information Technology Delhi, India

Dr. Asif Ekbal
IIT Patna, India

Dr. Amitava Das
Wipro Research, Bangalore, India

Parth Patwa
Indian Institute of Information Technology Sri City, India

Mohit Bhardwaj
Indraprastha Institute of Information Technology Delhi, India

Srinivas PYKL
Indian Institute of Information Technology Sri City, India

Shivam Sharma
Wipro Research, Bangalore, India

Vineeth Gupta
Wipro Research, Bangalore, India

Gitanjali Kumari
IIT Patna, India

The leaderboard is not working. You can calculate results on the validation data as the labels are released.

We will update this page once leaderboard starts working.

 

Validation Phase

Start: Oct. 1, 2020, midnight

Test Phase

Start: Dec. 1, 2020, midnight

Post Test Phase

Start: Dec. 11, 2020, midnight

Competition Ends

Never

You must be logged in to participate in competitions.

Sign In