Hope Speech Detection for Equality, Diversity, and Inclusion-EACL 2021
First phase
First phaseEnd
Competition EndsShared Task on Hope Speech Detection for Equality, Diversity, and Inclusion at LT-EDI 2021- EACL 2021
Hope is considered significant for the well-being, recuperation and restoration of human life by health professionals. Hope speech reflects the belief that one can discover pathways to their desired objectives and become roused to utilise those pathways[1-5]. Our work aims to change the prevalent way of thinking by moving away from a preoccupation with discrimination, loneliness or the worst things in life to building the confidence, support and good qualities based on comments by individuals. The goal of this task is to identify whether a given comment contains hope speech or not. We define the hope speech for our problem as "YouTube comments / posts that offer support, reassurance, suggestions, inspiration and insight". A comment / post within the corpus may contain more than one sentence but the average sentence length of the corpora is 1. The annotations in the corpus are made at a comment / post level.
The participants will be provided development, training and test dataset in English, Tamil, and Malayalam. To download the data and participate, go to codalab and click “Participate" tab. As far as we know, this is the first shared task on Hope Speech Detection.
Task:
This is a comment / post level classification task. Given a Youtube comment, the systems submitted by the participants should classify it into 'Hope speech', 'Not hope speech' and 'Not in intended language'. To download the data and participate, go to the Participate tab.
As far as we know, this is the first shared task on Hope Speech detection. More details at https://sites.google.com/view/lt-edi-2021
Results:
Paper name format should be: TEAM_NAME@LT-EDI-EACL2021: Title of the paper.
Example: NUIG_ULD@LT-EDI-EACL2021: Hope Speech Detection for Equality, Diversity, and Inclusion
For electronic submission of papers to LT-EDI workshop please use this link: https://www.softconf.com/eacl2021/LTEDI2021/
Evaluation Criteria
| id | text | label |
|---|---|---|
| 1 | காற்றில் பரவும் வாய்ப்பு உள்ளது | Non_hope_speech |
| 2 | Mrs.Sudha moorthy pathi pesungalen... | Hope_speech |
| 3 | Realme india product | not-Tamil |
- label column should only have labels in the form mentioned in the training data i.e. 'Hope_speech', 'Non_hope_speech', 'not-Tamil/not-English/not-Malayalam'.
- id column is the index of the row. (Retain the sequence of the Youtuebe comments in the same order as provided in the test data)
Classification system’s performance will be measured in terms of weighted averaged Precision, weighted averaged Recall and weighted averaged F-Score across all the classes. Weighted averaged are averaging the support-weighted mean per label. Participants are encouraged to check their system with Sklearn classification report https://scikit-learn.org/stable/modules/generated/sklearn.metrics.classification_report.html
Results
Terms and Conditions
By downloading the data or by accessing it any manner, you agree not to redistribute the data except for non-commercial and academic-research purposes. The data must not be used for providing surveillance, analyses or research that isolates a group of individuals or any single individual for any unlawful or discriminatory purpose.
You should cite this papers if you are using our data.
https://www.aclweb.org/anthology/2020.peoples-1.5/
@inproceedings{chakravarthi-2020-hopeedi,
title = "{H}ope{EDI}: A Multilingual Hope Speech Detection Dataset for Equality, Diversity, and Inclusion",
author = "Chakravarthi, Bharathi Raja",
booktitle = "Proceedings of the Third Workshop on Computational Modeling of People's Opinions, Personality, and Emotion's in Social Media",
month = dec,
year = "2020",
address = "Barcelona, Spain (Online)",
publisher = "Association for Computational Linguistics",
url = "https://www.aclweb.org/anthology/2020.peoples-1.5",
pages = "41--53",
abstract = "Over the past few years, systems have been developed to control online content and eliminate abusive, offensive or hate speech content. However, people in power sometimes misuse this form of censorship to obstruct the democratic right of freedom of speech. Therefore, it is imperative that research should take a positive reinforcement approach towards online content that is encouraging, positive and supportive contents. Until now, most studies have focused on solving this problem of negativity in the English language, though the problem is much more than just harmful content. Furthermore, it is multilingual as well. Thus, we have constructed a Hope Speech dataset for Equality, Diversity and Inclusion (HopeEDI) containing user-generated comments from the social media platform YouTube with 28,451, 20,198 and 10,705 comments in English, Tamil and Malayalam, respectively, manually labelled as containing hope speech or not. To our knowledge, this is the first research of its kind to annotate hope speech for equality, diversity and inclusion in a multilingual setting. We determined that the inter-annotator agreement of our dataset using Krippendorff{'}s alpha. Further, we created several baselines to benchmark the resulting dataset and the results have been expressed using precision, recall and F1-score. The dataset is publicly available for the research community. We hope that this resource will spur further research on encouraging inclusive and responsive speech that reinforces positiveness.",
}
@inproceedings{dravidianhopespeech-eacl,
title={Findings of the Shared Task on {H}ope {S}peech {D}etection for {E}quality, {D}iversity, and {I}nclusion},
author={Chakravarthi, Bharathi Raja and
Muralidaran, Vigneshwaran},
booktitle = "Proceedings of the First Workshop on Language Technology for Equality, Diversity and Inclusion",
month = April,
year = "2021",
publisher = "Association for Computational Linguistics",
year={2021}
}
Important Dates for shared task:
Task announcement: Nov 20, 2020
Release of Trail data: Nov 20, 2020
Release of Training data: Nov 20, 2020
Release of Test data: Jan 2, 2021
Run submission deadline: Jan 10, 2021
Results declared: Jan 15, 2021
Paper submission: Jan 30, 2021
Peer review notification: Feb 18, 2021
Camera-ready paper due: Feb 24, 2021
Workshop Dates: April 19-20, 2021
Bharathi Raja Chakravarthi, Insight SFI Research Centre for Data Analytics, Data Science Institute, National University of Ireland Galway
Vigneshwaran Muralidaran, School of Computer Science and Informatics, Cardiff University, United Kingdom
Email: equalitydiversityinclusion.lang@gmail.com and bharathiraja.akr@gmail.com
First phase
Start: Nov. 24, 2020, midnight
Competition Ends
Jan. 15, 2021, 8:07 a.m.
You must be logged in to participate in competitions.
Sign In