This is the website for the OffensEval 2019 shared task organized at SemEval 2019.
Offensive language is pervasive in social media. Individuals frequently take advantage of the perceived anonymity of computer-mediated communication, using this to engage in behavior that many of them would not consider in real life. Online communities, social media platforms, and technology companies have been investing heavily in ways to cope with offensive language to prevent abusive behavior in social media.
One of the most effective strategies for tackling this problem is to use computational methods to identify offense, aggression, and hate speech in user-generated content (e.g. posts, comments, microblogs, etc.). This topic has attracted significant attention in recent years as evidenced in recent publications (Waseem et al. 2017; Davidson et al., 2017, Malmasi and Zampieri, 2018, Kumar et al. 2018) and workshops such as AWL and TRAC.
In OffensEval we break down offensive content into three sub-tasks taking the type and target of offenses into account.
The data is retrieved from social media and distributed in tab separated format. The trial data is already available in the "Participate" tab under "Starting Kit". Please register to the competition to download the file.
The training data will be available on November 16, 2018.
Participants are allowed to use external resources and other datasets for this task. Please indicate which resources were used when submitting your results.
More information will be available with the training set release.
Davidson, T., Warmsley, D., Macy, M. and Weber, I. (2017) Automated Hate Speech Detection and the Problem of Offensive Language. Proceedings of ICWSM.
Kumar, R., Ojha, A.K., Malmasi, S. and Zampieri, M. (2018) Benchmarking Aggression Identification in Social Media. In Proceedings of the First Workshop on Trolling, Aggression and Cyberbullying (TRAC). pp. 1-11.
Malmasi, S., Zampieri, M. (2018) Challenges in Discriminating Profanity from Hate Speech. Journal of Experimental & Theoretical Artificial Intelligence. Volume 30, Issue 2, pp. 187-202. Taylor & Francis.
Waseem, Z., Davidson, T., Warmsley, D. and Weber, I. (2017) Understanding Abuse: A Typology of Abusive Language Detection Subtasks. Proceedings of the Abusive Language Online Workshop.
Classification systems will be evaluated using the macro-averaged F1-score.
Submission format information is available from the 'Participate' tab above.
Start: Dec. 12, 2018, midnight
Description: Submit practice predictions on the practice set. Use this to check your file format. A sample submission is available for download from the instructions page.
Start: Jan. 10, 2019, midnight
Description: Submit predictions for the test set.
Start: Feb. 1, 2019, midnight
Description: For evaluation after the competition ends. Submit additional test set predictions.
You must be logged in to participate in competitions.Sign In