**NOTE: The SemEval-2020 workshop has been accepted to co-locate with COLING-2020 and will take place on September 13-14. The evaluation will start on Feb 19 but not Jan 10. Please find all our new tentative dates below.
**NOTE: This is a Natural Language Processing task and No experience in Computer Vision or Graphic Design is needed.
**NOTE: Training and development sets are available to download.
Visual communication relies heavily on images and short texts. Whether it is flyers, posters, ads, social media posts or motivational messages, it is usually highly designed to grab a viewer’s attention and convey a message in the most efficient way. For text, word emphasis is used to better capture the intent, removing the ambiguity that may exist in plain text. Word Emphasis can clarify or even change the meaning of a sentence by drawing attention to some specific information, and it can be done with Colors, Backgrounds, or Fonts, Italic and Boldface. Our shared task is designed to invite research in this area. We are expecting to see a variety of traditional and modern NLP techniques to model emphasis. Whether you are an expert or new in Natural Language Processing, we encourage you to participate in this fun new task.
The purpose of this shared task is to design automatic methods for emphasis selection, i.e. choosing candidates for emphasis in short written text, to enable automated design assistance in authoring.
Here are some examples from our dataset:
No additional context from the user or the rest of the design such as background image is provided. The datasets contain very short texts, usually fewer than 10 words. Word emphasis patterns are author- and domain-specific. Without knowing the author’s intent and only considering the input text, multiple emphasis selections are valid. A good model, however, should be able to capture the inter-subjectivity or common sense within the given annotations and finally label words according to higher agreements.
We will announce the best paper award for each of the following categories:
We encourage all teams to describe their submission in a SemEval-2020 paper (ACL format), including teams with negative results.
We encourage all teams to open source their implementations.
During the evaluation phase, only the final valid submissions on CodaLab will be taken as the official submissions to the competition.
Feel free join the Google group for task-related news and discussions: email@example.com
Competition website: http://ritual.uh.edu/semeval2020-task10-emphasis-selection/
“Learning Emphasis Selection for Written Text in Visual Media from Crowd-Sourced Label Distributions”, 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019)
Reza (Amirreza) Shirani, University of Houston
Franck Dernoncourt, Adobe Research
Jose Echevarria, Adobe Research
Nedim Lipka, Adobe Research
Paul Asente, Adobe Research
Thamar Solorio, University of Houston
Matchm: For each instance X in the test set Dtest, we select a set Sm(x) of m ∊ (1. . .4) words with the top m probabilities according to the ground truth. Analogously, we select a prediction set set S^m(x) for each m, based on the predicted probabilities.
We define the metric Matchm as follows:
This page enumerated the terms and conditions of the competition.
Start: July 30, 2019, midnight
Start: Sept. 4, 2019, midnight
Start: Feb. 19, 2020, midnight
Start: March 11, 2020, midnight
You must be logged in to participate in competitions.Sign In