Automated Evaluation of Scientific Writing Shared Task 2016

Secret url: https://competitions.codalab.org/competitions/8711?secret_key=6bee2156-ccd2-45e3-827c-03b1f1f30cc2
Organized by vidas.daudaravicius - Current server time: Dec. 15, 2018, 5:10 a.m. UTC

Previous

Testing. Probabilistic estimation
Feb. 29, 2016, midnight UTC

Current

Testing. Probabilistic estimation
Feb. 29, 2016, midnight UTC

End

Competition Ends
March 14, 2016, midnight UTC

Task Objective and Definition

The objective of the AESW Shared Task is to promote the use of NLP tools to help ELL writers improve the quality of their scientific writing.

The main goals of the task are

  • to identify sentence-level features that are unique to scientific writing;
  • to provide a common ground for development and comparison of sentence-level automated writing evaluation systems for scientific writing;
  • to establish the state-of-the-art performance in the field.
Some interesting uses of sentence-level quality evaluations are the following:
  • automated writing evaluation of submitted scientific articles;
  • authoring tools for writing English scientific texts;
  • filtering out sentences that need quality improvement.
The task will examine automated evaluation of scientific writing at the sentence-level by using the output of professionally edited scientific texts. The dataset contains aligned text extracts before and after they were edited by native English speakers.

The goal of the task is to predict whether a given sentence needs language editing to improve it. The task is a binary classification task. Two types of predictions will be evaluated: binary prediction (False or True) and probabilistic estimation (between 0 and 1).

See more details at http://textmining.lt/aesw/index.html.

Detection score

The task is a binary classification task. Two types of predictions will be evaluated: binary prediction (False or True) and probabilistic estimation (between 0 and 1). We will measure the detection score of sentences that need improvement, i.e., were edited by language editors. The score will be an F-score of sentences that need improvement. See more details at http://textmining.lt/aesw/index.html.

Development. Boolean decision

Start: Dec. 7, 2015, midnight

Description: Submission of Boolean decision results. Submit zipped 'dev.bool' file, e.g., 'dev.bool.zip'. Each line should contain a pair of sentence ID and True|False separated with TAB in between.

Development. Probabilistic estimation

Start: Dec. 7, 2015, midnight

Description: Submission of Probabilistic estimation results. Submit zipped 'dev.prob' file, e.g., 'dev.prob.zip'. Each line should contain a pair of sentence ID and Float number separated with TAB in between.

Testing. Boolean decision

Start: Feb. 28, 2016, midnight

Description: Submission of Boolean decision results. Submit zipped 'test.bool' file, e.g., 'test.bool.zip'. Each line should contain a pair of sentence ID and True|False separated with TAB in between.

Testing. Probabilistic estimation

Start: Feb. 29, 2016, midnight

Description: Submission of Probabilistic estimation results. Submit zipped 'test.prob' file, e.g., 'test.prob.zip'. Each line should contain a pair of sentence ID and Float number separated with TAB in between.

Competition Ends

March 14, 2016, midnight

You must be logged in to participate in competitions.

Sign In
# Username Score
1 lunghaolee 0.7678
2 knowlet 0.7294
3 sweaglesw 0.7118