Natural Language to Code Generation on CoNaLa

Organized by edgarc - Current server time: May 23, 2019, 7:45 p.m. UTC


First phase
May 31, 2018, midnight UTC


Competition Ends


Welcome to the site of CoNaLa, the Code/Natural Language Challenge! This challenge was designed to test systems for generating programs from natural language. For example, if the input is "sort list x in reverse order", then the system would be required to output x.sort(reverse=True) in Python.

More information on the challenge, including data, etc. can be found here:


Submitting for Evaluation

In your submission, please include a zipped .json file containing a list of hypothesis code strings. To submit, upload a .zip file with answer.txt containing the .json content, and please make sure you set your submission as public so it will appear on the list of submissions.

Evaluation consists of computing the BLEU score between the reference and hypothesis tokens. The tokens are treated as words.  The code will then be tokenized and compared with the reference. 

First phase

Start: May 31, 2018, midnight

Competition Ends


You must be logged in to participate in competitions.

Sign In
# Username Score
1 YYY 33.51
2 lalala 32.54
3 NoSegFault 28.62