Dear organizers:
I saw the Submission Details in the page https://competitions.codalab.org/competitions/17184#participate-get_data.
Each line of the file should have 3 columns, separated by a comma. Each of the three columns should contain a number: The first column is the ID of a text, the second column the ID of the question for the text, and the third one the ID of the correct answer. The following example illustrates the format:
0,1,1
0,2,0
In the examples, the first question of text 0 has the correct answer 1. The second question has the correct answer 0.
Please note that you need to provide IDs for every question, otherwise the submission won't work. Also, the file should not contain any empty lines (including the end of the file!).
Should each line in the result file be sorted in the order of the articles and the questions of the articles？
for example:
0,1,1
0,2,0
0,3,1
0,4,1
0,5,0
1,1,1
1,2,1
1,3,0
.........
If I submit result in the following order：
6,5,1
1,2,0
3,4,1
3,3,1
3,2,0
6,1,1
..........
Does the result of above submission format conform to the requirements ？(I provide provide IDs for every question of all articles and ensure no empty line in result file, just a mess in the order. )
Thank you for your answer!
Hi,
changing the order should not be a problem and will work.
Best
Simon
Dear Simon, thank you for your answer!
Dear organizers:
I'm so sorry, today I found that the answer file I submitted contains a blank line at the end of the file , and the error is shown below：
Traceback (most recent call last):
File "/tmp/codalab/tmpg9VuOq/run/program/evaluation.py", line 32, in <module>
gold[spl[0].strip() + "," + spl[1].strip()] = spl[2].strip()
IndexError: list index out of range
Could you give me a chance to submit it again? I'm sorry to bother you.
Posted by: lanman @ Feb. 2, 2018, 6:41 a.m.