SemEval-2020 Task5: Modelling Causal Reasoning in Language: Detecting Counterfactuals Forum

Go back to competition Back to thread list Post in this thread

> Subtask 2 - evaluate2- submission format

we have already submitted our results for the competition.
However, to make sure our submission is ok, we have submitted about 10% of training data as a result as a practise submission. In the evaluation phase, the submission look to be ok (no error happened during submission). However, in practice phase, we ran into the following error:

Sorry, the sentence id is not matched, please check the sentence id in you submission.Practice stage: from 200000 to 203550 in increasing order, please do not shuffle.

Do sentence ids in the evaluation phase need to be sorted in increasing order as well? (Hopefully not.)
Best regards,
Martin, team BUT-FIT

Posted by: Martin @ March 9, 2020, 3:20 p.m.


Thanks for your feedback. According to our eval script, you could not submit part of the results, I guess that's why you saw that error msg in practice. And it will be fine if your results have the same sequential id order as the test set (if not, you'll see an error message and fail to submit).


Posted by: Ariel_yang @ March 9, 2020, 11:45 p.m.
Post in this thread