SemEval-2020 Task5: Modelling Causal Reasoning in Language: Detecting Counterfactuals Forum

Go back to competition Back to thread list Post in this thread

> Submission format subtask 1

I failed to submit a a valid file for the subtask 1 and get the following error:

WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
Traceback (most recent call last):
File "/tmp/codalab/tmpVRzQX4/run/program/evaluation.py", line 298, in
f1_scores, recall, precision, em = evaluate1(task1_submission_path, task1_solution)
File "/tmp/codalab/tmpVRzQX4/run/program/evaluation.py", line 79, in evaluate1
if row[0] != submission[i][0]:
TypeError: 'int' object has no attribute '__getitem__'

Here are the first lines of my test file:

sentenceID,pred_label
100000,0
100001,0
100002,0
100003,0

Posted by: PierreLudmann @ Dec. 4, 2019, 10:07 a.m.

Thank you so much for your feedback. Could you try it again? Please feel free to contact us if there are any submission problems.

Posted by: Ariel_yang @ Dec. 4, 2019, 5:23 p.m.

Don't know what you've done but now it's running.

Posted by: PierreLudmann @ Dec. 4, 2019, 6:51 p.m.

I updated the evaluation script yesterday ;)

Posted by: Ariel_yang @ Dec. 5, 2019, 3:59 p.m.

Hi Ariel

I have the same issue. I submitted the output tonight.

Posted by: pouria_babvey @ Dec. 27, 2019, 8:50 a.m.
Post in this thread