Forums

SemEval-2021 Task 2: Multilingual and Cross-lingual Word-in-Context Disambiguation (MCL-WiC) Forum

Go back to competition Back to thread list Post in this thread

> Failed Submissions

Hi
Are failed submissions also counted against the limit ? It takes some trial submissions to get the format right

Ehsan

Posted by: sehsanm @ Jan. 17, 2021, 8:52 p.m.

For example my submission failed because of this error:
Exception: Found invalid id name or id mismatch between gold and submission in /tmp/codalab/tmpywFABQ/run/input/res/test.en-fr: test.en-fr.26

Whereas such id (test.en-fr.26) exist in the test data file

Posted by: sehsanm @ Jan. 17, 2021, 8:56 p.m.

Hi,

Your ids must be in the correct natural order (test.en-fr.0, test.en-fr.1, test.fr.en-2 and so on). You have test.en-fr.24 and then test.en-fr.26, that's why you receive that message. I think that CodaLab counts only successful submissions.

Best regards,
The MCL-WiC team

Posted by: Federico_Martelli @ Jan. 17, 2021, 10:07 p.m.
Post in this thread