Sentence-level Direct Assessment QE Shared Task 2020 Forum

Go back to competition Back to thread list Post in this thread

> Submission error?

Hi,

I am receiving the following error for all of my submissions. Please help

--Jackie

Traceback (most recent call last):
File "/worker/worker.py", line 330, in run
if input_rel_path not in bundles:
TypeError: argument of type 'NoneType' is not iterable

Posted by: jackielo @ July 14, 2020, 12:12 p.m.

I have a predictions.txt and then zip it.
Is that the way to create the submission bundle?
Sorry for naive question, I am new to CodaLab.

Posted by: jackielo @ July 14, 2020, 1:07 p.m.

Hi Jackie,
the forum is here to help and there is no naive question, don't worry.

Does your method name contain a special character, e.g. an underscore?
Another participant has reported the same issue, which they fixed by removing
an underscore from their method name in the text file.

Best,
Fred

Posted by: fblain @ July 14, 2020, 2:01 p.m.

I have an "-" in the method name, indeed.
Thanks for the pointer.

--Jackie

Posted by: jackielo @ July 14, 2020, 2:32 p.m.

May I request to remove the three failed submissions?

Thanks for your understanding.

--Jackie

Posted by: jackielo @ July 14, 2020, 2:51 p.m.

Hi Jackie,
are you now able to submit?
I only see three failed submissions to the Russian-English subtask, is that correct?

Best,
Fred.

Posted by: fblain @ July 14, 2020, 2:59 p.m.

I am still not able to submit. The error message now is as follow.

WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
Traceback (most recent call last):
File "/tmp/codalab/tmpbl_TI1/run/program/evaluation.py", line 123, in
averaged_scores = getScores()
File "/tmp/codalab/tmpbl_TI1/run/program/evaluation.py", line 99, in getScores
predictions = parse_submission(submission_file.readlines())
File "/tmp/codalab/tmpbl_TI1/run/program/evaluation.py", line 68, in parse_submission
"Incorrect list of LPs, expecting: {}, given: {}.".format(language_pairs, lp_dict_keys)
AssertionError: Incorrect list of LPs, expecting: ['en-de', 'en-zh', 'ro-en', 'et-en', 'ne-en', 'si-en'], given: [u'ne-en', u'en-de', u'ro-en', u'si-en', u'ru-en', u'et-en', u'en-zh'].

--Jackie

Posted by: jackielo @ July 14, 2020, 3:10 p.m.

Hi Jackie,
the first part of the error is purely Codalab related and there is nothing to worry about.
The error that you have, is that you are submitting predictions for 7 language-pairs, while the multilingual sub-task expects 6.
This sub-task considers language pairs in the Wikipedia domain only, which excludes Russian-English (Reddit domain).
Assuming there is not formatting error in your file, remove the predictions for Russian-English and it will be fine.

Best,
Fred.

Posted by: fblain @ July 14, 2020, 3:15 p.m.

Thank you for all the assistance. I have some successful submissions now.

--Jackie

Posted by: jackielo @ July 14, 2020, 4:03 p.m.

I'm glad to hear that.
Thank you for the participating to our shared task and good luck!

Best,
Fred.

Posted by: fblain @ July 14, 2020, 4:18 p.m.
Post in this thread