For participation in the multilingual track in Task1, are we expected to submit the concatenation of the predictions of the 7 specific translation directions (en-de, en-zh, si-eni, et-en, ro-en, ne-en, ru-en) as a single file of predictions.txt and then zip? Or concatenation of 6 (witthout ru-en) translation directions?
Can you clarify some more? Thank you.
Posted by: bicici @ July 26, 2020, 9:18 a.m.
I sent a concatenation predictions.txt and received the following error:
AssertionError: Incorrect list of LPs, expecting: ['en-de', 'en-zh', 'ro-en', 'et-en', 'ne-en', 'si-en'], given: [u'ne-en', u'en-de', u'ro-en', u'si-en', u'ru-en', u'et-en', u'en-zh'].
Then, I prepared a submission zip file containing ['en-de', 'en-zh', 'ro-en', 'et-en', 'ne-en', 'si-en'] in the respective order and CodaLab accepted my submission.
Thank you.
Posted by: bicici @ July 26, 2020, 9:27 a.m.Hi Ergun,
as indicated in the shared task webpage, the multiligual sub-task consider all the language pairs
from the Wikipedia domain, which excludes Ru-En. It also specifies that the scoring will be an
average Pearson correlation across all the languages.
The order of the language pairs in the file does not matter.
I'm glad you managed to make it work.
Best,
Fred.