1. click on submit button.
2. select submission.txt file.
- on submitting the submission.txt file, getting an error as Invalid file type (text/plain).
Am I doing something wrong in submission?
Posted by: subh123 @ March 15, 2020, 8:33 p.m.Hi,
Yes, you need to upload a zipped txt file. So that the suffix of your file is ".zip".
Cheers,
Fernando
Thank you for the quick reply. In my understanding the zipped folder can contain 2 submission files 1 for each task with first line declaring the task.
Posted by: subh123 @ March 15, 2020, 8:48 p.m.Do you need any action from me? It seems your submissions were successful!
Cheers,
Fernando
Hi,
Can we make two different submission for subtask_a and subtask_b within a single zipped file? In that case do we have to provide 2 text files with different header as subtask_a and subtask_b.
Posted by: subh123 @ March 16, 2020, 8:08 a.m.You provide one submission file. We know which samples have zero-shot labels and test also against them for subtask b. The ones which do not have zero-shot samples are used for subtask_a. E.g.:
true:
sample1 A B C
predicted:
sample1 A B F
assume label C is zero shot, so for subtask A we do not count it, and for subtask B we count it when scoring.
Subtask A:
A==A, B==B, -F -> 2 true positives, 1 false positive
Subtask B:
A==A, B==B, +C, -F, -> 2 true positives, 1 false negative, 1 false positive
I will release soon the scoring script.
Cheers,
Fernando