Since we have the dev file and we do not have the test file. So, which file do we need to upload when submitting for the dev phase ?
Dear @abhinavbh08,
Please find our sample submission at https://github.com/YerevaNN/biorelex-codalab-submission
Thanks for the reply. So just to confirm, we need to upload the zip file containing the code for training and evaluation. Am I right?
Posted by: abhinavbh08 @ June 15, 2020, 9:58 a.m.The only thing you need to upload is the code to run on the test set.
I will strongly recommend that you train your model first (as you are given both train and dev splits) and ship submission.zip with pretrained weights and the code that builds your model with those weights and runs prediction on the test set.
Okay sure. Thanks for the reply.
Posted by: abhinavbh08 @ June 15, 2020, 10:20 a.m.Hey, So I uploaded the sumission.zip folder. But the status of my submission is coming as failed. Can you please check it? And also, the size of the trained model is around 850 MB. So, we are giving a google drive link in the ReadMe where we have uploaded the model file.
Posted by: abhinavbh08 @ June 15, 2020, 2:01 p.m.AFAIK CodaLab allows you to inspect detailed logs of your run. There is a "+" button alongside your submission line.
Just in case, you have an error:
Traceback (most recent call last):
File "/tmp/codalab/tmpiTOGVW/run/program/score.py", line 89, in
main()
File "/tmp/codalab/tmpiTOGVW/run/program/score.py", line 22, in main
scores = evaluate(truth_path, [prediction_path])
File "/tmp/codalab/tmpiTOGVW/run/program/evaluate.py", line 481, in evaluate
with open(path, 'r', encoding='utf-8') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/codalab/tmpiTOGVW/run/input/res/predictions.json'
Please note, that this is an automated scoring system.
You need to make submit your code/archive similar to our baseline.
Your archive has to include a file named `metadata` which states which file the grading system has to call.
Please look into files `metadata` and `code.py` in our sample submission to see how does this work.
https://github.com/YerevaNN/biorelex-codalab-submission