Hello,
This thread is for sharing / answering question and issues directly related to the Codalab platform.
Posted by: eustache @ May 17, 2021, 2:55 p.m.As of May 17th participants are encountering issues with the submission process.
The symptom is a submission in "Failed" with the following error:
"""
...
Traceback (most recent call last):
File "/worker/worker.py", line 313, in run
...
ReaderError: 'utf8' codec can't decode byte #xb1: invalid start byte
in "/tmp/codalab/tmp0Qasdr/run/input/ref/metadata", position 12
"""
We are currently trying to solve the issue with Codalab and will keep you posted.
Posted by: eustache @ May 17, 2021, 3:11 p.m.So we found the issue, reported it to Codalab and fixed it. It was related to a wrong filename when updating the reference score used to compute the "vs Oracle" and "vs Naive" metrics (which are for information only and not used in the ranking anyway).
To make sure that submissions that were submitted during the buggy period (May 17th-19th) were properly scored we re-run all submissions on the platform. You may see that the date of your submissions may have changed as a result of the process. The scores (click logloss or sales logloss) have not changed. Only failed submissions due to the bug have now a score and may appear in the leaderboard.
In the process we also added a dedicated worker queue so that your submissions will run faster (expect a run time in the range of 30s). Now that we have a dedicated queue we update the maximum daily submissions to 30 so that you can iterate faster on the problem. We will monitor the load and if everything is smooth we can continue with this limit going forward.
Posted by: eustache @ May 19, 2021, 9:26 a.m.Hi,
I am still facing the issue. I am getting the error "AssertionError: invalid submission: expected one of y_hat_click.txt or y_hat_sale.txt files to be present"
The zip folder, I am submitting does contain y_hat_click.txt. Many of my submissions are failed. What could be going wrong?
Thank you.
Posted by: learner @ May 20, 2021, 8:54 p.m.So in your case the problem is different. In fact, in the zip file of your submission there is a directory and then the file y_hat_click.txt is inside this directory.
The file y_hat_click.txt must be at the root of your zip file.
You can have a look at the create_submission() method in my_first_submission.py to see how to create a correct zip file.
OK, my bad. Thank you, it worked now.
Posted by: learner @ May 21, 2021, 10:26 p.m.