NeurIPS 2021 BEETL Competition - Benchmarks for EEG Transfer Learning Forum

Go back to competition Back to thread list Post in this thread

> Submission for Task 1

How to make correct submission?

I create answer.txt file with 25748 labels (0-5) by usage of code from start_kits:

np.savetxt("answer.txt", sleep_label, delimiter=',',fmt="%d")

Then I archive this file as .zip so I get answer.zip with single file inside - answer.txt.
I try to submit, status - Submitted but it doesn't have any score, just ---

Please help me to get some score)

Posted by: anton.v @ Aug. 25, 2021, 10:38 a.m.

One thing you could check is whether you are submitting to task 1 or task 2. Task 1 is with labels 0-5, maybe submitting the file to task 2 by accident would cause the problem you mentioned.

Best,
Xiaoxi Wei

Posted by: BEETLCompetition @ Aug. 25, 2021, 11:02 a.m.

Hi, Xiaoxi Wei.

Thank you for answer.

Seems initially I really submit it in other competition.
But unfortunately I get the same issue when try submit it to Task 1.

Shape of my answers before save to txt file: (25748,)

In text file it looks like:

2
3
0
0
1
....

Each number starts from new line without any delimeter after number.
May you clarify how numbers should looks like inside the file?

Posted by: anton.v @ Aug. 25, 2021, 11:42 a.m.

Your format should work. The status should be 'finished' after some time of 'submitted', then you will be able to see the score after refreshing the page. There may be some delay in the back end of the Codalab platform sometimes, what I usually do is to try to resubmit or wait for some time, it seems there is a delay of the Codalab server today.

Xiaoxi

Posted by: BEETLCompetition @ Aug. 25, 2021, 12:02 p.m.

yes, seems now it's finished. But my result is unseen on leaderboard. Hope that also just wait little bit)

Thanks for help!

Posted by: anton.v @ Aug. 25, 2021, 1:48 p.m.

There is a button beside your score that you can click to submit to the leaderboard. sometimes you need to refresh the page to see that button after you get the score.

Xiaoxi

Posted by: BEETLCompetition @ Aug. 25, 2021, 3:42 p.m.

I had the same issue and had to spend my resubmission right on this today. I think limiting daily submissions is really unfortunate for us. Can you increase the daily number of submissions from 3 to 4 maybe?

Posted by: andacdemir @ Aug. 26, 2021, 5:16 a.m.

I would suggest waiting for some time before resubmitting multiple times, the delay is normally caused by the platform queue. While it's a reasonable suggestion, I've increased it to 4 times a day.

Xiaoxi

Posted by: BEETLCompetition @ Aug. 26, 2021, 10:53 a.m.
Post in this thread