We hope that you have enjoyed the competition so far. We are surprised and deeply humbled by the popularity and people’s enthusiasm. In this email you will find the information about the final phase.
In the final phase, *each team is allowed to submit 3 submissions*. All 3 submissions will be considered for the final prize. When you make the submission, please include the name, email address, and affiliation (if any) of everyone on the team in the metadata. No participant should be associated with more than 3 submissions. We will only consider the first 3 submissions for the final prize if any team submits more than 3 times. Furthermore, any form of data probing (i.e. printing out the size of the data or printing out one image array) is strictly prohibited. Violation will result in disqualification. We recommend that you remove any form of print statement in your final code.
*The models in the final phase will still be feedforward and all input data will be square images but they can be colored (3 channel) or black-and-white (single channel)*. Please make sure that your submission works with these input formats. The complexity of the architecture will be similar so the scoring program will impose the same time limit (5 minutes per model); however, if your submission finishes within time on the phase 1 data but times out on the phase 2 data, send us the time-stamp of your submission. We will run them on the Phase 1 data to verify if it finishes within time, and run it on the phase 2 data with more time. We realize that sometimes the servers run into problems but please be ensured that these technical issues will not affect your final score.
Finally, given this information and the upcoming ICLR deadline, *we decided to push back the start date of phase 2 by 2 weeks (October 22nd)*. You can use this time to prepare and optimize your final submissions.
If you have any questions about this, please do not hesitate to contact us. Thank you and good luck!
PGDL Organizing Team
Posted by: ydjiang @ Sept. 30, 2020, 9:32 p.m.We have been notified that if the metadata is not formatted as a yaml file, it causes an error on codalab. If your team information does not work as yaml, you can also put a "team_info.txt" instead of putting this information in the metadata.
Posted by: ydjiang @ Oct. 21, 2020, 7:25 p.m.When does Phase-2 end? Also can we still make submissions to Phase-1 just to test our code?
Posted by: vamshichowdary @ Oct. 22, 2020, 5:14 p.m.The final phase ends on 11/1, which you can find on the official website.
Yes, you may still submit to the development phase, but please keep in mind that everyone is submitting to the final phase so the computation resource is a bit tight.
Thank you so much for the information. Regarding the submission to phase-1 however, when I navigate to "Participate -> Submit -> Development Phase", I can see my past submissions, but I don't a button to add new submission. I only see a submit button on "Final Phase". How can I make sure that my submission does NOT count toward the final phase submission?
Posted by: vamshichowdary @ Oct. 22, 2020, 6:49 p.m.Hi, can you please open development phase submissions again? As mentioned above, this seems to be closed.
Posted by: parthnatekar @ Oct. 23, 2020, 5:31 a.m.Turns out once phase 2 has started you can no longer submit to phase 1.
As a temporary solution, you can use this mirror competition:
https://competitions.codalab.org/competitions/27294?secret_key=6b69982d-234e-4f44-9c53-b8f60bb3a0db
Please only use this for TESTING out your solution instead of developing new solutions.
Test phase is the primary focus now. If compute becomes limited, we may cut down the compute for the mirror competition.
Can anyone confirm the exact time the competition ends on 11/1? Looks like the mirror competition ends on Nov. 1, 2020, 1:41 a.m. UTC. Is this the same time that main competition ends too?
Posted by: vamshichowdary @ Oct. 31, 2020, 2:47 a.m.The competition ends at 11/1 23:59 any where on earth.
Submissions will count as long as they are submitted before this time and run successfully (even if the deadline has passed).
Is it safe to assume that if the status of the program is "FInished" on the Codalab portal, then the submission ran within the time limit without any error on all the datasets and tasks?
Posted by: sumukhaithal6 @ Oct. 31, 2020, 8:19 a.m.There are a small number of submissions that finished but actually timed out.
All of yours finished in time properly so don't worry about it. I will be contacting the authors of the timed out submissions.
@ydjiang
We have submitted a solution on the mirror competition ~4hours ago. It's showing as still "Running" for 4 hours straight. Our model should not take that long to run. Is it possible that there's some issue server side? If its not, then can you just terminate our process and send us logs? We do not want to hog the server and keep it running wastefully.
Posted by: vamshichowdary @ Oct. 31, 2020, 8:13 p.m.