I'm a bit confused here.
So at the end of Phase 1, we get an updated leaderboard and need to submit a paper on the methodology.
However, between the end of Phase 1 (1st May) to Phase 2 (15th May), we get a test set that we iterate over for 15 days.
What happens if we change/modify our methodology? Do we submit an updated version of the paper?
You can explain your improvements in the camera ready version of the paper. The paper submissions being closed on May 1st is (i) to give committee time to review the papers and (ii) to prohibit over-optimizing the results specifically for the validation set.
Posted by: ilkedemir @ April 17, 2018, 7:04 p.m.Thanks for the reply, couple more follow-up questions
1) After May 1st, can we get feedback on validation set?
2) Is the test set a subset/superset of the validation set? Or that is something that will not be revealed.