Dear participants,
as the end of the competition is approaching, it is important to get prepared for the final stage (Phase 2). Let me briefly explain the process so that you have a clear picture on what will happen and how we will identify the final winners.
1. On Oct 2nd, the system will stop receiving submissions for Phase 1 and automatically transition to Phase 2.
2. Phase 2 will last until Oct 5th.
3. During Phase 2, you are supposed to submit the predictions of the best model according to Phase 1 on a new hold-out dataset from the same population (we allow for 3 attempts just in case you submit the wrong file and want to redo it).
4. The score will be visible on the leaderboard and we will use the last score from Phase 2 (shown on the leaderboard) to make the final winner selection. Phase 1 won't be used for the winner selection.
5. While you still have three attempts during Phase 2, we recommend making at most two distinct "best-bet" submissions to make sure that the last attempt is your best attempt.
If you have questions, please ask on the forum. Good luck with the final efforts!
Best,
CIKM Cup 2016 Organizing Committee
Hi, I am afraid that 3 submissions are not going to be enough with this system like that. Please consider that my last SEVEN submissions got stuck in the 'Submitting' status.
What if this happens in phase 2?
@joaopalotti, fair question. We will increase the number of allowed submissions if it happens.
Posted by: spirinus @ Sept. 30, 2016, 4:55 p.m.I still want to know whether you will release the ground truth data after the end of competition?
Posted by: tjy @ Oct. 1, 2016, 2:12 a.m.Yes, we just had a conference call with the data provider and they approved the release of the test labels. There will also be an opportunity to get access to the data set after the competition is over and test the algorithms in production if there is an interest. The A/B test will be discussed on a case by case basis and require additional formalization. We will keep the community informed.
Posted by: spirinus @ Oct. 1, 2016, 4:07 p.m.