Hi Radu,
Is it possible for a team to submit multiple results, or do we have to choose the best result?
For example, there are results from two different models from our team and we are curious if we can submit both.
Thank you
Posted by: SeungjunNah @ April 13, 2017, 8:29 a.m.Hi SeungjunNah,
There are two cases according to the difference between the methods that one team has.
=CASE 1: the team has a couple of variants of the same method
It is a challenge and for the final testing phase we expect a single submission from each team per each competition.
Therefore, you need to pick your best variant per each competition and submit its results.
It is not necessary to use the exact same method for all the competitions (at least the trained models could differ). However, the variant(s) should be described in the factsheet.
If a team submits more than one entry per each competition then we reserve the right to pick up the entries that will be used for the final ranking and awards and the team leader will be contacted.
Note that multiple results (on validation set or other sets) and analysis of the variants can be reported in the factsheet and/or in an eventual submitted paper describing the team's contribution/solutions.
=CASE 2: the team has a couple of clearly different methods (like CNN architectures, working principles)
In this case the team can have multiple entries corresponding to the different methods (by using different users/members registered in the Codalab system) and should clearly mention the method(s) in the readme.txt and the factsheet(s).
It is up to the team to decided if provides a factsheet per each method or a factsheet with multiple sections corresponding to each method for which the team submitted results in the final testing phase.
Best,
Radu