Hi!
I believe there can be an error in evaluation script, in particular if I submit file with same images, like 'tt0251160_0011', 'tt0251160_0255', 'tt0251160_0255' I got higher score, which I suppose should be handled by eval script and shouldn't be the case.
Thanks for your remind! That is the truth. We are quit sorry for that. We will update the code and rerun all the submissions within today. If you are eager to get a right result, you can just change line 26 of eval.py from "submission[key] = ret" to "submission[key] = unique_ret".
Posted by: wider @ July 18, 2018, 2:32 a.m.The bug has been fixed. The evaluation script has been updated and all the submissions have been re-run.
Posted by: wider @ July 18, 2018, 5:51 a.m.