Dear host,
we are doubt the GT file is as incorrect as we failed our test on it. We manually checked the GT
and found certain mis-matches in it.
We have also found some labeling errors, but the amount of which is very small. I think it would not have a big influences on the competition results.
If you find that there are a large amount of errors, could you please tell me more details?
Thanks very much.
We find that your submission file is a null file. Maybe there are something wrong with your submission. Could you please check again that your submission format is correct?
Posted by: wider @ June 13, 2018, 5:06 a.m.For instance, the cast "ZhengYiJian" with id "tt1611840_nm0155562", its given groudtruth list (in val_label.json) is:
tt1611840_nm0155562 tt1611840_0004, tt1611840_0016, tt1611840_0022, tt1611840_00040, tt1611840_0041, ...
We find that, "tt1611840_0004", "tt1611840_0016", "tt1611840_00040" do not exist in the candidates of validation; "tt1611840_0022" and "tt1611840_0041" do not contain the cast "Zhengyijian".
In face, we manually check all the groudtruth candidates of cast "tt1611840_nm0155562", none of them contains the cast "ZhengYiJian".
We also conduct certain local evaluations with different models, while the results remains in an abnormally low score range.
I check the cases about "tt1611840_nm0155562", but I find nothing wrong.
I guess that the problem may be that you mistake the "instance id" for "image name".
For example, id "tt1611840_0040" is associated with image "tt1611840/candidates/shot1049img0.jpg".
And you can check image "tt1611840/candidates/shot1049img0.jpg", which actually contains "Zheng Yijian".
Such association information is provided in the file "val.json".
Hope this reply will be helpful.
Thank you for your reply! It is helpful.
Posted by: shihailin @ June 16, 2018, 3:15 a.m.