Dear organizers,
In the competition website it is explicitly said that the dataset contains images "displaying a single individual".
Nevertheless, by a fast scanning through the test dataset, we have found about 20 images where the face of interest is not clearly distinguishable. You may refer to images 006793.jpg and 007170.jpg for representative examples.
We would like to highlight that we are not speaking about numerous images where multiple faces are presented but the central one is clearly visible for a human. In the mentioned examples, no preference can be given to one of 2 (or more) faces.
We have also noticed that in the validation and training datasets, the same problem caused ambiguities in provided apparent age annotations. Please, refer to the image 005152.jpg which is annotated with a mean age of 11.33 and a very high standard deviation while it contains a baby and an adult lady. Clearly, the age 11.33 does not correspond to neither of two persons presented in the image and some annotators annotated the image taking into account the baby's face while the others were focused on the lady's one.
What shall we do with these ambiguous images in the test dataset?
Can you guarantee that all images in the test dataset have been annotated taking into account one face per image?
Best regards,
OLE team
We missed some cases when clearing the data, sorry for the inconvenience.
We are currently working on fixing this, and will upload a new version of the test data later today. We'll send both an email to all participants and update this post later on, when the new validation data is available for download.
Thank you for your answer.
Could you, please, also publish the list of images which you are going to change/remove? So that we can safely work on other images without losing our time.
Otherwise, will you also correct similar errors in training and validation datasets? Will you update the validation leading board with respect to these eventual modifications?
Could you, please, clarify all these points as soon as possible since the competition deadline is just in 4 days!
We've already uploaded the corrected test data. It has two main changes. Firstly some participants had problems loading some of the images. These have been corrected so that there is no problem with the file structure,. If you had no problem loading them previously, then the system you were using already tolerated the problems with these files format.
Also, we've removed 22 test images displaying multiple faces. These are the removed images:
005851.jpg, 006004.jpg, 006026.jpg, 006240.jpg, 006575.jpg, 006670.jpg, 006708.jpg, 006793.jpg, 006816.jpg, 006872.jpg, 006874.jpg, 007036.jpg, 007069.jpg, 007073.jpg, 007094.jpg, 007132.jpg, 007170.jpg, 007206.jpg,, 007287.jpg, 007288.jpg, 007303.jpg, 007325.jpg
Please note that some cases where other faces appear cropped or obstructed are still in the database. These are intended and have not been removed.
Posted by: moliusimon @ March 11, 2016, 7:27 p.m.