Hi Guys,
The training_gt.csv file contains the continuous score computed for each personality trait for a given video. Could anyone explain how these continuous scores are calculated ? Is the submission file format same as this aforementioned file?
Thanks
Dear Participant,
Thanks for your question.
As you indicate, the continuous scores are computed for each video and the Big Five personality traits. In total we consider 10,000 videos and about 350,000 possible pairs of videos to compare each other, which have been annotated by AMT workers. The annotation task for a given video pairwise consisted of indicating who of the subjects appearing in the videos apparently looked to have each trait under the opinion of the observers (also using a neutral answer). Then, the continuous scores are computed taking into account a callibration problem produced when annotating pairwise rankings. You can see the description of the data, the personality traits, samples, and the scripts to download the data in the "Get Data" Section here: https://competitions.codalab.org/competitions/9181#participate-get_data
About the submission file format: yes, the format is exactly the same as it is explained in the evaluation details here: https://competitions.codalab.org/competitions/9181#learn_the_details-evaluation. However, note that at this stage of the competition the submission works for the 2,000 validation videos. Thus, the predictions.csv file that you generate for the submission with your computed scores must have the same video names and number of entries for validation (2,000) as the provided example file predictions.csv that you can find in the evaluation details. So you can use this file as a reference for the ground-truth of the validation data in order to generate your own predictions file, using the provided scripts if you want.
Have a nice competition.
Best,
Posted by: vponcel @ May 20, 2016, 11:08 a.m.Thank you
Posted by: ayushrai @ May 20, 2016, 3:21 p.m.