Dear participants,
We regret to inform you that we have recently detected an internal misalignment at the column tags shown in the leaderboard referring to bias, i.e., "Bias (positive pairs)" and "Bias (negative pairs)". Since face verification is a binary problem, for interpretability reasons, we internally choose the tag ‘0’ for negative samples and tag ‘1’ for positive samples. However, the leaderboard assumed the opposite assumption.
We apology for the mistake associated with the wrong order of labels "positive<->negative" shown in the leaderboard, that cannot be fixed on Codalab after the competition is released. Because of this, you have to consider "Bias (positive pairs)" as "Bias (negative pairs)" and vice-versa, i.e., "Bias (negative pairs)" as "Bias (positive pairs)".
That change does not affect your submissions nor the ranking at all, but the acknowledgment you have towards the case in which your algorithms may suffer the most from bias.
We hope that has not interfered in your work and we apologize for this inconvenience.
Sincerely,
The organizing team
Posted by: juliojj @ May 25, 2020, 6:50 p.m.