Welcome to the VoxCeleb speaker verification challenge! The goal of this challenge is to probe how well current methods can recognize speakers from speech obtained 'in the wild'. The dataset is obtained from YouTube videos of celebrity interviews, consisting of multi-speaker audio from both professionally edited and red carpet interviews as well as more casual conversational multi-speaker audio in which background noise, laughter, and other artefacts are observed in a range of recording environments.
The task of speaker verification is to determine whether two samples of speech are from the same person.
This the competition site for the Fixed training data, requiring participants to train only on the VoxCeleb2 dev set, for which we have already released speaker verification labels, see more details in the "Evaluation" section.
Arsha Nagrani, VGG, University of Oxford
Joon Son Chung, Naver, South Korea
Andrew Zisserman, VGG, University of Oxford
Ernesto Coto, VGG, University of Oxford
Weidi Xie, VGG, University of Oxford
Mitchell McLaren, Speech Technology and Research Laboratory, SRI International, CA
Douglas A Reynolds, Lincoln Laboratory, MIT
For more information, visit the VoxSRC Challenge page and read the rest of the sections under "Learn the Details".
This work is supported by the EPSRC programme grant Seebibyte EP/M013774/1: Visual Search for the Era of Big Data.
The test data for the challenge consists of pairs of audio segments, and the goal is to simply say whether they are from the same speaker or from different speakers. Teams are invited to create a system that takes the test data and produces a list of floating-point scores, with a single score for each pair of segments.
*****
All scores must be lying between the closed interval [0, 1], where 1 means the pair of segments correspond to the same speaker and 0 means the pair of segments correspond to different speakers.
*****
Via this CodaLab competition, teams will be able to compare the results of their system with the competition's blind set of results, during (and even after) the evaluation period. To participate, open the competition page and click on the "Participate" tab. Then accept the Terms and Conditions and click on the "Register" button. After this you will be able to upload files via the "Participate" tab. Note that a team must only submit to CodaLab the outputs of its system, not the system itself. Make sure you have read the "Submission Instructions" under the "Participate" tab before uploading any files.
In order to submit a file you need to click on the "Submit/View Results" link under the "Participate" tab. After this, you will be able to see two buttons, corresponding to the Competition Phases. Click on one of the buttons to choose the phase you want to submit to. The available phases are:
In order to prevent overfitting to the test data participants can only submit one result per day. There is also a limit over the total number of submissions for each phase, see more details about this under "Submit/View Results".
*****
Update 25 July 2019: Please note that any deliberate attempts to bypass the submission limit (for instance, by creating multiple accounts and using them to submit) will lead to automatic disqualification.
*****
System output will be evaluated using the Equal Error Rate (EER), a rate used to determine the threshold value for a system when its false acceptance rate (FAR) and false rejection rate (FRR) are equal, see below
FAR = FP/(FP+TN) | FRR = FN/(TP+FN) |
where
The code used for calculating the EER can be found in the development kit. Some examples for the ground truth and the predictions are also provided, but bear in mind that the format of these example files is a bit different to the format of the file that you need to submit in this competition. Make sure you have read the "Submission Instructions" under the "Participate" tab.
This competition site corresponds to the Fixed training data task, so participants can train only on the VoxCeleb2 dev set. The dev set contains 1,092,009 utterances from 5,994 speakers.
We encourage participants to validate their models using the VoxCeleb2 publicly released hard and easy test lists:
List of trial pairs - VoxCeleb1
List of trial pairs - VoxCeleb1 (cleaned)
List of trial pairs - VoxCeleb1-H
List of trial pairs - VoxCeleb1-H (cleaned)
List of trial pairs - VoxCeleb1-E
List of trial pairs - VoxCeleb1-E (cleaned)
These can also be found on the VoxCeleb2 website.
The test data is blind, i.e., it does not include annotations. It can only be used strictly for reporting of results alone. It cannot be used in any way to train or tune systems. You will find a link to download the data under "Learn the Details".
The results of this CodaLab competition will be announced at the challenge workshop, where we will invite presentations from the most exciting and novel submissions, as well as from the challenge winners. The challenge workshop will be held on the 14th of September 2019, in conjunction with Interspeech 2019 in Austria.
Participation in this competition is open to all who are interested and willing to comply with the rules laid out under the "Learn the Details" and "Participate" tabs. There is no cost to participate, although teams are encouraged to submit a paper to the corresponding Interspeech 2019 challenge workshop, to be held in Austria on the 14th of September 2019.
We also kindly ask you to associate the CodaLab account you will use for the competition to your institutional e-mail. We reserve the right to revoke your access to the competition otherwise.
*****
Update 25 July 2019: Please note that any deliberate attempts to bypass the submission limit (for instance, by creating multiple accounts and using them to submit) will lead to automatic disqualification.
*****
In case of any issues, all decisions made by the Organizing Committee will be final.
You can download a ZIP file with the test data from here.
The file contains 19154 .wav files and one text file called "list_pairs_test_data.txt". The text file contains the pairs that you are to evaluate for the competition. There are 208008 pairs to be evaluated. It is important to note the order of the pairs, as you need the keep the same order in the results file that you will submit to CodaLab.
For reference, we have added a baseline result to the leaderboard, submitted by the vgg user.
Good Luck!.
Start: July 15, 2019, midnight
Description: Submissions for the challenge workshop that will be held in conjunction with Interspeech 2019
Start: Aug. 30, 2019, 8 a.m.
Description: Submissions for comparison with previous ones. Not to be taken into account for the challenge workshop
Never
You must be logged in to participate in competitions.
Sign In# | Username | Score |
---|---|---|
1 | xx205 | 0.0069 |
2 | stones | 0.0075 |
3 | happy | 0.0084 |