The test phase has begun! Best of luck to everyone and do check out a recent tutorial, with links to tutorials on this task https://jvision.medium.com/families-in-the-wild-track-iii-b5651999385e?sk=df21210003d6515af84795d29750fd68.
Posted by: jvision @ Sept. 1, 2021, 4 a.m.Hello, in "Updated Challenge Phase Data" section, both "Test" and "Challenge" subsections contain test data. Which is used for this competition? The "Test Pairs" data in "Challenge" phase consists of 7 mat files, while I was expecting 1 csv file. Could you provide some details in how to use them? Thanks.
Posted by: jh3450 @ Sept. 1, 2021, 11:36 a.m.There should be a single CSV for the gallery and one for the probe list!
Here is probe list, https://www.dropbox.com/s/cxh61i6s91x01ni/probe_list.csv?dl=0
And gallery, https://www.dropbox.com/s/8xiahy4sl8qr1y8/gallery_list.csv?dl=0
Posted by: jvision @ Sept. 1, 2021, 1:42 p.m.The two files do not contain pairs of images though.
Posted by: jh3450 @ Sept. 2, 2021, 10:53 a.m.This task doesn’t involve pairs. There are probes and the gallery. Task 1 and 2 are based on pairs. See this tutorial and let me know if you have any questions. https://jvision.medium.com/families-in-the-wild-track-iii-b5651999385e?sk=df21210003d6515af84795d29750fd68
Posted by: jvision @ Sept. 2, 2021, 12:22 p.m.On this page https://competitions.codalab.org/competitions/21843, there is
"RFIW2021 will support 3 laboratory-style evaluation protocols:
Kinship Verification (one-to-one)
Tri-subject Verification (one-to-two)
Search and Retrieval (many-to-many)".
I registered for the first protocol "Kinship Verification", that's why I was looking for pairs of images. The probe and gallery files are for the third protocol "Search and Retrieval". It seem only the third protocol is supported in the Challenge phase. I was able to submit a result file following the first protocol during the Validation phase, though. Is there a competition for protocol 1, "Kinship Verification" ? Thanks.
Posted by: jh3450 @ Sept. 2, 2021, 12:52 p.m.Oh, my apologies! I thought this was the other portal. One sec, let me get those.
Posted by: jvision @ Sept. 2, 2021, 2:48 p.m.The top set of data at the bottom of "Participate" a link Pair List was added. Here is the direct link: should have everything you need (ignore what you do not). Thanks, https://www.dropbox.com/s/6ld90juxx1f2w12/test-full.csv?dl=0
Posted by: jvision @ Sept. 2, 2021, 2:53 p.m.Thank you. Is the leaderboard a mix of the three protocols? Since the test pairs for protocol 1 were provided today (9/2), it's impossible to have submitted a result for protocol 1 yesterday (9/1). The current top score on the leaderboard was submitted yesterday (9/1).
Posted by: jh3450 @ Sept. 3, 2021, 12:29 a.m.The scores are automatic and follow different formats. The different tasks are different competitions for this reason. All submissions here are using the verification data (the list was there, but perhaps you were looking at the wrong file so I added another).
Posted by: jvision @ Sept. 3, 2021, 12:05 p.m.Do we have any information about the data split for the test set in the kinship verification (track 1) task? Are the number of kin and non-kin relationships balanced?
Posted by: vaibhavbagri07 @ Sept. 3, 2021, 1:51 p.m.Thankfully you asked this; also, I am glad you asked for numbers, as I was on another computer, so realized the file was different when I went to download (and that the original list links were obsolete). Anyways, use these https://www.dropbox.com/sh/6yrzqr1vem5lnwn/AADC8HI-RL4X2zZ10EW0A3Zua?dl=0. I will add 1 day for the deadline to make up a tad. There are 19,890 positives and 19,853.
Furthermore, The splits are the same as last year: see white paper (https://arxiv.org/pdf/2002.06303.pdf) for additional stats and benchmarks. Perhaps these might help:
https://towardsdatascience.com/demo-for-rfiw-2020-task-1-kinship-verification-8a8ed7081bcc
https://towardsdatascience.com/pandas-tips-tricks-need-for-speed-54e217cc6aa0
Thanks and let me know if anything is needed that would help!
Sorry for the confusion: we have no idea why the other file lists disappeared (clearly, it was a mixup on our end, and we apologize). Best of luck and let us know how it goes!
Posted by: jvision @ Sept. 3, 2021, 3:51 p.m.