Recognizing Families In the Wild Data Challenge (4th Edition) in conjunction with FG 2020
Previous
ValidationCurrent
ChallengeEnd
Competition EndsFG 2020 Recognizing Families In the Wild (RFIW2020) Challenge
Tri-subject Verification (Track II)
Overview
The 4th RFIW large-scale kinship recognition data competition in conjunction with FG 2020. This is made possible with the release of the largest and most comprehensive image database for automatic kinship recognition, Families in the Wild (FIW).
RFIW2020 will support 3 laboratory-style evaluation protocols: (1) repeated; (2) and (3) supported for the first time.
- Kinship Verification (one-to-one)
- Tri-subject Verification (one-to-two)
- Search and Retrieval (many-to-many)
Additionally, we will have general paper and Brave New Idea tracks. Visit the RFIW workshop webpage to learn more about the other tasks, tracks, and the challenge workshop as a whole.
The best paper award will be awarded will be included in the 2020 IEEE Proceedings of Automatic Face and Gesture Recognition (AMFG). Note that individuals and/or teams can participate in just one or all tasks. In any case, standings will be determined for each task separately.
Important dates
- 2019.12.04 Team registration opens.
- 2019.12.04 Training and validation data made available (Phase I).
- 2019.12.04 Validation server online.
- 2019.12.05 Validation labels released (Phase II).
- 2020.01.13 Test "blind" set and labels for validation set are released; validation server closed (Phase III).
- 2020.01.20 Test results and README(s) (i.e., brief descriptions of each submission) are due.
- 2020.01.21 Results will be made public and standings listed on the leader-board.
- 2020.01.28 Paper submission for task evaluations and general paper submissions are due.
- 2020.02.05 Notification.
- 2020.02.26 Camera-ready due.
- 2020.05.[18-22] RFIW Challenge in conjunction with FG 2020.
Background
Automatic kinship recognition holds promise to an abundance of applications, like to aid forensic investigations as a powerful cue to narrow the search space (e.g., perhaps knowing that the Boston Bombers were brothers we may have identified the suspects sooner). There are many beneficiaries of such technologies, whether the consumer (e.g., automatic photo library management), the scholar (e.g., historic lineage & genealogical studies), the data analyzer (e.g., social-media-based analysis), or the investigator (e.g., cases of missing children and human trafficking).
A fair question to ask-- if so applicable, why is visual kinship recognition technology not found, or even prototyped, in real-world products? Reasons for this are two-fold:
- Other image datasets for kinship recognition tasks do not capture nor reflect the true data distributions of the families of the world. Furthermore, other collections are small in size.
- Kin-based relationships are less discriminant than other, more conventional face-based problems, as there exist many hidden factors that affect the facial appearances amongst different family members.
Both points were addressed with the introduction of our FIW database, with data distributions to properly represent real-world scenarios available at scales much larger than ever before. FIW now allows researchers and practitioners to employ complex, modern-day data-driven methods (i.e., deep learning) in ways not possible before.
In the end, we hope FIW serves as a rich resource to further bridge the semantic gap of facial recognition-based problems to the broader human-computer interaction incentive.
Participants can submit in one or all tasks-- note standings will be determined for each task separately. We next introduce the 3 tasks of 2020 RFIW.
T-1: Kinship Verification
Kinship verification aims to determine whether a pair of facial images are blood relatives of a certain type (e.g., parent-child). This is a classical Boolean problem with system responses being either KIN or NON-KIN (i.e., related or unrelated, respectfully). Thus, this task tackles the one-to-one view of automatic kinship recognition.
More details are found in the data section of the competition.
T-2: Tri-Subject Verification
Tri-Subject Verification focuses on a slightly different view of kinship verification– the goal is to decide whether a child is related to a pair of parents. This is a more realistic assumption, as having knowledge of one parent typically means knowledge of the other is accessible. This is the first time this was done using FIW data.
More details are found in the data section of the competition (register via the link above).
T-3: Search & Retrieval
Large-Scale Search and Retrieval of family members of missing children. The problem is posed as a many-to-many ranking problem, as one-to-multiple faces are provided for each unique individual. The goal is to find family members of the search subjects (i.e., the probes) in a search pool (i.e., the gallery). The task simulates the real-world problems of missing persons.
More details are found in the data section of the competition (register via the link above).
Provided Resources
- Scripts: Scripts and demos available on Github.
- Contact: You encourage using the forum provided as part of this challenge portal. recommended!) or directly contact the challenge organizers by email (robinson.jo@husky.neu.edu and khan.za@husky.neu.edu ) if you have doubts or any questions.
- For more information on the FIW dataset check out the project page: https://web.northeastern.edu/smilelab/fiw/
Contact
Joseph Robinson (robinson.jo [at] husky [dot] neu [dot] edu, robinson.jo@husky.neu.edu) and Zaid Khan (khan [dot] za [at] husky [dot] neu [dot] edu, khan.za@husky.neu.edu ) for all inquiries pertaining to 2020 RFIW and FIW.
RFIW Workshop and Challenge @ FG 2020
Tri-subject Verification (Track II)
Evaluation
Overview
Tri-Subject Verification focuses on a slightly different view of kinship verification-- the goal is to decide whether a child is related to a pair of parents. This is a more realistic assumption, as having knowledge of one parent typically means knowledge of the other is accessible. Following this, we propose adding this track, but with additional tri-subject pair types (e.g., given a couple of known siblings, determine if an unknown subject is also a sibling). Plus, this will be done at scales far greater than ever before possible.
Triplet pairs consist of Father (F) / Mother (M) - Child (C) (FM-C), where the child C could be either a Son (S) or a Daughter (D). Hence, the triplet pairs are FM-S and FM-D.
 Fig.1 - Depiction of Task 1, Kinship Verification. |
 Fig.2 - Depiction of Task 2 (this), Tri-Subject Verification. |
This is the first time tri-subject verification has been done using the large-scale FIW dataset. Efforts here were inspired by earlier work that introduced the tri-subject task to the machine vision community:
Qin, Xiaoqian, Xiaoyang Tan, and Songcan Chen. " Tri-subject kinship verification: Understanding the core of a family." IEEE Transactions on Multimediae 17.10 (2015): 1855-1867.
Data Splits
The data for tri-subject verification is split into 3 disjoint sets, train, val, and test. Ground truth for train is provided in Phase 1. This is for self-evaluation, as runs on the Validation can be submitted for scoring. Ground truth for and val is available for Phase 2. The "blind" test set will be released during Phase 3. Labels are not for the test set until after the challenge and when the results are made public. Teams will be asked to only process the Test set to generate submissions and, hence, any attempt of analyzing or understanding the Test set is prohibited. All sets will be made-up of an equal number of positive and negative pairs. Lastly, note that there is no family or subject identity overlapping between any of the sets.
For more information see data section.
More information about FIW dataset is provided on the project page: https://web.northeastern.edu/smilelab/fiw/
Evaluation Settings and Metrics
As conventional face verification, we offer 3 modes, which are listed & described as follows:
- Unsupervised: No labels are given, i.e., no prior knowledge about kinship or subject IDs.
- Image-restricted: Kin/ non-kin labels given for training set, with no family overlap between training and test sets.
- Image-unrestricted: Kinship labels & IDs given– allows mining for additional negative pairs.
Participants will be allowed to make up to 6 submissions of different runs for each mode (i.e., teams participating in all 3 settings will be allowed to submit up to 18 sets of results). Note that runs must be processed independently of one another.
The metric used is verification accuracy, which is provided per triplet-pair type (i.e., FM-D and FM-S). The overall averaged accuracy is used to determine the leaderboard. Also, ROC curves will also be upon requests for final papers.
Source code and trained models can be found on Github.
Special attention will be given to the submission that provides supporting code.
Contact Joseph Robinson (robinson.jo [at] husky [dot] neu [dot] edu, robinson.jo@husky.neu.edu) and Zaid Khan (khan [dot] za [at] husky [dot] neu [dot] edu, khan.za@husky.neu.edu ) for anything on 2020 RFIW and FIW.
RFIW Workshop and Challenge @ FG 2020
Tri-subject Verification (Track II)
Data Agreement
If you're using or participating in either Challenge or using FIW data please cite the following papers:
Joseph P. Robinson, Ming Shao, Yue Wu, Hongfu Liu, Timothy Gillis, Yun Fu, "Visual Kinship Recognition of Families in the Wild." In IEEE TPAMI, 2018.
title = {Visual kinship recognition of families in the wild},
author = {Robinson, Joseph P and Shao, Ming and Wu, Yue and Liu, Hongfu and Gillis, Timothy and Fu, Yun},
journal = {IEEE transactions on pattern analysis and machine intelligence},
volume = {40},
number = {11},
pages = {2624--2637},
year = {2018},
publisher = {IEEE}
}
Joseph P. Robinson, Ming Shao, Handong Zhao, Yue Wu, Timothy Gillis, Yun Fu, " Recognizing Families In the Wild (RFIW): Data Challenge Workshop in conjunction with ACM MM 2017." In RFIW '17: Proceedings of the 2017 Workshop on Recognizing Families In the Wild, 2017.
title = {Recognizing Families In the Wild (RFIW): Data Challenge Workshop in conjunction with ACM MM 2017},
author = {Robinson, Joseph P and Shao, Ming and Zhao, Handong and Wu, Yue and Gillis, Timothy and Fu, Yun},
booktitle = {RFIW '17: Proceedings of the 2017 Workshop on Recognizing Families In the Wild},
pages = {5--12},
location = {Mountain View, California, USA},
publisher = {ACM},
address = {New York, NY, USA},
year = {2016}
}
S. Wang, J. P. Robinson, and Y. Fu, "Kinship Verification on Families In The Wild with Marginalized Denoising Metric Learning." In 12th IEEE AMFG, 2017.
author = {Wang, Shuyang and Robinson, Joseph P and Fu, Yun},
title = {Kinship Verification on Families in the Wild with Marginalized Denoising Metric Learning},
booktitle = {Automatic Face and Gesture Recognition (FG), 2017 12th IEEE International Conference and Workshops on}
}
Joseph P. Robinson, Ming Shao, Yue Wu, and Yun Fu, "Families in the Wild (FIW): Large-scale Kinship Image Database and Benchmarks." In Proceedings of the ACM on Multimedia Conference, 2016.
title = {Families in the Wild (FIW): Large-Scale Kinship Image Database and Benchmarks},
author = {Robinson, Joseph P and Shao, Ming and Wu, Yue and Fu, Yun},
booktitle = {Proceedings of the 2016 ACM on Multimedia Conference},
pages = {242--246},
year = {2016},
organization = {ACM}
}
Submissions
See https://web.northeastern.edu/smilelab/rfiw2020/index.html for more information on submissions (i.e., continued terms and conditions, along with links to templates, a portal for authors, and more).
Contact Joseph Robinson (robinson.jo [at] husky [dot] neu [dot] edu, robinson.jo@husky.neu.edu) and Zaid Khan (khan [dot] za [at] husky [dot] neu [dot] edu, khan.za@husky.neu.edu ) for anything on 2020 RFIW and FIW.
RFIW Workshop and Challenge @ FG 2020
Tri-subject Verification (Track II)
Organizers
Honorary Chairs
 Matthew A. Turk
Toyota Technological Institute at Chicago (TTIC)
https://www.ttic.edu/mtur |
General Chair
 |
Workshop Chairs
Samson Timoner
ISMConnect
|
Yu Yin
Northeastern University
|
Web and Publicity Co-Chairs
 Zaid Khan
Northeastern University
|
RFIW2020 supports the traditional verification task, along with two new evaluations (i.e., Tri-Subject Verification and Search & Retrieval); also, General Paper Submission and Brave New Ideas tracks. See Challenge Page for more details.
Contact Joseph Robinson (robinson.jo [at] husky [dot] neu [dot] edu, robinson.jo@husky.neu.edu) and Zaid Khan (khan [dot] za [at] husky [dot] neu [dot] edu, khan.za@husky.neu.edu ) for anything on 2020 RFIW and FIW.
Training
Start: Dec. 3, 2019, midnight
Description: Training and validation data made available. Labels available for Training; sever will be open for scoring Validation
Validation
Start: Dec. 5, 2019, midnight
Description: Labels for Validation made available. Evaluation scripts provided to participants. Validation will still be open for those that rather upload results for automatic scoring and or those looking to make sure the submissions are formatted properly.
Challenge
Start: Jan. 13, 2020, midnight
Description: Test data release. Validation server closed. Open for final submissions.
Competition Ends
Jan. 20, 2020, midnight
You must be logged in to participate in competitions.
Sign In| # | Username | Score |
|---|---|---|
| 1 | ustc-nelslip | 0.78 |
| 2 | lemoner20 | 0.78 |
| 3 | Early | 0.77 |