Data Agreement
If you're using or participating in either Challenge or using FIW data please cite the following papers:
2018
Joseph P. Robinson, Ming Shao, Yue Wu, Hongfu Liu, Timothy Gillis, Yun Fu
IEEE Transactions on pattern analysis and machine intelligence (TPAMI)
Special Issue: Computational Face, 2018
Special Issue: Computational Face, 2018
@InProceedings{fiwpamiSI2018,
author = "Robinson, Joseph P and Shao, Ming and Wu, Yue and Liu, Hongfu and Gillis, Timothy and Fu, Yun",
title = "Visual Kinship Recognition of Families in the Wild",
journal = "IEEE Transactions on pattern analysis and machine intelligence",
Keywords = "Large-Scale Image Dataset, Kinship Verification, Family Classification, Semi-Supervised Clustering, Deep Learning",
year = "2018",
}
We present the largest database for visual kinship recognition, Families In the Wild (FIW), with over 13,000 family photos of 1,000 family trees with 4-to-38 members. It took only a small team to build FIW with efficient labeling tools and work-flow. To extend FIW, we further improved upon this process with a novel semi-automatic labeling scheme that used annotated faces and unlabeled text metadata to discover labels, which were then used, along with existing FIW data, for the proposed clustering algorithm that generated label proposals for all newly added data-- both processes are shared and compared in depth, showing great savings in time and human input required. Essentially, the clustering algorithm proposed is semi-supervised and uses labeled data to produce more accurate clusters. We statistically compare FIW to related datasets, which unarguably shows enormous gains in overall size and amount of information encapsulated in the labels. We benchmark two tasks, kinship verification and family classification, at scales incomparably larger than ever before. Pre-trained CNN models fine-tuned on FIW outscores other conventional methods and achieved state-of-the-art on the renowned KinWild datasets. We also measure human performance on kinship recognition and compare to a fine-tuned CNN.
2017
Joseph P. Robinson, Ming Shao, Handong Zhao, Yue Wu, Timothy Gillis, Yun Fu
Proceedings of the 2017 Workshop on Recognizing Families In the Wild, 2017
Bibtex | Abstract | PDF | Competition Page
Recognizing Families In the Wild (RFIW) is a large-scale, multi-track automatic kinship recognition evaluation, supporting both kinship verification and family classification on scales much larger than ever before. It was organized as a Data Challenge Workshop hosted in conjunction with ACM Multimedia 2017. This was achieved with the largest image collection that supports kin-based vision tasks. In the end, we use this manuscript to summarize evaluation protocols, progress made and some technical background and performance ratings of the algorithms used, and a discussion on promising directions for both research and engineers to be taken next in this line of work.
@Article{Robinson:2017:RFW:3134421.3134424,
author = "Robinson, Joseph P. and Shao, Ming and Zhao, Handong and Wu, Yue and Gillis, Timothy and Fu, Yun",
title = "Recognizing Families In the Wild (RFIW): Data Challenge Workshop in Conjunction with ACM MM 2017",
booktitle = "Proceedings of the 2017 Workshop on Recognizing Families In the Wild",
pages = "5--12",
numpages = "8",
isbn = "978-1-4503-5511-7",
series = "RFIW '17",
year = "2017",
location = "Mountain View, California, USA",
url = "http://doi.acm.org/10.1145/3134421.3134424",
doi = "10.1145/3134421.3134424",
acmid = "3134424",
publisher = "ACM",
address = "New York, NY, USA",
keywords = "acm mm workshop, algorithmic design, big data, convolutional neural networks, data challenge, deep learning, evaluation, facial recognition, family classification, kinship verification, large image database, large-scale kinship recognition, metric learning, visual understanding",
}
Recognizing Families In the Wild (RFIW) is a large-scale, multi-track automatic kinship recognition evaluation, supporting both kinship verification and family classification on scales much larger than ever before. It was organized as a Data Challenge Workshop hosted in conjunction with ACM Multimedia 2017. This was achieved with the largest image collection that supports kin-based vision tasks. In the end, we use this manuscript to summarize evaluation protocols, progress made and some technical background and performance ratings of the algorithms used, and a discussion on promising directions for both research and engineers to be taken next in this line of work.
Shuyang Wang, Joseph P. Robinson, Yun Fu
12th IEEE Conference on Automatic Face and Gesture Recognition, 2017
@InProceedings{kinFG2017,
author = "Wang, Shuyang and Robinson, Joseph P and Fu, Yun",
title = "Kinship Verification on Families In The Wild with Marginalized Denoising Metric Learning",
booktitle = "Automatic Face and Gesture Recognition (FG), 2017 12th IEEE International Conference and Workshops on",
year = "2017",
}
With our Families In the Wild (FIW) dataset, which consists of labels 1, 000 families in over 12, 000 family photos, we benchmarked the largest kinship verification experiment to date. FIW, with its quality data and labels for full family trees found worldwide, more accurately is the true, global distribution of blood relatives with a total 378, 300 face pairs of 9 different relationship types. This gives support to tackle the problem with modern-day data-driven methods, which are imperative due to the complex nature of tasks involving visual kinship recognition- many hidden factors and less discrimination when considering face pairs of blood relatives. For this, we propose a denoising auto-encoder based robust metric learning (DML) framework and its marginalized version (mDML) to explicitly preserve the intrinsic structure of data and simultaneously endow the discriminative information into the learned features. Large-scale experiments show that our method outperforms other features and metric based approaches on each of the 9 relationship types.
2016
Joseph P. Robinson, Ming Shao, Yue Wu, Yun Fu
ACM on Multimedia Conference, 2016
@InProceedings{robinson2016families,
author = "Robinson, Joseph P. and Shao, Ming and Wu, Yue and Fu, Yun",
title = "Families In the Wild (FIW): Large-Scale Kinship Image Database and Benchmarks",
booktitle = "Proceedings of the 2016 ACM on Multimedia Conference",
pages = "242--246",
publisher = "ACM",
year = "2016"
}
We present the largest kinship recognition dataset to date, Families in the Wild (FIW). Motivated by the lack of a single, unified dataset for kinship recognition, we aim to provide a dataset that captivates the interest of the research community. With only a small team, we were able to collect, organize, and label over 10,000 family photos of 1,000 families with our annotation tool designed to mark complex hierarchical relationships and local label information in a quick and efficient manner. We include several benchmarks for two image-based tasks, kinship verification and family recognition. For this, we incorporate several visual features and metric learning methods as baselines. Also, we demonstrate that a pre-trained Convolutional Neural Network (CNN) as an off-the-shelf feature extractor outperforms the other fea- ture types. Then, results were further boosted by fine-tuning two deep CNNs on FIW data: (1) for kinship verification, a triplet loss function was learned on top of the network of pre-train weights; (2) for family recognition, a family-specific softmax classifier was added to the network.
![Picture](https://web.northeastern.edu/smilelab/RFIW2018/uploads/4/5/1/8/45182585/published/raymond_2.jpeg?1509129410")