3D Face Alignment in the Wild Challenge

Organized by stulyakov - Current server time: May 23, 2018, 7 a.m. UTC

Previous

Challenge
July 13, 2016, midnight UTC

Current

Post-Challenge
Nov. 1, 2016, 2:26 p.m. UTC

End

Competition Ends
July 28, 2016, 11:59 p.m. UTC

3D Face Alignment in the Wild Challenge

 

3DFAW

 

Within the past 15 years, there has been increasing interest in automated facial alignment within the computer vision and machine learning communities. Face alignment – the problem of automatically locating detailed facial landmarks across different subjects, illuminations, and viewpoints – is critical to all face analysis applications, such as identification, facial expression and action unit analysis, and in many human computer interaction and multimedia applications.

The most common approach is 2D alignment, which treats the face as a 2D object. This assumption holds as long as the face is frontal and planar. As face orientation varies from frontal, however, this assumption breaks down: 2D annotated points lose correspondence. Pose variation results in self occlusion that confounds landmark annotation.

To enable alignment that is robust to head rotation and depth variation, 3D imaging and alignment has been explored. 3D alignment, however, requires special sensors for imaging or multiple images and controlled illumination. When these assumptions cannot be met, which is common, 3D alignment from 2D video or images has been proposed as a potential solution.

This challenge addresses the increasing interest in 3D alignment from 2D images. 3DFAW Challenge evaluates 3D face alignment methods on a large, diverse corpora of multi-view face images annotated with 3D information. The corpora includes images obtained under a range of conditions from highly controlled to in-the-wild:

  • Multi-view images of MultiPIE
  • Synthetically rendered images using BP4D Spontaneous dataset
  • "In-the-wild” images and videos collected on the Internet, including 3DTV content and time-slice videos captured with the use of camera arrays. The depth information has been recovered using a novel dense model-based Structure from Motion technique

All three sources have been annotated in a consistent way. 3D meshes that had large errors were eliminated. The participants of the 3DFAW Challenge will receive an annotated training set and a validation set without annotations. The best scoring methods will be required to provide their binaries for internal evaluation on a hidden testing set.

Top ranked participants will be invited to submit their work foloowing ECCV 2016 guidelines to appear in 3DFAW workshop proceedings

[1] Gross, R., Matthews, I., Cohn, J., Kanade, T., and Baker, S. (2010). Multi-pie. Image and Vision Computing, 28(5), 807-813.

[2] Zhang, X., Yin, L., Cohn, J. F., Canavan, S., Reale, M., Horowitz, A., Liu, P., and Girard, J. M. (2014). BP4D-Spontaneous: a high-resolution spontaneous 3D dynamic facial expression database. Image and Vision Computing, 32(10), 692-706.

[3] Jeni, L. A., Cohn, J. F., and Kanade, T. (2015). Dense 3D face alignment from 2D videos in real-time. In Automatic Face and Gesture Recognition (FG), 2015 11th IEEE International Conference and Workshops on (Vol. 1, pp. 1-8). IEEE.

Evaluation

Participants should submit their results as a single zip archive, containing predictions for every image in a set. For example, for the image 009nrfttm4x4byufp4dd6.jpg the file should be named 009nrfttm4x4byufp4dd6.csv

Each of the *.csv files in the archive must contain 66 rows and 3 columns, where the numbers are separated with a comma. Submissions lacking predictions for all the images will be discarded, as well as those containing NaN values.

Submissions are evaluated using the following evaluation metrics:

  • Ground Truth Error - GTE
  • Cross View Ground Truth Consistency Error - CVGTCE

GTE is the widely accepted 300-W evaluation metric, which is "the average point-to-point Euclidean error normalized by the inter-ocular distance (measured as the Euclidean distance between the outer corners of the eyes)", and is computed as follows:

 

GTE

 

where X is the prediction, Y is the ground truth di is the interocular distance for the ith image.

CVGTCE is intended to evaluate cross-view consistency of the predicted landmarks. Basically, the prediction is compared to the ground truth for another view of the subject, and is computed in the following way:

 

CVGTCE

 

where the transformation parameters P={s, R, t} are obrained in the following fashion:

 
Transform

 

Good luck!

Official Rules

Common terms used in these rules:

These are the official rules that govern how the 3D Face Alignment in the Wild (3DFAW) Challenge will operate. This challenge will be simply referred to as the “contest” or the “challenge” throughout the rest of these rules and may be abbreviated on our website, in our documentation, and other publications as 3DFAW.

In these rules, “organizers”, “we,” “our,” and “us” refer to the organizers of the 3DFAW Challenge; “Database” refer to all the distributed image and annotation data; "participant”, “you,” and “yourself” refer to an eligible contest participant.

Contest Description

This is a skill-based contest and chance plays no part in the determination of the winner(s).

  1. Focus of the Contest: thousands of 2D images obtained under a range of conditions with the objective of performing automatic 3D face alignment
  2. All eligible entries received will be judged using the criteria described below to determine winners

Data Description and Usage Terms

The data of this challenge consist of 2D images and 3D facial landmark annotations. The 2D images are derived from the following datasets:

  1. “MultiPIE” was developed in the Robotics Institute at Carnegie Mellon University, Pittsburgh, PA. The project was funded by the U.S. Technical Support Working Group.
  2. “BU-4DFE” (Binghamton University - 3D Dynamic Facial Expression Database) was developed in the graphics and image computing (GAIC) laboratory, directed by Dr. Lijun Yin, at the Department of Computer Science, State University of New York at Binghamton. The project was funded by the U.S. National Science Foundation and NYSTAR (New York State Office of Science, Technology and Academic Research).
  3. “BP4D-Spontaneous” (Binghamton-Pittsburgh 4D Spontaneous Facial Expression Database) was developed in the GAIC Lab, directed by Dr. Lijun Yin of the State University of New York at Binghamton and Dr. Jeffrey Cohn of the University of Pittsburgh. The project was funded by the United States National Science Foundation.
  4. “TimeSlice3D” contains annotated 2D images extracted from time-sliced videos collected from the web.
  5. The organizers make no warranties regarding the Database, including but not limited to warranties of non-infringement or fitness for a particular purpose. The copyright of the images remain the property of their respective owners. By downloading and making use of the data, you accept full responsibility for using the data. You shall defend and indemnify the organizers, including their employees, Trustees, officers and agents, against any and all claims arising from your use of the data.
  6. Test data: The organizers will use test data to perform the final evaluation, hence the participants’ final entry will be based on test data.
  7. Training and validation data: The contest organizers will make available to the participants a training dataset with truth values, and a validation set with no truth values. The validation data will be used by the participants for practice purposes to validate their systems. It will be similar in composition to the test set (validation labels will be provided in the final test stage of the challenge).

The landmark annotations consist of 66 3D fiducial points that define the shape of permanent facial features. In the case of “MultiPIE” and “TimeSlice3D” datasets, the depth information has been recovered using a model-based Structure from Motion technique (ZFace, www.zface.org). The same technique was used to acquire consistent annotation from the 3D ground truth data of “BU-4DFE” and “BP4D-Spontanoues” datasets.

Three out of four datasets to be used for the 3DFAW Challenge of ECCV 2016 are subsets of the “MultiPIE”, “BU-4DFE” and “BP4D-Spontaneous” datasets. We will refer these as “MultiPIE_Subset”, “BU4D_Subset” and “BP4D_Subset”. The “MultiPIE_Subset” consist of 3-5 views with one illumination condition per subject from the original “MultiPIE” dataset. The “BU4D_Subset” and “BP4D_Subset” consist of rendered images with different poses and the corresponding 3D feature points.

The datasets may be used for the 3DFAW Challenge of ECCV 2016 only. The recipient of the datasets must be a full-time faculty, researcher or employee of an organization (not a student) and must agree to the following terms:

  1. The participants receive a non-exclusive, non-transferable license to use the Database for internal research purposes. You may not sell, rent, lease, sublicense, lend, time-share or transfer, in whole or in part, or provide third parties access to the Database.
  2. The data will be used for non-for-profit research only. Any use of the Database in the development of a commercial product is prohibited.
  3. If this Database is used, in whole or in part, for any publishable work, the following papers must be referenced:

     

    For the Multi-PIE Database:

    Ralph Gross, Iain Matthews, Jeffrey Cohn, Takeo Kanade, and Simon Baker. "Multi-pie." Image and Vision Computing 28, no. 5 (2010): 807-813.

    For the BU-4DFE Database:

    Lijun Yin, Xiaochen Chen, Yi Sun, Tony Worm, and Michael Reale, “A High-Resolution 3D Dynamic Facial Expression Database” The 8th International Conference on Automatic Face and Gesture Recognition (FGR08), Amsterdam, The Netherlands, 2008.

    For the BP4D-Spontaneous Database:

    Xing Zhang, Lijun Yin, Jeffrey Cohn, Shaun Canavan, Michael Reale, Andy Horowitz, Peng Liu, and Jeffrey Girard, BP4D-Spontaneous: A high resolution spontaneous 3D dynamic facial expression database, Image and Vision Computing, 32 (2014), pp.692-706 (special issue of The Best of Face and Gesture 2013)

    For the landmark annotation:

    László A. Jeni, Jeffrey F. Cohn, and Takeo Kanade. "Dense 3D face alignment from 2D video for real-time use." Image and Vision Computing (2016). doi:10.1016/j.imavis.2016.05.009

Eligibility criteria

  1. You are an individual or a team of people desiring to contribute to the tasks of the challenge and accepting to follow its rules; and
  2. You are NOT a resident of any country constrained by US export regulations included in the OFAC sanction page http://www.treasury.gov/resource-center/sanctions/Programs/Pages/Programs.aspx. Therefore residents of these countries / regions are not eligible to participate; and
  3. You are not involved in any part of the administration and execution of this contest; and
  4. You are not an immediate family (parent, sibling, spouse, or child) or household member of a person involved in any part of the administration and execution of this contest.

This contest is void within the geographic area identified above and wherever else prohibited by law.

Entry

  1. All members of the team meets eligibility criteria.
  2. To be considered in the competition, provide a description of your approach. To be considered for publication in the proceedings, submit a workshop paper. All participants are invited to submit a maximum 8-page paper for the proceedings of the ECCV 2016 - 3D Face Alignment in the Wild (3DFAW) Challenge workshop.
  3. Workshop report: The organizers may write and publish a summary of the results.
  4. Submission: The entries of the participants will be submitted on-line via the Codalab web platform. During the development period, the participants will receive immediate feed-back on validation data released for practice purpose. For the final evaluation, the results will be computed automatically on test data submissions. The performances on test data will not be released until the challenge is over.
  5. Original work, permissions: In addition, by submitting your entry into this contest you confirm that, to the best of your knowledge:
    • Your entry is your own original work; and
    • Your entry only includes material that you own, or that you have permission from the copyright / trademark owner to use.

On-line notification

We will post changes in the rules or changes in the data as well as the names of confirmed winners (after contest decisions are made by the judges) online on the 3DFAW webpage

Conditions.

By entering this contest you agree all terms of use. You understand that the violation of the use will be pursued.

This contest is void within the geographic area identified above and wherever else prohibited by law.

Entry

  1. All members of the team meets eligibility criteria.
  2. To be considered in the competition, provide a description of your approach. To be considered for publication in the proceedings, submit a workshop paper. All participants are invited to submit a maximum 8-page paper for the proceedings of the ECCV 2016 - 3D Face Alignment in the Wild (3DFAW) Challenge workshop.
  3. Workshop report: The organizers may write and publish a summary of the results.
  4. Submission: The entries of the participants will be submitted on-line via the Codalab web platform. During the development period, the participants will receive immediate feed-back on validation data released for practice purpose. For the final evaluation, the results will be computed automatically on test data submissions. The performances on test data will not be released until the challenge is over.
  5. Original work, permissions: In addition, by submitting your entry into this contest you confirm that, to the best of your knowledge:
    • Your entry is your own original work; and
    • Your entry only includes material that you own, or that you have permission from the copyright / trademark owner to use.

On-line notification

We will post changes in the rules or changes in the data as well as the names of confirmed winners (after contest decisions are made by the judges) online on http://mhug.disi.unitn.it/workshop/3dfaw/.

Conditions.

By entering this contest you agree all terms of use. You understand that the violation of the use will be pursued.

Training

Start: June 13, 2016, midnight

Challenge

Start: July 13, 2016, midnight

Post-Challenge

Start: Nov. 1, 2016, 2:26 p.m.

Competition Ends

July 28, 2016, 11:59 p.m.

You must be logged in to participate in competitions.

Sign In