MCS 2018. Adversarial Attacks on Black Box Face Recognition
First phase
Main PhaseEnd
Competition EndsWelcome!
Faces provide a natural means to recognize our friends, colleagues and relatives as well as to make ourselves recognized by others. Recent computer vision technology enables to scale face recognition to millions of faces. Modern methods of face recognition largely bypass human performance while relying on machine learning and neural networks. Despite this power, such methods can be vulnerable to attacks aiming to modify network outputs. Indeed, arbitrary changes to the network output can be produced by small and well-designed modifications of the network input, as known under Adversarial Examples. Applied to face recognition, adversarial examples imply that an attack can be designed to force a network to identify a person in the original image (Im1) as any other person on the planet (e.g. Im2) by making small modifications to the original image G(Im1) as shown below.
While attacks on open networks are relatively straightforward, the design of attacks on “black-box” systems, e.g. networks with unknown structure and parameters, is more difficult.
Our challenge ”Adversarial Attacks on Black-box Face Recognition” aims to test the vulnerability of black-box face recognition systems. Participants will be given an opportunity to compete and design the best attack forcing the system to recognize an image of a person A as person B, by applying small modifications to images of A.
Evaluation Criteria
We are given a black-box neural network that generates 512-dimentional descriptors D(I) for face images I. The distance between two faces is defined as ||D(I1)-D(I2)||2 and the network is designed to produce small distances for face images of the same person and large distances for face images of different people.
Participants are given 1000 pairs of source and target identities (IDs , IDt) where each pair corresponds to two different people. Each source identity is represented by five face images Is(1..5) . Each target identity is represented by five public It(1..5) and five private It(6..10) face images. The goal of participants is to design an image transformation G, such that
||D(G(Is))-D(It)||2 → min,
under a constraint that G makes small changes to the image, i.e.
SSIM(G(Is), Is) ≥ 0.95,
where SSIM is a Structural Similarity Measure.
The goal of the challenge is to minimize the distance ||D(G(Is))-D(It)||2 for all provided pairs of identities and corresponding images while keeping modified images close to originals. The score for the leaderboard will be calculated as
Score = 1/Npairs * 1/25 * Σk=1..Npairs Σi=1..5 Σj=6..10 ||D(G(Is(k,i)))-D(It(k,j))||2
Public leaderbord (Main phase) will be calculated on 25% of image pairs, remaining 75% will determine final competition results.
Participants are asked to submit modified images G(Is). Submissions with images violating the SSIM criteria above will be discarded.
Terms and Conditions
GENERAL RULES.
1. One account per participant. Submitting from multiple accounts is not allowed.
2. No private code sharing outside teams.
3. External data is allowed but source should be listed on the competition forum.
4. No competition data/models sharing outside competition.
WINNERS RULES.
Competition winners or their authorized representatives must attend the Machines Can See 2018 conference to receive prizes.
A prize winner must also fulfill the following obligations:
1. Deliver to the Competition Organizer the final model’s software code as used to generate the winning Submission and associated documentation. The delivered software code must be capable of generating the winning Submission and contain a description of resources required to build and/or run the executable code successfully.
2. Grant to Competition Organizer the Non-exclusive license to the winning model’s software code and represent that you have the unrestricted right to grant that license.
3. Sign and return all Prize acceptance documents as may be required by Competition Organizer.
Prizes
The winners will share a cash pool of 300,000 RUB and 3 NVIDIA GPU’s. The prizes will be announced at the annual conference MachinesCanSee organized on June 8, 2018 in Moscow.
1st place - 150,000 RUB + 1080Ti
2nd place - 75,000 RUB + 1080Ti
3rd place - 36,000 RUB + 1080Ti
4th place - 24,000 RUB
5th place - 15,000 RUB
Timeline
Competition starts - May 14 12:00 MSK
Private test phase - June 5 23:59 MSK *
Competition ends - June 6 23:59 MSK
* Participants will either submit final result or transfer 1 from public stage.
Main Phase
Start: May 14, 2018, midnight
Private Test Phase
Start: June 5, 2018, 9 p.m.
Competition Ends
June 20, 2018, 10 p.m.
You must be logged in to participate in competitions.
Sign In