CVPM2020 challenge: The 1st Challenge on Remote Physiological Signal Sensing (RePSS)

Organized by XuesongNiu - Current server time: July 9, 2020, 7:41 a.m. UTC

First phase

Final test phase
Feb. 20, 2019, midnight UTC

End

Competition Ends
March 6, 2020, midnight UTC

Challenge on Remote Physiological Signal Sensing

Paper submission instruction

Challenge results have been announced and now paper submission is open at CMT: https://cmt3.research.microsoft.com/RePSS2020
All participants are welcomed to write a paper introducing their approaches and results for submission to the RePSS change. Other contents that are related to the challenge topic are also welcomed for submission. All papers will be evaluated through a double-blind review process. The paper should be no longer than 8 pages (including figures, tables, and references), and following all CVPR2020 guidelines and templates. The submission deadline is Mar. 30th, 11:59 p.m. Pacific Standard Time, 2020.

Challenge overview

Remote measurement of physiological signals from videos is an emerging topic. The topic draws great interests from both researchers and companies, and the number of published papers is growing every year. Despite the thriving research interests, the lack of publicly available benchmark databases and a fair validation platform are the major issues that hinder its further development. Kin researchers have to make repetitive efforts on self-collecting small datasets to test proposed methods, which makes it difficult to fairly compare and evaluate the actual strength and weakness of each proposed method, as self-collected data are of different recording conditions and qualities.

For this concern, we organize the first challenge on Remote Physiological Signal Sensing (RePSS) in conjunction with the CVPM 2020 workshop, which will be held in CVPR 2020 at Seattle, USA. As the first open challenge on remote physiological signal sensing, we will be focusing on measuring the average heart rate from color facial videos, which is the most fundamental problem in this field.

We will provide a large dataset (500 people’s data for training, and 200 people’s data for testing) as the benchmark for this competition. We hope this will serve as a fair evaluation platform for researchers who are kin for this topic. More data is under construction, and in future challenges, we hope to add more data to cover various challenging conditions, and also to cover more complex physiological sensing problems, e.g., measuring breathing rate, heart rate variability, and so on.

Important Dates

  • Jan.15 Training data release
  • Feb.10 Team register deadline
  • Feb.20 Validation data release (no more registration accepted)
  • Mar.05 Validation submission deadline
  • Mar.25   Mar.30 Paper submission deadline (11:59 pm PST)
  • Apr.05 Notification to authors
  • Apr.15 Camera-ready (date TBD)

Notification about the test data

The test data for the challenge are ready for release now and we will email all registered teams. 
The test data contains 1000 pieces of 10s videos from 200 persons (100 from VIPL-HR, 100 from OBF). 
The eyes and mouth areas are covered with mosaics according to the data regulation policy of University of Oulu to protect subjects' privacy. 
To compensate, we provide the face bounding box location and 68 facial landmarks' coordinate info of every frame in .txt files: [bbox_x1, bbox_y1, bbox_x2, bbox_y2, p1_x, p1_y, p2_x, p2_y, … p68_x, p68_y].

Evaluation

Each sample contains a 10-second face video, and you must estimate the average heart rate of the subject. You can either use learning-based methods or hand-crafted methods to get the final results.

Evaluation Metrics

We provide three metrics to evaluate the performance for heart rate, which includes the mean absolute error (MAE), root mean square error (RMSE), and the Pearson correlation coefficient (R). The final ranking is based on the MAE metric.

Results Submission

Participants need to submit their results in form of text files ('.txt'). The name of the text file must be "answers.txt". The submission must be one number of average heart rate in beat per minute (bpm) for each video sample, in corresponding to the video ID, in the form of: i.e.,

(first column: video ID) (second column: estimated average heart rate)

Examples:

0001.avi, 81.33
0002.avi, 86.01
....
1000.avi, 77.00

Data

Data Access

Data for the challenge include two parts, one part is provided by the Institute of Computing Technology, Chinese Academy of Sciences (ICT, CAS) and the other part is provided by the University of Oulu. Two license agreements (LA) must be signed before participants can get access to any data. The license agreements can be downloaded from Baidu Cloud Drive or Google Cloud Drive. Please follow the following instructions carefully for preparing and signing the license agreements:

1). The LA must be signed by a person with an email that affiliated with an institution or company (e.g., xx.xx@mit.edu, xx.xx@microsoft.com ), which means that the person has a fixed position in the institution or company (e.g., a professor from a university, or an employee from a company). Personal emails (e.g., xxx@gmail.com, xxx@qq.com ) are NOT valid. For students please ask your supervisor to sign the LA.

2). One signer can be associated with multiple registered competition IDs, i.e., one professor can sign the LA and multiple students from his/her group can register to the competition.

3). The signer must read through the LA carefully, and only sign the document when he/she fully understand and agree to all the items listed in the LA.

4). The signed LA will be scanned into PDF format, named as CVPM 2020 data agreement_Yourname.pdf, and sent to Dr. Hu Han (hanhu[at]ict.ac.cn) and Dr. Xiaobai Li (xiaobai.li[at]oulu.fi). We will send you the download link and password after receiving the signed LA.

5). The signer is fully responsible (for all users whose IDs are associated with him/her) to make sure that all associated ID users are fully aware of the LA contents, and the data is accessed and used in the proper way according to LA. Data users have no right to distribute the data in any form.

6). The data is shared only for research purpose of this competition usage but not for any other usage. All data must be deleted after the competition by 15.04.2020.

Test data

The test data for the challenge are ready for release now and we will email all registered teams.
The test data contains 1000 pieces of 10s videos from 200 persons (100 from VIPL-HR, 100 from OBF).
The eyes and mouth areas are covered with mosaics according to the data regulation policy of University of Oulu to protect subjects' privacy.
To compensate, we provide the face bounding box location and 68 facial landmarks' coordinate info of every frame in .txt files: [bbox_x1, bbox_y1, bbox_x2, bbox_y2, p1_x, p1_y, p2_x, p2_y, … p68_x, p68_y].

Final test phase

Start: Feb. 20, 2019, midnight

Competition Ends

March 6, 2020, midnight

You must be logged in to participate in competitions.

Sign In