The challenge has been completed, and the results have been announced on our website.
Congratulations to team AcRec@UniKoblenz and RAAR3D!
In the HEART-MET Gesture Recognition Challenge, the task is to recognize gestures from videos. The gestures are meant to communicate intentions or commands to the robot, and include the stop sign, nodding, pointing etc. Some examples can be seen in this video.
HEART-MET is one of the competitions in the METRICS project, which has received funding from European Union’s Horizon 2020 research and innovation program under grant agreement No 871252. The competition aims to benchmark assistive robots performing healthcare-related tasks in unstructured domestic environments.
Gesture recognition is one of the functional benchmarks by which assistive robots are evaluated. Gesture recognition is necessary for non-verbal communication with a user. The datasets for this challenge are collected from real robots performing gesture recognition in domestic environments with several different volunteers performing the gestures.
Other challenges presented by HEART-MET include Activity Recognition, Object Detection and Handover Failure Detection. More details can be found on the HEART-MET website.
In order to participate:
Your request will be approved within 24 hours.
If you are participating as a team, the team leader can create a new team by clicking the Team tab. Team members can then request to join the team. A team member must first be registered to participate in the challenge before they can request to join a team. See Competition Teams for more details.
The challenge consists of two phases:
During the validation phase, you are provided with a subset of the dataset with which you can validate your models, and test the submission procedure. No training data is provided since it is expected that you use existing public datasets for training.
The competition phase will start at the end of the validation phase, and you will be provided with a larger competition set. The labels for the validation set will be released as well; hence you can use the validation set for fine-tuning models if required. The competition phase will have a limit on the total number of submissions and the maximum submissions per day, and the duration is shorter compared to the validation phase.
You must submit a submission bundle (a .zip archive) with a single text file named submission.txt. This file has the following format:
video0000.mp4 3 video0001.mp4 7 video0002.mp4 2 ... ...
Each line consists of the name of the video file and the recognized gesture ID.
A sample submission file has been provided in the starting kit.
The gesture IDs are from 0-8 (inclusive), and can be found under Dataset.
The evaluation is based on the true positive rate = number of true positives / total number of videos
The dataset consists of video clips up to 7 seconds long, of a person performing a gesture. The gesture may be performed once (for e.g. stop sign) or multiple times (for e.g. waving). The videos are recorded from multiple cameras; hence the dataset may contain videos of the same volunteer performing a gesture from different viewpoints.
There are a total of 9 gestures in this challenge. There is some (intentional) overlap with the classes in the 20BN-jester dataset and the NVIDIA Dynamic Hand Gesture Dataset, with some additional classes not present in either dataset.
The gesture IDs and the corresponding classes contained in this dataset are listed below:
0 | Nodding |
---|---|
1 | Stop sign |
2 | Thumb down |
3 | Waving |
4 | Pointing |
5 | Pulling hand in |
6 | Thumb up |
7 | Pushing hand out |
8 | Shaking head |
The labels for the validation set will be released at the end of the validation phase. It will consist of a single text file with each line consisting of the name of the video and the gesture ID, as seen in the example below:
video0000.mp4 3 video0001.mp4 7 video0002.mp4 2 ... ...
The validation set consists of 100 videos and the competition set consists of 314 videos.
Start: April 8, 2021, midnight
Start: May 15, 2021, 11:59 p.m.
June 30, 2021, 11:59 p.m.
You must be logged in to participate in competitions.
Sign In