Welcome to the EPIC-Kitchens 2018 Action Recognition challenge. EPIC-Kitchens is an unscripted egocentric action dataset collected from 32 different people from 4 cities across the world.
This challenge is part of the ECCV 2018 workshop.
Classify trimmed action segments from seen and unseen kitchens by action verb and noun.
Submissions are evaluated across 2 test sets:
We evaluate model performance across two sets of metrics:
These metrics are micro-averaged giving weight to each class proportionately to their frequency in the test set under evaluation.
These metrics are macro-averaged giving equal weight to all classes regardless of their prevalence. We compute these for many-shot classes only: these are verb/noun classes that appear more than 100 times in training, and in the case of actions the cross product between the many shot verb and many shot noun classes given that the action appears at least once in training.
To submit your results to the leaderboard you must construct a submission zip file containing two files:
seen.json- Model inference on the seen kitchens test set (S1)
unseen.json- Model inference on the unseen kitchens test set (S2)
Both of these files follow the same format detailed below:
The JSON submission format is composed of a single JSON object containing entries for every action in the test set. Specifically, the JSON file should contain:
'version'property, set to
'0.1'(the only supported version so far);
'challenge'property, which can assume the following values, depending on the challenge:
'results'object containing entries for every action in the test set (e.g.
'1924'is the first action ID in the seen test set).
Each action segment entry is a nested object composed of two entries:
'verb', specifying the class score for every verb class and the other,
'noun' specifying the score for every noun class. Action scores are automatically computed by applying softmax to the verb and noun scores and computing the probability of each possible action.
If you wish to compute your own action scores, you can augment each segment submission with exactly 100 action scores with the key
The keys of the
action object are of the form
You can provide scores in any float format that numpy is capable of reading (i.e. you do not need to stick to 3 decimal places).
If you fail to provide your own action scores we will compute them by
p(a = (v, n)) = p(v) * p(n)
To upload your results to CodaLab you have to zip both files into a flat zip archive (they can’t be inside a folder within the archive).
You can create a flat archive using the command providing the JSON files are in your current directory.
$ zip -j my-submission.zip seen.json unseen.json
Start: June 30, 2018, midnight
You must be logged in to participate in competitions.Sign In