Evaluation Criteria

Submissions are evaluated across 2 test sets:

• Seen kitchens (kitchens present in the training set)
• Unseen kitchens (kitchens not present in the training set)

We evaluate model performance on the mean average precision (mAP) metric from PASCAL VOC using IoU thresholds of 0.05, 0.5, 0.75 for many shot and few shot classes.

Terms and Conditions

• You agree to us storing your submission results for evaluation purposes.
• You agree that if you place in the top-10 at the end of the challenge you will submit your code so that we can check for cheating.
• You agree not to distribute the EPIC-Kitchens dataset without prior written permission.

Submissions

To submit your results to the leaderboard you must construct a submission zip file containing two files:

• seen.json - Model inference on the seen kitchens test set (S1)
• unseen.json - Model inference on the unseen kitchens test set (S2)

Both of these files follow the same format detailed below:

JSON Submission Format

The JSON submission format is composed of a single JSON object containing detections for every image in the test set. The JSON file should consist of a list of of each detected bounding box with the corresponsing video and frame, category and score. Each entry is should have the form:

{
  "version": "0.1",
  "challenge": "object_detection",
  "results": [{
    "video_id"          : str,
    "frame"             : int,
    "category_id"       : int,
    "bbox"              : [ymin, xmin, ymax, xmax],
    "score"             : float
  }]

You should submit 300 detections per image. Note: box coordinates should be in the range [0,1] in terms of the image. We recommend rounding coordinates to 5 decimal places to reduce the size of the JSON files.

Submission archive

To upload your results to CodaLab you have to zip both files into a flat zip archive (they can’t be inside a folder within the archive).

You can create a flat archive using the command providing the JSON files are in your current directory.

$ zip -j my-submission.zip seen.json unseen.json