[Deprecated, check description] CVPR 2020 BDD100K Multiple Object Tracking Challenge

Organized by 28 - Current server time: Nov. 30, 2020, 5:25 p.m. UTC

First phase

val2020
Jan. 1, 2020, midnight UTC

End

Competition Ends
May 21, 2020, 6:59 a.m. UTC

[Deprecated] CVPR 2020 BDD100K Multiple Object Tracking Challenge

 

animated

 

[Update 0521] This challenge has been moved to here. We now employ mean MOTA (mean MOTA value of each of the 8 categories) as the metric for ranking. We also included an example submission file in the description for reference.

Evaluation

Phases

There are two phases for the challenge: val phase and test phase. The final ranking will be based on the test phase.

Pre-training

It is a fair game to pre-train your network with ImageNet or COCO, but if other datasets are used, please note in the submission description. We will rank the methods without using external datasets except ImageNet and COCO.

Ignoring distractors

"other person", "trailer", and "other vehicle" are considered detractors in this challenge. As a preprocessing step, all predicted boxes are matched and the ones matched to distractor ground-truth boxes are ignored.

Crowd region

After bounding box matching, we ignore all detected false-positive boxes that has >50% overlap with the crowd region (ground-truth boxes with the "Crowd" attribute).

Super-category

In addition to the evaluation of all 8 classes, we merge ground truth and prediction categories into 3 super-categories specified above, and evaluate the results for each super-category. The super-category evaluation results will be provided only for the purpose of reference.

Metrics

It is a fair game to pre-train your network with ImageNet or COCO, but if other datasets are used, please note in the submission description. We will rank the methods without using external datasets except ImageNet and COCO.

We employ Multiple Object Tracking Accuracy (MOTA) as our primary evaluation metric for ranking. All metrics are detailed below. See this paper for more details.

  • MOTA (%): Multiple object tracking accuracy, reported in percentage.
  • MOTP (%): Multiple object tracking precision, reported in percentage.
  • Misses: The total number of missed ground-truth boxes.
  • FP: Number of false positive matches after global min-cost matching.
  • Switch: An identity switch is counted when a ground-truth object is matched with a track that is different from the last known assigned track.
  • Mostly Tracked: Number of objects with at least 80 percent of its lifespan tracked.
  • Mostly Lost:Number of objects with less than 20 percent of its lifespan tracked.
  • Partially Tracked: Number of objects with at least 20 percent and less than 80 percent of its lifespan tracked.

Please refer to BDD data website for terms to use BDD data.

val2020

Start: Jan. 1, 2020, midnight

Description: The val phase evaluates 200 sequences in the BDD100K MOT validation set. The data can be downloaded from https://bdd-data.berkeley.edu/. Evaluation usually takes about 5 minutes (there might be a roughly 10 to 15 minutes delay before the evaluation starts). Click "Submit to Leaderboard" to submit your results to the leaderboard in Results page.

test2020

Start: Jan. 1, 2020, midnight

Description: The test phase evaluates 400 sequences in the BDD100K MOT testing set. The data can be downloaded from https://bdd-data.berkeley.edu/. Evaluation usually takes about 10 minutes (there might be a roughly 10 to 15 minutes delay before the evaluation starts). Click "Submit to Leaderboard" to submit your results to the leaderboard in Results page.

Competition Ends

May 21, 2020, 6:59 a.m.

You must be logged in to participate in competitions.

Sign In