Mouse Behavior Challenge MBC2020

Organized by tttamaki - Current server time: July 9, 2020, 7:21 a.m. UTC

Current

test-dev
Dec. 1, 2019, 3 p.m. UTC

Next

test-challenge
Aug. 31, 2020, 2:59 p.m. UTC

End

Competition Ends
Sept. 1, 2020, 2:59 p.m. UTC

Mouse Behavior Challenge: MBC2020

Normal or Parkinson's disease ? Classifying mice from their trajectories.

Parkinson's disease (PD) is one of major diseases for people over the age of 60, and causes disorders of body motions due to the affected motor system. It is important to tackle PD because there are more than 6 million patients and 100k death in a year globally. Many neuroscience research works have been done by using mice as a model animal for investigating and understanding PD of humans. In this challenge, your task is to classify mice trajectories into normal or PD, provided training mice trajectories.

Data: mouse in a box

A normal or PD mouse moves freely in a box surrounded by black walls. A camera mounted above the box takes top-view movies in 30 fps. 2D coordinates of three points on the mouse (the tip of the nose, the point between the ears, the root of the tail) are tracked and recorded as trajectories using DeepLabCut.

Examples of normal and PD trajectories with videos

The first video is a normal mouse, and the second is a PD mouse. Three points on the mouses are shown. These videos are not included in the dataset and this challenge is based on trajectory (i.e., 2D coordinates).


About us

Systems Science of Bio-Navigation

Grant-in-Aid for Scientific Research on Innovative Areas from 2016 to 2021

Navigation is a fundamental behavior of animals including human. In navigation, the following three functions are required: the acquisition of dynamically-changing information from external and internal environment, the choice of route and destination based on the information, and the behavioral regulation to reach the destination. We aim for systems science of bio-navigation to understand the "algorithms" for the navigation of animals. To this end, we bring together experts from control engineering, data science, animal ecology, and neuroscience, and jointly work on how to measure, analyze, understand, and verify bio-navigation.

By hosting a series of competitions "Animal Behavior Challenge (ABC2018)" "Human Behavior Challenge (HBC2018)" "Mouse Behavior Challenge (MBC2020)", we hope to boost our understanding the algorithms for animal navigation.

Please visit our website.

Terms and Conditions

This challenge is governed by the general ChaLearn contest rules.

Rules

You may submit 5 submissions every day and 50 in total in the test-dev phase.

Metric

Submitted results are evaluated by the accuracy, widely used for binary classification problems.

  • acc = (number of correctly classified test trajectories) / (total number of test trajectories)

It is 0 if all predictions are wrong, while 1 if perfect.

Phases

  1. test-dev phase: scores are shown with randomness (1/50 of your submission labels are randomly modified for avoiding cheat and overfit) on the leaderboard, which is NOT the final score.
  2. test-challenge phase: scores are shown with no randomness. This score is the final one.

Protocol

  • During the test-dev phase, You can submit a csv file up to 50 times, with 5 maximum submissions per day. (Caution! if you submit 5 times a day for 10 days, then you run out the allowed number of 50 submissions.)
  • The submitted csv file is the predictions for "all 1538 samples" in the test set.
  • If the file is successfully submitted, the result score is shown in your private view (with randomness during the test-dev phase). It may take few minutes. Be patient ! (If something wrong, please check the log by clicking "View scoring output log" link.)
  • Once your submissions has scores, choose one of the results to show on the public leaderboard, by clicking "Submit to Leaderboard" button. The selected one is used for the evaluation in the test-challenge phase. If you want to show another submission on the leaderboard (and hence for the evaluation in the test-challenge phase), simply click "Submit to Leaderboard" button of the submission you want to show.
  • In the test-challenge phase, you have to nothing. The score of the submission selected in the test-dev phase is shown on the leaderboard. The score is computed without randomness.

Submission Format of the result

There are 1538 test trajectory, and classification results should be a 1538-line text file, which contains 0 (normal) or 1 (PD) for each of test trajectory.

Each line has the label of the corresponding test trajectory; that is, line 0 is the label of the test trajectory 0000.csv. There is no header line.

Predicted label is binary (character):

  • normal: 0
  • PD: 1

Here is an example:

=================
1
1
1
0
1
0
0
...
=================

file name

The file name should be "test_prediction.csv", which should be zipped as a single zip file.

The zip file can have any name, but should contain a single file named "test_prediction.csv" without any folders.

You can create and check your submission zip file by using zip and unzip as follows:

$ zip yoursubmission.zip test_prediction.csv
updating: test_prediction.csv (deflated 79%)
$ unzip -l yoursubmission.zip
Archive:  yoursubmission.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
   357417  03-22-2019 10:17   test_prediction.csv
---------                     -------
   357417                     1 file

Task

Classifying trajectories of mice into normal or PD

Trajectory file format

A single CSV file (0000.csv, 0001.csv, ...) contains a trajectory. The first line is a header, and following lines represent the information of the location of a mouse.

  • frame: frame number (starting from 0)
  • tip_x: x coordinates of 2D location of the tip of the mouse
  • tip_y: y coordinates of 2D location of the tip of the mouse
  • ear_x: x coordinates of 2D location of the point between the ears of the mouse
  • ear_y: y coordinates of 2D location of the point between the ears of the mouse
  • tail_x: x coordinates of 2D location of the root of the tail of the mouse
  • tail_y: y coordinates of 2D location of the root of the tail of the mouse

Float values are of the format %.01f, and fields are separated by a single comma.

Here is an example:

frame,top_x,top_y,ear_x,ear_y,tail_x,tail_y
0,342.0,402.3,342.4,374.3,324.4,305.4
1,325.4,407.3,331.9,379.3,326.0,320.8
2,321.4,409.1,327.7,379.3,328.2,321.9
3,314.3,410.7,324.0,379.0,329.9,323.7
...
  • Trajectories are obtained by tracking mice in 30fps videos of size 640x480. See videos at the top page. In the video, top-left corner is the origin (0, 0), and the positive direction of the x axis is rightward, and the positive direction of the y axis is the downward (this is common in image processing).

Labels: normal or PD

  • Training trajectories of files from 0.csv to 4.csv are normal.
  • Training trajectories of files from 5.csv to 9.csv are PD.

The dataset

The training and test trajectories are different in length; training is long (15min) and unprocessed, while test is short (4 to 8 sec) and post-processed.

  • Training set
    • 5 normal trajectories
    • 5 PD trajectories
    • each are 15 minutes long
      • that is, each has 30fps * 60 sec * 15 min = 27000 lines.
      • 2D coordinates are obtained by tracking points in 30fps videos of size 640x480, as mentioned above. Coordinate ranges are therefore from 0 to 639 for x and 0 to 439 for y.
  • Test set
    • 1539 trajectories
    • each are between 4 and 8 seconds long (between 120 and 240 lines)
      • 2D coordinates are obtained by the same process mentioned above.
      • HOWEVER post-processed. Coordinates in a single test trajectory file is
        • first translated so that the tip of frame 0 is the origin (0, 0),
        • then randomly rotated (from -pi to +pi) about the origin (hence the frame 0 tip remains at (0, 0)).
      • Coordinate ranges are therefore not positive anymore.

Prizes

UPDATE: Due to the pandemic of COVID-19, the following symposium MAY be cancelled in future. In such case, unfortunately the travel grant will not be given to winners and we will PDF certificates to winners as a prize.

The winner will receive the following travel grant in addition to the winner certificate.

Travel grant

The first winner will receive

  • Flight/surface ticket (+ hotel for a few days) to Nagoya for attending the award seremony and presenting the method in the poster session of Symposium on Systems Science of Bio-Navigation 2020, November 25-26, 2020, Nagoya, Japan
    • Venue:Sakata and Hirata Hall (Science South Building), Nagoya University. MAP

Note that

  • to recieve the prizes, winners must publish their methods and/or codes available online (for example, as a technical report on arXiv and github) upon request by the organizer.
  • In case of team submission, the travel grant goes to a single person who is a representative.
  • The prize will go to the next winner if the winner withdraws the attendance and the poster presentation for any reasons (e.g., personal or VISA issues). Winners will be received PDF certificates by email in case of withdrawal.
  • We will provide an invitation letter and the winner have to obtain a VISA by himself/herself.

Terms and Conditions

This challenge is governed by the general ChaLearn contest rules.

test-dev

Start: Dec. 1, 2019, 3 p.m.

Description: Submit your results on the test set. Show one of results to the leaderboard, which is used for the test-challenge phase.

test-challenge

Start: Aug. 31, 2020, 2:59 p.m.

Description: Final phase. you don't submit any results. Instead, the result shown in the leaderboard of the test-dev phase is used for the final evaluataion.

Competition Ends

Sept. 1, 2020, 2:59 p.m.

You must be logged in to participate in competitions.

Sign In