Rules

You may submit 5 submissions every day and 50 in total in the test-dev phase.

Metric

Submitted results are evaluated by the accuracy, widely used for binary classification problems.

• acc = (number of correctly classified test trajectories) / (total number of test trajectories)

It is 0 if all predictions are wrong, while 1 if perfect.

Phases

1. test-dev phase: scores are shown with randomness (1/50 of your submission labels are randomly modified for avoiding cheat and overfit) on the leaderboard, which is NOT the final score.
2. test-challenge phase: scores are shown with no randomness. This score is the final one.

Protocol

• During the test-dev phase, You can submit a csv file up to 50 times, with 5 maximum submissions per day. (Caution! if you submit 5 times a day for 10 days, then you run out the allowed number of 50 submissions.)
• The submitted csv file is the predictions for "all 1538 samples" in the test set.
• If the file is successfully submitted, the result score is shown in your private view (with randomness during the test-dev phase). It may take few minutes. Be patient ! (If something wrong, please check the log by clicking "View scoring output log" link.)
• Once your submissions has scores, choose one of the results to show on the public leaderboard, by clicking "Submit to Leaderboard" button. The selected one is used for the evaluation in the test-challenge phase. If you want to show another submission on the leaderboard (and hence for the evaluation in the test-challenge phase), simply click "Submit to Leaderboard" button of the submission you want to show.
• In the test-challenge phase, you have to nothing. The score of the submission selected in the test-dev phase is shown on the leaderboard. The score is computed without randomness.

Submission Format of the result

There are 1538 test trajectory, and classification results should be a 1538-line text file, which contains 0 (normal) or 1 (PD) for each of test trajectory.

Each line has the label of the corresponding test trajectory; that is, line 0 is the label of the test trajectory 0000.csv. There is no header line.

Predicted label is binary (character):

• normal: 0
• PD: 1

Here is an example:

=================
1
1
1
0
1
0
0
...
=================

file name

The file name should be "test_prediction.csv", which should be zipped as a single zip file.

The zip file can have any name, but should contain a single file named "test_prediction.csv" without any folders.

You can create and check your submission zip file by using zip and unzip as follows:

$ zip yoursubmission.zip test_prediction.csv
updating: test_prediction.csv (deflated 79%)
$ unzip -l yoursubmission.zip
Archive:  yoursubmission.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
   357417  03-22-2019 10:17   test_prediction.csv
---------                     -------
   357417                     1 file

Task

Classifying trajectories of mice into normal or PD

Trajectory file format

A single CSV file (0000.csv, 0001.csv, ...) contains a trajectory. The first line is a header, and following lines represent the information of the location of a mouse.

• frame: frame number (starting from 0)
• tip_x: x coordinates of 2D location of the tip of the mouse
• tip_y: y coordinates of 2D location of the tip of the mouse
• ear_x: x coordinates of 2D location of the point between the ears of the mouse
• ear_y: y coordinates of 2D location of the point between the ears of the mouse
• tail_x: x coordinates of 2D location of the root of the tail of the mouse
• tail_y: y coordinates of 2D location of the root of the tail of the mouse

Float values are of the format %.01f, and fields are separated by a single comma.

Here is an example:

frame,top_x,top_y,ear_x,ear_y,tail_x,tail_y
0,342.0,402.3,342.4,374.3,324.4,305.4
1,325.4,407.3,331.9,379.3,326.0,320.8
2,321.4,409.1,327.7,379.3,328.2,321.9
3,314.3,410.7,324.0,379.0,329.9,323.7
...

• Trajectories are obtained by tracking mice in 30fps videos of size 640x480. See videos at the top page. In the video, top-left corner is the origin (0, 0), and the positive direction of the x axis is rightward, and the positive direction of the y axis is the downward (this is common in image processing).

Labels: normal or PD

• Training trajectories of files from 0.csv to 4.csv are normal.
• Training trajectories of files from 5.csv to 9.csv are PD.

The dataset

The training and test trajectories are different in length; training is long (15min) and unprocessed, while test is short (4 to 8 sec) and post-processed.

• Training set
• 5 normal trajectories
• 5 PD trajectories
• each are 15 minutes long
  • that is, each has 30fps * 60 sec * 15 min = 27000 lines.
  • 2D coordinates are obtained by tracking points in 30fps videos of size 640x480, as mentioned above. Coordinate ranges are therefore from 0 to 639 for x and 0 to 439 for y.
• Test set
• 1539 trajectories
• each are between 4 and 8 seconds long (between 120 and 240 lines)
  • 2D coordinates are obtained by the same process mentioned above.
  • HOWEVER post-processed. Coordinates in a single test trajectory file is
    • first translated so that the tip of frame 0 is the origin (0, 0),
    • then randomly rotated (from -pi to +pi) about the origin (hence the frame 0 tip remains at (0, 0)).
  • Coordinate ranges are therefore not positive anymore.

Prizes

UPDATE 2020/Sep/1: Due to the pandemic of COVID-19, the following symposium has been cancelled. We are very sorry but the travel grant will not be given to winners, and we will PDF certificates to winners as a prize.

UPDATE: Due to the pandemic of COVID-19, the following symposium MAY be cancelled in future. In such case, unfortunately the travel grant will not be given to winners and we will PDF certificates to winners as a prize.

The winner will receive the following travel grant in addition to the winner certificate.

Travel grant

The first winner will receive

• Flight/surface ticket (+ hotel for a few days) to Nagoya for attending the award seremony and presenting the method in the poster session of Symposium on Systems Science of Bio-Navigation 2020, November 25-26, 2020, Nagoya, Japan
  • Venue:Sakata and Hirata Hall (Science South Building), Nagoya University. MAP

Note that

• to recieve the prizes, winners must publish their methods and/or codes available online (for example, as a technical report on arXiv and github) upon request by the organizer.
• In case of team submission, the travel grant goes to a single person who is a representative.
• The prize will go to the next winner if the winner withdraws the attendance and the poster presentation for any reasons (e.g., personal or VISA issues). Winners will be received PDF certificates by email in case of withdrawal.
• We will provide an invitation letter and the winner have to obtain a VISA by himself/herself.

Terms and Conditions

This challenge is governed by the general ChaLearn contest rules.

Test-challenge phase top three winners:

1. vaibhav29498 0.711300
2. Alex92 0.628700
3. darn0163 0.570200

1st place solution:

Mouse Behavior Challenge at github

3rd place solution: