Rules

You may submit 5 submissions every day and 50 in total in the test-dev phase.

Metric

Submitted results are evaluated by the accuracy, widely used for binary classification problems.

• acc = (number of correctly classified test trajectories) / (total number of test trajectories)

It is 0 if all predictions are wrong, while 1 if perfect.

Phases

1. test-dev phase: scores are shown with randomness (1/50 of your submission labels are randomly modified for avoiding cheat and overfit) on the leaderboard, which is NOT the final score.
2. test-challenge phase: scores are shown with no randomness. This score is the final one.

Protocol

• During the test-dev phase, You can submit a csv file up to 50 times, with 5 maximum submissions per day. (Caution! if you submit 5 times a day for 10 days, then you run out the allowed number of 50 submissions.)
• The submitted csv file is the predictions for "all 382 samples" in the test set.
• If the file is successfully submitted, the result score is shown in your private view (with randomness during the test-dev phase). It may take few minutes. Be patient ! (If something wrong, please check the log by clicking "View scoring output log" link.)
• Once your submissions has scores, choose one of the results to show on the public leaderboard, by clicking "Submit to Leaderboard" button. The selected one is used for the evaluation in the test-challenge phase. If you want to show another submission on the leaderboard (and hence for the evaluation in the test-challenge phase), simply click "Submit to Leaderboard" button of the submission you want to show.
• In the test-challenge phase, you have to nothing. The score of the submission selected in the test-dev phase is shown on the leaderboard. The score is computed without randomness.

Submission Format of the result

There are 382 test trajectory, and classification results should be a 382-line text file, which contains 0 (not including screen-play) or 1 (including screen-play) for each of test trajectory.

Each line has the label of the corresponding test trajectory; that is, line 0 is the label of the test trajectory 0000.csv. There is no header line.

Predicted label is binary (character):

• not including screen-play: 0

• including screen-play: 1

Here is an example:

=================
1
1
1
0
1
0
0
...
=================

file name

The file name should be "test_prediction.csv", which should be zipped as a single zip file.

The zip file can have any name, but should contain a single file named "test_prediction.csv" without any folders.

You can create and check your submission zip file by using zip and unzip as follows:

$ zip yoursubmission.zip test_prediction.csv
updating: test_prediction.csv (deflated 79%)
$ unzip -l yoursubmission.zip
Archive: yoursubmission.zip
 Length Date Time Name
--------- ---------- ----- ----
 357417 03-22-2019 10:17 test_prediction.csv
--------- -------
 357417 1 file

Task

Classifying multi-agent trajectories in basketball into screen-play or non-screen play

Trajectory file format

A single CSV file (0000.csv, 0001.csv, ...) contains a trajectory. The first line is a header, and following lines represent the information of the location of the related players and the ball.

• frame: frame number (starting from 0)
• scr_x: x coordinates of 2D location of the screener
• scr_y: y coordinates of 2D location of the screener
• usr_x: x coordinates of 2D location of the user of screen-play
• usr_y: y coordinates of 2D location of the user of screen-play
• uDF_x: x coordinates of 2D location of the defender of the user
• uDF_y: y coordinates of 2D location of the defender of the user
• bal_x: x coordinates of 2D location of the ball
• bal_y: y coordinates of 2D location of the ball

Float values are of the format %.02f, and fields are separated by a single comma.

Here is an example:

frame scr_x scr_y usr_x usr_y uDF_x uDF_y bal_x bal_y
0 2.89 4.74 5.49 1.5 2.78 5.22 6.98 12.7
1 2.88 4.7 5.52 1.51 2.8 5.2 7.08 12.52
2 2.87 4.67 5.54 1.53 2.82 5.19 7.13 12.35
3 2.86 4.65 5.56 1.54 2.84 5.17 7.08 12.37
...

• Trajectories are obtained by SportVU system in 25 fps. See videos at the top page. In the video, bottom-left corner is the origin (0, 0), and the positive direction of the x axis is rightward, and the positive direction of the y axis is the upward. Coordinate ranges are from 0 to 14 [m] for x and 0 to 15 [m] for y.

Labels: including screen-play or not

• Training trajectories of files from 0.csv to 399.csv include screen-play.

• Training trajectories of files from 400.csv to 1527.csv include non screen-play.

The dataset

The training and test trajectories are different in length (from 23 to 242 frames).
Both the training and test trajectories are pre-processed in the same way.

• Training set
• 400 screen-play trajectories
• 1128 non-screen play trajectories
• Test set
• 382 trajectories

Prizes

UPDATE 2020/Sep/1: Due to the pandemic of COVID-19, the following symposium has been cancelled. We are very sorry but the travel grant will not be given to winners, and we will PDF certificates to winners as a prize.

UPDATE: Due to the pandemic of COVID-19, the following symposium MAY be cancelled in future. In such case, unfortunately the travel grant will not be given to winners and we will PDF certificates to winners as a prize.

The winner will receive the following travel grant in addition to the winner certificate.

Travel grant

The first winner will receive

• Flight/surface ticket (+ hotel for a few days) to Nagoya for attending the award seremony and presenting the method in the poster session of Symposium on Systems Science of Bio-Navigation 2020, November 25-26, 2020, Nagoya, Japan
  • Venue:Sakata and Hirata Hall (Science South Building), Nagoya University. MAP

Note that

• to recieve the prizes, winners must publish their methods and/or codes available online (for example, as a technical report on arXiv and github) upon request by the organizer.
• In case of team submission, the travel grant goes to a single person who is a representative.
• The prize will go to the next winner if the winner withdraws the attendance and the poster presentation for any reasons (e.g., personal or VISA issues). Winners will be received PDF certificates by email in case of withdrawal.
• We will provide an invitation letter and the winner have to obtain a VISA by himself/herself.

Terms and Conditions

This challenge is governed by the general ChaLearn contest rules.

Test-challenge phase top three winners:

1. shotaro.ishihara 0.871700 (Basketball Behavior Challenge BBC2020, 1st Place Solution)
2. tishikawa 0.853400 (code, blog in JP)
3. Alex92 0.806300