ICCV DeeperAction Challenge - FineAction Track on Temporal Action Localization Forum

Go back to competition Back to thread list Post in this thread

> Confused about the testing leaderboard ~

##Competition organizers have chosen to keep the leaderboard private. Please check back later.##

Q:When will the testing leaderboard to be open? It`s important that everything in the competition be fair and open.

Posted by: flyDeeper @ Sept. 1, 2021, 6 a.m.

Until the end of the testing competition.
Only competition organizers can see the leaderboard during the testing competition.

Posted by: yiliu @ Sept. 1, 2021, 6:38 a.m.

""Questions and Suggestions on Keeping the Leaderboard Private""

Dear Sir or Madam
  I have some questions and suggestions about keeping the leaderboard private.
Question 1: Keeping the leaderboard private will leave the participants without supervision, how can we ensure the fairness of the competition? Also, what is the point of keeping the leaderboard private and why?
Question 2: How can I ensure that the results file submitted is in the correct format? The test results we submit cannot even be seen by ourselves. How can we ensure that the series of optimizations are performance-enhancing for the Challenege?Isn't this contrary to mining the map cap of the Fineaction dataset?
Suggestion: keep the leaderboard open, and you can make the team names anonymous. The second most important thing is to show the scores of the results you submitted.



Posted by: flyDeeper @ Sept. 3, 2021, 4:12 a.m.

Dear flyDeeper and all,

First thanks for participating this challenge. We have to clarify this some points:

1. We have divide the challenge phase into validation and testing phase. Validation phase is for your method development and model tuning. You can get feedback from our leaderboard to improve your method and parameter tuning. After validation phase, testing phase is mainly for testing your method on the whole testing dataset and report the final performance based on your best models discovered in validation phase. This is the difference between validation and testing phase.

2. Dividing the chalenge into validation phase and testing phase and hidding the testing result aims to prevent the participants from over-fitting the testing set. This is a common practice in many challenges such as ImageNet, WebVision, and ActivityNet.

3. Finally, please be noted the process is open and automatcially handled by the codalab platform. Keep the leaderboard hidden will not affect your performance.

Organizer of DeeperAction challenge

Posted by: 07wanglimin @ Sept. 3, 2021, 5:14 a.m.

Sincerely suggest the organizers to clarify the operation of different phase in next year's Challenge-Overview.
I'm a TAL direction fresh with no experience in ImageNet & Anet participation. It's heartbreaking and regrettable to spend so much effort to devote to FineAction research...

Posted by: flyDeeper @ Sept. 3, 2021, 7:09 a.m.
Post in this thread