UIoU Dark Zurich at Vision for All Seasons Workshop, CVPR 2020 Forum

Go back to competition Back to thread list Post in this thread

> Execution time limit exceeded!

The evaluation stopped before it was finished showing the following error message: Execution time limit exceeded!
Is there any way around this?

Many thanks in advance!

Posted by: attilalengyel @ Nov. 11, 2020, 12:59 p.m.

Hi Attila,

You can use the validation set, for which the ground truth has been made available, to debug locally the evaluation of your predictions. This will most likely help you to fix the issue you have on the test set too. The time limit for the execution of the evaluation script on each submission on the CodaLab server is 30 minutes. This should be enough, as the execution usually takes only 5 minutes in our machines.

In order to run evaluation on the validation set locally, you need to download all three packages that are made available on the Files page https://competitions.codalab.org/competitions/23553#participate-get_starting_kit :
1) the starting kit, containing the evaluation code (you may have to adjust slightly for your configuration),
2) the public data from the development phase, containing the RGB images of the validation set, and
3) the public data from the testing phase, containing the ground truth annotations of the validation set.

Best regards,
Christos Sakaridis

Posted by: sakaridis @ Nov. 11, 2020, 3:27 p.m.

Hi Christos,
Thanks for the quick reply.
As per your suggestion I ran the evaluation code on the validation set and the script indeed finished in ~5 minutes. I then again uploaded the test set predictions to Codalab but unfortunately the evaluation has been stuck in "Submitted" for more than 3 hours. Not sure if you can check what exactly is going on, I guess Codalab is just handling a lot of CVPR traffic at the moment. ;)

Posted by: attilalengyel @ Nov. 12, 2020, 2:55 p.m.

Hi Attila,

I checked your new submission and the evaluation runs properly, but the time limit is reached before the evaluation is completed (just a few images are left). So, I increased the general time limit for the execution of the evaluation to 45 minutes. I am now re-running your submission and there should be enough time for the evaluation. Let me know if there is still a problem.


Posted by: sakaridis @ Nov. 13, 2020, 8:29 a.m.

Hi Christos,
Thanks again for your efforts; unfortunately the evaluation timed out again near the end. Would it be possible to have all my submissions so far (all failed) deleted so that I can give it a couple of more tries?

Posted by: attilalengyel @ Nov. 13, 2020, 5:53 p.m.


Posted by: sakaridis @ Nov. 16, 2020, 8:41 a.m.
Post in this thread