NTIRE 2020 Video Quality Mapping - Track 1: Supervised Forum

Go back to competition Back to thread list Post in this thread

> test phase, submittion is failed

Can you help me?

WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
Traceback (most recent call last):
File "/tmp/codalab/tmpkWeCOj/run/program/evaluation.py", line 95, in
raise Exception('Expected %d .png images'%len(ref_pngs))
Exception: Expected 0 .png images

Posted by: lj1995 @ March 18, 2020, 2:29 p.m.

Could you please explain how you get the error? I guess it might be caused by the large file submission. During the test, participants are not encouraged to use the online server.

Posted by: Zhiwu_Huang @ March 20, 2020, 10:27 a.m.

I face this problem too. I uploaded a .zip file with 96 test images and 1 readme file in it. I don't know where I get it wrong.

Posted by: jojo23333 @ March 26, 2020, 2:59 p.m.

Hi,

According to the test guideline we sent you before, participants are not encouraged to submit their test results directly to the Codalab system. Instead, you are expected to submit the results by email. The body of the email shall include the following:
a) the challenge full name
b) team name
c) team leader's name and email address
d) rest of the team members
e) team members with NTIRE2020 sponsors
f) team name and user names on NTIRE2020 CodaLab competitions
g) executable/source code attached or download links
h) factsheet attached (you should use the updated factsheet of the competition!)
i) download link(s) to the FULL results of ALL of the test frames (corresponding to the last entry in the online Codalab test server and provided codes/exe)

For more submission details, I copy the following instruction we sent you before:
####################################################################################
Dear NTIRE 2020 challenge participant,

Here are general guidelines for the final test phase.

By now the necessary details/data/factsheet template for the final test phase were provided on each competition codalab page. (check Participate->Get Data tab)

You are considered participant in the final test phase if you have a valid entry in the online test server (or submitted as indicated by the challenge organizers).
The online test server provides flat results for all entries (all metrics are set to 0) and the final results will be computed offline by the organizers and released to the test phase participants after the end of the challenge.
The validation data and/or the validation server are meant only for validation and not for training.

In order your solution to be ranked and included in the report of the challenge and the authors of the solution to be invited to co-author the challenge report, you need to assure the reproducibility of the results, provide full test results, codes/executables and instructions, and a detailed description of your solution in a factsheet.

Unless the challenge organizers provided another guidelines, the final submission should be emailed to all organizers of the respective challenge (see Learn the details -> organizers tab)

In this case, the title of the mail should follow the pattern: [COMPETITION_NAME] - [TEAM_NAME]
For example, for
COMPETITION_NAME = NTIRE 2020 NonHomogeneous Dehazing Challenge
TEAM_NAME = Superman
the email title is: [NTIRE 2020 NonHomogeneous Dehazing Challenge] - [Superman]

The body of the email shall include the following:

a) the challenge full name
b) team name
c) team leader's name and email address
d) rest of the team members
e) team members with NTIRE2020 sponsors
f) team name and user names on NTIRE2020 CodaLab competitions
g) executable/source code attached or download links
h) factsheet attached (you should use the updated factsheet of the competition!)
i) download link(s) to the FULL results of ALL of the test frames (corresponding to the last entry in the online Codalab test server and provided codes/exe)

The executable/source code should include trained models and/or necessary parameters so that we could run it and reproduce results. There should be a README or descriptions that explains how to execute the executable/code. Factsheet must be a zip with the corresponding .tex source files and compiled pdf file. Please provide a detailed explanation.

Note that you can still submit to the validation server in those competitions for which the validation ground truth images were not released.

Please check carefully the competition pages (including "Participate->Get Data") and be sure that your are following the protocol, the right templates and provide the results for the correct test images.

Please use carefully the limited number of submissions to the codalab test server. Only the last entry will be considered for the final ranking.

As participant in the challenge you are also invited to submit a paper related to the challenge (by topic and/or using challenge data/results) to NTIRE 2020 workshop. For this please follow the instructions from https://data.vision.ee.ethz.ch/cvl/ntire20/ , the deadline is April 04, 2020.

Note that writing & submitting a paper to NTIRE2020 is not necessary for getting ranked and invited to co-author the challenge report. However, assuring the reproducibility of the results and providing a detailed factsheet of the proposed solution is a must in order to be ranked.

Success,
Challenge organizers

Posted by: Zhiwu_Huang @ March 26, 2020, 3:08 p.m.
Post in this thread