Hi, I submitted my submission yesterday. The status of the submission is "Submitted". But my submission is not reflected in the LeaderBoard. Can you please look into it
Posted by: tulasi @ Feb. 8, 2021, 11:47 a.m.Hi the evaluation server is back alive. The submission should’ve be evaluated properly. Please check the status again. Thank you.
Posted by: HsienKaiKuo @ Feb. 8, 2021, 1:43 p.m.Hi, My submission got failed. The error log says there should be 2258 images. But my submission contains 2258 png images. I cross-checked again. Don't know why the server is throwing an exception. Please look into it. I need to submit few more submissions pending from the last 3-4 days.
Posted by: tulasi @ Feb. 8, 2021, 3:22 p.m.Hi,
Please follow the folder structure listed in
https://competitions.codalab.org/competitions/28054#learn_the_details-evaluation
All files are at the root folder, NOT in an additional subfolder.
====
5. To get the submission run successfully by the evaluation server, the structure of your submission should like the following:
submission.zip
├─ 0.png
├─ 1.png
├─ ...
├─ 2257.png
├─ readme.txt
└─ model.tflite
Posted by: HsienKaiKuo @ Feb. 9, 2021, 1:17 a.m.Hi I submitted as per the instructions. But my submission is failed. Please find the error log. It seems there is a bug from your side.
INFO: Init
INFO: BuildGraph
INFO: AddOpsAndTensors
INFO: AddOpsAndTensors done
INFO: total_input_byte_size: 8355840
INFO: total_output_byte_size: 25067520
INFO: NeuronModel_identifyInputsAndOutputs
INFO: NeuronModel_relaxComputationFloat32toFloat16: 1
INFO: NeuronModel_finish
INFO: BuildGraph done
INFO: NeuronCompilation_setPreference: 1
INFO: NeuronCompilation_setPriority: 110
Segmentation fault (core dumped)
Traceback (most recent call last):
File "/tmp/codalab/tmpZDViJy/run/program/evaluation.py", line 173, in
file = open(LOG_NAME, 'r')
IOError: [Errno 2] No such file or directory: '/tmp/codalab/tmpZDViJy/run/output/output.csv'
Hi, I have the same error as you:
'IOError: [Errno 2] No such file or directory: '/tmp/codalab/tmpqdAPkV/run/output/output.csv''
Have you already solved this problem?
Posted by: rhwang @ Feb. 9, 2021, 6:59 a.m.No, It seems this is an issue in the server. Organizers should resolve it.
Posted by: tulasi @ Feb. 9, 2021, 7:07 a.m.Hi,
From the ERROR log, it shows that the submitted TFLite model had been putting into the evaluation device (The format of folder structure is correct).
But, one the device side, something unexpected happens. Please help to check if TF 1.15.0 is used for the conversion from .pb to .tflite. (listed in https://competitions.codalab.org/competitions/28054#learn_the_details-evaluation)
====
4. Export a TFLite model (with Tensorflow 1.15.0) of your solution and add it to the abovementioned ZIP archive; we will run this model to perform the evaluation on the target platform, MediaTek Dimensity 1000+.
Please create the TFLite which has input shape [1, 544, 960, 4] and output shape [1, 1088, 1920, 3]
The runtime of your TFLite model will be automatically evaluated on MediaTek Dimensity 1000+ platform and updated in the leaderboard of runtime per image [s].
Participants are welcome to use our tutorial codes to export TFLite models.
====
or directly apply our conversion script (w/ TF 1.15.0 installed) in https://github.com/MediaTek-NeuroPilot/mai21-learned-smartphone-isp
Posted by: HsienKaiKuo @ Feb. 9, 2021, 3:31 p.m.Hi, I am sure that I used TF 1.1.5 for conversion. I followed the mentioned instructions
Posted by: tulasi @ Feb. 9, 2021, 3:43 p.m.Hi tulasi，Sorry to disturb you again，have you already solve this problem？
Posted by: rhwang @ Feb. 10, 2021, 8:38 a.m.No I am waiting for Organizers reply
Posted by: tulasi @ Feb. 10, 2021, 8:50 a.m.Hi tulasi,
It seems that your model uses dynamic weight for convolution, which is not supported by the SW stack and device.
You could also refer to "Optimization & Preference Guide" in the Get Data page for more operation suggestion.
Thank you.