Since the ground-truth (GT) of the verification set has been released, it is possible to use it as a training set.
So is it better to have a part of the test set as an online validation assembly?
Hi zheng222,
As mentioned in the evaluation tab, we do not recommend using the validation set for training purposes.
https://competitions.codalab.org/competitions/28051#learn_the_details-evaluation
While the official benchmarks on the test data can be used for comparison, most of the analysis including the ablation study is done on validation data.
If validation data is used for training, each method is not eligible to compare with the others.
I believe using validation GT for training is not much helpful for the community.
To check what is done during the training, we are going to look into the details on the fact sheet the source code.
The test set accuracy will not be noticed on CodaLab benchmarks and the participants will not get intermediate feedbacks in terms of PSNR and SSIM. (unless noticed otherwise)
In my experiences, the validation set accuracy had a close correlation with the test set accuracy.
I expect the test set evaluation results will not be necessary to fine-tune the models.
Best,
Seungjun