Hello:
If my model's performance is better than the baseline model in the Val phase, but it is worse than the baseline model in the testing phase. Should my model consider as an invalid model and doesn't rank in the final phase? If the model's performance is very close to the baseline and it's hard to control the model ability. Thanks.
If the model is 0.2dB worse than baseline model on testing dataset, we may not rank the model.
Posted by: cszn @ July 7, 2020, 3:13 p.m.Dear organizers:
we are puzzled about some problems:
1. Can we train a model with validation data set?
2. Based on your above comment, we have taken some experiments that train our model with training data set and validation data set, even only trained on validation data set. We found that the PSNR can easily meet the the condition that supass 29 dB on validation data set and worse than baseline model on testing dataset in 0.2dB.
So can you give a more detailed explanation of the above issues? Thanks.
For a fair comparison, you should not use validation dataset during training.
Posted by: cszn @ July 8, 2020, 1:56 p.m.How did you know your performance in testing data？
Posted by: qiuzhangTiTi @ July 8, 2020, 4:25 p.m.We split 100 pictures from the training set as the testing set. Besides, we made many experiments and came a conclusion that it was impossible to supass 29dB on the validation set but 0.2dB worse than baseline model on testing set. Actually, 0.02dB decrease is more common.
Posted by: Airia_CG @ July 9, 2020, 4:01 a.m.Thanks.
Dear organizers:
What if my model performance is worse than the baseline on Val set and within 0.2dB worse than the baseline model on the testing dataset? Will my model be valid?
Thanks.
In this case, I would like to change 0.2dB to 0.05dB.
Posted by: cszn @ July 9, 2020, 6:23 p.m.Dear Organizers:
According to "In this case, I would like to change 0.2dB to 0.05dB", can we regard our model with 28.95dB on validation set as a valid solution?
Thanks.