The val video of fixed-QP is the same as fixed-bit-rate. But the video quality of fixed-QP val video is much higher than fixed-bit-rate. In this case, someone may cheat by using val data of fixed-QP on track of fixed-bitrate. So I think the val/test video of two track should be different.
Posted by: lixin41 @ Jan. 10, 2021, 3:14 a.m.Dear participant,
I have double checked the data, the val for fixed rate and fixed QP are not the same:
https://data.vision.ee.ethz.ch/reyang/validation_fixed-rate.zip
https://data.vision.ee.ethz.ch/reyang/validation_fixed-QP.zip
Could you please check, and feel free to contact me if there's still problem
Posted by: RenYang @ Jan. 10, 2021, 9:12 a.m.I think, that what lixin41 meant is that the content in val videos in both tracks is exactly the same (which is true). It means that participants from the track1 can potentially abuse track3 results by submitting outputs using val videos from track1. The concern is whether organisers will be aware of this issue during test phase.
Posted by: koshelev @ March 12, 2021, 11:19 a.m.Yes, we aware of that. So we will first start track 3 (March 14 to March 17) and then start tracks 1 and 2 (March 17 to March 20) after track 3 ends on March 17. (All time points are 11:59 p.m. UTC on each day)
Posted by: RenYang @ March 12, 2021, 11:27 a.m.I think someone has used this point to cheat：
Track3：chaowentao 03/12/21 29.8888 (1) 0.8514 (1) 1.00 (8) 1.00 (1) 0.00 (2), which exceeded the other results by 2dB.
Track1：TouchFish submitted a result of 32.5 in March 11, 2021, which exceeded the other results by 2+dB. I suspect that he picked some good results from the fixed-rate track and submitted to the fixed-QP track.
32.5 is not from us (TouchFish).
Posted by: TouchFish @ March 14, 2021, 1:38 p.m.