ETCI 2021 Competition on Flood Detection Forum

Go back to competition Back to thread list Post in this thread

> Please check the evaluation method.

Hi, administrator,

Thank you for organizing this interesting competition.
I am very happy to join this.

I am also wondering how the models are evaluated?

As mwlan-san mentioned, all zero array shows high IOU score (val: 0.7010, test: 0.785),
I get the same results as well.

On the other hand, all one array (means all flooded area) shows "0" score.
I guess, the evaluation procedure would calculate the IoU of "not flood region:0".

If my suggestion is correct, please modify the procedure for us.


Posted by: Skawakita @ May 22, 2021, 9:12 a.m.

Hi Shiro,

We are aware of this possible hack and we will make an announcement about it soon. Thanks for bringing it up!

Posted by: Shubhankar @ May 22, 2021, 12:03 p.m.
Post in this thread