I would like to ask whether the uploaded test results should be 124 categories or 125 categories (including "other") for each pixel. My model now predicts 125 categories (including "other"). If it is 124 classification results, what should I do with pixels that have been predicted to be "other"? Also, in the test code, how do you deal with pixels that should be "other" categories? for example, there is a pixel that is originally "other", but because the model output is 124classes, it must be assigned to another category. In this way, for the assigned categories, the union set will certainly be increased, then the mIOU will certainly be reduced, which will affect the final score, if this problem cannot be solved. I think this is an unreasonable test scheme. Please explain it.
WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
Traceback (most recent call last):
File "/tmp/codalab/tmp2d_6id/run/program/score.py", line 139, in
eval_.add_batch(tar_,pred_)
File "/tmp/codalab/tmp2d_6id/run/program/score.py", line 59, in add_batch
self.confusion_matrix += self._generate_matrix(gt_image, pre_image)
File "/tmp/codalab/tmp2d_6id/run/program/score.py", line 53, in _generate_matrix
confusion_matrix = count.reshape(self.num_class, self.num_class)
ValueError: cannot reshape array of size 15384 into shape (124,124)
1. The test results should contain 124 categories.
The number range of your submission must be 0-123. The category-number dictionary "label_num_dic_final.json" in VSPW dataset shows that 0 denotes "others" , 1 denotes "wall", 2 denotes "ceiling" ... However, "others" is not evaluated. Thus, the corresponding number of category for submission should be origin number minus 1, i.e., 0 denotes "wall", 1 denotes "ceiling"...
2. The "others" pixels are not counted for the final results, which means whatever category you predict for the "other" pixels, it doesn't affect the final result.
Posted by: miaomm @ July 13, 2021, 10:44 a.m.Do you mean that I can classify pixels that are actually "other" within the range of [0-123] without affecting the final score?
Posted by: Marco_JR @ July 13, 2021, 10:57 a.m.Yes.
Posted by: miaomm @ July 13, 2021, 10:57 a.m.