Hey everyone,
I'm having a lot of fun with this competition and I hope you all are too.
I noted that there were a lot of errors in the ground truth labels for the dataset (as did some other competitors as can be seen in these threads: https://competitions.codalab.org/forums/16267/2566/, https://competitions.codalab.org/forums/16267/2586/)
So I wrote a script that "should" fix it, no guarantees.
Example problems with dataset:
2383d8aafd/00070.png
7ff84d66dd/00115.png
As you can see in these images there are small annotated regions that are incorrect. There is literally thousands of these mistakes throughout the dataset.
My vos method for this challenge calculates bounding boxes from masks, and with these tiny way off pixels present in the ground truth masks, the "ground truth" bounding boxes were often very incorrect.
In order to fix this I used cv2.connectedcomponents to find each connected component and if it's below a threshold size I removed the annotation there.
This fixes many errors in the annotations, however, occasionally it also results in parts of correct annotations being removed, but these parts are always very small, so I imagine it doesn't matter too much.
An example of my method removing something when it should not:
80db581acd/00010.png
The script I used is pasted below.
Feel free to use it. I imagine it should help.
import numpy as np
from PIL import Image
import glob
import time
import cv2
import os
pascal_colormap = [
0 , 0, 0,
0.5020, 0, 0,
0, 0.5020, 0,
0.5020, 0.5020, 0,
0, 0, 0.5020,
0.5020, 0, 0.5020,
0, 0.5020, 0.5020,
0.5020, 0.5020, 0.5020,
0.2510, 0, 0,
0.7529, 0, 0,]
def save_with_pascal_colormap(filename, arr):
colmap = (np.array(pascal_colormap) * 255).round().astype("uint8")
palimage = Image.new('P', (16, 16))
palimage.putpalette(colmap)
im = Image.fromarray(np.squeeze(arr.astype("uint8")))
im2 = im.quantize(palette=palimage)
im2.save(filename)
limit = 100 # pixels e.g. approx 10x10
input = 'path_to_data/train/Annotations/'
output = 'path_to_data/train/CleanedAnnotations/'
folders = sorted(glob.glob(input + '*/'))
for vid_id,folder in enumerate(folders):
if not os.path.exists(folder.replace(input,output)):
os.makedirs(folder.replace(input,output))
files = sorted(glob.glob(folder + '*.png'))
for frame_id,file in enumerate(files):
ann = np.array(Image.open(file))
ids = np.unique(ann)
ids = ids[ids != 0]
cleaned_ann = ann.copy()
for id in ids:
mask = (ann==id)*id
num_con, connected_mask = cv2.connectedComponents(mask)
part_ids = np.unique(connected_mask)
part_ids = part_ids[part_ids != 0]
for part_id in part_ids:
if np.count_nonzero(connected_mask == part_id) < limit:
cleaned_ann[connected_mask == part_id] = 0
print(vid_id,frame_id,id,part_id, np.count_nonzero(connected_mask == part_id), file.split('/')[-2:])
save_with_pascal_colormap(file.replace(input,output),cleaned_ann)
print("Video",vid_id,folder.split("/")[-2], "done")
Above the indenting wasn't working, one way I thought to get around that was to comment out the code and then paste it. Might work, I guess I can try below.
Regards,
Jonathon Luiten
# import numpy as np
# from PIL import Image
# import glob
# import time
# import cv2
# import os
#
# pascal_colormap = [
# 0 , 0, 0,
# 0.5020, 0, 0,
# 0, 0.5020, 0,
# 0.5020, 0.5020, 0,
# 0, 0, 0.5020,
# 0.5020, 0, 0.5020,
# 0, 0.5020, 0.5020,
# 0.5020, 0.5020, 0.5020,
# 0.2510, 0, 0,
# 0.7529, 0, 0,]
#
# def save_with_pascal_colormap(filename, arr):
# colmap = (np.array(pascal_colormap) * 255).round().astype("uint8")
# palimage = Image.new('P', (16, 16))
# palimage.putpalette(colmap)
# im = Image.fromarray(np.squeeze(arr.astype("uint8")))
# im2 = im.quantize(palette=palimage)
# im2.save(filename)
#
# limit = 100 # pixels e.g. approx 10x10
# input = '/home/luiten/vision/youtubevos/ytvos_data/train/Annotations/'
# output = '/home/luiten/vision/youtubevos/ytvos_data/train/CleanedAnnotations/'
# folders = sorted(glob.glob(input + '*/'))
# for vid_id,folder in enumerate(folders):
# t = time.time()
#
# if not os.path.exists(folder.replace(input,output)):
# os.makedirs(folder.replace(input,output))
#
# files = sorted(glob.glob(folder + '*.png'))
# for frame_id,file in enumerate(files):
#
# ann = np.array(Image.open(file))
# ids = np.unique(ann)
# ids = ids[ids != 0]
# cleaned_ann = ann.copy()
# for id in ids:
# mask = (ann==id)*id
#
# num_con, connected_mask = cv2.connectedComponents(mask)
# part_ids = np.unique(connected_mask)
# part_ids = part_ids[part_ids != 0]
# for part_id in part_ids:
# if np.count_nonzero(connected_mask == part_id) < limit:
# cleaned_ann[connected_mask == part_id] = 0
# print(vid_id,frame_id,id,part_id, np.count_nonzero(connected_mask == part_id), file.split('/')[-2:])
# save_with_pascal_colormap(file.replace(input,output),cleaned_ann)
# print("Video",vid_id,folder.split("/")[-2], "done")
Okay it seems like pasting code here doesn't work, so I have uploaded it somewhere for people to download, I have also included the annotations that I mentioned in the post as easy reference to what I am talking about, and I have included an MIT licence with the code, because I know a lot of you care about that sought of thing.
Here's the good stuff:
https://drive.google.com/drive/folders/1nzux8BikpACX52KzrwG5DsmCOYKIb85f?usp=sharing
Regards,
Jonathon Luiten
*sort of thing.
Posted by: Jono @ Aug. 16, 2018, 6:32 p.m.By the way, this problem is also very persistent on the annotated first frame annotations for both the validation set and the test set.
Posted by: Jono @ Aug. 18, 2018, 7:31 p.m.