I just found the case where the meta-info indicates the object exists, and they do exist on the images, but those objects are not annotated.
The case I found is "011ac0a06f". See the code below.
In [1]: from PIL import Image
In [2]: import numpy as np
In [3]: x = np.array(Image.open('/home/user/data/youtube-vos/train/Annotations/011ac0a06f/00160.png'))
In [4]: np.unique(x,return_counts=True)
Out[4]: (array([0, 1], dtype=uint8), array([780832, 140768]))
where meta-information for this video is:
"011ac0a06f": {
"objects": {
"1": {
"category": "ape",
"frames": [
"00000",
"00005",
"00010",
"00015",
"00020",
"00025",
"00030",
"00035",
"00040",
"00045",
"00050",
"00055",
"00060",
"00065",
"00070",
"00075",
"00080",
"00085",
"00090",
"00095",
"00100",
"00105",
"00110",
"00115",
"00120",
"00125",
"00130",
"00135",
"00140",
"00145",
"00150",
"00155",
"00160",
"00165",
"00170",
"00175"
]
},
"2": {
"category": "ape",
"frames": [
"00130",
"00135",
"00140",
"00145",
"00150",
"00155",
"00160",
"00165",
"00170",
"00175"
]
},
"3": {
"category": "ape",
"frames": [
"00130",
"00135",
"00140",
"00145",
"00150",
"00155",
"00160",
"00165",
"00170",
"00175"
]
},
"4": {
"category": "ape",
"frames": [
"00130",
"00135",
"00140",
"00145",
"00150",
"00155",
"00160",
"00165",
"00170",
"00175"
]
},
"5": {
"category": "ape",
"frames": [
"00130",
"00135",
"00140",
"00145",
"00150",
"00155",
"00160",
"00165",
"00170",
"00175"
]
}
}
I am not sure whether this case is the only case in the dataset. I will report when I found more of this. I hope that it does not happen on the validation/challenge set. Thanks!
Posted by: hi.nine @ July 30, 2018, 3:46 a.m.It seems that also meta information is not reliable. There exist many cases where meta information indicates that segmentation is available, but actually, no segmentation can be found. Is the "frames" property is set for the all the rest frames from the frame when it first appears? Even though it is true, I can still notice some obvious error like the case of "27659fa7d6" where "frisbee" shows up in the first frame, but meta-information indicates it first shows up in the 20th frame.
Posted by: hi.nine @ July 30, 2018, 8:10 a.m.Anyway, I forget to say thank you for the wonderful dataset! It would be better if some of the minor error can be clarified or corrected.
I attached the code I used for verification below. Thanks!
```
import json
import pathlib
from PIL import Image
import numpy as np
cats = set()
meta = json.load(open('meta.json'))
for vid,vid_info in meta['videos'].items():
for oid, object_info in vid_info['objects'].items():
cats.add(object_info['category'])
print( sorted(list(cats)) )
imgs = pathlib.Path('JPEGImages/')
segs = pathlib.Path('Annotations/')
try:
from tqdm import tqdm
print_fn = lambda *args: tqdm.write(' '.join([str(arg) for arg in args]))
except ImportError:
tqdm = lambda obj: obj
print_fn = print
for vid,vid_info in tqdm(meta['videos'].items()):
for seg_file in sorted(list((segs/vid).glob('*.png'))):
# Check whether Image file exsist
assert (imgs/vid/(seg_file.stem+'.jpg')).exists()
seg = np.array(Image.open(seg_file))
object_ids = list(np.unique(seg))
meta_object_ids = [0]+sorted([int(oid) for oid,o_info in vid_info['objects'].items() if seg_file.stem in o_info['frames']])
if object_ids != meta_object_ids:
print_fn(vid,seg_file.stem,object_ids,'!=',meta_object_ids)
```
Hi,
Thanks a lot for your feedback! Regarding your questions, for this dataset, the "frames" property is set for the all the frames from the frame when the object first appears. It is not for indicating whether there is a corresponding object mask in the current frame because the object can be occluded / out-of-view. On the other hand, we do not include very small objects in the mete data ( less than 100 pixels with image resized to 256p). So your observation that some object appear early and meta json shows the first frame is from a later frame is due to this reason. We also did the same procedure to validation and test set. Let us know if you have more questions. Thanks!
For the sequence "011ac0a06f", your are right, there is some mistake with the labeling. For the mean time you can ignore 011ac0a06f in your training. We have tried our best to check the data but we still have a few mistakes in this dataset. Any help on spotting the incorrect labelings are very appreciated. We will correct these labels in the near future, but will not change them for this challenge. Thanks!
Posted by: Chestnut @ July 30, 2018, 5:51 p.m.