Competition

Please note: Submissions have been deactivated on this server. For new submissions, please see the current LSC version on the new CodaLab server.

Welcome to the Leaf Segmentation Challenge

This is the CodaLab version of the Leaf Segmentation Challenge from the CVPPP2017, the third workshop on Computer Vision Problems in Plant Phenotyping held in conjunction with ICCV2017. Submissions have been deactivated on this server. For new submissions, please see the current LSC version on the new CodaLab server. We set up this CodaLab version to meet the communities high interest in these challenges, as e.g. visible in the high download numbers (see Fig.1). For further information please refer to our dataset page.

Fig. 1. Strong growth in downloads.

To advance the state of the art in leaf segmentation and to demonstrate the difficulty of segmenting all leaves in an image of plants, we organized the Leaf Segmentation and Counting Challenges (LSC and LCC). This is the 3^rd LSC after the successful LSC 2014 and 2015 and the 2^nd LCC. Examples of methods stemming from these challenges or using the data are http://link.springer.com/article/10.1007/s00138-015-0737-3 , https://arxiv.org/abs/1605.09410 , https://arxiv.org/abs/1511.08250 ). The major difference of the 2017 challenge is the expansion of the data that we focus on leaf segmentation accuracy and as such ground truth foreground segmentation masks are provided for training and testing.

For the challenges we release training sets (containing raw images and annotations) and testing sets (containing raw images, only).

How to participate

Please read first the challenge terms and conditions

1. If you do not have the training data please download them from here: How to download data or download the data from the 'Participate' tab, once you registered to the challenge.
2. The provided data has been collected in our laboratories (datasets A1 -- A3) or derived from a public dataset (A4, public data kindly shared by Dr Hannah Dee from Aberystwyth) of top-view images of rosette plants. All images were hand labelled. The archive contains evaluation functions (in MATLAB and Python) for comparing segmentation and counting outcomes between ground truth and algorithm results. The MATLAB function works with folders containing images files, as described in the original challenge call document LSC 2017. The Python version has beed designed to work with CodaLab and uses HDF5 files, where the original folder structure has been reproduced.
3. Training and testing results can be submitted independently. Results need to be submitted as a single zipped HDF5 file with the extension '.h5'. The folder structure in the HDF5 file needs to be exactly as in the provided HDF5 files. Please examine the provided 'sample_submission.h5' file. The participants descide by button click, which of their submitted results should be presented at the leaderboards.

About the data

We share images of tobacco plants and arabidopsis plants (download links see 'Participate' tab). Tobacco images were collected using a camera which contained in its field of view a single plant. Arabidopsis images were collected using a camera with a larger field of view encompassing many plants, which were cropped. The images released are either from mutants or wild types and have been taken in a span of several days. Plant images are encoded as tiff files.

All images were hand labelled to obtain ground truth masks for each leaf in the scene. These masks are image files encoded in PNG where each segmented leaf is identified with a unique integer value, starting from 1, where 0 is background. For the counting problem, annotations are provided in the form of a png image where each leaf center is denoted by a single pixel. Additionally a CSV file with image name and number of leaves is provided.

For further information on the ground truth annotation process, please refer to:

1. M. Minervini, A. Fischbach, H.Scharr, and S.A. Tsaftaris. Finely-grained annotated datasets for image-based plant phenotyping. Pattern Recognition Letters, pages 1-10, 2015, doi:10.1016/j.patrec.2015.10.013 [PDF] [BibTex]
2. Hanno Scharr, Massimo Minervini, Andreas Fischbach, Sotirios A. Tsaftaris. Annotated Image Datasets of Rosette Plants. Technical Report No. FZJ-2014-03837, Forschungszentrum Jülich, 2014
3. Bell, Jonathan, & Dee, Hannah M. (2016). Aberystwyth Leaf Evaluation Dataset [Data set]. Zenodo. http://doi.org/10.5281/zenodo.168158

or the challenge documents on LSC 2017 or LCC 2017.

File types and naming conventions:

Originally, plant images were encoded as PNG files and their size vary. Plants appear centered in the cropped image. Segmentation masks are image files encoded in PNG where each segmented leaf is identified with a unique (per image) integer value, starting from 1, where 0 is background. A color index palette is included within the file for visualization reasons.  The filenames have the form:

plantXXX_rgb.png is the raw color image in RGB
plantXXX_label.png  is the labeled image as indexed PNG file

plantXXX_fg.png is the foreground (plant segmentation) as binary PNG file

where XXX is a 3 or 4  digit integer number. Note that plants are not numbered continuously.

If you are interested to work with this format, please visit https://www.plant-phenotyping.org/CVPPP2017-challenge for further information. For CodaLab the images have been stored in a few HDF5 files. The folder structure in the HDF5 files resembles the original folder structure:

In the training images file as well as in the testing images file one finds

AY/plantXXX/rgb : the raw color image in RGB
AY/plantXXX/fg : the foreground (plant segmentation)

In the training truth file

AY/plantXXX/label : the labeled image

where Y is a number between 1 and 4 (training set) or 5 (testing set), and again XXX is a 3 or 4  digit integer number. Note that plants are not numbered continuously.

Training set

We provide 27 images of tobacco and 783 Arabidopsis images and label images to the registered users.

Testing set

Here, we will not share ground truth leaf segmentations. We share two different versions of the testing set:

1.    [SPLIT] images are split according to the origin i.e. following the A1,…, A4 nomenclature.

2.    [WILD] images are included in one folder (A5) only and may vary in size. This tries to emulate a leaf counting in the wild scenario where data from different sources are pooled in the testing phase.  If you want to perform well in this testing set we advise that you aim to pool data from A1 to A4 together.

Please note that IT IS STRICTLY FORBIDDEN to attempt to use the testing set in any other manner, e.g., to label testing data for improved training, to check algorithmic performance visually on the testing data, etc.  The organizers reserve the right to release a new testing set prior to the challenge for verifying the reported average performance of participants.