Grass Clover Dataset Semantic Segmentation and Biomass Composition Challenges

Organized by sskovsen - Current server time: Nov. 17, 2019, 9:47 a.m. UTC

Current

First phase

Sept. 1, 2019, midnight UTC

End

Competition Ends

Never

Overview
Evaluation
Terms and Conditions

Grass Clover Image Dataset Challenges

Welcome to the Grass Clover Image Dataset challenges. This competitions consists of the challenges associated with the Grass Clover Image Dataset presented at CVPPP 2019 workshop at CVPR 2019. The dataset was collected from May to October of 2017 and 2018 in field conditions. Grass and clover species were varied across the mixtures. Weeds were naturally present in the organic trials. The dataset consists of:

1: Synthetic images of grass and clover mixtures with pixelwise species labels in a hierarchy.

2: Collected images of grass and clover mixtures with biomass composition labels (grass, clover, weeds, (if availble: red clover, white clover)).

3: Unlabeled images of grass and clover mixtures.

The dataset is available for download here.

The challenges associated with the dataset are the Semantic Segmentation challenge and the Biomass Composition Prediction challenge. In the Semantic Segmentation challenge, participants must perform pixel-wise classifiction on hand-labeled collected images. In the Biomass Composition Prediction challenge, participants must predict the biomass composition (in fractions) of a subset of the biomass-labeled images.

Semantic Segmentation challenge

The intersection over union (IoU) (or Jaccard index) is used as evaluation metric for the semantic segmentation challenge. For each submission, the mean IoU and per class IoU are used as evaluation metrics.

Let

$c_{o,p}$

be the number of instances of class o predicted as class p, then the IoU for class k is given by:

$IoU_k = \frac{c_{k,k}}{- c_{k,k} + \sum_{o=1}^K{c_{o,k}} + \sum_{p=1}^K{c_{k,p}}}$

The mean IoU is then given by:

$mean~IoU = \frac{1}{K}\sum_{k=1}^K{IoU_k}$

NOTE: Pixels annotated as "Unknown", typically heavily occluded leaves below the canopy, are ignored during evaluation. Clover leaves that could not be identified at species level with certainty, are annotated as clover, and does not impact the IoU of the per class IoU of white clover and red clover.

Biomass Composition Prediction challenge

The biomass composition prediction challenge is evaluated using the root mean square error (RMSE) and the mean absolute error (MAE) for each fraction: grass, clover, weeds, white clover and red clover. The RMSE and MAE are given by:

$RMSE = \sqrt{\frac{1}{N}\sum_{i=1}^N{\left(t_i - y_i \right)^2}}$

and

$MAE = \frac{1}{N}\sum_{i=1}^N{\left|t_i - y_i\right|}$

Copyright and license

The images and annotations are distributed under the Creative Commons BY-SA license.

All use of the data and derived work, including, but not limited to, trained algorithms and machine learning models requires full citation. THE IMAGES AND ANNOTATIONS ARE PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS DATA, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Contact

http://au.dk/ssk@eng

Citation

If you use this dataset in your research or elsewhere, please cite the following paper: The GrassClover Image Dataset for Semantic and Hierarchical Species Understanding in Agriculture

@InProceedings{Skovsen_2019_CVPR_Workshops,
	author = {Skovsen, Soren and Dyrmann, Mads and Mortensen, Anders K. and Laursen, Morten S. and Gislum, Rene and Eriksen, Jorgen and Farkhani, Sadaf and Karstoft, Henrik and Jorgensen, Rasmus N.},
	title = {The GrassClover Image Dataset for Semantic and Hierarchical Species Understanding in Agriculture},
	booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
	month = {June},
	year = {2019}
}

First phase

Start: Sept. 1, 2019, midnight

Description: The competition only consists of one phase, which never ends. In this phase, participants can submit methods for both the semantic segmentation challenge and the biomass composition prediction challenge. See the "Learn the Details" and "Participte" tabs for more information.

Competition Ends

Never

You must be logged in to participate in competitions.