News! Sept 4, 2019: following Codalab's crash, SegTHOR competition is back ☺
Our challenge was an official challenge of IEEE ISBI in April 2019 and remains open for new submissions.
Our challenge addresses the problem of organs at risk segmentation in Computed Tomography (CT) images. In lung and esophageal cancer, radiation therapy is a treatment of choice, and the irradiation planning begins with the delineation of the target tumor and healthy organs located near the target tumor, called Organs at Risk (OAR) on CT images. Routinely, the delineation is largely manual which is tedious and may be source of reproducibility errors. For some organs (e.g. esophagus), the segmentation is especially challenging: shape and position vary greatly between patients; the contours in CT images have low contrast, and can be absent. In this challenge, we focus of 4 OAR: heart, aorta, trachea, esophagus, which are shown on the figure below.
The goal of the SegTHOR challenge is to automatically segment 4 OAR: heart, aorta, trachea, esophagus. We provide a training set 40 CT scans with manual segmentation. The test set includes 20 CT scans.
September 2019: We are now writing a paper describing the data, the submitted methods and summarizing the results.
We ask the teams using our dataset to sign adata usage agreement form and to acknowledge the SegTHOR dataset by citing the following paper:
R Trullo, C Petitjean, B Dubray, S Ruan. Multiorgan segmentation using distance-aware adversarial networks, Journal of Medical Imaging 6 (1), 014001, 2019
Predicted contours are evaluated against the ground truth using:
DM and HD are complementary metrics and provide a good idea of the global accuracy of a segmentation method. They will be computed independently for each of the 4 organs at risk. Evaluation code will be open source and easily accessible since these metrics are standard and already implemented in numerous libraries. We will use the library SimpleITK in our evaluation code. To this regard, 8 metrics will be computed and each one will be assigned to a designated row. The average of each of the 8 rows will give the challenger’s ranking. Moreover, as we wish to pay particular attention to esophagus segmentation, the leaderboard will display the average row metrics calculated for the esophagus. While it will not be taken into account in the final score, computational complexity and running time will be a point of discussion, in the papers, but also on the day of the challenge at ISBI.
The results must be submitted through this CodaLab competition site in the section Participate. The participants can make up to 3 submissions per day.
A result.zip for this competition would look like this:
result.zip |- Patient_01.nii (Contains: segmented CT scan in nifty format)
Warning: make sur to zip just the contents of the directory result not the directory itself
Please be patient. Once you've submitted your results, it may take a while for the scoring program to compute your scores.
In order to participate to this competition, it is mandatory to sign up and fill in a data usage agreement. The form should be returned to us completed and signed for you to be allowed in this competition.
Download the form here.
Fill it, sign it and return it to caroline.petitjean AT univ-rouen.fr
Computed Tomography images (with or without IV contrast) have been retrieved from the medical records of 60 patients with non small cell lung cancer referred for curative-ntent radiotherapy at the Centre Henri Becquerel, Rouen, France (CHB, regional anti-cancer center) between February 2016 and June 2017. Patients with tumor extension distorting the mediastinum anatomy were not eligible. All data were fully anonymized. The institution (CHB) board reviewed and approved the protocol in February 2016.
The CT scans have 512 x 512 pixels size with in-plane resolution varying between 0.90 mm and 1.37 mm per pixel, depending on the patient. The number of slices varies from 150 to 284 with a z-resolution between 2mm and 3.7mm. The most frequent resolution is 0.98x0.98x2.5 mm3.
The whole SegTHOR dataset (60 patients and 11084 slices) has been randomly split into:
We have carefully checked the distribution to ensure that the training is representative of the testing data. No specific preprocessing will be made. Participants are fully allowed to use their own training data, in addition to the ones provided by the challenge. As described in the general organization of the challenge in section Overwiew we plan to release the training data and their associated manual contours in early January, and the testing data by end-February (results submission will be expected by end-March).
The reference standard for our problem is manual segmentation. On each CT scan, the OARs have been delineated by Pr Bernard Dubray, an experienced radiation oncologist, using a SomaVision platform, Varian Medical Systems, Inc, Palo Alto, USA. The body and lung contours were segmented with the automatic tools available on the platform. The esophagus was manually delineated from the 4th cervical vertebra to the esophago-gastric junction. The heart was delineated as recommended by the Radiation Therapy Oncology Group 2. The trachea was contoured from the lower limit of the larynx to 2cm below the carena excluding the lobar bronchi. The aorta was delineated from its origin above the heart down to below the diaphragm pillars.
The proceedings of the SegTHOR challenge at IEEE ISBI 2019 have been published as Vol 2349 of CEUR-WS and may be found here.
The following participants presented their work during the SegTHOR challenge at IEEE ISBI in Venice Italy:
|S. Kim et al||Yonsei University, Yonsei University College of Medicine||South Korea||34|
|V. Kondratenko et al||Skolkovo Institute of Science and Technology||Russia||26|
|D. Lachinov||Intel, Nizhny Novgorod||Russia||15|
|L. van Harten et al||Image Sciences Institute & Department of Radiotherapy, UMC Utrecht||The Netherlands||12|
|S. Vesal et al||Friedrich-Alexander-Universit\"at Erlangen-N\"urnberg||Germany||9|
|Q. Wang et al||Chinese University of HK, Sun Yat-sen University, University of Hong Kong||HK, China||5|
|M. Han et al||Shanghai United Imaging Intelligence Inc.||China||1|
|05-Jan, 2019||Release of the training set|
|28-Feb, 2019||Release of the test set|
|30-Mar, 2019||Last date for submission of results on the test set with a detailed paper|
|08-Apr, 2019||Workshop in Venise, Italy at IEEE International Symposium on Biomedical Imaging (ISBI) conference|
Start: Jan. 5, 2019, midnight
Description: Training phase (with optional result scoring)
Start: Feb. 28, 2019, midnight
Description: Test phase with result scoring
You must be logged in to participate in competitions.Sign In