LiTS - Liver Tumor Segmentation Challenge

Organized by PatrickChrist - Current server time: Oct. 15, 2018, 9:39 a.m. UTC

Previous

MICCAI 2017
June 29, 2017, midnight UTC

Current

Open leaderboard
Aug. 4, 2017, 11 p.m. UTC

End

Competition Ends
Never

Overview

The liver is a common site of primary (i.e. originating in the liver like hepatocellular carcinoma, HCC) or secondary (i.e. spreading to the liver like colorectal cancer) tumor development. Due to their heterogeneous and diffusive shape, automatic segmentation of tumor lesions is very challenging. Until now, only interactive methods achieved acceptable results segmenting liver lesions.

With our challenge we encourage researchers to develop automatic segmentation algorithms to segment liver lesions in contrast­-enhanced abdominal CT scans. The data and segmentations are provided by various clinical sites around the world. The training data set contains 130 CT scans and the test data set 70 CT scans. The challenge is organised in conjunction with ISBI 2017 and MICCAI 2017. For MICCAI 2017 we added tasks for liver segmentation and tumor burden estimation.

Get started

1. Register here to get access

2. Download the data after approval

3. Check our Google Groups and FAQ

5. Submit your results

6. Win the Challenge

Important Dates

  • 01.12.16 Release of training data batch 1
  • 15.01.17 Release of all training data
  • 04.03.17 Release of Test data
  • 18.03.17 Submission Deadline ISBI 2017
  • 16.04.17 Results and Workshop on ISBI 2017 
  • 30.06.17 Submission for MICCAI 2017
  • 21.07.17 Submission Deadline for MICCAI 2017
  • EXTENDED SUBMISSION DEADLINE FOR MICCAI 2017 Due to technical issues
  • 14.09.17 Results and Workshop on MICCAI 2017
  • 15.09.17 Open Leaderboard

Submission Guide

Please find here a guide to successfully master the submission to the LITS challenge.

Sponsors

 

Evaluation measures

The main goal of this challenge is to automatically segment liver lesions in CT volumes. We also evaluate liver segmentation and tumor burden estimation.

We evaluate detection and segmentation performance, separately. Segmentation metrics are evaluated only for detected lesions, except for a global Dice score (combines all data sets into one) and an average Dice per volume score. A lesion is considered detected by a contiguous predicted object which has a greater than 50% intersection over union with the reference lesion. Per-lesion segmentations metrics for detected lesions are the mean values for Dice score, as well as Jaccard and volume overlap error (VOE), relative volume difference (RVD), average symmetric surface distance (ASSD), and maximum symmetric surface distance (MSSD). Liver segmentation is evaluated with the same segmentation metrics. If any liver prediction is not provided, all average liver segmentation metrics will default to the worst possible score (0 or infinity, depending on the metric). The tumor burden estimation is evaluated as a a root mean square (RMS) and a max difference/error with respect to to the true tumour burden. Please have a look at our metrics notebook once it becomes available.

Publication

The top 10 participating teams and individuals will be invited to contribute to a joint journal paper describing and summarising the methods used and results found in this challenge. The paper will be submitted to a high-impact journal in the field no later than 6 months after the challenge. In order to be included in the joint journal paper the participants must send to the organisers or submit to arXiv a 4 page paper in appropriate LNCS latex describing their methods. The organisers will review the paper for sufficient detail to be able to understand and reproduce the method and hold the right to exclude participants from the joint journal paper in case their method description is not adequate.

Attribution

Appropriate citation is to be made in scientific publications (journal publications, conference papers, technical reports, presentations at conferences and meetings) that use the data shared in this challenge. Currently, this citation must refer to this website, and later to the publication that will describe the results of this challenge. Teams must notify the organisers of the challenge about any publication that is (partly) based on the results or data published on this site in order for us to maintain a list of publications associated with the challenge.

Data license

Creative Commons Licence This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. Feel free to contact us if you have any questions or need clarification regarding the rules of the challenge or the licensing of the data.

Organizer and Data Contributors

Technical University of Munich

Patrick Christ, Florian Ettlinger, Felix Gruen, Sebastian Schlecht, Jana Lipkova, Georgios Kassis, Sebastian Ziegelmayer, Fabian Lohöfer, Rickmer Braren & Bjoern Menze

Ludwig Maxmilian University of Munich

Julian Holch, Felix Hofmann, Wieland Sommer & Volker Heinemann

Radboudumc

Colin Jacobs, Gabriel Efrain HumpireMamani & Bram van Ginneken

Polytechnique Montréal & CHUM Research Center

Gabriel Chartrand, Eugene Vorontsov, An Tang, Michal Drozdzal & Samuel Kadoury

Tel Aviv University & Sheba Medical Center

Avi Ben-Cohen, Eyal Klang, Marianne M. Amitai, Eli Konen & Hayit Greenspan.

IRCAD

Johan Moreau, Alexandre Hostettler & Luc Soler

The Hebrew University of Jerusalem & Hadassah University Medical Center

Refael Vivanti, Adi Szeskin, Naama Lev-Cohain, Jacob Sosna & Leo Joskowicz

Special thanks to the CodaLab Team for helping us

Eric Carmichael & Flavio Alexander

MICCAI 2017

The LITS Challenge Workshop will take place on 14.09.2017. We are happy to announce our Keynote Speaker Olaf Ronneberger (Deepmind).

Time

Title

Presenter

13.30-14.00

Intro & Motivation

Patrick Christ (TUM)

14.00-14.30

Keynote

Olaf Ronneberger (Deepmind)

14.30-14.50

Keynote

NVIDIA

14.50-15.10

Participants Talk

Eugene Vorontsov (MILA)

15.10-15.30

Participants Talk

Hayit Greenspan (Tel Aviv University)

15.30-16.00

Coffee Break

 

16.00-16.20

Participants Talk

Grzegorz Chlebus (Fraunhofer)

16.20-16.40

Participants Talk

Hao Chen (CUHK)

16.40-17.00

Participants Talk

Jiang Xuan (Lenovo)

17.00-end

Award Ceremony

LITS Organizers

ISBI 2017

The LITS Challenge Workshop at ISBI 2017 took place on 18.04.2017 in Melbourne, Australia. Please find attached to official leaderboard from ISBI 2017 LITS Challenge.

Ranking

Submission Name

Name

Institution

DICE

VOE

RVD

ASD

MSD

1

Elehanx

X. Han

Elekta Inc.

0,67

0,45

0,04

6,66

57,93

2

Medical

E. Vorontsov et al.

MILA

0,65

0,47

-0,21

7,12

51,96

2

Gchlebus

G. Chlebus et al.

Fraunhofer

0,65

0,46

17,41

17,75

57,64

3

Lei

L. Bi et al.

Uni Sydney

0,64

0,46

1,90

21,19

72,80

4

IBBM

F. Ettlinger et al.

TU Munich

0,58

0,53

2,09

13,76

63,59

4

Chunliang

C. Wang et al.

KTH Sweden

0,58

0,54

3,93

26,02

85,38

5

Woshialex-

N.A

N.A.

0,52

0,60

270,44

30,66

100,64

6

Janal

J. Lipkova et al.

TU Munich

0,48

0,63

103,31

32,32

105,82

7

Njust768

J. Ma et al.

NJUST

0,47

0,65

-0,35

11,49

64,31

8

Digvijay

N.A.

N.A.

0,44

0,67

246,55

43,74

137,87

9

Konop

T. Konopczynski et al.

Uni Heidelberg

0,42

0,69

103,74

32,54

116,60

10

Miriambellver

M. Beliver

ETH Zurich

0,41

0,69

3,60

36,29

130,46

11

Kmaninis

K. Maninis

ETH Zurich

0,38

0,71

1,23

19,74

86,83

12

Jinqi

J. Qi et al.

UESTC

0,19

0,87

1985,20

40,61

95,63

FAQ

If you have questions please have a look at our Google Group. If you encounter any issues during the submission process please have a look at the CodaLab Documentation.

MICCAI 2017

Start: June 29, 2017, midnight

Open leaderboard

Start: Aug. 4, 2017, 11 p.m.

Competition Ends

Never

You must be logged in to participate in competitions.

Sign In
# Username Score
1 phil666 0.7260
2 xjqi 0.7220
3 mastermind 0.6870