Evaluation

The problem is to predict a semantic label for each 3D point. Each input is a lidar point cloud dataset in ASCII. You are intended to submit semantic labels in ASCII with the same number of rows as the input ASCII file.

The contest will consist of two phases:

• Phase 1: Development phase. We provide you with labeled training data and unlabeled validationn data. You can submit prediction results for the validation set to Codalab to get feedback on the performance. The performance of your LAST submission will be displayed on the leaderboard.
• Phase 2: Final phase. The test data set (without the corresponding ground truth) will be released on March 7, 2019. You can submit your predictions of semantic labels from March 7 to March 22, 2019. In parallel, you need to submit a short paper of 2 pages describing the used approach by March 26, 2019. Please send a paper to iadf_chairs@grss-ieee.org using the IGARSS paper template. After evaluation of the results, we will announce two winners on March 29, 2019.

Performance is assessed using the pixel-wise mean Intersection over Union (mIoU). The evaluation code is available at https://github.com/pubgeo/dfc2019.

The winners will have approximately one month (submission deadlline: April 26, 2019) to write their manuscript that will be included in the IGARSS 2019 proceedings. Manuscripts are 4-page IEEE-style formatted. Each manuscript describes the addressed problem, the proposed method, and the experimental results.

Terms and Conditions

Participants of this challenge acknowledge that they have read and agree to the following Contest Terms and Conditions:

• The owners of the data and of the copyright on the data are DigitalGlobe, IARPA and Johns Hopkins University.
• Any dissemination or distribution of the data packages by any registered user is strictly forbidden.
• The data can be used in scientific publications subject to approval by the IEEE GRSS Image Analysis and Data Fusion Technical Committee and by the data owners on a case-by- case basis. To submit a scientific publication for approval, the publication shall be sent as an attachment to an e-mail addressed to iadf_chairs@grss-ieee.org.
• In any scientific publication using the data, the data shall be identified as “grss_dfc_2019” and shall be referenced as follows: “[REF. NO.] 2019 IEEE GRSS Data Fusion Contest. Online: http://www.grss-ieee.org/community/technical-committees/data-fusion”.
• Any scientific publication using the data shall include a section "Acknowledgement". This section shall include the following sentence: "The authors would like to thank the Johns Hopkins University Applied Physics Laboratory and IARPA for providing the data used in this study, and the IEEE GRSS Image Analysis and Data Fusion Technical Committee for organizing the Data Fusion Contest."
• Any scientific publication using the data shall refer to the following papers:

  @InProceedings{US3D,
     author = {Bosch, Marc and Foster, Kevin and Christie, Gordon and Wang, Sean and Hager, Gregory D and Brown, Myron},
     title = {Semantic Stereo for Incidental Satellite Images},
     booktitle = {The IEEE Winter Conference on Applications of Computer Vision},
     month = {January},
     year = {2019}
  }
  @Article{IADF_GRSM_201903,
     author = {Saux, Bertrand Le and Yokoya, Naoto and Hänsch, Ronny and Brown, Myron and Hager, Greg},
     title = {2019 IEEE GRSS Data Fusion Contest: Large-Scale Semantic 3D Reconstruction [Technical Committees]},
     booktitle = {IEEE Geoscience and Remote Sensing Magazine},
     month = {March},
     year = {2019}
  }