Evaluation

The problem is to predict semantic labels and stereo disparities. Each input is a pair of epipolar rectified satellite images. You are intended to submit 2D semantic segmentation and stereo disparities in TIFF with the same height and width as the input image.

The contest will consist of two phases:

• Phase 1: Development phase. We provide you with training data (which includes ground truth) and validationn data (without ground truth). You can submit prediction results for the validation set to Codalab to get feedback on the performance. The performance of your LAST submission will be displayed on the leaderboard.
• Phase 2: Final phase. The test data set (without the corresponding ground truth) will be released on March 7, 2019. You can submit your predictions of semantic labels and stereo disparities from March 7 to March 22, 2019. In parallel, you need to submit a short paper of 2 pages describing the used approach by March 26, 2019. Please send a paper to iadf_chairs@grss-ieee.org using the IGARSS paper template. After evaluation of the results, we will announce two winners on March 29, 2019.

Performance is assessed using the pixel-wise mean Intersection over Union (mIoU) combined with the percentage of erroneous pixels with a threshold of 3 pixels for disparity values. We call this metric mIoU-3. The evaluation code is available at https://github.com/pubgeo/dfc2019.

The winners will have approximately one month (submission deadlline: April 26, 2019) to write their manuscript that will be included in the IGARSS 2019 proceedings. Manuscripts are 4-page IEEE-style formatted. Each manuscript describes the addressed problem, the proposed method, and the experimental results.

Terms and Conditions

Participants of this challenge acknowledge that they have read and agree to the following Contest Terms and Conditions:

• The owners of the data and of the copyright on the data are DigitalGlobe, IARPA and Johns Hopkins University.
• Any dissemination or distribution of the data packages by any registered user is strictly forbidden.
• The data can be used in scientific publications subject to approval by the IEEE GRSS Image Analysis and Data Fusion Technical Committee and by the data owners on a case-by- case basis. To submit a scientific publication for approval, the publication shall be sent as an attachment to an e-mail addressed to iadf_chairs@grss-ieee.org.
• In any scientific publication using the data, the data shall be identified as “grss_dfc_2019” and shall be referenced as follows: “[REF. NO.] 2019 IEEE GRSS Data Fusion Contest. Online: http://www.grss-ieee.org/community/technical-committees/data-fusion”.
• Any scientific publication using the data shall include a section "Acknowledgement". This section shall include the following sentence: "The authors would like to thank the Johns Hopkins University Applied Physics Laboratory and IARPA for providing the data used in this study, and the IEEE GRSS Image Analysis and Data Fusion Technical Committee for organizing the Data Fusion Contest."
• Any scientific publication using the data shall refer to the following papers:

  @InProceedings{US3D,
     author = {Bosch, Marc and Foster, Kevin and Christie, Gordon and Wang, Sean and Hager, Gregory D and Brown, Myron},
     title = {Semantic Stereo for Incidental Satellite Images},
     booktitle = {The IEEE Winter Conference on Applications of Computer Vision},
     month = {January},
     year = {2019}
  }
  @Article{IADF_GRSM_201903,
     author = {Saux, Bertrand Le and Yokoya, Naoto and Hänsch, Ronny and Brown, Myron and Hager, Greg},
     title = {2019 IEEE GRSS Data Fusion Contest: Large-Scale Semantic 3D Reconstruction [Technical Committees]},
     booktitle = {IEEE Geoscience and Remote Sensing Magazine},
     month = {March},
     year = {2019}
  }