Holistic 3D Vision Challenge - Structured3D General Room Layout Estimation Track @ ECCV 2020

Organized by zhengjia - Current server time: Nov. 30, 2020, 5:52 p.m. UTC


July 24, 2020, midnight UTC


May 25, 2020, midnight UTC


Competition Ends
July 31, 2020, 11:59 p.m. UTC


This is the submission site for the Holistic 3D Vision Challenge of Structured3D General Room Layout Estimation Track. In this track, your algorithm needs to reconstruct the enclosing structure of the indoor scene, consisting of walls, floor, and ceiling. In this challenge, we do not make any assumption on the room structure, such as cuboid-shaped or Manhattan layouts.


  • [July 22, 2020] The data for the test-challenge phase is available now!
  • [July 16, 2020] The mean metric in the leaderboard is mistakenly measured by the ranking. We still use the average score of F (Junction), F (wireframe), and F (Plane).


The download link of the dataset can be found under "Participate / Get Data".


The competition consists of two phases: Test-Development and Test-Challenge. Test-Development is the default test set for testing under general circumstances and is used to maintain a public leaderboard. Test-Challenge is used for the workshop competition.

Important Dates

May 25, 2020 Challenge Launch Date
July 24, 2020 Test-Challenge Submission Starts
July 31, 2020 Test-Challenge Submission Deadline
August 7, 2020 Paper Submission Deadline
August 10, 2020 Paper Notification Date
August 23, 2020 Workshop Date (Co-located with ECCV 2020)

All deadlines are at midnight UTC.

Evaluation Criteria

Different from the current layout estimation approaches, we only evaluate the performance on the visible layout, i.e., ignore the occluded layout by the front wall. For this task, we use the following three metrics for room layout estimation evaluation, and the final metric is the mean of these three metrics.

  • Junction: We consider the predicted corner is correct only if the distance between this junction and its nearest ground-truth is within a threshold. The threshold is set to 5, 10, and 20 pixels.
  • Wireframe: Similarly, we consider the predicted line segment is correct only if the distance between this line segment and its nearest ground-truth is within a threshold. The threshold is set to 5, 10, and 20 pixels.
  • Plane: We consider the predicted plane is correct only if the intersection-over-union between this plane and its nearest ground-truth is larger than 0.5.

The evaluation code is available at Github.

Annotation Format

We provide the amodal layout annotation as an ordered list of 2D positions of the junctions. In our dataset, the cameras of the panoramas are aligned with the gravity direction. Thus, a pair of ceiling-wall and floor-wall junctions share the same x-axis coordinates.

Submission Format

The predicted layout should be represented by an Nx2 integer array with a 512x1024 image resolution and only contains the visible layout (shown in the figure above). Thus, the x coordinates of the layout is a monotonically non-decreasing sequence. To submit your results to the leaderboard, you have to construct a submission zip file containing prediction text files (".txt") with the same name as the corresponding testing images:

$ zip -rj submission.zip folder-of-your-results

General Rules

  • Each entry must be associated with a team and provide its affiliation.
  • The limit in the number of submissions is per team and not per user. We define a team as a group of people that would coauthor the final publication. Please follow the instruction to create the competition team.
  • You are allowed to use other annotations in the Structured3D dataset. If you use additional annotations (e.g., depth, normal) in the Structured3D dataset, please specify in the "method description" when submitting your results.
  • The best entry of each team will be public in the leaderboard at all times.
  • We reserve the right to remove one of the entry methods to the competition when the entry is incomplete or violates the rules.


The Structured3D dataset is released under the Structured3D Terms of Use. Please cite the Structured3D if you participate in the challenge or find this dataset helpful in your research:

  title     = {Structured3D: A Large Photo-realistic Dataset for Structured 3D Modeling},
  author    = {Jia Zheng and Junfei Zhang and Jing Li and Rui Tang and Shenghua Gao and Zihan Zhou},
  booktitle = {Proceedings of The European Conference on Computer Vision (ECCV)}),
  year      = {2020}


Start: May 25, 2020, midnight


Start: July 24, 2020, midnight

Competition Ends

July 31, 2020, 11:59 p.m.

You must be logged in to participate in competitions.

Sign In
# Username Score
1 dongho.choi 2.67
2 Fromandtozh 1.33
3 sunset1995 2.67