CVPR’18 Paper “Learning 3D Shape Completion from Laser Scan Data with Weak Supervision”

In this CVPR’18 paper, based on my master thesis, we propose a weakly-supervised and learning-based approach to 3D shape completion of sparse and noisy point clouds. We show that, using a learned shape prior, shape completion can be learned without access to ground truth shapes — only by knowing the object category at hand. This article provides the paper and its supplementary material.

Update. Code and data now available on GitHub, see davidstutz/cvpr2018-shape-completion.

Update. The LaTeX source of the paper is available on GitHub: davidstutz/cvpr2018-shape-completion.


Figure 1 (click to enlarge): Overview of the proposed, weakly-supervised 3D shape completion approach; see the paper for details.

3D shape completion from partial point clouds is a fundamental problem in computer vision and computer graphics. Recent approaches can be characterized as either data-driven or learning-based. Data-driven approaches rely on a shape model whose parameters are optimized to fit the observations. Learning-based approaches, in contrast, avoid the expensive optimization step and instead directly predict the complete shape from the incomplete observations using deep neural networks. However, full supervision is required which is often not available in practice. In this work, we propose a weakly-supervised learning-based approach to 3D shape completion which neither requires slow optimization nor direct supervision. While we also learn a shape prior on synthetic data, we amortize, ie, learn, maximum likelihood fitting using deep neural networks resulting in efficient shape completion without sacrificing accuracy. Tackling 3D shape completion of cars on ShapeNet and KITTI, we demonstrate that the proposed amortized maximum likelihood approach is able to compete with a fully supervised baseline and a state-of-the-art data-driven approach while being significantly faster. On ModelNet, we additionally show that the approach is able to generalize to other object categories as well.

Paper (∼ 2.7MB)Supplementary (∼ 4.9MB)

  title = {Learning 3D Shape Completion from Laser Scan Data with Weak Supervision},
  author = {David Stutz and Andreas Geiger},
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  publisher = {IEEE Computer Society},
  year = {2018}
What is your opinion on this article? Let me know your thoughts on Twitter @davidstutz92 or LinkedIn in/davidstutz92.