IAM

OPENSOURCEFAN STUDYING
STUDYINGCOMPUTERSCIENCEANDMATH COMPUTERSCIENCE

Check out the latest superpixel benchmark — Superpixel Benchmark (2016) — and let me know your opinion! @david_stutz

ARTICLE

Bachelor Thesis “Superpixel Segmentation Using Depth Information”

My bachelor thesis, written at the Computer Vision Group at RWTH Aachen University, discusses superpixel segmentation utilizing depth information. Based on our own implementation of SEEDS [1], we examine the influence of depth information on the performance and compare several variants to other state-of-the-art approaches to superpixel segmentation.

See also my Bachelor Thesis Proposal and the slides of my Introductory Talk.

Update. LaTeX source of the bachelor thesis and the slides, as well as the code for evaluation is now available on GitHub.

Update. Superpixels Revisited (GitHub) bundles several of the evaluated superpixel algorithms in a single library.

Update. An updated version of the thesis is available below. Thanks to Rudolf Mester, a mistake in figure 7.11 on page 57 was corrected.

Update. A throrough overview over all superpixel algorithms evaluated in [2], as well as the evaluation reslts, can now be found here: Superpixels / SEEDS.

Our implementation of the superpixel algorithm SEEDS [1], referred to as SEEDS Revised, is now available on GitHub:

SEEDS Revised on GitHub

Introduction

The term superpixel was coined by Ren and Malik [2] and describes a group of pixels similar in color and other low-level features. Compared to pixels, which are the result of discretization, superpixels are used as more natural entities [2] - grouping pixels which perceptually belong together. Since their introduction, superpixels have actively been used for a wide range of applications, as for example tracking [3], classical segmentation [2] or semantic segmentation [4]. Independent of the application, most authors agree on the following requirements for superpixels [5, 6]:

  • Superpixels should adhere to object boundaries;
  • Superpixels should be generated as efficiently as possible and not lower the performance of subsequent steps;

Additionally, some applications may require compactness or connectivity [7].

Several superpixel algorithms have been proposed to meet the above requirements. However, some applications are based on depth information as provided by RGB-D cameras and therefore incorporating depth into superpixel segmentation may be beneficial as well. Based on the superpixel algorithm proposed by Van den Bergh et al. in [1], called SEEDS, this thesis aims to examine the benefit of using depth information. Our goal is to preserve the excellent runtime while further increasing performance.

An additional focus of this thesis is a comparison of all superpixel algorithms providing source code. In our opinion, this is necessary because of the rising number of superpixel algorithms - only few publications offer detailed comparison in a consistent framework [8, 9, 10]. In addition, evaluation measures as for example the Undersegmentation Error [8] are not defined consistently and details concerning the actual implementations are often not reported. Therefore, we find it difficult to define the state-of-the-art and choose algorithms suited for specific applications. We base our experiments on both the Berkeley Segmentation Dataset [11] and the NYU Depth Dataset V2 [12]. Here, the indoor scenes provided by the NYU Depth Dataset can be seen as fresh contrast to the natural images of the Berkeley Segmentation Dataset. In addition, the NYU Depth Dataset allows to evaluate superpixel algorithms requiring depth information.

Contributions

These are the main contributions of this thesis:

  • An implementation of SEEDS including several variants utilizing depth information.
  • A thorough evaluation of several superpixel algorithms on the Berkeley Segmentation Dataset and the NYU Depth Dataset using an extended version of the Berkeley Segmentation Benchmark providing additional measures to evaluate superpixel algorithms.

Download

The thesis can be downloaded below. Due to the extensive amount of evaluation data, appendix B is available separately. Additionally, our benchmark and our source code will be made publicly available during the next months.

Loading the below documents may take some time ...

Thesis (~57MB)Appendix B (~98MB)

How to cite this thesis?

As BibTex does not support an @bachelorthesis entry yet, use the following suggestion:

@misc{Stutz:2014,
    author = {David Stutz},
    title = {Superpixel Segmentation using Depth Information},
    month = {September},
    year = {2014},
    institution = {RWTH Aachen University},
    address = {Aachen, Germany},
    howpublished = {http://davidstutz.de/},
}

References

  • [1] M. van den Bergh, X. Boix, G. Roig, B. de Capitani, L. van Gool. SEEDS: Superpixels extracted via energy-driven sampling. Proceedings of the European Conference on Computer Vision, pages 13–26, 2012.
  • [2] X. Ren, J. Malik. Learning a classification model for segmentation. Proceedings of the International Conference on Computer Vision, pages 10 - 17, 2003.
  • [3] S. Wang, H. Lu, F. Yang, M.-H. Yang. Superpixel tracking. Proceedings of the International Conference on Computer Vision, pages 1323–1330, 2011.
  • [4] S. Gupta, P. Arbeláez, J. Malik. Perceptual organization and recognition of indoor scenes from RGB-D images. Proceedings of the Conference on Computer Vision and Pattern Recognition, pages 564–571, 2013.
  • [5] R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, S. Süsstrunk. SLIC superpixels. Technical report, École Polytechnique Fédérale de Lausanne, 2010.
  • [6] M. Y. Lui, O. Tuzel, S. Ramalingam, R. Chellappa. Entropy rate superpixel segmentation. Proceedings of the Conference on Computer Vision and Pattern Recognition, pages 2097–2104, 2011.
  • [7] A. Levinshtein, A. Stere, K. N. Kutulakos, D. J. Fleet, S. J. Dickinson, K. Siddiqi. TurboPixels: Fast superpixels using geometric flows. Transactions on Pattern Analysis and Machine Intelligence, volume 31, number 12, pages 2290–2297, 2009.
  • [8] R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, S. Süsstrunk. SLIC superpixels compared to state-of-the-art superpixel methods. Transactions on Pattern Analysis and Machine Intelligence, volume 34, number 11, pages 2274 – 2281, 2012.
  • [9] A. Schick, M. Fischer, R. Stiefelhagen. Measuring and evaluating the compactness of superpixels. Proceedings of the International Conference on Pattern Recognition, pages 930–934, 2012.
  • [10] P. Neubert, P. Protzel. Superpixel benchmark and comparison. Forum Bildverarbeitung, 2012.
  • [11] P. Arbeláez, M. Maire, C. Fowlkes, J. Malik. Contour detection and hierarchical image segmentation. Transactions on Pattern Analysis and Machine Intelligence, volume 33, number 5, pages 898–916, 2011.
  • [12] N. Silberman, D. Hoiem, P. Kohli, R. Fergus. Indoor segmentation and support inference from RGBD images. Proceedings of the European Conference on Computer Vision, pages 746–760, 2012.

What is your opinion on this article? Did you find it interesting or useful? Let me know your thoughts in the comments below or using the following platforms:

@david_stutz