IAM

OPENSOURCEFAN STUDYING
STUDYINGCOMPUTERSCIENCEANDMATH COMPUTERSCIENCE

Check out the latest superpixel benchmark — Superpixel Benchmark (2016) — and let me know your opinion! @david_stutz

ARTICLE

Running VLFeat’s SLIC Superpixels using CMake and C++

There are several superpixel algorithms out there. One popular approach is called “Simple Linear Iterative Clustering” – short SLIC [1]. One implementation of SLIC is provided as part of the VLFeat library [2] which can be used from MatLab, but provides the C source code as well. This article shows how to use VLFeat’s implementation of SLIC from C++ using CMake.

For my bachelor thesis I am concerned with superpixel algorithms - algorithms used to oversegment an image into groups of pixels using low-level features. Of course, I had a look at existing approaches, as for example "Simple Linear Iterative Clustering" (short SLIC) [1]. There are multiple implementations of SLIC available, among which is the original implementation which can be found here and an implementation which is part of the VLFeat Library [2]. More information on the actual algorithm can be found in the original paper [1].

Although VLFeat can easily be used from MatLab, I needed to use SLIC from C++. Therefore, I used CMake to add VLFeat as library and wrote a simple command line tool to be able to evaluate a whole bunch of images. A simplified version of the command line tool can be found below. In addition, the example is available on Github:

GitHub Repository

// OpenCV can be used to read images.
#include <opencv2/opencv.hpp>

// The VLFeat header files need to be declared external.
extern "C" {
    #include "vl/generic.h"
    #include "vl/slic.h"
}

int main() {
    // Read the Lenna image. The matrix 'mat' will have 3 8 bit channels
    // corresponding to BGR color space.
    cv::Mat mat = cv::imread("Lenna.png", CV_LOAD_IMAGE_COLOR);
    
    // Convert image to one-dimensional array.
    float* image = new float[mat.rows*mat.cols*mat.channels()];
    for (int i = 0; i < mat.rows; ++i) {
        for (int j = 0; j < mat.cols; ++j) {
            // Assuming three channels ...
            image[j + mat.cols*i + mat.cols*mat.rows*0] = mat.at<cv::Vec3b>(i, j)[0];
            image[j + mat.cols*i + mat.cols*mat.rows*1] = mat.at<cv::Vec3b>(i, j)[1];
            image[j + mat.cols*i + mat.cols*mat.rows*2] = mat.at<cv::Vec3b>(i, j)[2];
        }
    }

    // The algorithm will store the final segmentation in a one-dimensional array.
    vl_uint32* segmentation = new vl_uint32[mat.rows*mat.cols];
    vl_size height = mat.rows;
    vl_size width = mat.cols;
    vl_size channels = mat.channels();
            
    // The region size defines the number of superpixels obtained.
    // Regularization describes a trade-off between the color term and the
    // spatial term.
    vl_size region = 30;        
    float regularization = 1000.;
    vl_size minRegion = 10;
          
    vl_slic_segment(segmentation, image, width, height, channels, region, regularization, minRegion);
            
    // Convert segmentation.
    int** labels = new int*[mat.rows];
    for (int i = 0; i < mat.rows; ++i) {
        labels[i] = new int[mat.cols];
                
        for (int j = 0; j < mat.cols; ++j) {
            labels[i][j] = (int) segmentation[j + mat.cols*i];
        }
    }
    
    int label = 0;
    int labelTop = -1;
    int labelBottom = -1;
    int labelLeft = -1;
    int labelRight = -1;
    
    for (int i = 0; i < mat.rows; i++) {
        for (int j = 0; j < mat.cols; j++) {
            
            label = labels[i][j];
            
            labelTop = label;
            if (i > 0) {
                labelTop = labels[i - 1][j];
            }
            
            labelBottom = label;
            if (i < mat.rows - 1) {
                labelBottom = labels[i + 1][j];
            }
            
            labelLeft = label;
            if (j > 0) {
                labelLeft = labels[i][j - 1];
            }
            
            labelRight = label;
            if (j < mat.cols - 1) {
                labelRight = labels[i][j + 1];
            }
            
            if (label != labelTop || label != labelBottom || label!= labelLeft || label != labelRight) {
                mat.at<cv::Vec3b>(i, j)[0] = 0;
                mat.at<cv::Vec3b>(i, j)[1] = 0;
                mat.at<cv::Vec3b>(i, j)[2] = 255;
            }
        }
    }
    
    cv::imwrite("Lenna_contours.png", mat);
    
    return 0;
}

Using the labels, contours can be drawn around the superpixels. Figure 1 shows the resulting superpixel segmentation for different values of the regularization option. Figure 2 illustrates how the regionSize influences the total number of computed superpixels.

Lenna_contours_1 Lenna_contours_100 Lenna_contours_1000_20 Lenna_contours_1000 Lenna_contours_1000_40

Figure 1: Final superpixel segmentation generated by the SLIC implementation as part of the VLFeat Library. From left to right: regularization set to $1$, region size set to $30$; regularization set to $100$, region size set to $30$; regularization set to $1000$, region size set to $20$; regularization set to $1000$, region size set to $30$; regularization set to $1000$, region size set to $40$.

Update: Figure 2 shows images of the validation set of the Berkeley Segmentation Dataset [3] oversegmented using VLFeat's implementation of SLIC

bsd_1_vlslic_regularization_1 bsd_1_vlslic_regularization_1 bsd_1_vlslic_regularization_1 bsd_1_vlslic_regularization_1 bsd_1_vlslic_regularization_1
bsd_1_vlslic_regularization_1 bsd_1_vlslic_regularization_1 bsd_1_vlslic_regularization_1 bsd_1_vlslic_regularization_1 bsd_1_vlslic_regularization_1
bsd_1_vlslic_regularization_1 bsd_1_vlslic_regularization_1 bsd_1_vlslic_regularization_1 bsd_1_vlslic_regularization_1 bsd_1_vlslic_regularization_1

Figure 1: Several images from the validation set of the Berkeley Segmentation Dataset oversegmented into roughly $600$ superpixels using a region size of $10$. From top to bottom: regularization set to $1$; regularization set to $100$; regularization set to $400$.

References

VLFeat is distributed under the BSD license, see https://github.com/vlfeat/vlfeat. For license details of Lenna.png, see the corresponding Wikipedia entry.

  • [1] R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, S. Susstrunk. SLIC Superpixels. Technical report, EPFL, Lausanne, 2010.
  • [2] A. Vedaldi, B. Fulkerson. VLFeat: An open and portable library of computer vision algorithms. http://www.vlfeat.org/, 2008.
  • [3] P. Arbeláez, M. Maire, C. Fowlkes, J. Malik. Contour detection and hierarchical image segmentation. Transactions on Pattern Analysis and Machine Intelligence, volume 33, number 5, pages 898–916, 2011.

What is your opinion on this article? Did you find it interesting or useful? Let me know your thoughts in the comments below or get in touch with me:

@david_stutz  

  • Pingback: "SLIC Superpixels Compared to State-Of-The-Art Superpixel Methods", Achanta et al. - David Stutz()

  • Good tutorial, Thanks.

  • 苏超腾

    Thank you for your excellent tutorial,but the regularization value will be [0, 1000] in RGB color space will be better, because the paper says the regularization value should be [0, 40] in CIELAB color space, may be you should try to change the image color space to CIELAB, will be more closer to the paper result.

    • davidstutz

      Yes you are right, the original paper uses Lab color space. However, the code can easily be adapted using cv::cvtColor(mat, mat, CV_BGR2Lab);

      • Khaled Elrifaay

        undefined reference to `vl_slic_segment’
        i want some help

  • Khaled Elrifaay

    undefined reference to `vl_slic_segment’
    pls help!!