IAM

ARTICLE

Inspecting Tensorflow’s Tensors using C++ and Bazel

Currently it is difficult to successfully link C++ projects with Tensorflow. However, to compile and run smaller code snippets based on Tensorflow, it might be convenient to put the code inside the tensorflow code base and compile an individual executable using Bazel.

In order to extend Tensorflow by implementing new operations in C++, a profound understanding of Tensorflow's inner workings is beneficial. For example, to use operations already implemented on C++ arrays, I was most interested in Tensorflow's tensors — specifically their memory layout and how to convert form and to C/C++ arrays.

Initially, I tried to link to Tensorflow using CMake, following the guide provided here. However this turned out to be quite complicated and error prone. Instead, I decided to follow this guide written by Jim Fleming to compile C++ code within Tensorflow using Bazel. As also stated by him, this method has some disadvantages including long building time and large binary size. Still it is the easiest working method I found.

Update. This StackOverflow question might also be helpful for linking to TensorFlow using g++. Also check the comments for details.

Setup

In the following, I assume that Tensorflow has been successfully build manually. Thorough documentation on installing Tensorflow from sources can be found in the official documentation. For simplicity, a new directory inside tensorflow-master/tensorflow should be created (assuming Tensorflow was installed in tensorflow-master). For the following, this directory is called arrays_from_tensors. In the following we will put a arrays_from_tenors.cc file as well as a BUILD file in this directory.

C++ for Understanding Tensors

The following listing, arrays_from_tensors.cc, shows a simple example investigating the underlying memory layout of a Tensorflow tensor:

/// \brief Trying to create Eigen tensors, and Tensorflow tensors from
/// simple C++ arrays in different alignment (row-major, col-major, 3d etc.).
/// \author David Stutz
/// \file tensors_from_array.cc

#include "tensorflow/core/framework/tensor.h"
#include "tensorflow/core/framework/tensor_shape.h"

using namespace tensorflow;

/// \brief Trying to create Eigen/Tensorflow tensors from arrays.
/// \param argc
/// \param argv
/// \return 
int main(int argc, char** argv) {
  
  const int batch_size = 1;
  const int depth = 5;
  const int height = 5;
  const int width = 5;
  const int channels = 3;
  
  // trying to create and fill a Tensorflow tensor, with at least 4 dimensions
  // to see how the data is stored in memory;
  // hopefully able to convert it to the Eigen tensor and from there to an array.
  Tensor tensor(DataType::DT_INT32, TensorShape({batch_size, depth, height, width, channels}));
  
  // get underlying Eigen tensor
  auto tensor_map = tensor.tensor<int, 5>();
  
  // fill and print the tensor
  for (int n = 0; n < batch_size; n++) {
    for (int d = 0; d < depth; d++) {
      std::cout << d << " --" << std::endl;
      for (int h = 0; h < height; h++) {
        for (int w = 0; w < width; w++) {
          for (int c = 0; c < channels; c++) {
            tensor_map(n, d, h, w, c) = (((n*depth + d)*height + h)*width + w)*channels + c;
            std::cout << tensor_map(n, d, h, w, c) << ",";
          }
          std::cout << " ";
        }
        std::cout << std::endl;
      }
    }
  }
  
  // get the underlying array
  auto array = tensor_map.data();
  int* int_array = static_cast<int*>(array);
  
  // try to print the same to see the data layout
  for (int n = 0; n < batch_size; n++) {
    for (int d = 0; d < depth; d++) {
      std::cout << d << " --" << std::endl;
      for (int h = 0; h < height; h++) {
        for (int w = 0; w < width; w++) {
          for (int c = 0; c < channels; c++) {
            std::cout << int_array[(((n*depth + d)*height + h)*width + w)*channels + c] << ",";
          }
          std::cout << " ";
        }
        std::cout << std::endl;
      }
    }
  }
  
  return 0;
}

The main insight of this example is that Tensorflow tensors can be easily created, the underlying pointer can be accessed and the tensor data can be manipulated by other libraries. For my application, I found the following to be a useful utility:

/// Some array conversion tools.
/// \author David Stutz
template<typename T, int NDIMS>
class TensorConversion {
public:
  
  /// Access the underlying data pointer of the tensor.
  /// \param tensor
  /// \return
  static T* AccessDataPointer(const tensorflow::Tensor &tensor) {
    // get underlying Eigen tensor
    auto tensor_map = tensor.tensor<T, NDIMS>();
    // get the underlying array
    auto array = tensor_map.data();
    return const_cast<T*>(array);
  }
};

Building using Bazel

To build the C++ code, the following BUILD file is used:

cc_binary(
    name = "tensors_from_arrays",
    srcs = ["tensors_from_arrays.cc"],
    deps = [
        "//tensorflow/core:tensorflow",
    ]
)

The code can then be built and run using:

bazel build :tensors_from_arrays
bazel run :tensors_from_arrays

The compiled executable can also be executed directly, it can usually be found in bazel-bin.

Conclusion

Building C++ executables inside the Tensorflow project using Bazel is currently the easiest method. However, build times may be longer as the whole Tensorflow project needs to be built or loaded from the cache. Similarly the executable becomes very big. Therefore, this method might be suboptimal for larger endeavours or comfortable debugging. However, to quickly try smaller code snippets to get started, it is fine.

What is your opinion on this article? Let me know your thoughts on Twitter @davidstutz92 or LinkedIn in/davidstutz92.