In order to extend Tensorflow by implementing new operations in C++, a profound understanding of Tensorflow's inner workings is beneficial. For example, to use operations already implemented on C++ arrays, I was most interested in Tensorflow's tensors — specifically their memory layout and how to convert form and to C/C++ arrays.
Initially, I tried to link to Tensorflow using CMake, following the guide provided here. However this turned out to be quite complicated and error prone. Instead, I decided to follow this guide written by Jim Fleming to compile C++ code within Tensorflow using Bazel. As also stated by him, this method has some disadvantages including long building time and large binary size. Still it is the easiest working method I found.
Update. This StackOverflow question might also be helpful for linking to TensorFlow using g++. Also check the comments for details.
Setup
In the following, I assume that Tensorflow has been successfully build manually. Thorough documentation on installing Tensorflow from sources can be found in the official documentation. For simplicity, a new directory inside tensorflow-master/tensorflow
should be created (assuming Tensorflow was installed in tensorflow-master
). For the following, this directory is called arrays_from_tensors
. In the following we will put a arrays_from_tenors.cc
file as well as a BUILD
file in this directory.
C++ for Understanding Tensors
The following listing, arrays_from_tensors.cc
, shows a simple example investigating the underlying memory layout of a Tensorflow tensor:
/// \brief Trying to create Eigen tensors, and Tensorflow tensors from /// simple C++ arrays in different alignment (row-major, col-major, 3d etc.). /// \author David Stutz /// \file tensors_from_array.cc #include "tensorflow/core/framework/tensor.h" #include "tensorflow/core/framework/tensor_shape.h" using namespace tensorflow; /// \brief Trying to create Eigen/Tensorflow tensors from arrays. /// \param argc /// \param argv /// \return int main(int argc, char** argv) { const int batch_size = 1; const int depth = 5; const int height = 5; const int width = 5; const int channels = 3; // trying to create and fill a Tensorflow tensor, with at least 4 dimensions // to see how the data is stored in memory; // hopefully able to convert it to the Eigen tensor and from there to an array. Tensor tensor(DataType::DT_INT32, TensorShape({batch_size, depth, height, width, channels})); // get underlying Eigen tensor auto tensor_map = tensor.tensor<int, 5>(); // fill and print the tensor for (int n = 0; n < batch_size; n++) { for (int d = 0; d < depth; d++) { std::cout << d << " --" << std::endl; for (int h = 0; h < height; h++) { for (int w = 0; w < width; w++) { for (int c = 0; c < channels; c++) { tensor_map(n, d, h, w, c) = (((n*depth + d)*height + h)*width + w)*channels + c; std::cout << tensor_map(n, d, h, w, c) << ","; } std::cout << " "; } std::cout << std::endl; } } } // get the underlying array auto array = tensor_map.data(); int* int_array = static_cast<int*>(array); // try to print the same to see the data layout for (int n = 0; n < batch_size; n++) { for (int d = 0; d < depth; d++) { std::cout << d << " --" << std::endl; for (int h = 0; h < height; h++) { for (int w = 0; w < width; w++) { for (int c = 0; c < channels; c++) { std::cout << int_array[(((n*depth + d)*height + h)*width + w)*channels + c] << ","; } std::cout << " "; } std::cout << std::endl; } } } return 0; }
The main insight of this example is that Tensorflow tensors can be easily created, the underlying pointer can be accessed and the tensor data can be manipulated by other libraries. For my application, I found the following to be a useful utility:
/// Some array conversion tools. /// \author David Stutz template<typename T, int NDIMS> class TensorConversion { public: /// Access the underlying data pointer of the tensor. /// \param tensor /// \return static T* AccessDataPointer(const tensorflow::Tensor &tensor) { // get underlying Eigen tensor auto tensor_map = tensor.tensor<T, NDIMS>(); // get the underlying array auto array = tensor_map.data(); return const_cast<T*>(array); } };
Building using Bazel
To build the C++ code, the following BUILD
file is used:
cc_binary( name = "tensors_from_arrays", srcs = ["tensors_from_arrays.cc"], deps = [ "//tensorflow/core:tensorflow", ] )
The code can then be built and run using:
bazel build :tensors_from_arrays bazel run :tensors_from_arrays
The compiled executable can also be executed directly, it can usually be found in bazel-bin
.
Conclusion
Building C++ executables inside the Tensorflow project using Bazel is currently the easiest method. However, build times may be longer as the whole Tensorflow project needs to be built or loaded from the cache. Similarly the executable becomes very big. Therefore, this method might be suboptimal for larger endeavours or comfortable debugging. However, to quickly try smaller code snippets to get started, it is fine.