COMPUTERVISION RESEARCHSCIENTIST

RESEARCHSCIENTIST

04^{th}APRIL2017

Many recent deep learning frameworks such as Tensorflow, PyTorch, Theano or Torch are based on dense tensors. However, deep learning on non-tensor data structures is also interesting – especially for sparse, three-dimensional data. This article summarizes some of my experiences regarding deep learning on custom data structures in the mentioned libraries.

06^{th}MARCH2017

This blog series collects useful insights for working with Ubuntu. In this article, I describe how to avoid building problems when building Tensorflow on an NFS-mounted `$HOME`

directory and how to install CUDA 8.0 without a manual driver installation.

03^{rd}MARCH2017

Currently it is difficult to successfully link C++ projects with Tensorflow. However, to compile and run smaller code snippets based on Tensorflow, it might be convenient to put the code inside the tensorflow code base and compile an individual executable using Bazel.

24^{th}FEBRUARY2017

A simple implementation of batch normalization in Tensorflow. Note that the implementation is meant for training — i.e. for testing on individual samples, the mean and variance should be fixed in advance. Also note that `tf.nn.moments`

is replaced by a custom method for computing mean and variance to avoid the problem described here.

23^{th}FEBRUARY2017

In specific cases, `tf.nn.moments`

cannot be run on the GPU (see here). This is problematic when training (convolutional) neural networks where moments are part of the computation graph (e.g. for normalization). This snippet is a simple work around, computing mean and variance along the provided dimensions manually.

16^{th}FEBRUARY2017

Slightly adapted example for adding new operations in Tensorflow taken from the official documentation. The files should be copied to `tensorflow/core/user_ops`

. The new operation is compiled using `bazel build -c opt //tensorflow/core/user_ops:zero_out.so`

from the Tensorflow root. The generated `.so`

file can usually be found by searching `bazel-bin`

. This code does not include the corresponding gradient function yet.

04^{th}FEBRUARY2017

In this article, I discuss a simple Tensorflow operation implemented in C++. While the example mostly builds upon the official documentation, it includes trainable parameters and the gradient computation is implemented in C++, as well. As such, the example is slightly more complex compared to the simple `ZeroOut`

operation discussed in the documentation.

14^{th}JANUARY2017

Recently, I started working with Tensorflow — a deep learning library developed by Google. Unfortunately, Tensorflow did not work with the installed Version of CUDA. Therefore, I decided to upgrade to CUDA 8.0 and also install CuDNN. This article describes the installation process.