As part of the online course Creative Applications of Deep Learning with TensorFlow, and to get started with TensorFlow, I implemented some experiments on MNIST. Specifically, I tested different architectures, activation functions and initialization schemes. While these experiments are not systematic enough for reliable results, they can be useful as an introduction to TensorFlow. In this article, I want to share the code and the corresponding presentation.
Many recent deep learning frameworks such as Tensorflow, PyTorch, Theano or Torch are based on dense tensors. However, deep learning on non-tensor data structures is also interesting – especially for sparse, three-dimensional data. This article summarizes some of my experiences regarding deep learning on custom data structures in the mentioned libraries.
In this article, I discuss a simple Tensorflow operation implemented in C++. While the example mostly builds upon the official documentation, it includes trainable parameters and the gradient computation is implemented in C++, as well. As such, the example is slightly more complex compared to the simple
ZeroOut operation discussed in the documentation.