Tensorboard is a great tool to monitor and debugg deep neural network training. Originally developed for TensorFlow, Tensorboard is now also supported by other libraries such as PyTorch. While the integration in PyTorch was shaky in the beginning, it got better and better with more recent releases. In this article, I want to discuss how to use Tensorboard for monitoring training with PyTorch. The article’s code is available on GitHub.
As part of the online course Creative Applications of Deep Learning with TensorFlow, and to get started with TensorFlow, I implemented some experiments on MNIST. Specifically, I tested different architectures, activation functions and initialization schemes. While these experiments are not systematic enough for reliable results, they can be useful as an introduction to TensorFlow. In this article, I want to share the code and the corresponding presentation.
Many recent deep learning frameworks such as Tensorflow, PyTorch, Theano or Torch are based on dense tensors. However, deep learning on non-tensor data structures is also interesting – especially for sparse, three-dimensional data. This article summarizes some of my experiences regarding deep learning on custom data structures in the mentioned libraries.
In this article, I discuss a simple Tensorflow operation implemented in C++. While the example mostly builds upon the official documentation, it includes trainable parameters and the gradient computation is implemented in C++, as well. As such, the example is slightly more complex compared to the simple
ZeroOut operation discussed in the documentation.