Many recent deep learning frameworks such as Tensorflow, PyTorch, Theano or Torch are based on dense tensors. However, deep learning on non-tensor data structures is also interesting – especially for sparse, three-dimensional data. This article summarizes some of my experiences regarding deep learning on custom data structures in the mentioned libraries.
Following the Theano documentation, this snippet illustrates the creation of a new Theano type, namely the Double type. Based on this type, the add operation is implemented. Originally, I intended this as a quick tutorial on how to define more complex types with differentiable operations. However, as also discussed here, this turned out to be more involved than expected.