Seminar Paper “Understanding Convolutional Neural Networks”

In my sixth semester at RWTH Aachen University, I am currently attending a seminar offered by the Computer Vision Group headed by Prof. Leibe on “Current Topics in Computer Vision and Machine Learning”. In the course of this seminar, I wrote a seminar paper entitled “Understanding Convolutional Neural Networks”. Both the seminar paper as well as the slides of the corresponding talk can be found here.

Update. The LaTeX source of the seminar paper is now available on GitHub.

Note. The Beamer theme used for the presentation can be found on geekStack.net. I removed the logo of RWTH Aachen University within the header due to copyright issues.


This seminar paper focusses on convolutional neural networks and a visualization technique allowing further insights into their internal operation. After giving a brief introduction to neural networks and the multilayer perceptron, we review both supervised and unsupervised training of neural networks in detail. In addition, we discuss several approaches to regularization.

The second section introduces the different types of layers present in recent convolutional neural networks. Based on these basic building blocks, we discuss the architecture of the traditional convolutional neural network as proposed by LeCun et al. [1] as well as the architecture of recent implementations.

The third section focusses on a technique to visualize feature activations of higher layers by backprojecting them to the image plane. This allows to get deeper insights into the internal working of convolutional neural networks such that recent architectures can be evaluated and improved even further.

Seminar PaperPresentation Slides

Table of Contents



  • [1] Y. LeCun, O. Matan, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, L. D. Jackel, H. S. Baird. Handwritten zip code recognition with multilayer networks. Proceedings of the International Conference on Pattern Recognition, pages 35 - 40, Atlantic City, 1990.
  • [2] David Stutz. Understanding Convolutional Neural Networks. Seminar on Current Topics in Computer Vision and Machine Learning, 2014. PDF
What is your opinion on this article? Let me know your thoughts on Twitter @davidstutz92 or LinkedIn in/davidstutz92.