Seminar Paper “Introduction to Neural Networks”

During my fifth semester at RWTH Aachen University I participated in a seminar offered by the Chair of Computer Science 6 on “Selected Topics in Human Language Technology and Pattern Recognition”. As result, I wrote a seminar paper on neural networks for pattern recognition titled “Introduction to Neural Networks”. At the end of the seminar, all participants had to give a short talk on their topic. Both my seminar paper and my slides of the talk can be found here.

In addition to the seminar paper, I used a simple two-layer perceptron trained with gradient descent and error backpropagation (see the seminar paper for details) to recognize handwritten digits based on the MNIST dataset. The implementation was done in MatLab. I plan to publish the code on GitHub and give a short introduction in a separate article. The code and some of the results can be found in my seminar paper.

Update. The slides of the seminar paper are part of Prof. Schiele's and Dr. Mario Fritz' lecture slides on deep learning.

Update. The LaTeX source of the seminar paper can now be found on GitHub.

Update. The MatLab code is now available on GitHub. A short introduction can be found in my article "Recognizing Handwritten Digits using a Two-Layer Perceptron and the MNIST Dataset".


In this seminar paper we study artificial neural networks, their training and application to pattern recognition. We start by giving a general definition of artificial neural networks and introduce both the single-layer and the multilayer perceptron. After considering several activation functions we discuss network topology and the expressive power of multilayer perceptrons. The second section introduces supervised network training. Therefore, we discuss gradient descent and Newton's method for parameter optimization. We derive the error backpropagation algorithm for evaluating the gradient of the error function and extend this approach to evaluate its hessian. In addition, the concept of regularization will be introduced. The third section introduces pattern classification. Using maximum likelihood estimation we derive the cross-entropy error function. As application, we train a two-layer perceptron to recognize handwritten digits based on the MNIST dataset.

Seminar PaperPresentation Slides

Table of Contents



  • [1] David Stutz. Introduction to Neural Networks. Seminar on Selected Topics in Human Language Technology and Pattern Recognition, 2014. PDF
What is your opinion on this article? Let me know your thoughts on Twitter @davidstutz92 or LinkedIn in/davidstutz92.