Tutorials for (deep convolutional) neural networks.
In 2019, I interviewed for research internships at DeepMind and Google AI. I have been asked repeatedly about my preparation for and experience with these interviews. As internship applications at DeepMind have been opened recently, I thought it could be valuable to summarize my experience and recommendations in this article.
Python implementation of probabilistic principal component analysis (PPCA).
Last week, I had the pleasure to give a talk at the recently started Seminar on Distribution-Free Statistics organized by Anastasios Angelopoulos. Specifically, I talked about conformal training, a procedure allowing to train a classifier and conformal predictor end-to-end. This allows to optimize arbitrary losses defined directly on the confidence sets obtained through conformal prediction and can be shown to improve inefficiency and other metrics for any conformal predictor used at test time. In this article, I want to share the corresponding recording.
With the rising success of deep neural networks, their reliability in terms of robustness (for example, against various kinds of adversarial examples) and confidence estimates becomes increasingly important. Bayesian neural networks promise to address these issues by directly modeling the uncertainty of the estimated network weights. In this article, I want to give a short introduction of training Bayesian neural networks, covering three recent approaches.
The Heidelberg Laureate Forum brings together young researchers and laureates in computer science and mathematics. During lectures, workshops, panel discussions and social events, the forum fosters personal and scientific exchange with other young researchers as well as laureates. I was incredibly lucky to have the opportunity to participate in the 7th Heidelberg Laureate Forum 2019. In this article, I want to give a short overview of the forum and share some of my impressions.
A variational auto-encoder trained on corrupted (that is, noisy) examples is called denoising variational auto-encoder. While easily implemented, the underlying mathematical framework changes significantly. As the second article in my series on variational auto-encoders, this article discusses the mathematical background of denoising variational auto-encoders.
In the third article of my series on variational auto-encoders, I want to discuss categorical variational auto-encoders. This variant allows to learn a latent space of discrete (e.g. categorical or Bernoulli) latent variables. Compared to regular variational auto-encoders, the main challenge lies in deriving a working reparameterization trick for discrete latent variables — the so-called Gumbel trick.
As part of my master thesis, I made heavy use of variational auto-encoders in order to learn latent spaces of shapes — to later perform shape completion. Overall, I invested a big portion of my time in understanding and implementing different variants of variational auto-encoders. This article, a first in a small series, will deal with the mathematics behind variational auto-encoders. The article covers variational inference in general, the concrete case of variational auto-encoder as well as practical considerations.