Koushik, based on the work of Mallat [][], illustrates the connection between convolutional neural networks (at least the kind of convolutional neural networks considering single-channel convolutions) and the scattering transform by Mallat []. Essentially, considering a convolutional neural network, computing
When unrolling the individual layers ($\ast$ is the discrete convolution), Koushik argues that this can equivalently be expressed using wavelets. Specifically, there exists a sequence $\lambda_1,\ldots,\lambda_m$ with
where $\phi_J$ is an averaging filter and $\psi_{\lambda_i}$ are suitably chosen wavelet filters. Unfortunately, examples and illustrations are missing such that it is quite hard to extract any useful conclusions from Koushik's paper. For details, I refer to [] and [].
[] Stéphane Mallat. Group invariant scattering. Communications on Pure and Applied Mathematics, 65(10):1331–1398, 2012.
Koushik, based on the work of Mallat [][], illustrates the connection between convolutional neural networks (at least the kind of convolutional neural networks considering single-channel convolutions) and the scattering transform by Mallat []. Essentially, considering a convolutional neural network, computing
$x_J(u, k_j) = \rho(\rho(\ldots \rho(x \ast W_{1})\ast \ldots )\ast W_J)$
When unrolling the individual layers ($\ast$ is the discrete convolution), Koushik argues that this can equivalently be expressed using wavelets. Specifically, there exists a sequence $\lambda_1,\ldots,\lambda_m$ with
$x_J(u,k_J) = S_J[p]x(u) = (U[p] x \ast \phi_J)(u) = (\rho(\rho(\ldots \rho(x\ast \psi_{\lambda_1})\ast \ldots )\ast\psi_{\lambda_m})(u)$
where $\phi_J$ is an averaging filter and $\psi_{\lambda_i}$ are suitably chosen wavelet filters. Unfortunately, examples and illustrations are missing such that it is quite hard to extract any useful conclusions from Koushik's paper. For details, I refer to [] and [].