Stability Analysis of Denoising Autoencoders Based on Dynamical Projection System

Abstract

In this study, we give a stability analysis of denoising autoencoder(DAE) from the novel perspective of dynamical systems when the input density is defined as a distribution on a manifold. We demonstrate the connection between the corrupted distribution and the learned reconstruction function of a nonlinear DAE, which motivates the use of a dynamic projection system (DPS) associated with the learned reconstruction function. Utilizing the constructed DPS, we prove that the high-density region of the corrupted data distribution asymptotically converges to the data manifold. Then, we show that the region is the attracting stable equilibrium manifold of the DPS which is completely stable. These results serve a theoretical basis of the DAE in recognizing the high-density region of the highly corrupted data with large deviations through the DPS. The effectiveness of this analysis is verified by conducting experiments on several toy examples and real image datasets with various types of noise.