Exploring theoretical perspective of Lévy itô model

Abstract

Lévy-Itô Models belong to the category of score-based generative models, utilizing isotropic a-stable Lévy processes. Isotropic a-stable Lévy processes follows heavy-tailed distribution and possess dis- continuous paths, setting them apart from the light-tailed and continuous-path attributes of Brownian motion, which is commonly employed for injecting noise in traditional diffusion models. This paper focuses on the theoretical foundations required to derive the Lévy-Itô Models. It derives exact time- reversal formula of Stochastic differential equations (SDEs) driven by Lévy processes. It expands the concept of denoising score matching to incorporate the fractional score function version, which plays a crucial role in the drift term of the time-reversal formula for SDEs driven by an isotropic a-stable Lévy process. By computing the fractional score function of the transition density function, it can stabilizes the training of the fractional score models. Furthermore, this paper introduces a sampling method by leveraging the semi-linear structure in time-reversal SDEs. The theoretical analysis demonstrates that this sampling method converges for the Wasserstein-2 distance to a data distribution. In experiments, we show that Lévy-Itô models perform comparably to existing diffusion models on various image datasets while exhibiting a faster convergence rate and mitigating mode-collapse issues.