AI & ML
Recursively Trained Diffusion Models: Limiting Collapse Distribution and Spectral Characterization
arXiv stat.ML0 views
arXiv:2606.13796v2 Announce Type: replace Abstract: Recursive training of generative models on their own outputs can lead to model collapse, a compounding drift away from the true data distribution. Existing theoretical works bound finite-round error accumulation in the context of diffusion models, but two questions remain open:~what distribution does the recursion converge to, and how fast? We answer both, isolating a mechanism distinct from imperfect learning: even with perfect score estimation and exact sampling, the early stopping of the reverse diffusion (required for numerical stability) drives a progressive drift away from the data distribution. We prove that this recursion converges geometrically to a unique limiting distribution, which admits a closed-form characterization as an infinite mixture of increasingly Gaussian-smoothed versions of the data distribution. A Hermite spectral decomposition of this limit reveals that recursive training acts as a low-pass filter: higher-order modes, which encode fine non-Gaussian structure, are attenuated much more strongly than coarse modes. This spectral picture motivates annealed truncation schedules that progressively shrink truncation times across retraining rounds; we prove that any schedule converging to $0$ asymptotically eliminates recursive compounding. Finally, we show our idealized characterization is robust: in the presence of discretization and score estimation errors, the learned distribution remains in a Wasserstein-2 ball around the ideal limit, with mode-dependent contraction rates that contract high-order errors faster than low-order ones. We validate the theory on synthetic Gaussian mixtures and CIFAR-10.
Source:https://arxiv.org/abs/2606.13796PreviousTight Generalization Bounds for Noiseless Inverse OptimizationNextDiffusion-Driven State Space Models
Related posts
Where an e-commerce explainer video belongs, and what to show in it
0dev.to
How to explain a crypto product on video without sounding like a pitch
0dev.to
Why Europe's Freezer Hacks Spread So Far, So Fast
0Dark Reading

Protesters storm Microsoft AI data center construction site
0Tom's Hardware

The Download: a humanoid cleans house and Samsung builds for the AI boom
0MIT Technology Review

How Remedy designed Control Resonant’s wildest mission: ‘Anything kind of goes’
0The Verge