AI & ML
High-probability convergence of adaptive, proximal, and clipped Adam-type methods for nonconvex stochastic optimization
arXiv cs.NA0 views
arXiv:2611.17736v1 Announce Type: cross Abstract: This paper develops a unified high-probability analysis of adaptive, proximal, and clipped first-order methods for nonconvex stochastic optimization. For a smooth objective whose stochastic gradient error has sub-Gaussian norm, the normalized adaptive gradient method attains the standard stochastic nonconvex complexity of the square root of the number of iterations, with an additional logarithmic dependence on the inverse confidence level that is inversely proportional to the number of iterations. The proof uses an adaptive descent decomposition together with Freedman and Bernstein concentration inequalities. The same guarantee is established for the full adaptive moment method under an explicit momentum-alignment condition. The analysis is extended to projected and proximal updates through a stationarity measure based on the subdifferential distance, to objectives satisfying the Polyak-Lojasiewicz condition, and to heavy-tailed noise through truncation. The framework is applied to stochastic inverse problems in elasticity, and numerical experiments illustrate the theoretical scaling and the practical behavior of the methods.
Source:https://arxiv.org/abs/2611.17736PreviousProximal Adam algorithm for nonsmooth composite optimization and linear convergence under proximal PL geometryNextScalar Entropic Damage Transport: Well-posed softening with a single damage variable
Related posts
Your Manager's Manager Has Booked a Skip Level Meeting With You
0dev.to

How to Make a Character Reference Sheet with AI (Prompts Included)
0dev.to Design
How Virtual Assistants Help Businesses Manage Their Work More Efficiently
0dev.to Productivity
DiffusionSSD: Achieving Extreme Visual Compression with Dual-stage Diffusion and Cross-level Near-Data Processing
0arXiv eess.IV
D2ToF: One-Step Diffusion for Depth Super-Resolution from Lightweight Single-Photon LiDAR Waveforms
0arXiv eess.IV
Few-Shot Open-Set Hyperspectral Pixel-Wise Classification with Spatial Aware Label Propagation
0arXiv eess.IV