AI & ML
Proximal Adam algorithm for nonsmooth composite optimization and linear convergence under proximal PL geometry
arXiv cs.NA0 views
arXiv:2611.17727v1 Announce Type: cross Abstract: We analyze Adam-type methods for nonsmooth composite optimization problems $\min_x\Phi(x)=f(x)+h(x)$, where $f$ is smooth and $h$ is convex, proper, and lower semicontinuous. The nonsmooth term is treated by a metric-weighted proximal map, while the smooth term is preconditioned by the Adam second-moment matrix. We prove a composite descent lemma whose constants depend explicitly on the spectral bounds of the adaptive metric and a residual--step equivalence that converts proximal step decay into subgradient stationarity. These yield an $O(1/N)$ bound on the minimum proximal stationarity residual for the deterministic proximal normalized gradient method without convexity of $f$, and the same $O(1/N)$ rate for proximal Adam under quantitative momentum-alignment and residual-coherence conditions together with a metric-compatibility condition. By estimating the momentum error directly, the same $O(1/N)$ rate is also obtained without the momentum-alignment and residual-coherence assumptions. When the composite objective satisfies a proximal Polyak--{\L}ojasiewicz inequality, both algorithms and the relaxed proximal Adam scheme converge linearly with explicit contraction factors. Numerical experiments on a diagonal Lasso problem, two-dimensional image TV denoising, and two-dimensional elastic parameter identification illustrate the predicted stationarity and linear rates.
Source:https://arxiv.org/abs/2611.17727PreviousA graph discretization of the Steklov and Neumann problems on domains in complete Riemannian manifoldsNextHigh-probability convergence of adaptive, proximal, and clipped Adam-type methods for nonconvex stochastic optimization
Related posts
Your Manager's Manager Has Booked a Skip Level Meeting With You
0dev.to

How to Make a Character Reference Sheet with AI (Prompts Included)
0dev.to Design
How Virtual Assistants Help Businesses Manage Their Work More Efficiently
0dev.to Productivity
DiffusionSSD: Achieving Extreme Visual Compression with Dual-stage Diffusion and Cross-level Near-Data Processing
0arXiv eess.IV
D2ToF: One-Step Diffusion for Depth Super-Resolution from Lightweight Single-Photon LiDAR Waveforms
0arXiv eess.IV
Few-Shot Open-Set Hyperspectral Pixel-Wise Classification with Spatial Aware Label Propagation
0arXiv eess.IV