Backend
A Modified Quasi-Newton Method for Unconstrained Optimization with Nonpositive Curvature Detection and Structured Complexity
arXiv cs.NA0 views
arXiv:2611.25133v1 Announce Type: cross Abstract: We consider the convergence, worst-case complexity, and implementation of R2N, a modified quasi-Newton method for large-scale unconstrained smooth optimization. At each iteration, we compute a step that approximately minimizes the sum of a quadratic model of the objective with an adaptive quadratic regularization, and that results in sufficient decrease. The objective is only assumed to be $C^1$, although second derivatives may be used if they are available. The implementation features a broad range of step-computation strategies, from sparse factorization to matrix-free Krylov methods, that satisfy global convergence requirements. When detected, directions of nonpositive curvature are exploited. Our analysis allows unbounded model Hessians, provided they satisfy a mild growth condition. We derive a worst-case evaluation complexity bound that separates the progress made along positive- and nonpositive-curvature directions. By forcing the regularization parameter to vanish near isolated local minimizers, we obtain superlinear local convergence. The implementation specializes naturally to nonlinear least-squares problems. We provide a modular Julia implementation, and use it in a numerical study on the CUTEst test set that assesses the treatment of nonpositive curvature, the subproblem solver, the update of the regularization parameter, and direct versus iterative subproblem solves. An Armijo-Goldstein line search along nonpositive-curvature directions, MINRES-QLP as subproblem solver, and a regularization parameter driven to zero near a solution form our recommended configuration. With exact second derivatives, R2N is competitive with a trust-region method, solving slightly more problems with fewer objective and gradient evaluations. With convex models, the trust-region method performs as well or better.
Source:https://arxiv.org/abs/2611.25133PreviousA GPU Interior-Point Solver with Macro-Element Condensation for Equilibrium-Based PlasticityNextAccelerating Diffusion Maps on a Quantum Computer: Spectral Filtering and Galerkin PDE Solvers
Related posts

Data Science with Python Certification for Non-Programmers: Is It Possible?
0dev.to
How to Get Upcoming Luma (lu.ma) Events for Any City or Calendar with Python (No API Key)
0dev.to
Ranking 60 LLM APIs by Quality per Dollar: What a Value Score Can and Cannot Tell You
0dev.to

npm ERESOLVE: --legacy-peer-deps vs --force (What Each Really Does)
0dev.to

Semantic Cache with Redis to Cut AI API Costs
0dev.to
Before Automating Everything, Ask This: Is Your Business Process Actually Ready for AI?
0dev.to