Backend
Descending Everest, Part II: Oracle Complexity of Minimizing a Sum of Convex $(L_0,L_1)$-Smooth Functions
arXiv stat.ML0 views
arXiv:2611.04171v2 Announce Type: replace-cross Abstract: We study the minimization of the average of $n$ convex $(L_0,L_1)$-smooth functions: the Hessian norm of each function is bounded by an affine function of its gradient norm. We assume known upper bounds $R$ on the distance from the starting point to a minimizer and $\delta$ on the difference between the minimum of the average and the average of the infima of the components. We derive upper and lower bounds on the oracle complexity of reaching accuracy $\varepsilon$ in function value in expectation. Up to logarithmic factors, both bounds are of order $n(1+(L_1R)^{2/3}) +\sqrt{\frac{n(L_0+L_1^2\delta)R^2}{\varepsilon}+n(L_1R)^2}$. The upper bound is attained by the proposed method YETI. The method combines ideas of accelerated proximal methods and variance reduction methods with a heuristic for choosing additional points. The number of oracle calls made by the heuristic is bounded, which preserves the upper bound on the total number of queries needed to reach the target accuracy. In the experiments, YETI is compared with first-order methods on synthetic problems and Poisson regression problems. We also examine the stopping criterion and the effect of the heuristic separately. The lower bound is obtained by an explicit construction of families of instances that are hard for any first-order method.
Source:https://arxiv.org/abs/2611.04171PreviousSimple Extensions of Single-Objective Acquisition Functions and Hedge Strategies for Multi-Objective Bayesian OptimizationNextNovel technique based on L\'eja Points Approximation for Log-determinant Estimation of Large matrices
Related posts
How to fix Salesforce import errors
0dev.to Data
Wenn Proteomik das „Was“ liefert, aber VR das „Wie“ begreifbar macht
0dev.to Data
AI PFP generator 2026: identity, speed and the 96px problem
0dev.to Design
The Shadow Backlog: How Agentic Code Generation Overwhelms System Architecture
0dev.to Discuss

The Day After the Vendor Demo, When Everyone Returns to Their Actual Jobs
0dev.to Discuss
从 NBitcoin 到 NKaspa:.NET 开发者的 Kaspa 迁移对照(alpha)
0dev.to C#