AI & ML
Optimal Deep Nonparametric Regression with Uniformly Bounded Lipschitz Constants
arXiv stat.ML0 views
arXiv:2611.23979v1 Announce Type: new Abstract: Stability under input perturbations is necessary for deep neural network estimators in many applications involving noisy covariates or estimated features. Weight norm constraints are widely used to control sensitivity to input perturbations by bounding the Lipschitz constant of neural networks. However, existing approximation results under fixed bounds on standard norm products are insufficient to establish optimal statistical rates. In this paper, we establish a sharp approximation theory for deep ReLU networks under a split spectral norm product constraint. This theoretical result shows that sufficiently smooth H\"{o}lder functions can be uniformly approximated to arbitrary accuracy as the network depth increases, with the norm-product bound held fixed. We then propose a penalized least squares estimator for deep nonparametric regression. The resulting estimator has a uniformly bounded Lipschitz constant while still achieving the optimal nonparametric rate up to a logarithmic factor. As applications, we establish optimal prediction rates for two-stage learning and provide robustness guarantees under adversarial input perturbations. Numerical studies show that the penalized estimator has well-controlled sensitivity to input perturbations and prediction accuracy comparable to that of classical deep network methods.
Source:https://arxiv.org/abs/2611.23979PreviousImproved Leverage Score Sampling for Constrained Active Linear RegressionNextReliFlow: Reliability-Calibrated Residual Transport for Combinatorial Single-Cell Perturbation Prediction
Related posts
Prompt, Workflow, Agent or Agentic System: Which One Does Your Process Need?
0dev.to
Running Claude Code and Codex from your phone: the official options, and the self-hosted app I built
0dev.to
Unlocking Enhanced Visuals: Dlss5 Without Extra Load --An unofficial side-story version
0dev.to
Fiona the Hippo Died After Eating a Hat. But How Does a Hat Kill a Hippo?
0Wired

Why ChatGPT Cites Review Sites Instead of Your Company Website
0dev.to
Basalt walkthrough is up: why 776 tok/s needs the whole card to itself
0dev.to