Backend
Blind Noise Equalization for Large Data Matrices under General Variance Profiles
arXiv stat.ML0 views
arXiv:2611.08784v1 Announce Type: new Abstract: Low-rank approximation is a basic tool for inference from large and noisy data matrices. However, when noise variances differ across entries, a setting known as heteroskedasticity, the noise can distort the observed spectrum, producing spurious components and obscuring weak signals. To address these challenges, we develop BRINE (Blind Resolvent-based Iterative Noise Equalizer), a data-driven procedure that iteratively rescales rows and columns to equalize the average noise variance across both dimensions. Under suitable conditions, this procedure restores the Marchenko-Pastur law for the noise spectrum and can amplify signal components without prior knowledge of the signal or noise structure. A key idea underlying our approach is to equalize the diagonal entries of the data resolvent as a proxy for equalizing the unknown noise variances. At the population level, we identify a potential function that characterizes this equalization and motivates an iterative minimization procedure. We prove convergence of the iterations to the true equalizer for arbitrary positive variance profiles, with a geometric rate under mild conditions. The updates can be estimated directly from noisy data, and we establish probabilistic error bounds for the scaling factors. Simulations and experiments across diverse biological data modalities and preprocessing pipelines illustrate the effectiveness of our approach.
Source:https://arxiv.org/abs/2611.08784PreviouspESTO: Iterative p-greedy kernel regression with risk-driven Early STOppingNextHigh-Dimensional Off-Policy Evaluation without Sample Splitting using an Onsager Correction
Related posts
Same Request, Different Answer: Quantization Amplifies Cache-Induced Divergence in LLM Serving
0arXiv cs.DC
A structure-preserving semi-implicit finite volume scheme on vertex-staggered unstructured meshes
0arXiv cs.NA
Descending Everest, Part II: Oracle Complexity of Minimizing a Sum of Convex $(L_0,L_1)$-Smooth Functions
0arXiv stat.ML
Black-box model classification under the discriminative factorization
0arXiv stat.ML
Matrix Factorization Framework for Community Detection under the Degree-Corrected Block Model
0arXiv stat.ML
Guidance for Prior Change via Density Ratio Estimation
0arXiv stat.ML