Backend
Blind Noise Equalization for Large Data Matrices under General Variance Profiles
arXiv stat.ML0 views
arXiv:2611.08784v1 Announce Type: new Abstract: Low-rank approximation is a basic tool for inference from large and noisy data matrices. However, when noise variances differ across entries, a setting known as heteroskedasticity, the noise can distort the observed spectrum, producing spurious components and obscuring weak signals. To address these challenges, we develop BRINE (Blind Resolvent-based Iterative Noise Equalizer), a data-driven procedure that iteratively rescales rows and columns to equalize the average noise variance across both dimensions. Under suitable conditions, this procedure restores the Marchenko-Pastur law for the noise spectrum and can amplify signal components without prior knowledge of the signal or noise structure. A key idea underlying our approach is to equalize the diagonal entries of the data resolvent as a proxy for equalizing the unknown noise variances. At the population level, we identify a potential function that characterizes this equalization and motivates an iterative minimization procedure. We prove convergence of the iterations to the true equalizer for arbitrary positive variance profiles, with a geometric rate under mild conditions. The updates can be estimated directly from noisy data, and we establish probabilistic error bounds for the scaling factors. Simulations and experiments across diverse biological data modalities and preprocessing pipelines illustrate the effectiveness of our approach.
Source:https://arxiv.org/abs/2611.08784PreviouspESTO: Iterative p-greedy kernel regression with risk-driven Early STOppingNextHigh-Dimensional Off-Policy Evaluation without Sample Splitting using an Onsager Correction
Related posts
Building a Private AI Control Plane for PS5 Porting: What We Learned After SQLite Locked Us Out at 2 AM
0dev.to
Monolith vs Microservices: A Founder's Guide
0dev.to
Sentinel Dev Diary: Read First, Run Second
0dev.to
A website for the café down the street, prepared before they ask
0dev.to

Where Temporary Files Come From and Why They Can Leak Office Data
0dev.to

How to Accept USDC Payments on EVM: A Reorg-Safe Stablecoin Payments Detector in Node.js
0dev.to