AI & ML
Geometry-Aware Score-Repellent Monte Carlo
arXiv stat.ML0 views
arXiv:2611.08909v1 Announce Type: new Abstract: The recently proposed Score-Repellent Monte Carlo (SRMC) method improves MCMC sampling by tilting the target density $\pi$ in $\mathbb{R}^d$ with the factor $\exp(-\alpha\,\theta_n^\top s(x))$, where the score function $s(x) = \nabla_x \log \pi(x)$, and $\theta_n$ is the score average from past samples. However, a single scalar repellence strength $\alpha\ge0$ cannot handle both the steep and the flat directions in anisotropic energy landscapes at once. We introduce \emph{Geometry-Aware SRMC} (GA-SRMC), which generalizes the Euclidean alignment to $\theta^\top M s(x)$ for any symmetric positive-definite matrix $M$. A stochastic-approximation central limit theorem under a trace constraint on $M$ identifies the trace-normalized inverse Fisher $M_{\rm opt}\propto S^{-1}$ as the unique minimizer of the worst-direction covariance bound, elevating the Fisher information $S=\text{Cov}_\pi(s,s)$ from a proposal-side preconditioner (e.g., FisherMALA, natural gradient) to the bound-optimal repulsion-side matrix, with a single online estimate $\widehat S_n$ serving both roles at no additional computational cost. The full-Fisher online metric $M_n$ converges almost surely to $M_{\rm opt}$ and satisfies a CLT, providing a theoretical guarantee for the online implementation of the GA-SRMC framework. On an $8$-target simulation spanning the condition number $\kappa\in[1.3,100]$ and dimension $d\in[2,785]$, GA-SRMC dominates SRMC in ESS by $\boldsymbol{4.3\times}$ on anisotropic Gaussian ($\kappa=100$) and $\boldsymbol{6.8\times}$ on MNIST under the MALA baseline, by $\boldsymbol{16\text{--}32\%}$ on three well-conditioned logistic-regression posteriors, and by up to $\boldsymbol{+48\%}$ under FisherMALA; gains peak on the most anisotropic targets, and posterior-mean MSE is lower than with SRMC in all $16$ full-Fisher implementations.
Source:https://arxiv.org/abs/2611.08909PreviousBeyond the Lower-Star Filtration: Alternative Filtration Methods for Persistent HomologyNextKernel-Based Learning of Analytic Operators and Parametric PDEs
Related posts
An Empirical Analysis of Static Analysis Methods for Detection and Mitigation of Code Library Hallucinations
0arXiv cs.SE
TATG: Tracking-Aware Testing Objective for LLM-based Test Generation
0arXiv cs.SE
MCP Server Architecture Patterns for LLM-Integrated Applications
0arXiv cs.SE
Introducing Cluster Efficiency to LLM Serving: Pooling Economical Accelerators over Fast Interconnects
0arXiv cs.DC
Evidence-Linked Radiology Reporting: A Human-Supervised Reference Architecture for Structured Imaging Intelligence
0arXiv cs.HC
Intentmaking and Sensemaking: Human Interaction with AI-Guided Mathematical Discovery
0arXiv cs.HC