Backend
MCGI: Manifold-Consistent Graph Indexing for Billion-Scale Disk-Resident Vector Search
arXiv cs.IR0 views
arXiv:2601.01930v4 Announce Type: replace Abstract: Graph-based Approximate Nearest Neighbor (ANN) search often suffers from performance degradation in high-dimensional spaces due to the Euclidean-Geodesic mismatch, where greedy routing diverges from the underlying data manifold. To address this challenge, this paper presents Manifold-Consistent Graph Indexing (MCGI), a geometry-aware and disk-resident indexing method that leverages Local Intrinsic Dimensionality (LID) to allocate the search effort of a graph index according to the intrinsic geometry of data. Specifically, MCGI estimates the LID around each query from the distances that its own search has already computed and resizes the candidate list of the query once, so that queries in high-LID regions receive longer lists while queries in low-LID regions terminate earlier; the estimate itself costs no additional disk reads. Theoretical analysis shows that the success probability of any fixed search list decays exponentially with the LID of the query. At matched high recall, MCGI raises the query throughput of DiskANN on the same standard Vamana index by up to $1.43\times$ on three million-scale benchmarks and by up to $1.24\times$ on the billion-scale SIFT1B and Deep1B.
Source:https://arxiv.org/abs/2601.01930PreviousENGRAM: From Agent Experience to Weights through Verified ConsolidationNextBenchmarking Patent Embeddings: A Multi-Task Evaluation of 22 Models Across Retrieval, Classification, and Clustering
Related posts
Same Request, Different Answer: Quantization Amplifies Cache-Induced Divergence in LLM Serving
0arXiv cs.DC
A structure-preserving semi-implicit finite volume scheme on vertex-staggered unstructured meshes
0arXiv cs.NA
Descending Everest, Part II: Oracle Complexity of Minimizing a Sum of Convex $(L_0,L_1)$-Smooth Functions
0arXiv stat.ML
Black-box model classification under the discriminative factorization
0arXiv stat.ML
Matrix Factorization Framework for Community Detection under the Degree-Corrected Block Model
0arXiv stat.ML
Guidance for Prior Change via Density Ratio Estimation
0arXiv stat.ML