Backend
MCGI: Manifold-Consistent Graph Indexing for Billion-Scale Disk-Resident Vector Search
arXiv cs.IR0 views
arXiv:2601.01930v4 Announce Type: replace Abstract: Graph-based Approximate Nearest Neighbor (ANN) search often suffers from performance degradation in high-dimensional spaces due to the Euclidean-Geodesic mismatch, where greedy routing diverges from the underlying data manifold. To address this challenge, this paper presents Manifold-Consistent Graph Indexing (MCGI), a geometry-aware and disk-resident indexing method that leverages Local Intrinsic Dimensionality (LID) to allocate the search effort of a graph index according to the intrinsic geometry of data. Specifically, MCGI estimates the LID around each query from the distances that its own search has already computed and resizes the candidate list of the query once, so that queries in high-LID regions receive longer lists while queries in low-LID regions terminate earlier; the estimate itself costs no additional disk reads. Theoretical analysis shows that the success probability of any fixed search list decays exponentially with the LID of the query. At matched high recall, MCGI raises the query throughput of DiskANN on the same standard Vamana index by up to $1.43\times$ on three million-scale benchmarks and by up to $1.24\times$ on the billion-scale SIFT1B and Deep1B.
Source:https://arxiv.org/abs/2601.01930PreviousENGRAM: From Agent Experience to Weights through Verified ConsolidationNextBenchmarking Patent Embeddings: A Multi-Task Evaluation of 22 Models Across Retrieval, Classification, and Clustering
Related posts
How do AI agents stay logged in between runs?
0dev.to AI
The AI Didn't Break In. It Knocked on Every Door: What the BOSSAI Mesh Actually Tells Us
0dev.to AI
Attacker used artificial intelligence to steal data from 11 countries
0dev.to AI
HTML Can Open Dialogs Without JavaScript- Here's How command and commandfor Work
0dev.to WebDev

Announcing LightningChart JS v9.1.0
0dev.to WebDev
Building Introworth: Recurring Payments Are a Net-Worth Story in Slow Motion — What Worked, What Broke
0dev.to WebDev