Backend
Beyond Overlap: Hierarchical Graph Abstraction for Out-of-Core Vector Search
arXiv cs.IR0 views
arXiv:2512.22838v3 Announce Type: replace-cross Abstract: At billion scale, approximate nearest neighbor search (ANNS) often becomes an out-of-core problem: the full vector collection and index structures exceed memory capacity, making query performance dominated by SSD accesses and DRAM-SSD data movement. Existing systems struggle to strike a balance between accuracy and efficiency: physical-overlap methods replicate vectors or index entries across partitions, enlarging the SSD-resident index and incurring extra I/O; quantization-based methods reduce memory usage, but their approximate distances are less reliable and often require costly raw-vector reranking from SSD to preserve recall. We present GANNS (Graph-Abstraction ANNS), a hierarchical out-of-core ANNS engine that improves accuracy and I/O efficiency simultaneously. GANNS stores each cluster as a disjoint SSD partition with scale-aware indexes, while a memory-resident graph abstraction provides logical overlap before SSD access. During serving, GANNS uses query hotness and cluster priorities from the graph abstraction to prune low-value clusters and improve access locality. Across five datasets under strict memory constraints, GANNS delivers representative QPS improvements of 2.0--4.7x over state-of-the-art baselines, while preserving accuracy.
Source:https://arxiv.org/abs/2512.22838PreviousKnowledge Graph-based Retrieval-Augmented Generation for Schema MatchingNextRACORN: Robust Predicate-Agnostic Filtered Vector Search under Extreme Selectivity
Related posts
Building a Private AI Control Plane for PS5 Porting: What We Learned After SQLite Locked Us Out at 2 AM
0dev.to
Monolith vs Microservices: A Founder's Guide
0dev.to
Sentinel Dev Diary: Read First, Run Second
0dev.to
A website for the café down the street, prepared before they ask
0dev.to

Where Temporary Files Come From and Why They Can Leak Office Data
0dev.to

How to Accept USDC Payments on EVM: A Reorg-Safe Stablecoin Payments Detector in Node.js
0dev.to