AI & ML
Towards Online Adaptive LLM-based Semantic Filtering
arXiv cs.DB0 views
arXiv:2611.24518v1 Announce Type: new Abstract: Large language models (LLMs) enable semantic filtering over unstructured documents using natural language predicates, but invoking an oracle LLM on every document incurs prohibitive monetary cost and online latency. Existing model-cascade approaches reduce this cost by using a lightweight proxy model to score each document and forwarding only uncertain documents to the oracle. However, their proxy models have limited generalization capability, and they typically adopt fixed sampling budgets for proxy training and threshold calibration, which insufficiently adapts to diverse online queries. To address this, we present OASIS, an online adaptive framework for LLM-based semantic filtering. OASIS combines a query-agnostic document similarity prior with query-adaptive online learning and sampling. In the offline phase, OASIS utilizes query-agnostic document similarity as a prior and constructs a document navigating spread-out graph (NSG) over document embeddings. For each online query, OASIS trains a query-conditioned graph attention network over the document NSG to obtain proxy scores through semi-supervised node classification. To allocate labels effectively, OASIS adopts density-aware active learning to explore uncertain score regions, selectively annotate informative documents, retrain the proxy, and stop early when additional labels are unlikely to improve the cascade. During threshold calibration, OASIS progressively samples an adaptively sized calibration set, adapting calibration cost to the query's score distribution. Extensive experiments demonstrate that OASIS achieves a 1.75x-5.42x reduction in online latency and reduces the number of LLM-processed documents by 35%-75% relative to the state-of-the-art methods at an achieved accuracy of 0.9.
Source:https://arxiv.org/abs/2611.24518PreviousCascadeability and the Realization Gap: An Empirical Study of LLM-Powered Semantic FilteringNextProvCards: Artifact-Oriented Provenance for Heterogeneous Scientific Workflows
Related posts
Causal Effect Estimation Under Uncertain Statistical Surrogacy
0arXiv stat.TH
A note on the learning coefficient of exceptional singularities
0arXiv stat.TH
Differentially Private Score Estimation with Applications to Score-Based Generative Models
0arXiv stat.TH
Improved Constants in the Kautz-Singleton Construction for Probabilistic Group Testing
0arXiv stat.TH
Faithful Generative Modeling for Censored Time-to-Event Data: From Distributional Fidelity to Statistical Inference
0arXiv stat.TH
Asymptotic efficiency comparison of explicit Dirichlet shape estimators
0arXiv stat.TH