AI & ML
LEGIT: Live Eligibility Gating over Identifier Tries for Constrained Decoding of Heterogeneous Pages
arXiv cs.IR0 views
arXiv:2611.10357v2 Announce Type: replace Abstract: A generative recommender names each candidate with a short token sequence, a Semantic ID, and recommends by generating those tokens. Some generative recommenders, including the Uber Eats search engine we are building, write a whole results page, stores with their menu items, in one decoding pass. Every Semantic ID on such a page must name a real store or item that is eligible for the search, such as a store that delivers to the user's location, and that is not already on the page. Each item must also belong to the store above it. A trie over the catalog keeps identifiers real, but eligibility and repetition change with every request and token, and the grammar-constrained decoder we tried could not enforce these changing rules exactly and quickly. We call this constraint LEGIT, for live eligibility gating over identifier tries. Numbering trie leaves depth first turns each rule into a range lookup over one store trie and one item trie, and every generated page passes an independent catalog check. On one NVIDIA B200 GPU with one or two requests per decode step, a direct implementation that walks the trie on the GPU makes each token about 5 times as slow as unconstrained decoding. Across 36 implementations, hardware counters and ablations are consistent with kernel launches and CPU-GPU waits as the source of this slowdown. Our fastest implementation walks the trie on the CPU while the GPU runs the model, needs no custom kernel, and cuts time per output token by 65.9 percent against the direct GPU implementation, to about 1.7 times the unconstrained cost. This implementation is also the fastest on a page layout that puts each item before its store.
Source:https://arxiv.org/abs/2611.10357PreviousPrompt-Based Abstention Fails Under Misleading Context: A Controlled Study of Small Frozen RAG ModelsNextChi-Square Wavelet Graph Neural Networks for Heterogeneous Graph Anomaly Detection
Related posts

SBOM ที่ทำแล้วไม่มีใครเปิด: เปลี่ยนมันเป็น re-scan รายวันที่จับ CVE ใหม่ได้ โดยไม่ต้อง rebuild image
0dev.to Kubernetes

Add ChatGPT-style AI chat to your Expo app in 5 minutes
0dev.to OpenSource
sheet_route: نوافذ bottom sheet في Flutter كـ routes حقيقية مع snap points و go_router
0dev.to OpenSource
Every family WhatsApp group in India has the same problem
0dev.to OpenSource

How to Use Git Stash Step by Step
0dev.to Git
GenTUS: Table Union Search via Constrained Generation of Semantic Table Identifiers
0arXiv cs.DB