AI & ML
From Token Changes to Task Failures: Bit Flips in a Shared KV-Cache Block
arXiv cs.CR0 views
arXiv:2611.22243v1 Announce Type: new Abstract: In LLM serving systems with prefix caching, each KV block of a shared system prompt is held in GPU memory as a single physical copy, which every request with that prompt reads. Our earlier work measured change and showed that one bit flip in such a block changes many later responses without making them visibly degenerate. This paper extends that work by measuring how often the changed responses are wrong. Using software fault injection into vLLM serving Qwen3-8B, we flip the most significant mantissa bit or the sign bit of a value in such a block. We score responses on four tasks: code generation, short-answer questions, multiple-choice reasoning, and refusal under a policy. The fraction of changes that turned a correct response wrong depended strongly on the task: 0.023 on MBPP+ and 0.59 on held-out TriviaQA at the mantissa bit, and 0.019 and 0.60 at the sign bit. Harm fell on requests whose sampled answers are sometimes wrong, so the damage of a flip depends on how common such requests are. Harmed responses looked like ordinary errors, so a flip should be detected by checking cached blocks rather than by monitoring outputs.
Source:https://arxiv.org/abs/2611.22243PreviousGapLeak: When Privacy-Preserving LLM-based Web Agents Become a Privacy RiskNextDressed in Authority: Practical Black-Box Jailbreaking of MLLMs via Visual Trust Bias
Related posts

The Download: AI learns to smell and hydropower struggles
0MIT Technology Review

Nvidia isn’t selling any more AI chips to China
0The Verge
Nvidia clarifies when it will — and won’t —keep financing neoclouds
0TechCrunch

Google Home speakers get a major Gemini upgrade ‘soon,’ but only for paid subscribers
09to5Google

NVIDIA's CEO Shake-Up Exposes a Costly Prediction Market Deadline
0HackerNoon
Lumi-4B AI Model: Everything You Need to Know
0HackerNoon