AI & ML
Evolution Under Fire: Evolving Agent Safety Harnesses with Adaptive Injection Stressors
arXiv cs.CR0 views
arXiv:2611.22583v1 Announce Type: new Abstract: Tool-using language-model agents must act on information from untrusted emails, webpages, and documents. This creates a central security challenge: malicious instructions and legitimate task directives can arrive through the same interaction channel. Existing defenses are largely hand-designed and static, making further improvements dependent on repeated manual redesign. We introduce Evolution Under Fire, a staged framework that couples safety-harness evolution with adaptive stressor evolution. Utility-constrained, Pareto-scheduled tree search evolves complete executable safety harnesses from execution evidence while keeping the task model, benchmark, tools, and evaluator fixed. Adaptive stressor evolution then discovers utility-preserving attacks that guide a subsequent stage of harness evolution. On AgentDyn with GPT-4o-mini, an evolved harness reduces attack success from 50.4% to 3.0%, improves clean and attacked utility over the initial harness, and offers a stronger security-utility trade-off than reproduced defenses. On Agent-SafetyBench with Qwen2.5-7B-Instruct, harness evolution reduces attack success from 82.6% to 62.0% while retaining helpfulness and runtime validity. Evolved harnesses also transfer without further evolution to Qwen3.5-35B-A3B, reducing attack success from 32.1% to 0.9% while improving attacked-task utility.
Source:https://arxiv.org/abs/2611.22583PreviousHow Well Is My Artwork Protected? Predicting and Tailoring Protection against Style Mimicry by PersonalizationNextAdaptive LLM-Driven Honeypots vs. LLM-Driven Attackers: Does Narrative Coherence Improve Deception?
Related posts
Microsoft Built Its Own Coding Model, and the Price Is the Point
0dev.to

El diario económico coreano Maeil Business cuenta 12 primeros puestos de Darwin en Hugging Face: cuéntalos tú con Python
0dev.to
Can Peer Review Survive AI-Generated Text? [pdf]
0Hacker News
Toward Planning Representations That Separate Task Structure from Visual Content
0arXiv cs.RO
Spectral Steering: Local Inverse Geometry for Frozen Diffusion Generators
0arXiv cs.RO
Representing Differential Inclusions as Monads: A Framework for Compositional Controller Synthesis
0arXiv cs.RO