AI & ML
PSM: Prompt Sensitivity Minimization via LLM-Guided Black-Box Optimization
arXiv cs.CR0 views
arXiv:2511.16209v3 Announce Type: replace Abstract: System prompts are critical for guiding the behavior of Large Language Models (LLMs), yet they often contain proprietary logic or sensitive information, making them a prime target for extraction attacks. Adversarial queries can successfully elicit these hidden instructions, posing significant security and privacy risks. Existing defense mechanisms frequently rely on heuristics, incur substantial computational overhead, or are inapplicable to models accessed via black-box APIs. This paper introduces a novel framework for hardening system prompts through shield appending, a lightweight approach that adds a protective textual layer to the original prompt. Our core contribution is the formalization of prompt hardening as a utility-constrained optimization problem. We leverage an LLM-as-optimizer to search the space of possible SHIELDs, seeking to minimize a leakage metric derived from a suite of adversarial attacks, while simultaneously preserving task utility above a specified threshold, measured by semantic fidelity to baseline outputs. This black-box, optimization-driven methodology is lightweight and practical, requiring only API access to the target and optimizer LLMs. We demonstrate empirically that our optimized SHIELDs significantly reduce prompt leakage against a comprehensive set of extraction attacks, outperforming established baseline defenses without compromising the model's intended functionality. Our work presents a paradigm for developing robust, utility-aware defenses in the escalating landscape of LLM security. The code is made public on the following link: https://github.com/psm-defense/psm
Source:https://arxiv.org/abs/2511.16209PreviousHE-LRM: Encrypted Deep Learning Recommendation Models using Fully Homomorphic EncryptionNextFrom Strategy Search to Policy Learning: MCTS-Guided Adversarial Co-Evolution for LLM Safety Alignment
Related posts

Audit Trails and Explainability for Agentic AI in Code Events
0dev.to
By 3 p.m. the daily high was already final on 78% of days. In Madrid, 32% (49 airports, one year)
0dev.to

This smell startup says its AI nose is as good as a human’s
0MIT Technology Review

Debugging Claude Fable 5.2 Code Art: Truncated SVG, Broken Canvas, Working Render
0dev.to
Hydra · Coeziune OM–IA — the complete DOI registry (2026-11-18): 76 published works
0dev.to OpenSource
Your Rust Code Sets an Environment Variable and the 2024 Edition Calls That Unsafe
0dev.to Rust