AI & ML
Shielded Trajectory Sampling for Personalized Relational Safety in Conversational AI
arXiv cs.HC0 views
arXiv:2609.30780v2 Announce Type: replace Abstract: Anthropomorphic conversational systems can provide companionship and emotional support, but repeated, seemingly harmless interactions can encourage excessive dependence and reduced human contact. We formulate safety in user-AI relationships as a sequential decision problem, using conversational cues to track the relationship and selecting responses according to their anticipated effects over time. A conditional Generative Flow Network learns from sampled response plans to select replies that preserve the largest estimated mass of safe and useful continuations. Model-based checks screen plans during training and responses during interaction, requiring at least one sequence of future response strategies that meets a prescribed safety bound under every candidate user model, independently of the learned network. Strategies are selected directly from learned first-step probabilities, avoiding online plan sampling. We evaluate the approach in simulation against reactive safeguards, filters with and without memory, and alternative planners, using exact computation to validate the learned approximation. We also evaluate relational cue extraction from real user messages in WildChat. To our knowledge, this is the first approach to select conversational replies by anticipating their effects on the user-AI relationship while imposing an explicit model-based safety bound on future response sequences. Results show close agreement with exact planning, lower decision costs at longer horizons, and improved safety-utility trade-offs when protective responses are less effective. The approach balances immediate support with the longer-term effects of repeated interaction. The source code is available at https://github.com/benedettapicano/ANTHROPOMORPHIC_SAFETY_TRAJ.
Source:https://arxiv.org/abs/2609.30780PreviousHuman-AI Writing as Distributed Cognition: Reassessing Collaborative Writing Theories in the Age of LLMsNextAI Simulations Grounded in Rich Self-Reports Increase Accuracy and Reduce Distortion
Related posts
Introduction to CUDA Beyond the Syntax
0Hacker News New
Physics AI: The illusion of ground truth
0Hacker News New
RiddleBell Puzzle Alarm Clock
0Hacker News New
Open WebUI – Open Terminal: A Computer for Your AI Agents
0Hacker News New
LLMs abandon correct climate facts when users push back (UNESCO/UCL)
0Hacker News New
Standard Support: How AI agents ask for support
0Hacker News New