AI & ML
Great Listeners, Poor Decision-Makers: Rethinking Agentic Audio AI
arXiv cs.SD0 views
arXiv:2611.23847v1 Announce Type: new Abstract: Audio artificial intelligence (AI) now recognizes speech, sounds, and music remarkably well, yet voice agents still act when they should not: they execute commands addressed to others, act on misheard content, and commit before the user finishes speaking. This position article argues that the bottleneck in agentic audio AI has shifted from recognition to decision-making. We propose a reference architecture with an explicit decision layer, separate from perception and reasoning, that chooses whether to act, wait, ask, or remain silent from evidence on addressedness, speaker identity, content confidence, tool reliability, utterance completeness, and action risk. We motivate the design through six recurring decision failures and a decision-theoretic argument that transcript confidence alone is insufficient to justify an action. We implement the layer with fuzzy inference, evolutionary optimization, and sequential decision-making, and evaluate it through simulations, ablations, and recorded audio, including egocentric recordings. Addressedness and speaker evidence matter most, risk-aware gating suppresses costly errors, and well-tuned policies perform similarly, with fuzzy gates more robust to overconfident addressedness estimates. On recorded audio, explicit gating cuts executed side talk from 41\% to 7--10\% while serving 83--85\% of valid requests under successful clarification, outperforming prompting-based baselines. These results support explicit action gating as an architectural requirement for reliable voice agents.
Source:https://arxiv.org/abs/2611.23847PreviousSIGMA: speaker-isolated, symptom-structured multimodal estimation of depression severity from clinical interviewsNextConstructing Dynamic Conversations with Neural Back-channel Generation and Voice-line Scheduling
Related posts
How to detect user frustration in chatbot conversations
0dev.to
Mình Đã Biến Gemini Thành Trợ Lý Cá Nhân Như Thế Nào (Lịch, Drive, Gmail và Những Lần Suýt Toang)
0dev.to
Im looking fr any free ai models / tools are there?, leaVE UR Lights💡
0dev.to
I checked 10 SEO agency websites for AI-search readiness. The real bugs were the boring ones.
0dev.to

How much GPU memory does an LLM need? The whole sum, worked out on real cards
0dev.to

Routing Upscaling Jobs by Output Size, Not Input Size
0dev.to