AI & ML
Constructing Dynamic Conversations with Neural Back-channel Generation and Voice-line Scheduling
arXiv cs.SD0 views
arXiv:2611.24086v1 Announce Type: new Abstract: We present a novel approach to the problem of combining two voice-line channels of dyadic conversational audio into a unified conversation with controllable back-channeling and turn-taking dynamics. In voice performances for the game industry, the two tracks of voice-lines are typically recorded or synthesized in isolation. Our method takes two such unsynchronized tracks and uses a neural scheduler to sequence them into a single dynamic conversation, modifying both tracks by inserting or removing pauses and introducing voice overlap as appropriate. We further propose a back-channel predictor that is able to predict opportunity for back-channels for a given conversational turn's audio and content, which the scheduler can then incorporate. We evaluate our scheduler and predictor independently and in combination by: various quantitative comparison; a qualitative and perceptual comparison to prior art and naive baselines; a compelling set of results showing animator control over the diversity, extent and nature of back-channeling and turn dynamics.
Source:https://arxiv.org/abs/2611.24086PreviousGreat Listeners, Poor Decision-Makers: Rethinking Agentic Audio AINextRespCCT and FreqCCT: Compact, Explainable and Cost-Aware Convolutional Transformers for Multi-Class Respiratory Disease Classification
Related posts
The Heater Room So Far, in Seven Posts
0dev.to TypeScript
How I Built a Zero-Signup, Ad-Free Image Host on Cloudflare R2 (and Stopped Fighting Cluttered Image Sites)
0dev.to JavaScript
Why the link box is a plain input, not a drag-and-drop zone
0dev.to JavaScript
Re-Testing LLM-Based C-to-Rust Translations Against Their Original C
0arXiv cs.SE
Reliability is a user-specific property of large language models
0arXiv cs.SE
Safe Personal RSI: Evaluating and Repairing Personal-Agent Updates
0arXiv cs.SE