AI & ML
Constructing Dynamic Conversations with Neural Back-channel Generation and Voice-line Scheduling
arXiv cs.SD0 views
arXiv:2611.24086v1 Announce Type: new Abstract: We present a novel approach to the problem of combining two voice-line channels of dyadic conversational audio into a unified conversation with controllable back-channeling and turn-taking dynamics. In voice performances for the game industry, the two tracks of voice-lines are typically recorded or synthesized in isolation. Our method takes two such unsynchronized tracks and uses a neural scheduler to sequence them into a single dynamic conversation, modifying both tracks by inserting or removing pauses and introducing voice overlap as appropriate. We further propose a back-channel predictor that is able to predict opportunity for back-channels for a given conversational turn's audio and content, which the scheduler can then incorporate. We evaluate our scheduler and predictor independently and in combination by: various quantitative comparison; a qualitative and perceptual comparison to prior art and naive baselines; a compelling set of results showing animator control over the diversity, extent and nature of back-channeling and turn dynamics.
Source:https://arxiv.org/abs/2611.24086PreviousGreat Listeners, Poor Decision-Makers: Rethinking Agentic Audio AINextRespCCT and FreqCCT: Compact, Explainable and Cost-Aware Convolutional Transformers for Multi-Class Respiratory Disease Classification
Related posts
Razorwire: a proxy that blocks an AI agent's tool calls before they run
0Rust Users Forum

Gemini can now analyze your spending through Google Wallet
0TechRadar

Surfshark updates its iPhone VPN app to filter out scam callers — but you need a premium plan
0TechRadar

Hackers steal over 1 million credit card details across 20 countries — attackers hit old Windows bug to infect over 700 point-of-sale terminals
0TechRadar

Volkswagen reveals the electric camper van we've all been waiting for
0TechRadar

iOS 27 gave AirPods Pro a convenient new setting, here’s how to use it
09to5Mac