Backend
DialInstruct-TTS: Interactive Expressive Speech Generation via Multi-Turn Dialogue with Fine-Grained Temporal Control
arXiv eess.AS0 views
arXiv:2611.13875v1 Announce Type: new Abstract: Existing instruction-based TTS systems primarily support one-shot, utterance-level control, where a single instruction determines the speaking style of an entire utterance. As a result, even minor refinements require rewriting the instruction and regenerating the full utterance, limiting fine-grained interactive control. We propose DialInstruct-TTS, an interactive multi-turn expressive TTS framework that enables users to iteratively refine generated speech through natural-language dialogue until the desired style is achieved. At each dialogue turn, a Qwen-based planner integrates the dialogue history and latest user instruction into a complete TemporalPlan, while a temporal acoustic control module provides fine-grained control over emotion, speaking rate, volume, and phonation. Reference-latent revision preserves non-target regions across turns, allowing users to modify only the requested attributes or temporal spans while retaining previously satisfactory speech. Subjective evaluations demonstrate improved global instruction following, accurate local style realization, and effective non-target preservation. Compared with conventional one-shot interaction requiring full instruction restatement and utterance regeneration, DialInstruct-TTS substantially improves perceived convenience and interaction comfort. Audio samples are available at https://anonymous.4open.science/w/eacl-demo-17C5/.
Source:https://arxiv.org/abs/2611.13875PreviousWalrus Technical ReportNextPassive ladder-network representation of the radiation impedance of a spherical cap on a rigid sphere
Related posts
Postgres 14 hits end of life on November 12
0dev.to

MySQL to Snowflake: Binlog CDC Without Kafka or Debezium, Step by Step
0dev.to
Node.js Template Ownership: 3 PDF Watermark API Decisions Before Sharing SaaS Reports
0dev.to
I Built an Open-Source Tool to Test What Happens When MCP Servers Fail
0dev.to
Streamlining complex Helm tooling reduces team overhead and administrative friction
0dev.to

100% annotation agreement can mean one labelled record out of twelve
0dev.to