Updated: November 18, 2026
Disclosure: I work on Miragea, the voice engine this open-source project calls to render voices. The project itself is MIT-licensed, and every step below comes from its public README and example files. This post was drafted with AI assistance and I checked it against the repository.
Short answer: Install the open-source YouTube Dubbing MCP server in Claude Code, Claude Desktop, Cursor, Codex or VS Code, then ask your agent: "Dub this video into English, keep each speaker's voice, and make the YouTube upload package." The server pulls the captions or transcript, works out who is speaking, cuts a short voice sample of each speaker and clones it, and gives your agent a time budget for every line to translate into. It then voices each line in that speaker's cloned voice, mixes a track exactly as long as the video, and writes an SRT plus a per-language upload folder for YouTube Studio.
Why "one voice per speaker" is the hard part
Most AI dubbing demos use one narrator. Real videos (interviews, podcasts, tutorials with a guest) have two or more people, and viewers notice straight away when everyone suddenly sounds the same. Doing it properly means three things:
- Knowing who said what (speaker separation)
- A clean sample of each person to clone from
- Translated lines that fit the original timing. Translations usually run longer, so you either rewrite them or speed the audio up.
The workflow below handles all three and keeps the language model (your agent) in charge of the translation, which is the part it's best at.
What you need
- An MCP client: Claude Code, Claude Desktop, Cursor, Windsurf, OpenAI Codex or VS Code
-
uv, which provides theuvxcommand (or Docker) - A voice-engine API key from Miragea (
MIRAGEA_API_KEY) - Permission from the people in the video to clone their voices
You don't need a GPU or any local models, and ffmpeg comes bundled.
Step 1: Add the server to your agent
Claude Code (one line):
claude mcp add youtube-dubbing -s user -e MIRAGEA_API_KEY=<your-key> -- \ uvx --from git+https://github.com/RTY2345RR/youtube-dubbing-mcp youtube-dubbing-mcp Claude Desktop / Cursor / Windsurf (claude_desktop_config.json or ~/.cursor/mcp.json):
{ "mcpServers": { "youtube-dubbing": { "command": "uvx", "args": ["--from", "git+https://github.com/RTY2345RR/youtube-dubbing-mcp", "youtube-dubbing-mcp"], "env": { "MIRAGEA_API_KEY": "<your-key>" } } } } Docker:
docker run -i --rm -e MIRAGEA_API_KEY -v $HOME/YouTubeDubbing:/data miragea/youtube-dubbing-mcp To check the setup without creating a job, run the same uvx command with --check. Projects are saved to ~/YouTubeDubbing by default (DUBBING_OUTPUT_DIR).
Step 2: Start a project from a URL, a file or an SRT
Ask your agent to start, or call the tool directly:
dubbing_start_project(source="https://youtu.be/…", source_language="es") The server uses YouTube's manual or automatic captions when they exist, because they're fast and avoid guesswork. When there are none, it transcribes the audio and splits it at pauses. Local MP4, MOV, MKV, MP3 and WAV files work too, and so do existing SRT or VTT subtitles.
Step 3: Name the speakers
Speakers come from caption markers such as Name:, [Name] or >>. If there aren't any, the server groups lines by voice pitch. You can fix the result with one call:
dubbing_assign_speakers({"1-2": "Marco", "3": "Lucia"}) Step 4: Cut a voice sample per speaker and clone it
dubbing_extract_voice_samples(share=true) This cuts a clean 4–14 second sample of each speaker from the source audio, with its transcript, and clones each one automatically. If you'd rather use a saved voice or a public reference clip, use dubbing_set_voice instead.
Step 5: Let the agent translate inside a time budget
Each line gets a character budget for its time slot in the target language. Your agent writes the translation to fit that budget and saves it:
dubbing_set_translation("en", {"1": "Hi everyone.", ...}, title="…") Lines that are still too long come back to the agent to shorten. This is the step most dubbing pipelines skip, and it's why their dubs either drift out of sync or play back at chipmunk speed.
Step 6: Render the dub track
dubbing_render("en") Every line is voiced in its speaker's cloned voice and placed at its original start time. Lines are sped up only when they have to be, with pitch kept and a cap of 1.25×. Then everything is mixed into a track that is exactly as long as the video, which YouTube requires. You also get an SRT and an optional preview MP4. If a job is interrupted, run it again: finished lines are reused, so nothing gets voiced twice.
Step 7: Package it for YouTube
dubbing_export_youtube_package() This gives you one folder per language with audio.m4a (AAC, 48 kHz), captions.srt and a localized title and description, plus an UPLOAD-TO-YOUTUBE.md with the steps.
Important: YouTube's public Data API can't upload extra audio tracks, so the last step happens in YouTube Studio:
- Studio → Content → your video → Languages
- Add language → Dub → Add → upload
audio.m4a→ Publish - Subtitles → Add → Upload file → With timing →
captions.srt - Title & description → Add → paste the localized text
Multi-language audio tracks are only available on channels YouTube has enabled for the feature.
A real example: Spanish interview → English, two voices
The repository includes a real run in examples/spanish-to-english-two-speakers. The prompt was:
Dub
interview-es.mp4into English. Marco and Lucia should keep their own voices. Then make the YouTube upload package.
| Step | Tool call | Result reported in the repo |
|---|---|---|
| 1 | dubbing_start_project | 3 sentences, 11.3 s, 2 voices found by pitch (117 Hz / 182 Hz) |
| 2 | dubbing_assign_speakers | Speakers named Marco and Lucia |
| 3 | dubbing_extract_voice_samples(share=true) | 5.5 s and 5.1 s samples, both voices set |
| 4 | dubbing_set_translation("en", …) | 3/3 lines fit their slots |
| 5 | dubbing_render("en") | Track 11.29 s = video 11.29 s, 0 lines sped up |
| 6 | dubbing_export_youtube_package() | youtube-upload/en/audio.m4a, captions.srt, title.txt |
The output MP3, the dubbed MP4 and the speaker-labelled SRT are all in the repo, and you can play them on the project's Hugging Face demo page. It's a short clip, so treat it as proof that the pipeline works, not as a quality benchmark. Test it on your own footage.
How this compares with other approaches
This comparison is taken from the project README (November 2026). Corrections are welcome there.
| YouTube Dubbing MCP | elevenlabs-mcp | Auto-Synced-Translated-Dubs | YouTube auto-dubbing | |
|---|---|---|---|---|
| Runs from an AI agent (MCP) | Yes | Yes | No (Python scripts) | No |
| Dedicated dubbing workflow | Yes (12 tools) | No dubbing tool (TTS, STT, cloning) | Yes | Yes |
| One cloned voice per speaker | Automatic, from the video | Manual | No (stock voices) | No |
| Input from a YouTube URL | Yes | No | No (needs SRT) | Own channel only |
| Agent edits translation before voicing | Yes, with time budgets | — | Manual | No |
| License | MIT | MIT | AGPL-3.0 | Proprietary |
When not to use this:
- You want a point-and-click editor with no agent involved: a hosted dubbing studio will suit you better.
- You need a language the voice engine doesn't speak. Miragea currently covers 10: Chinese, English, Japanese, Korean, German, French, Russian, Portuguese, Spanish and Italian.
Tips for better dubs
- Clean source audio matters most. Background music under speech makes worse voice samples. If you can, start from a version without music.
- Ask the agent for shorter, spoken-style translations. Written-style translations rarely fit the timing.
- Check names and numbers in the SRT before you render. They're the most common errors.
- Get consent from everyone whose voice you clone, and say in the video description that the audio track is AI-dubbed.
FAQ
Can AI dubbing keep the original speaker's voice?
Yes. This workflow cuts a few seconds of each speaker from the original audio and clones it, so each person is dubbed in their own voice.
Can I upload multiple audio tracks to YouTube through the API?
No. The YouTube Data API has no endpoint for extra audio tracks. You add them in YouTube Studio → Languages, and this tool prepares the exact files for that.
Does it need a GPU?
No. Voices are rendered by a hosted voice engine. You only need internet access and an API key.
Can I dub from an SRT instead of a video?
Yes: dubbing_start_project(source="talk.srt"). Then pick the voices with dubbing_set_voice.
Is it free?
The code is free and MIT-licensed. Voicing and transcription use the Miragea API, so check its current plans.
Source code, issues and examples: github.com/RTY2345RR/youtube-dubbing-mcp. Related: tts-mcp-server (script → voiceover + SRT/VTT) and miragea-agent-skills (Claude Code skills for voiceovers, podcasts and dubbing).
