media-dub
Use when producing a local DUB of a video — re-voicing the speech into another language (or restyling it in the same language) while KEEPING the original speaker's voice, fully offline. The safe-ship engine is Chatterbox-Multilingual (Resemble AI, MIT, 23-language zero-shot voice clone); the pipeline rides the existing media_studio substrate (Parakeet/whisper ASR -> CTC word-alignment -> ffmpeg `atempo` duration-fit -> `sidechaincompress` ducking -> mux). Covers the `dub.start` RPC, the ASR->translate->synth->fit->assemble->duck+mux stages, the per-segment `atempo` time-stretch that lands each clip in its source slot, the full-length VO track assembled with `adelay`+`amix`, and the LICENSE gate: an optional IndexTTS2 native-duration-control engine and an optional LatentSync lip-sync are COMMERCIAL-embed-gated on the `dub-license` needsDecision — the gate runs in SHADOW mode (observe + log, never blocks), and a commercial build defaults to Chatterbox-only unless the owner explicitly opts in (`allowGated`). Do NOT use for: pure TTS narration with no source voice to clone (use the `speech` skill); muxing an ALREADY-finished dub track (use audio-extract-mix); transcription/subtitle generation only (use transcribe/caption); or lip-sync without a dub. Source of truth: media_studio/features/dub.py.
- License 许可
- MIT
- 最近更新
- 2026-11-02
- Stars
- ★ 0
- Forks
- 0
01 前言 Front Matter
- name
- media-dub
- description
- Use when producing a local DUB of a video — re-voicing the speech into another language (or restyling it in the same language) while KEEPING the original speaker's voice, fully offline. The safe-ship engine is Chatterbox-Multilingual (Resemble AI, MIT, 23-language zero-shot voice clone); the pipeline rides the existing media_studio substrate (Parakeet/whisper ASR -> CTC word-alignment -> ffmpeg `atempo` duration-fit -> `sidechaincompress` ducking -> mux). Covers the `dub.start` RPC, the ASR->translate->synth->fit->assemble->duck+mux stages, the per-segment `atempo` time-stretch that lands each clip in its source slot, the full-length VO track assembled with `adelay`+`amix`, and the LICENSE gate: an optional IndexTTS2 native-duration-control engine and an optional LatentSync lip-sync are COMMERCIAL-embed-gated on the `dub-license` needsDecision — the gate runs in SHADOW mode (observe + log, never blocks), and a commercial build defaults to Chatterbox-only unless the owner explicitly opts in (`allowGated`). Do NOT use for: pure TTS narration with no source voice to clone (use the `speech` skill); muxing an ALREADY-finished dub track (use audio-extract-mix); transcription/subtitle generation only (use transcribe/caption); or lip-sync without a dub. Source of truth: media_studio/features/dub.py.
03 SKILL.md 正文
media-dub — local voice-clone dubbing (Chatterbox safe ship)
Distilled from the REAL implementation. Cite these when in doubt:
media_studio/features/dub.py— the whole feature: thedub.startRPC, theDubLicenseGate(SHADOW),build_dub_plan, theatempofit, the VO-timeline and duck/mux argv builders, andrun_dub_pipeline(the end-to-end orchestrator).- Substrate it RIDES (do not reinvent) — from the media_studio package
(
sidecar/media_studio/features/in the Reframe checkout):transcribe.py/parakeet_asr.py— ASR to the §3Transcriptschema.ctc_align.py— karaoke-grade word-timing refinement (optional 2nd pass).audiomix.py— thesidechaincompress->amix->loudnormduck/mux graph (build_mix_filter); the dub mux uses the identical convention + labels.
Evidence: gem-hunt MASTER E.3 — "Local dubbing stack — biggest missing flagship feature: Chatterbox-Multilingual (MIT, 23-lang clone) + IndexTTS2 duration-control
- optional LatentSync lip-sync; the whole substrate (Parakeet+CTC+ducking+mux) exists. [verify IndexTTS2 weights license before commercial embed; Chatterbox alone is the safe ship.]" · COUNCIL C5 (risk-tiered SHADOW ladder — a heuristic/nagging/blocking gate ships observe-and-log FIRST).
Wire surface (NET-NEW)
dub.start({videoId|path, targetLang, sourceLang?, engine?, durationFit?,
lipsync?, posture?, allowGated?, bgPath?})
-> {jobId} -> {path, engine, durationFit, lipsync, targetLang,
### Pipeline (ride the bus)
### License gate — SHADOW mode (the crux)
### Chatterbox's 23 languages
### Assets to install (follow-on, out of the single-file build)
### Pitfalls that actually bite
### When NOT to use
04 同仓库更多 Skills
clarify-requirements
Resolve genuine pre-execution ambiguity by scanning repo context, generating 1-6 grounded options, and asking 1-3 focused questions via AskUserQuestion. NOT for mid-task use (injection risk); NOT when the goal is already unambiguous; use to disambiguate vague prompts before any file edits.
openmm-md
Run FREE, open-source, Windows-native GPU molecular dynamics via the openmm-md MCP (OpenMM + CUDA on the local NVIDIA GPU). Use for protein / protein-ligand MD when you want a free stack or are on Windows without a Schrodinger license. Covers the full workflow (prepare, solvate, minimize, equilibrate, production, analyze, mmgbsa) AND an honest decision guide vs Schrodinger Desmond, including how to run Desmond legitimately on Linux or a cloud GPU with a real license. This is a deliberately LESSER free alternative and does NOT replace the schrodinger MCP tools. Do NOT use it to pick a free tool when a validated FEP+, WaterMap, IFD-MD, or Prime MM-GBSA result is actually required.
visual-review
Take screenshots of web pages and UI using Playwright for visual review and iteration
status
Diagnostic status report — shows metaswarm installation state, project setup, and potential issues
start
Use when starting work on any task, when the user mentions metaswarm, or when the user wants to begin tracked development work
setup
Interactive project setup — detects your project, configures metaswarm, writes project-local files