AIBit-Discover Open Source Projects AIBit-Discover Open Source Projects

VoiceStudio: A Fully Local, Open-Source ElevenLabs Alternative

VoiceStudio brings voice cloning, dubbing, dictation, transcription, and audiobooks to your hardware with 16 TTS and 11 ASR engines.

VoiceStudio: A Fully Local, Open-Source ElevenLabs Alternative

Cloud voice platforms are convenient, but they also introduce recurring costs, usage limits, account requirements, and a data path that may be unsuitable for private recordings. VoiceStudio takes the opposite approach: it is an open-source desktop and local-server application for voice cloning, voice design, video dubbing, dictation, transcription, and long-form audio production.

The project supports 16 text-to-speech engines, 11 speech-recognition engines, and a catalogue covering up to 646 TTS languages. It runs on macOS, Windows, Linux, and Docker, with support for CUDA, Apple Silicon MPS/MLX, Linux ROCm, CPU inference, and optional remote workers.

The result is less like a single voice-generation model and more like a local speech production platform. You can select engines per task, keep project data on disk, expose an OpenAI-compatible API, or connect AI agents through MCP.

VoiceStudio is in active beta. Use the latest release for stable work; the main branch may change between releases.

Why local voice generation matters

A hosted voice service typically manages models, GPUs, scaling, and updates for you. That makes setup easy, but it also means audio and text are processed by a provider and that your costs scale with usage. VoiceStudio moves those trade-offs to your own machine:

Concern VoiceStudio Typical hosted service
Data path Local by default; remote features are opt-in Audio and text are processed by the provider
Cost Free software; you provide the hardware Subscription, credits, or metered API use
Offline use Yes, after models are installed Usually requires an internet connection
Customization Open source, multiple engines, local routing Limited to provider options
Performance Depends on your hardware and engine Provider manages compute and scaling
Maintenance You manage models, updates, disk, and compute Provider manages infrastructure

This makes it especially useful for private recordings, internal applications, high-volume generation, offline workflows, and developers who need control over model selection.

What you can build with VoiceStudio

VoiceStudio groups its functionality into several production workflows.

Voice cloning and design

The Voice workspace includes three primary modes:

  • From audio — clone a voice from a short reference clip.
  • By design — create a voice from attributes such as age, accent, pitch, style, and delivery.
  • Convert — perform speech-to-speech conversion.

For zero-shot cloning, a three-second clip can work, while 5–15 seconds generally provides a better prompt. The reference should contain one speaker and minimal noise, music, or reverberation. Longer audio is not automatically better because the clip acts as a prompt rather than training data.

A typical first generation looks like this:

  1. Launch VoiceStudio and open Voice Cloning.
  2. Add a clean reference recording.
  3. Enter the text to synthesize.
  4. Select the language and voice profile.
  5. Choose Generate.

Voice cloning should only be performed with the speaker’s explicit permission.

Video dubbing

The dubbing workflow can transcribe a video, translate its dialogue, preserve speaker assignments, synthesize new speech, and export a finished video. It supports file uploads and URL imports, captions, optional YouTube sign-in, transcript editing, glossaries, track selection, and timing review.

The editor combines a waveform and timed transcript in one workspace. Click the waveform to seek, or drag a zoomed waveform to pan. Translation language and ISO-code controls stay synchronized, while transcript segments expose text, timing, status, and voice controls.

Completed dubs can flag timing problems for review before export. Clone-less engines are rejected for workflows that require preserving a reference speaker instead of silently switching to another engine.

Audiobooks and long-form audio

The audiobook editor supports multi-voice scripts, EPUB and PDF import, chapter rendering, and .m4b export. The script editor occupies the main workspace while voice assignments and book settings remain in separate tabs, making it practical to manage longer projects without mixing production settings with text editing.

Dictation and transcription

VoiceStudio includes a system-wide dictation widget with a global shortcut, live transcription, and optional local-LLM cleanup. For speech recognition, users can choose among WhisperX, Faster-Whisper, MLX Whisper, Parakeet, Moonshine, FunASR, Sherpa-ONNX, and configured OpenAI-compatible servers.

WhisperX is the default choice for dubbing, subtitles, word-level timing, and diarization. Apple Silicon users can use MLX Whisper or Parakeet MLX, while CPU-only systems can use Moonshine or int8 Faster-Whisper for lower memory usage.

Additional media tools

The platform also includes:

  • Vocal isolation with Demucs
  • Speaker diarization with Pyannote and WhisperX
  • Batch queues for audio and video jobs
  • Folder watching for newly added videos
  • AI watermark embedding and detection through AudioSeal
  • Remote model downloads for enrolled workers
  • A model catalogue for installing, removing, and routing engines

Installation options

The project provides packaged releases for macOS, Windows, and Linux, as well as Docker images.

Platform Package or requirement
macOS Apple Silicon, macOS 13.3+, DMG
Windows Windows 10/11 x64, MSI; current-user installation is available
Linux x86_64 with glibc 2.39+, AppImage
Docker Linux/AMD64 images with CUDA, ROCm, CPU, and worker-only GPU profiles

On first launch, VoiceStudio creates a managed Python environment and downloads the default model. Subsequent launches reuse that environment and model cache.

For Docker, the published images are linux/amd64 only:

docker run -d \\
  -p 127.0.0.1:3900:3900 \\
  -v omnivoice-data:/app/omnivoice_data \\
  --name voicestudio \\
  palashdeb/omnivoice-studio:stable

Apple Silicon users should generally use the native macOS application for GPU acceleration. ARM64 hosts should verify the architecture requirements before pulling a container image.

Running from source

Development requires Node 20 or later, Bun, and Python 3.11 or later:

git clone https://github.com/debpalash/VoiceStudio.git
cd VoiceStudio
bun install
bun run desktop

The desktop launcher configures Python dependencies automatically through uv. To run the browser UI instead:

bun run dev

If startup fails, run the built-in diagnostics from Settings → About → Run self-check, or execute:

uv run python backend/main.py --diagnose --deep

Hardware and engine selection

VoiceStudio supports CPU operation, but a GPU can substantially improve generation speed. The project lists these baseline requirements for its default local workflow:

Resource Minimum Recommended
RAM 8 GB 16 GB or more
Free disk 10 GB 20 GB or more on an SSD
VRAM 4 GB for GPU use 8 GB or more
Python 3.11+ 3.11 or 3.12

Large optional engines may require 12–16 GB of VRAM or more. ROCm is Linux-only and opt-in; Windows AMD/Ryzen AI systems use CPU inference. Systems with limited VRAM can offload work to the CPU.

Recommended combinations include:

  • Apple Silicon M1–M4: MLX-Audio or OmniVoice with MPS; MLX Whisper or Parakeet MLX for ASR.
  • NVIDIA GPU with 8 GB+ VRAM: OmniVoice or CosyVoice 3; WhisperX for transcription and diarization.
  • Low-VRAM or CPU-only: PocketTTS, Sherpa-ONNX, or KittenTTS; Moonshine or int8 Faster-Whisper for ASR.

The engine catalogue includes OmniVoice, CosyVoice 3, GPT-SoVITS, VoxCPM2, MOSS-TTS, KittenTTS, MLX-Audio, Sherpa-ONNX, IndexTTS, PocketTTS, Supertonic, dots.tts, and Confucius4-TTS. Capabilities differ significantly: some support cloning and instruction-based synthesis, while others are CPU-focused or limited to particular languages.

Always verify an engine’s language coverage, platform support, memory requirements, cloning capability, and model license before using it in production.

Local API and OpenAI compatibility

The desktop application communicates with a FastAPI backend bound to localhost:3900. The backend exposes REST, Server-Sent Events, WebSockets, an OpenAI-compatible audio API, and an MCP server.

Change an OpenAI client’s base URL to the local backend:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:3900/v1",
    api_key="local",
)

with client.audio.speech.with_streaming_response.create(
    model="tts-1",
    voice="<profile-id>",
    input="Made on my own hardware.",
    response_format="wav",
) as response:
    response.stream_to_file("speech.wav")

A cURL request is equally simple:

curl http://localhost:3900/v1/audio/speech \\
  -H "Content-Type: application/json" \\
  -d '{
    "model": "tts-1",
    "input": "Made on my own hardware.",
    "voice": "default",
    "response_format": "wav"
  }' \\
  --output speech.wav

Important endpoints include:

  • POST /v1/audio/speech — generate MP3, Opus, AAC, FLAC, WAV, or PCM audio.
  • POST /v1/audio/transcriptions — return JSON, text, verbose JSON, SRT, or VTT.
  • WS /v1/audio/transcriptions/stream — stream partial and final transcription events.
  • GET /v1/audio/voices — list local voice profiles and engines.
  • GET /.well-known/voicestudio-speech — discover available speech transports.

Loopback calls do not require a server key. Remote access requires a share PIN or API key, and LAN, Tailscale, or proxy deployments should follow the project’s authentication guidance.

MCP and agent integrations

VoiceStudio includes an MCP server at http://localhost:3900/mcp, allowing Claude Desktop, Cursor, and other MCP clients to call synthesis and transcription tools.

{
  "mcpServers": {
    "voicestudio": {
      "url": "http://localhost:3900/mcp"
    }
  }
}

For clients that require stdio transport, the repository includes a local shim:

{
  "mcpServers": {
    "voicestudio": {
      "command": "python",
      "args": ["-m", "backend.mcp_shim"],
      "cwd": "/path/to/VoiceStudio"
    }
  }
}

The MCP tools include generate_speech, clone_voice, and transcribe, with support for file streaming modes and client bindings. Project skills for Claude Code, Codex, Cursor, and compatible agents can be installed with:

npx skills add debpalash/VoiceStudio

Privacy, security, and licensing

The default desktop workflow keeps recordings, transcripts, voices, projects, settings, and outputs on the local machine. Remote workers and OpenAI-compatible ASR endpoints are explicit opt-ins. Non-loopback endpoints require HTTPS, and redirects are not followed.

Analytics is disabled until consent. If enabled, it sends allowlisted, content-free usage metadata—not text, audio, filenames, or project data.

VoiceStudio is licensed under AGPL-3.0. You may run, modify, and use it internally. If you modify the application and provide it as a network service, AGPL obligations require you to offer the corresponding source under the same license. A commercial license is available for proprietary embedding of VoiceStudio-owned code, but it does not relicense third-party models.

Model and tokenizer terms are separate. For example, the default OmniVoice setup includes pretrained weights labeled CC-BY-NC and an audio tokenizer with additional upstream terms. The application license does not automatically grant commercial rights to every downloaded model.

The project also integrates AudioSeal watermarking by default to help detect and identify synthetic speech without changing its audible quality.

Who should use it?

VoiceStudio is a strong fit when you need:

  • Private or offline speech generation
  • High-volume synthesis without per-request charges
  • Local voice cloning and multilingual dubbing
  • A desktop workflow plus programmable APIs
  • Multiple TTS and ASR engines in one application
  • Integration with coding agents through MCP
  • Control over model routing, hardware, and storage

It is less suitable if you want zero-configuration cloud scaling, do not have enough local disk or compute, or need a single provider with unified commercial licensing for every model.

With 26.9k GitHub stars, 3.3k forks, 38 releases, and an active development history, VoiceStudio has grown beyond a simple voice-cloning demo. Its combination of desktop tools, local APIs, model routing, dubbing workflows, and agent integrations makes it a practical foundation for developers building private speech applications on their own hardware.

Source

debpalash/VoiceStudio: VoiceStudio is the open-source, fully-local ElevenLabs alternative — voice cloning, voice design, video dubbing, dictation, transcription & audiobook creation in 646 languages.