StemDeck: Free, Local 6-Stem AI Audio Separation with Demucs and Tauri

Cloud-based stem splitters like Moises, LALAL.AI, and Splitter.ai have made track deconstruction accessible for musicians, remixers, and producers. However, they almost always come with restrictive usage quotas, privacy concerns regarding unreleased audio, and ongoing monthly subscriptions.

StemDeck addresses these issues by delivering a local-first audio separation studio. It extracts up to six distinct stems (vocals, drums, bass, guitar, piano, and other) directly on your machine with zero cloud dependencies, no accounts, and no data upload.

The Architecture Behind StemDeck

StemDeck combines high-performance ML models with an accessible web-native studio UI, wrapped in a lightweight cross-platform desktop shell.

+-------------------------------------------------------------+
|                 StemDeck Desktop (Tauri v2)                 |
|  +-------------------------------------------------------+  |
|  |            Frontend (Vanilla JS + Web Audio)          |  |
|  |   - Multi-lane Canvas Waveforms  - Per-stem VU Meters |  |
|  |   - Loop Regions & Scrubbing     - Live Mixer Bus     |  |
|  +-------------------------------------------------------+  |
|                             │ SSE / REST API                |
|  +──────────────────────────▼────────────────────────────+  |
|  |                 FastAPI Backend (Python 3.12)         |  |
|  |   - Demucs (htdemucs_6s) [CUDA / MPS / CPU]           |  |
|  |   - UVR-MDX-NET Karaoke 2 (Lead/Backing Vocal Split)  |  |
|  |   - Librosa (BPM & Key) | Pyloudnorm (BS.1770 LUFS)   |  |
|  |   - FFmpeg Transcoding & Custom Mixdown Engine        |  |
|  +-------------------------------------------------------+  |
+-------------------------------------------------------------+

Separation Engine: Leverages Meta AI’s htdemucs_6s model via PyTorch. It automatically detects and binds to NVIDIA CUDA on Linux/Windows, Apple Silicon MPS on macOS, or falls back gracefully to CPU threads.
Vocal Splitting: Supports an optional secondary pass using the UVR-MDX-NET Karaoke 2 model (via audio-separator) to isolate lead and backing vocals.
Audio Analysis: Automatically extracts track metadata, including BPM (via librosa), key and scale confidence (via Albrecht-Shanahan pitch profiles), and integrated loudness measurements (via pyloudnorm ITU-R BS.1770).
Frontend & DAW Mixer: Built without heavyweight frontend frameworks. It uses vanilla JavaScript with HTML5 <canvas> min/max sample rendering and the Web Audio API for post-gain RMS monitoring, solo/mute logic, and phase-accurate loop playback.
Desktop Shell: Powered by Tauri v2 (Rust + native system WebViews: WKWebView on macOS and WebView2 on Windows).

Core Capabilities

6-Stem Isolation: Drag-and-drop support for standard uncompressed and compressed formats (MP3, WAV, FLAC, OGG/Opus, MP4, M4A) or direct YouTube URLs.
Smart Subset Extraction: Select a subset of stems (e.g., just Bass and Drums) and StemDeck automatically creates an Original backing track lane containing the complementary mix (full mix minus your selection), avoiding accidental doubling.
Non-destructive In-Browser Mixing: Real-time gain control, solo, mute, and monitor controls per stem track with live VU metering.
Instant Mix Export: Generates combined mix.wav stems on the fly through FFmpeg amix or individual isolated audio files.

Quickstart & Installation

1. Native Desktop Binaries

Pre-built binaries for macOS (Apple Silicon arm64 and Intel x64) and Windows (CPU or pre-packaged CUDA bundles) are available directly on GitHub Releases. On first launch, the app sets up the required environment and downloads model weights (~170 MB) into a local data directory.

2. Running Locally with Python and UV

If you prefer running the web server directly:

# Clone the repository
git clone https://github.com/stemdeckapp/stemdeck && cd stemdeck

# macOS / Linux automated setup (requires ffmpeg and uv)
./run.sh setup
./run.sh start

For Windows PowerShell with an NVIDIA GPU:

# Install dependencies with uv
uv sync
uv pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124

# Set device and launch the FastAPI server
$env:STEMDECK_DEMUCS_DEVICE = "cuda"
uv run uvicorn app.main:app --host 127.0.0.1 --port 8000 --timeout-graceful-shutdown 5

3. Docker Deployment

Run the official GHCR container with GPU passthrough enabled:

docker run -d --name stemdeck \
  -p 8000:8000 \
  --runtime=nvidia \
  -e NVIDIA_VISIBLE_DEVICES=all \
  -e STEMDECK_PERSIST_LIBRARY=1 \
  -v /local/jobs:/app/jobs \
  -v /local/cache:/cache \
  ghcr.io/stemdeckapp/stemdeck:edge

REST API & Job Workflow

StemDeck exposes a clean REST interface alongside Server-Sent Events (SSE) for job monitoring:

Method	Endpoint	Description
`POST`	`/api/jobs`	Submit a local file multipart upload or `{ "url": "...", "stems": [...] }`
`GET`	`/api/jobs/{id}/events`	SSE stream emitting job progress (download, analyze, separate, mix)
`GET`	`/api/jobs/{id}/stems/{stem}.wav`	Stream specific stem audio (`vocals.wav`, `bass.wav`, `drums.wav`, etc.)
`POST`	`/api/jobs/{id}/cancel`	Terminate active subprocess pipelines immediately and prune scratch disk
`PATCH`	`/api/jobs/{id}/sections`	Save marker boundaries and waveform annotations

Summary

If you need heavy mobile integrations or proprietary 10-stem separation models, commercial SaaS platforms remain an option. But for producers, transcriptionists, and musicians who want complete privacy, zero usage caps, and local offline processing, StemDeck offers an impressive open-source DAW-style workflow.

Source

stemdeckapp/stemdeck: Stemdeck is an modern stem extraction platform for musicians,producers and hobbyists, designed to isolate vocals, drums, bass, piano and guitar for practice, transcription, remixing, and creative audio workflows through a modern and interactive interface