ComfyUI MiniMax H3 Timeline Director for Long-Form Video

Generating a short clip from a reference image is relatively straightforward. Generating a coherent, synchronized video that runs for a minute or longer is much harder: identity drifts, audio seams become noticeable, and every extension often requires another hand-built sampler chain.

ComfyUI-MiniMaxH3-TimelineDirector addresses that workflow problem with an editable timeline director for ComfyUI’s native MiniMax H3 Reference to Video pipeline. It combines reference images, videos, paired soundtracks, guides, prompts, and generation windows in one interface, then expands the timeline into a segmented generation graph.

The project is released under GPL-3.0 and, at the referenced release, has 275 stars, 31 forks, and a single contributor. Version 0.6.0 focuses on lightweight unlimited-length generation and adds long-form digital humans and character singing with locked source audio.

Why the timeline matters

The plugin treats generation as an arrangement problem rather than a collection of disconnected inputs. You can position clips, trim them, split them, define numeric ranges, and establish cyan generation windows. Only media that intersects the active generation range is included in the current H3 reference or Guide plan.

That makes it possible to use one workflow for several otherwise different tasks:

• Text-to-video with no uploaded media.
• Single- and multi-segment image-reference generation.
• Audio-driven video and digital-human lip sync.
• Character replacement and motion transfer.
• Editable video references.
• Manually segmented long-video generation.
• Long-form character singing with an unchanged soundtrack.

The timeline state is serialized into the ComfyUI workflow JSON, so a project can be saved, reopened, and reproduced without reconstructing the media arrangement manually.

The headline feature: segmented, unlimited-length generation

The plugin divides a target duration into overlapping segments and completes the process in one ComfyUI execution. Instead of relying on generic Loop nodes or duplicated sampler chains, it uses a finite expansion graph that performs:

1. Per-segment generation with one shared seed.
2. Direct continuation from the previous segment’s sampled AV-latent tail.
3. Adaptive Drift-Control video masking.
4. Soft AV audio continuity.
5. Overlap removal.
6. Final synchronized assembly.

The practical duration limit is determined by local VRAM, RAM, disk capacity, and ComfyUI execution limits—not by a hardcoded number of segments. The repository includes two examples generated in one execution, each approximately 52.625 seconds long at 1,263 frames and 24 fps.

Why direct latent continuation is useful

A conventional approach might decode the previous segment to RGB, feed that result into the next stage, and encode it again. That introduces an unnecessary decode/re-encode round trip and can amplify visual discontinuities. Finite Segment Sampling instead carries the previous sampled AV latent tail directly into the next opening.

The requested overlap is aligned down to MiniMax H3’s legal temporal grid. For example, a requested overlap of 24 frames may become 22, while 48 may become 39. The aligned value is used consistently for latent carry, decoded trimming, and final assembly, preventing different stages from disagreeing about the seam.

Drift-Control adapts its mask to both the aligned overlap’s video-token count and the connected sampler’s sigma schedule. This includes accelerated four-step and eight-step schedules. Only the disposable video prefix is dynamically re-noised, while the seam-side latent remains clean.

For audio continuation, the carried overlap remains exact until the final eight audio-latent ticks. A half-cosine Soft AV mask then gradually releases the overlap into newly generated audio. During assembly, the incoming Soft AV overlap replaces the preceding audio tail so the transition remains present in the final soundtrack.

Every segment uses the exact seed displayed on the sampling node. The older generic-loop helper nodes and the PR #15923 dependency have been removed.

Locked audio for long-form digital humans

One of the most useful additions is support for a long soundtrack that must remain unchanged across multiple generation windows. A reference image preserves the character’s identity, while the locked soundtrack is decoded once, sliced by exact sample position, and injected into every segment’s AV latent.

The audio denoise mask is set to zero for the locked source, allowing the original signal to pass through rather than being regenerated independently for each segment. The final output restores the continuous soundtrack after segment assembly.

Automatic fitting removes a common source of failure:

• If the timeline is shorter than the source audio, the source is truncated.
• If the timeline extends beyond the source, all available audio is preserved and only the excess is padded with silence.
• Users do not need to match MP3 metadata, H3-valid durations, or codec priming delays manually.

This enables long-form lip sync and character singing workflows that are much easier to control than independently generated audio per segment. Bound source audio can follow video edits and can also be disabled independently.

Reference modes

Each video clip can be assigned one of three purposes:

Fixed Guide

The clip anchors the overlap at its generated-frame positions. This is useful when the source motion or framing should be treated as a strong structural guide.

Editable Reference

The clip is passed as a numbered H3 video reference without hard-locking the original subject. This is generally the preferred mode for character replacement and flexible reference-driven generation.

Boundary Only

Only the first and last overlap frames are used as anchors. This is useful when the endpoints should remain stable but the generated interior should be allowed to change.

Video soundtracks can be enabled or disabled independently. Standalone images and audio files are also supported, with stable <Picture N>, <Video N>, and <Audio N> ordering from the UI through to the H3 inputs.

Prompt behavior and planning

The primary workflow exposes both a Global Prompt and per-segment prompts. The global prompt is reused only when all segment prompts are empty. Once any segment prompt is entered, every segment must have a prompt, and the global prompt is disabled.

This rule prevents an easy-to-miss ambiguity where some segments inherit global instructions while others silently override them. It is especially important for long-form generation, where each segment should describe the overlap as the opening shot before introducing new content.

The project also includes an optional Omni prompt bridge. The MiniMax H3 Omni Media-Bundle Prompt Bridge sends the ordered image, video, and audio bundle to an installed Prompt Rewriter Omni backend and returns only rewritten_prompt. This variant is useful when multimodal material understanding should happen before H3 prompt expansion. It requires MiniMax-H3-Prompt-Rewriter-ComfyUI and its associated model, quantization, and VRAM setup.

Core nodes

The recommended architecture uses two main nodes:

• MiniMax H3 Material Planner — edits media and produces a compact H3 plan plus an ordered Omni media bundle.
• MiniMax H3 Finite Segment Sampling — expands the plan into continuation, masking, sampling, deduplication, and assembly operations.

The optional MiniMax H3 Omni Media-Bundle Prompt Bridge handles prompt rewriting. MiniMax H3 Timeline Director (Compatibility) preserves the original all-in-one workflow and supports older saved workflows.

Long-video generation needs only:

Material Planner Segment Plan -> Finite Segment Sampling

The Plan Encoder remains registered as a hidden internal node for expanded execution graphs and backward compatibility. This split architecture also avoids a ComfyUI dependency cycle.

Installation

Clone the repository into ComfyUI’s custom node directory:

cd ComfyUI/custom_nodes
git clone https://github.com/Songssx/ComfyUI-MiniMaxH3-TimelineDirector.git

Restart ComfyUI and search for MiniMax H3.

The source UI is English. Simplified Chinese is supplied through ComfyUI’s official localization system and follows the language selected in ComfyUI settings. Reload or restart the frontend after changing the locale.

Requirements

You need:

• A recent ComfyUI build with native MiniMax H3 nodes.
• MiniMaxH3AddGuide when using Guides.
• MiniMax H3 Ref2VA model, CLIP, video VAE, and audio VAE.
• Python 3.10 or newer.
• ComfyUI’s imageio-ffmpeg package for low-resolution preview proxies.

No additional pip dependency is declared. The plugin relies on packages normally present in a compatible ComfyUI installation, including PyAV, Pillow, NumPy, PyTorch, torchaudio, aiohttp, and imageio-ffmpeg.

For VRAM protection, preview proxies can be generated silently at up to 480×270 and 12 fps. Decode-time resizing also scales output to the node’s configured width and height.

A practical workflow

1. Open the All-in-One Full Timeline Director workflow.
2. Set the width, height, and generation_seconds.
3. Add images, videos, and audio from the toolbar or by dropping files into the UI.
4. Move, trim, or split clips as needed.
5. Drag the cyan range over the interval to generate.
6. Assign each video a mode: Fixed Guide, Editable Reference, or Boundary Only.
7. Decide whether paired video audio should be included.
8. Add a global prompt or complete prompts for every segment.
9. For long-form generation, create overlapping GEN windows and choose the desired seam overlap.
10. Connect the Material Planner’s Segment Plan directly to Finite Segment Sampling.

The plugin handles legal-frame alignment, Drift-Control, Soft AV, overlap removal, tail trimming, and final AV assembly. It does not automatically segment media based on reference duration or infer whether character identity should continue. Segment ranges, overlaps, and assignments remain explicit user decisions.

For long-video references, use the previous segment’s final shot as the next segment’s opening Guide. Describe that overlap as Shot 1 before introducing new content, then remove the repeated Guide interval from the later segment during assembly.

Related work and licensing

Drift-Control AV is adapted from ComfyUI-MiniMaxH3-Contex-Loop under GPL-3.0 and remains experimental, particularly for same-shot long-chain comparisons. The Omni bridge and prompt-generation workflow reference and adapt pytraveler/MiniMax-H3-Prompt-Rewriter-ComfyUI.

The project also credits the maintainers of ComfyUI’s native MiniMax H3 and Guide nodes and points users to MiniMax-AI/MiniMax-H3 for official model guidance.

For developers already using ComfyUI’s MiniMax H3 nodes, Timeline Director is valuable because it turns a one-off reference-to-video graph into a reusable editing and continuation system. Its strongest idea is not merely adding another node, but making media selection, segment boundaries, overlap policy, prompt scope, audio continuity, and final assembly explicit parts of one reproducible workflow.

Source

Songssx/ComfyUI-MiniMaxH3-TimelineDirector: Editable reference-media timeline director for ComfyUI MiniMax H3 Reference to Video