diff --git a/apps/ComfyUI-vLLM-Omni/README.md b/apps/ComfyUI-vLLM-Omni/README.md index b848a0716..8f445635d 100644 --- a/apps/ComfyUI-vLLM-Omni/README.md +++ b/apps/ComfyUI-vLLM-Omni/README.md @@ -139,6 +139,80 @@ You can configure per-stage sampling parameters for multi-stage models. > > Do not use `frame` and `references` together. Task routing is automatic from which inputs you connect. +For MiniMax-H3 Ref2VA, **Video References** accepts up to 9 images (`image_1`–`image_9`), +3 videos (`video_1`–`video_3`), and 3 audio clips (`audio_1`–`audio_3`), with at most +12 connected inputs in total. Any mixture containing at least one image or video +is supported; empty and audio-only references are rejected. For example, you can +combine 6 images, 3 videos, and 3 audio clips in one request. + +Within each media type, references follow slot-number order, skipping unconnected +slots. For example, connecting `image_2` and `image_9` sends `image_2` as the first +image and `image_9` as the second. Each image slot uses the first image in its batch. +Existing connections to `image_1`, `image_2`, `audio_1`, `audio_2`, `video_1`, and +`video_2` remain valid in saved workflows. + +#### MiniMax-H3 Reference to Video + +Load [MiniMax-H3 Reference to Video](example_workflows/vLLM-Omni%20MiniMax-H3%20Reference%20to%20Video.json) +from **Templates → ComfyUI-vLLM-Omni**, or drag the JSON onto the canvas. +It adapts the [official ComfyUI R2V workflow](https://github.com/Comfy-Org/workflow_templates/blob/main/templates/video_minimax_h3_r2v.json) +to remote vLLM-Omni execution. Model loading, sampling, and VAE decoding run on +the server; ComfyUI loads references and saves the returned video. + +Use an extension version containing the audio-preserving output fix from +[#7456](https://github.com/vllm-project/vllm-omni/pull/7456). Older versions can +drop the server's audio while decoding the response. + +1. Start a Ref2VA-capable service using the [MiniMax-H3 recipe](../../recipes/MiniMaxAI/MiniMax-H3.md). + Set **Generate Video** to its `/v1` URL and served model name. +2. Select your own image in **Load Image**. The template's filenames are placeholders + relative to ComfyUI's input directory; no sample assets or model weights are bundled. +3. Connect **Load Video** and **Load Audio** when needed. Duplicate the loaders to + fill more reference slots, or disconnect the image when using video-only input. + Keep the references connected to **Generate Video** and leave `frame` disconnected. +4. Match prompt tags such as ``, `