anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

[Frontend] Add MiniMax H3 video upscale workflow (WF-07) (#7472)

Browse Source 
Signed-off-by: xuezhihao <89690814+xuexueligao@users.noreply.github.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
This commit is contained in:
xuexueligaoand汪志鹏 authored and GitHub committed 2026-09-15 03:46:13 -07:00
1 parent 92715f3f7b
commit 4a8ac29775
3 files changed
+749
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

apps/ComfyUI-vLLM-Omni/README.md
+4
View File
@@ -139,6 +139,10 @@ You can configure per-stage sampling parameters for multi-stage models.
>
> Do not use `frame` and `references` together. Task routing is automatic from which inputs you connect.
#### H3 video upscale (WF-07)
The **vLLM-Omni MiniMax H3 Video Upscale** template generates video remotely, upscales it with SeedVR2, and saves the original and upscaled videos with the generated audio and FPS. See [workflow setup](docs/wf07-h3-upscale.md).
#### FastH3 text-to-video
FastH3 is fused into MiniMax-H3 when the vLLM-Omni server starts; it is not a request-switchable LoRA. Download one adapter variant and start a non-offloaded FL2VA server. For the Dense / Data-Free profile:
apps/ComfyUI-vLLM-Omni/docs/wf07-h3-upscale.md
+21
View File
@@ -0,0 +1,21 @@
# WF-07: H3 video upscale with SeedVR2
Generate video and audio through vLLM-Omni H3, upscale the decoded frames with SeedVR2, and save both the original and upscaled videos. The generated audio and FPS connect directly to Create Video.
## Requirements
- ComfyUI with ComfyUI-vLLM-Omni and [ComfyUI-SeedVR2_VideoUpscaler](https://github.com/numz/ComfyUI-SeedVR2_VideoUpscaler#-installation) installed.
- An H3 service using the FL2VA checkpoint partition. Follow the [H3 recipe](../../../recipes/MiniMaxAI/MiniMax-H3.md) for server setup.
- `seedvr2_ema_3b_fp16.safetensors` and `ema_vae_fp16.safetensors` from [SeedVR2's ComfyUI weights](https://huggingface.co/numz/SeedVR2_comfyUI/tree/main), placed in `ComfyUI/models/SEEDVR2`. The SeedVR2 nodes can also download them on first use.
## Use
Open **vLLM-Omni MiniMax H3 Video Upscale.json** from the example workflows. An API-format version is provided alongside it. Set the Generate Video URL/model and the LoRA path as seen by the H3 server.
The preset uses the LightX2V v1.0 768p Turbo adapter in Diffusers format: `minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors`, five sigma points, video/audio flow shifts 6/3, and LoRA scale 1. Preload the same adapter on the server using the [Turbo LoRA recipe](../../../recipes/MiniMaxAI/MiniMax-H3.md#turbo-lora). For base H3, disconnect LoRA, set inference steps to 50, and set video flow shift to 12.
The H3 preset requests 1344 x 768 at 24 FPS for 5.167 seconds, which gives 124 frames. SeedVR2's `resolution=1536` sets the target short edge, producing 2688 x 1536 for this input. Update this value when changing the input size; it is an output resolution, not a fixed 2x multiplier.
The SeedVR2 preset uses 3B FP16, five-frame batches, one overlapping frame, uniform batch padding, and LAB color correction. The DiT loader uses CPU offload with 32 swapped blocks; the VAE uses tiled encoding and decoding. Select the local GPU in both model loaders. See the [SeedVR2 settings](https://github.com/numz/ComfyUI-SeedVR2_VideoUpscaler#-usage) when adjusting memory use or batch size.
Run the workflow. The two Save Video nodes write MP4s under `WF07/generated` and `WF07/upscaled` in ComfyUI's output directory.
apps/ComfyUI-vLLM-Omni/example_workflows/vLLM-Omni MiniMax H3 Video Upscale.json
+724
View File
@@ -0,0 +1,724 @@
{
"version": 0.4,
"last_node_id": 12,
"last_link_id": 13,
"nodes": [
{
"id": 1,
"type": "VLLMOmniGenerateVideo",
"pos": [
360,
20
],
"size": [
330,
480
],
"flags": {},
"order": 1,
"mode": 0,
"inputs": [
{
"name": "frame",
"type": "IMAGE",
"shape": 7,
"link": null
},
{
"name": "references",
"type": "VIDEO_REFERENCES",
"shape": 7,
"link": null
},
{
"name": "sampling_params",
"type": "SAMPLING_PARAMS",
"shape": 7,
"link": 11
},
{
"name": "lora",
"type": "REMOTE_LORA",
"shape": 7,
"link": 12
},
{
"name": "model_params",
"type": "VIDEO_PARAMS",
"shape": 7,
"link": 10
},
{
"name": "fast_h3",
"type": "FASTH3_DEPLOYMENT",
"shape": 7,
"link": null
}
],
"outputs": [
{
"name": "video",
"type": "VIDEO",
"links": [
1,
9
]
}
],
"properties": {
"Node name for S&R": "VLLMOmniGenerateVideo"
},
"widgets_values": [
"http://127.0.0.1:8091/v1",
"MiniMaxAI/MiniMax-H3",
"A small brass bell hanging from a wooden beam in a quiet garden. A gentle breeze swings the bell and it rings clearly. Static camera, natural daylight, continuous motion, synchronized ringing and soft birdsong.",
"",
1344,
768,
24,
5.167
]
},
{
"id": 2,
"type": "GetVideoComponents",
"pos": [
730,
20
],
"size": [
250,
125
],
"flags": {},
"order": 1,
"mode": 0,
"inputs": [
{
"name": "video",
"type": "VIDEO",
"link": 1
}
],
"outputs": [
{
"name": "images",
"type": "IMAGE",
"links": [
2
]
},
{
"name": "audio",
"type": "AUDIO",
"links": [
6
]
},
{
"name": "fps",
"type": "FLOAT",
"links": [
5
]
},
{
"name": "bit_depth",
"type": "INT",
"links": [
7
]
}
],
"properties": {
"Node name for S&R": "GetVideoComponents"
}
},
{
"id": 3,
"type": "SeedVR2LoadDiTModel",
"pos": [
730,
290
],
"size": [
340,
245
],
"flags": {},
"order": 4,
"mode": 0,
"inputs": [
{
"localized_name": "torch_compile_args",
"name": "torch_compile_args",
"shape": 7,
"type": "TORCH_COMPILE_ARGS",
"link": null
}
],
"outputs": [
{
"localized_name": "SEEDVR2_DIT",
"name": "SEEDVR2_DIT",
"type": "SEEDVR2_DIT",
"links": [
3
]
}
],
"properties": {
"Node name for S&R": "SeedVR2LoadDiTModel"
},
"widgets_values": [
"seedvr2_ema_3b_fp16.safetensors",
"cuda:0",
32,
false,
"cpu",
false,
"sdpa"
]
},
{
"id": 4,
"type": "SeedVR2VideoUpscaler",
"pos": [
1120,
290
],
"size": [
340,
415
],
"flags": {},
"order": 8,
"mode": 0,
"inputs": [
{
"localized_name": "image",
"name": "image",
"type": "IMAGE",
"link": 2
},
{
"localized_name": "dit",
"name": "dit",
"type": "SEEDVR2_DIT",
"link": 3
},
{
"localized_name": "vae",
"name": "vae",
"type": "SEEDVR2_VAE",
"link": 13
}
],
"outputs": [
{
"localized_name": "IMAGE",
"name": "IMAGE",
"type": "IMAGE",
"links": [
4
]
}
],
"properties": {
"Node name for S&R": "SeedVR2VideoUpscaler"
},
"widgets_values": [
42,
"fixed",
1536,
0,
5,
true,
"lab",
1,
0,
0.0,
0.0,
"cpu",
false
]
},
{
"id": 5,
"type": "CreateVideo",
"pos": [
1490,
290
],
"size": [
250,
175
],
"flags": {},
"order": 4,
"mode": 0,
"inputs": [
{
"name": "images",
"type": "IMAGE",
"link": 4
},
{
"name": "fps",
"type": "FLOAT",
"widget": {
"name": "fps"
},
"link": 5
},
{
"name": "audio",
"type": "AUDIO",
"shape": 7,
"link": 6
},
{
"name": "bit_depth",
"type": "INT",
"widget": {
"name": "bit_depth"
},
"link": 7
}
],
"outputs": [
{
"name": "VIDEO",
"type": "VIDEO",
"links": [
8
]
}
],
"properties": {
"Node name for S&R": "CreateVideo"
},
"widgets_values": [
24,
8
]
},
{
"id": 6,
"type": "SaveVideo",
"pos": [
1790,
290
],
"size": [
330,
280
],
"flags": {},
"order": 5,
"mode": 0,
"inputs": [
{
"name": "video",
"type": "VIDEO",
"link": 8
},
{
"name": "filename_prefix",
"type": "STRING",
"widget": {
"name": "filename_prefix"
},
"link": null
},
{
"name": "format",
"type": "COMBO",
"widget": {
"name": "format"
},
"link": null
},
{
"name": "codec",
"type": "COMFY_DYNAMICCOMBO_V3",
"widget": {
"name": "codec"
},
"link": null
}
],
"outputs": [
{
"name": "video",
"type": "VIDEO",
"links": []
}
],
"properties": {
"Node name for S&R": "SaveVideo"
},
"widgets_values": [
"WF07/upscaled",
"mp4",
"auto"
]
},
{
"id": 7,
"type": "VLLMOmniMiniMaxH3Params",
"pos": [
0,
20
],
"size": [
320,
120
],
"flags": {},
"order": 7,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "MiniMax-H3 video params",
"type": "VIDEO_PARAMS",
"links": [
10
]
}
],
"properties": {
"Node name for S&R": "VLLMOmniMiniMaxH3Params"
},
"widgets_values": [
3.0,
6.0
]
},
{
"id": 8,
"type": "VLLMOmniDiffusionSampling",
"pos": [
0,
200
],
"size": [
320,
280
],
"flags": {},
"order": 8,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "diffusion sampling params",
"type": "SAMPLING_PARAMS",
"links": [
11
]
}
],
"properties": {
"Node name for S&R": "VLLMOmniDiffusionSampling"
},
"widgets_values": [
1,
5,
1.0,
1.0,
false,
true,
42,
"fixed"
]
},
{
"id": 9,
"type": "VLLMOmniRemoteLoRA",
"pos": [
0,
530
],
"size": [
320,
210
],
"flags": {},
"order": 9,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "lora",
"type": "REMOTE_LORA",
"links": [
12
]
}
],
"properties": {
"Node name for S&R": "VLLMOmniRemoteLoRA"
},
"widgets_values": [
"models/minimax-h3-turbo/minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors",
"h3-turbo-v1.0",
1.0,
0
]
},
{
"id": 10,
"type": "SaveVideo",
"pos": [
1120,
20
],
"size": [
330,
190
],
"flags": {},
"order": 10,
"mode": 0,
"inputs": [
{
"name": "video",
"type": "VIDEO",
"link": 9
},
{
"name": "filename_prefix",
"type": "STRING",
"widget": {
"name": "filename_prefix"
},
"link": null
},
{
"name": "format",
"type": "COMBO",
"widget": {
"name": "format"
},
"link": null
},
{
"name": "codec",
"type": "COMFY_DYNAMICCOMBO_V3",
"widget": {
"name": "codec"
},
"link": null
}
],
"outputs": [
{
"name": "video",
"type": "VIDEO",
"links": []
}
],
"properties": {
"Node name for S&R": "SaveVideo"
},
"widgets_values": [
"WF07/generated",
"mp4",
"auto"
],
"title": "Save generated video before upscale"
},
{
"id": 11,
"type": "Note",
"pos": [
360,
560
],
"size": [
330,
260
],
"flags": {},
"order": 11,
"mode": 0,
"inputs": [],
"outputs": [],
"properties": {
"Node name for S&R": "Note"
},
"widgets_values": [
"WF-07: H3 + SeedVR2\n\nInstall ComfyUI-SeedVR2_VideoUpscaler. The DiT and VAE loaders use SeedVR2 3B FP16 and ema_vae_fp16.\n\nSet the H3 service URL/model and the server-side Turbo LoRA path.\n\n1536 is the target short edge for this 1344 x 768 preset (2x). Original audio and FPS connect directly to Create Video.\n\nFor base H3: disconnect LoRA, use 50 steps and video flow shift 12."
]
},
{
"id": 12,
"type": "SeedVR2LoadVAEModel",
"pos": [
730,
590
],
"size": [
340,
315
],
"flags": {},
"order": 5,
"mode": 0,
"inputs": [
{
"localized_name": "torch_compile_args",
"name": "torch_compile_args",
"shape": 7,
"type": "TORCH_COMPILE_ARGS",
"link": null
}
],
"outputs": [
{
"localized_name": "SEEDVR2_VAE",
"name": "SEEDVR2_VAE",
"type": "SEEDVR2_VAE",
"links": [
13
]
}
],
"properties": {
"Node name for S&R": "SeedVR2LoadVAEModel"
},
"widgets_values": [
"ema_vae_fp16.safetensors",
"cuda:0",
true,
1024,
128,
true,
768,
128,
"false",
"cpu",
false
]
}
],
"links": [
[
1,
1,
0,
2,
0,
"VIDEO"
],
[
2,
2,
0,
4,
0,
"IMAGE"
],
[
3,
3,
0,
4,
1,
"SEEDVR2_DIT"
],
[
4,
4,
0,
5,
0,
"IMAGE"
],
[
5,
2,
2,
5,
1,
"FLOAT"
],
[
6,
2,
1,
5,
2,
"AUDIO"
],
[
7,
2,
3,
5,
3,
"INT"
],
[
8,
5,
0,
6,
0,
"VIDEO"
],
[
9,
1,
0,
10,
0,
"VIDEO"
],
[
10,
7,
0,
1,
4,
"VIDEO_PARAMS"
],
[
11,
8,
0,
1,
2,
"SAMPLING_PARAMS"
],
[
12,
9,
0,
1,
3,
"REMOTE_LORA"
],
[
13,
12,
0,
4,
2,
"SEEDVR2_VAE"
]
],
"groups": [],
"config": {},
"extra": {
"ds": {
"scale": 0.52,
"offset": [
100,
100
]
}
}
}