diff --git a/recipes/MiniMaxAI/MiniMax-H3-4090.md b/recipes/MiniMaxAI/MiniMax-H3-4090.md new file mode 100644 index 000000000..1e22f5d2e --- /dev/null +++ b/recipes/MiniMaxAI/MiniMax-H3-4090.md @@ -0,0 +1,169 @@ +# MiniMax-H3 on RTX 4090 + +This recipe uses BF16 weights, tiled VAE decode, tensor parallelism, optional +Ulysses sequence parallelism, and distributed layerwise offload (DLO). It is the +24 GiB sibling of [MiniMax-H3-5090.md](MiniMax-H3-5090.md): the two-GPU topology +matches that recipe, but the resident DiT-block count is lower so the per-rank +peak fits in 24 GiB. Lower resident counts reduce HBM use and increase +CPU-to-GPU transfer time. + +## Capacity requirements + +| Resource | Two RTX 4090s | Four RTX 4090s | +| --- | ---: | ---: | +| GPU HBM | 24 GiB per GPU | 24 GiB per GPU | +| Checkpoint storage | 135 GiB per partition | 135 GiB per partition | +| Available system RAM | 200 GiB minimum | 200 GiB minimum | +| Recommended system RAM | 384 GiB | 384 GiB | + +`FL2VA` and `Ref2VA` are separate 135 GiB checkpoint partitions. Start one +server at a time. DLO keeps rank-local weights in pinned host memory; increasing +`--dlo-resident-layers` improves latency but does **not** reduce host RAM in the +current implementation because resident layers retain pinned CPU master copies. + +A single RTX 4090 is not covered here. The one-GPU DLO profile in the RTX 5090 +recipe peaked at 26.50 GiB with 12 resident layers, which exceeds 24 GiB. On one +4090, use the model-level CPU offload command in +[MiniMax-H3.md](MiniMax-H3.md#single-gpu-accuracy-and-memory-first) instead, or +lower the resident count and re-measure before trusting it. + +> **Modular H3:** after #5720 lands, preserve this recipe's one-partition +> behavior with `--task-type fl2va` or `--task-type ref2va`. + +## Two RTX 4090s: 1024x576, 5 seconds + +Use TP2 and 12 resident DiT layers. This is the `rtx4090` profile from the main +recipe. On two RTX 4090s it peaked at 15,620 MiB per rank for T2VA and +14,918 MiB per rank for Ref2VA, leaving roughly 9 GiB of headroom per card. See +[Target-hardware validation](#target-hardware-validation) for the full numbers. + +```bash +export VLLM_WORKER_MULTIPROC_METHOD=spawn +export VLLM_OMNI_VIDEO_SYNC_TIMEOUT=14400 + +CUDA_VISIBLE_DEVICES=0,1 vllm serve /path/to/MiniMax-H3/FL2VA \ + --omni --trust-remote-code --host 0.0.0.0 --port 8000 \ + --num-gpus 2 --tensor-parallel-size 2 --text-encoder-tp-size 2 \ + --usp 1 --ring 1 --vae-patch-parallel-size 2 \ + --vae-parallel-mode tile --vae-use-tiling \ + --enable-distributed-layerwise-offload --dlo-no-use-allgather \ + --dlo-resident-layers 12 --enforce-eager \ + --diffusion-attention-backend CUDNN_ATTN +``` + +For Ref2VA, stop the FL2VA server and restart the same command with +`/path/to/MiniMax-H3/Ref2VA`. Ref2VA reference image count and prompt length can +increase activation memory; begin with one request at a time. + +`VLLM_OMNI_VIDEO_SYNC_TIMEOUT` matters more here than on larger cards. DLO +streams non-resident DiT blocks over PCIe on every denoising step, so a 60-step +request takes several minutes and would otherwise hit the 600-second default. + +## Four RTX 4090s: 1024x576, 5 seconds + +Keep TP2 and raise Ulysses sequence parallel to 2 so the world size is +`TP × USP = 4`. Text-encoder TP and VAE patch parallel follow the GPU count. +Resident layers stay at 12: adding USP does not shard DiT weights further, so +per-GPU peak HBM stays close to the two-GPU profile while wall-clock improves. + +```bash +export VLLM_WORKER_MULTIPROC_METHOD=spawn +export VLLM_OMNI_VIDEO_SYNC_TIMEOUT=14400 + +CUDA_VISIBLE_DEVICES=0,1,2,3 vllm serve /path/to/MiniMax-H3/FL2VA \ + --omni --trust-remote-code --host 0.0.0.0 --port 8000 \ + --num-gpus 4 --tensor-parallel-size 2 --text-encoder-tp-size 4 \ + --usp 2 --ring 1 --vae-patch-parallel-size 4 \ + --vae-parallel-mode tile --vae-use-tiling \ + --enable-distributed-layerwise-offload --dlo-no-use-allgather \ + --dlo-resident-layers 12 --enforce-eager \ + --diffusion-attention-backend CUDNN_ATTN +``` + +For Ref2VA, stop the FL2VA server and restart the same command with +`/path/to/MiniMax-H3/Ref2VA`. Pin four dedicated GPUs; if the launcher exposes +fewer than four devices, startup fails with +`Stage 0 requires 4 device(s) based on parallel_config`. + +## Target-hardware validation + +Measured on RTX 4090 (24,564 MiB each, driver 580.126.09) with the serve +commands above, at vLLM-Omni `0.26.1.dev55+g81b48e83e`, vLLM `0.26.0`, and +PyTorch `2.11.0+cu130`. GPUs used by each server were dedicated. `Client E2E` +and `Peak per GPU` come from the `/v1/videos/sync` response headers +`x-inference-time-s` and `x-peak-memory-mb`; the latter is the rank-0 CUDA +reserved high-water mark, so it is a per-GPU figure. + +| GPUs | Topology | Task | Shape | Frames | Steps | Client E2E | Peak per GPU | Output validation | +| ---: | --- | --- | ---: | ---: | ---: | ---: | ---: | --- | +| 2 | TP2 × USP1 | T2VA (`FL2VA`) | 1024x576 | 124 at 24 FPS | 60 | 7 min 9 s | 15.3 GiB | H.264 video + 32 kHz stereo AAC; full `ffmpeg` decode passed | +| 2 | TP2 × USP1 | Ref2VA | 1024x576 | 124 at 24 FPS | 60 | 14 min 52 s | 14.6 GiB | H.264 video + 32 kHz stereo AAC; full `ffmpeg` decode passed | +| 4 | TP2 × USP2 | T2VA (`FL2VA`) | 1024x576 | 124 at 24 FPS | 60 | 4 min 29 s | 15.2 GiB | H.264 video + 32 kHz stereo AAC; full `ffmpeg` decode passed | +| 4 | TP2 × USP2 | Ref2VA | 1024x576 | 124 at 24 FPS | 60 | 9 min 5 s | 16.1 GiB | H.264 video + 32 kHz stereo AAC; successful `/v1/videos/sync` 200 | + +All rows use `seed=1101`, `flow_shift=12`, and `audio_flow_shift=3.0`. + +Two-GPU repeats: T2VA measured 434.9 s and 429.3 s; Ref2VA measured 891.9 s and +892.1 s. Peak memory was identical across repeats for both tasks, and the two +outputs of each pair agreed to within 115 bytes. + +Four-GPU T2VA repeats: 273.8 s and 269.5 s, peaking at 15,560 MiB and +15,612 MiB. Four-GPU Ref2VA is a single successful end-to-end run (545.2 s, +16,448 MiB). Follow-up Ref2VA requests on this build completed diffusion but +then hit the engine's hardcoded 30 s async output wait +(`_ASYNC_OUTPUT_TIMEOUT` in `diffusion_engine.py`) during post-compute D2H, +so they are not reported as repeats. + +Relative to the two-GPU baseline on the same shape, four-GPU TP2×USP2 was about +1.6× faster for both tasks, while per-GPU peak HBM stayed in the mid-15 GiB +range because USP does not further shard resident DiT weights. These are +single-request validation runs, not concurrent throughput benchmarks. + +## Request examples + +`t2va` requires an explicit `aspect_ratio`. Without it the request fails with +`t2va requires an explicit aspect_ratio`, even when `width` and `height` are set. + +Text to video and audio, against the `FL2VA` server: + +```bash +curl -sS -D headers_t2va.txt -o out_t2va.mp4 --max-time 14400 \ + -X POST "http://127.0.0.1:8000/v1/videos/sync" \ + -F 'prompt=傍晚的小厨房，夕阳余晖从窗边洒进来，旧木桌上放着洗到一半的马克杯和起雾的玻璃瓶，悬挂的抹布轻轻晃动。画面带有手持拍摄的轻微晃动和逆光曝光波动，生活感十足。环境音是安静的厨房底噪。' \ + -F 'aspect_ratio=16:9' \ + -F 'width=1024' -F 'height=576' -F 'fps=24' \ + -F 'num_inference_steps=60' -F 'flow_shift=12' -F 'seed=1101' \ + -F 'extra_params={"task":"t2va","duration":5,"audio_flow_shift":3.0}' +``` + +Reference to video and audio, against the `Ref2VA` server: + +```bash +curl -sS -D headers_ref2va.txt -o out_ref2va.mp4 --max-time 14400 \ + -X POST "http://127.0.0.1:8000/v1/videos/sync" \ + -F 'prompt=傍晚的小厨房，夕阳余晖从窗边洒进来，旧木桌上放着洗到一半的马克杯和起雾的玻璃瓶，悬挂的抹布轻轻晃动。画面带有手持拍摄的轻微晃动和逆光曝光波动，生活感十足。环境音是安静的厨房底噪。' \ + -F 'input_reference=@/path/to/reference.jpg;type=image/jpeg' \ + -F 'audio_reference=