From 615de4e041d1eeb78b3017845ab2f0600f2c9c41 Mon Sep 17 00:00:00 2001 From: yiminghub2024 <482890@qq.com> Date: Sun, 9 Aug 2026 23:52:40 +0800 Subject: [PATCH] [doc]Add recipe for MiniMax-H3 on RTX PRO 6000 (#5863) Signed-off-by: yiminghub2024 <482890@qq.com> --- recipes/MiniMaxAI/MiniMax-H3-RTX-PRO-6000.md | 346 +++++++++++++++++++ 1 file changed, 346 insertions(+) create mode 100644 recipes/MiniMaxAI/MiniMax-H3-RTX-PRO-6000.md diff --git a/recipes/MiniMaxAI/MiniMax-H3-RTX-PRO-6000.md b/recipes/MiniMaxAI/MiniMax-H3-RTX-PRO-6000.md new file mode 100644 index 000000000..c82a03050 --- /dev/null +++ b/recipes/MiniMaxAI/MiniMax-H3-RTX-PRO-6000.md @@ -0,0 +1,346 @@ +# MiniMax-H3 on RTX PRO 6000 Blackwell GPUs + +This recipe runs MiniMax-H3 in BF16 on 96 GiB RTX PRO 6000 Blackwell GPUs. +The additional HBM over the RTX PRO 5000 profile removes the four-GPU +minimum: two GPUs are enough to keep the model resident with TP2. Four and +eight GPUs raise the Ulysses degree to shard the attention sequence further. +CPU offload and distributed layerwise offload are not required in any of +these configurations. + +Validated on: +- Host: YLX Y762 +- GPUs: 8 × RTX PRO 6000 Blackwell (96 GiB) +- Device order: default (`CUDA_VISIBLE_DEVICES` not set) +- Driver / CUDA: 580.105.08 / 13.0 +- Container image: `vllm/vllm-omni:minimax-h3` +- Workload: T2VA, 1344×768, `duration=5.0`, 50 steps, `flow_shift=12`, `seed=1101` + +## Capacity requirements + +| Resource | Two GPUs | Four GPUs | Eight GPUs | +| --- | ---: | ---: | ---: | +| GPU HBM | 96 GiB per GPU | 96 GiB per GPU | 96 GiB per GPU | +| Checkpoint storage | 135 GiB per partition | 135 GiB per partition | 135 GiB per partition | +| Available system RAM | 200 GiB minimum | 200 GiB minimum | 200 GiB minimum | +| Recommended system RAM | 384 GiB | 384 GiB | 384 GiB | +| Measured peak HBM per GPU | 77.49 GiB | 66.44 GiB | 61.07 GiB | + +`FL2VA` and `Ref2VA` are separate checkpoint partitions. Start one server at +a time on a host sized for the minimum system-memory requirement, or pass +`--task-type fl2va` or `--task-type ref2va` to download and load a single +partition. The two-server layout described in the eight-GPU section doubles +both the storage and the host-memory requirement. + +## Parallelism summary + +| Configuration | TP | Ulysses | Text-encoder TP | VAE patch parallel | +| --- | ---: | ---: | ---: | ---: | +| Two GPUs | 2 | 1 | 2 | 2 | +| Four GPUs | 2 | 2 | 4 | 4 | +| Eight GPUs | 2 | 4 | 8 | 8 | +| Eight GPUs, headroom variant | 4 | 2 | 8 | 8 | + +Per-GPU weight residency is set by the tensor-parallel degree alone. Raising +the Ulysses degree lowers activation memory and per-step latency but leaves +weight residency unchanged. Ring parallelism stays at 1, and H3 is +CFG-distilled, so `--cfg-parallel-size` must also remain 1. + +## PCIe topology and GPU order + +RTX PRO 6000 Blackwell does not provide NVLink. Before starting the server, +inspect the host topology: + +```bash +nvidia-smi topo -m +nvidia-smi nvlink -s +``` + +Tensor-parallel groups are consecutive ranks. Ulysses groups are strided by +the tensor-parallel degree. Confirm this grouping against the server log +before tuning the device order on a new host. + +For the two-GPU configuration, pick a single `PXB` pair on one NUMA node. +For the four-GPU configuration, pick two `PXB` pairs on the same node and +order them so that each Ulysses group lands inside one pair. If physical +GPUs `(0,1)` and `(2,3)` are the two local pairs, the order is +`CUDA_VISIBLE_DEVICES=0,2,1,3`. Do not copy these IDs blindly: reproduce the +same relationship on the target host. + +Device-order pinning and NUMA binding are tuning knobs, not prerequisites. +The measurements in this recipe were taken without either, so any gain from +them is additional to the numbers reported below and is host-specific. + +## Two-GPU serving configuration + +Two 96 GiB GPUs hold the BF16 model with TP2 alone. There is no Ulysses +group, so the DiT attention sequence is not sharded and activation memory is +higher than in the four-GPU profile at the same output shape. Re-measure +peak memory before increasing the output shape, the reference-input length, +or concurrency. + +```bash +export MODEL_ROOT=/path/to/MiniMax-H3 +export MODEL="${MODEL_ROOT}/FL2VA" +export PORT=8000 + +VLLM_OMNI_VIDEO_SYNC_TIMEOUT=1800 \ +vllm serve "${MODEL}" \ + --omni \ + --host 0.0.0.0 \ + --port "${PORT}" \ + --trust-remote-code \ + --num-gpus 2 \ + --tensor-parallel-size 2 \ + --usp 1 \ + --ring 1 \ + --text-encoder-tp-size 2 \ + --vae-patch-parallel-size 2 \ + --vae-parallel-mode tile \ + --vae-use-tiling \ + --diffusion-attention-backend CUDNN_ATTN +``` + +## Four-GPU serving configuration + +Four GPUs keep TP2 for the weight shard and add Ulysses2, which shards the +attention sequence and lowers per-GPU activation memory. Text-encoder TP4 +and VAE patch parallelism 4 spread the Qwen3-VL encoder and tiled decode +across all four GPUs. TP1 is not an option on this card: an unsharded BF16 +partition does not fit in 96 GiB. + +```bash +export MODEL_ROOT=/path/to/MiniMax-H3 +export MODEL="${MODEL_ROOT}/FL2VA" +export PORT=8000 + +VLLM_OMNI_VIDEO_SYNC_TIMEOUT=1800 \ +vllm serve "${MODEL}" \ + --omni \ + --host 0.0.0.0 \ + --port "${PORT}" \ + --trust-remote-code \ + --num-gpus 4 \ + --tensor-parallel-size 2 \ + --usp 2 \ + --ring 1 \ + --text-encoder-tp-size 4 \ + --vae-patch-parallel-size 4 \ + --vae-parallel-mode tile \ + --vae-use-tiling \ + --diffusion-attention-backend CUDNN_ATTN +``` + +## Eight-GPU serving configuration + +Eight GPUs extend the Ulysses degree to 4 while keeping TP2. This does not +lower per-GPU weight residency relative to the four-GPU profile; it lowers +activation memory and per-step latency by sharding the attention sequence +four ways. + +```bash +export MODEL_ROOT=/path/to/MiniMax-H3 +export MODEL="${MODEL_ROOT}/FL2VA" +export PORT=8000 + +VLLM_OMNI_VIDEO_SYNC_TIMEOUT=1800 \ +vllm serve "${MODEL}" \ + --omni \ + --host 0.0.0.0 \ + --port "${PORT}" \ + --trust-remote-code \ + --num-gpus 8 \ + --tensor-parallel-size 2 \ + --usp 4 \ + --ring 1 \ + --text-encoder-tp-size 8 \ + --vae-patch-parallel-size 8 \ + --vae-parallel-mode tile \ + --vae-use-tiling \ + --diffusion-attention-backend CUDNN_ATTN +``` + +### Rank ordering across two sockets + +With TP2 and Ulysses4 the two Ulysses groups are ranks `(0,2,4,6)` and +`(1,3,5,7)`, and the four tensor-parallel pairs are `(0,1)`, `(2,3)`, +`(4,5)`, and `(6,7)`. On a dual-socket host these two collectives cannot +both be socket-local. If you choose to pin the device order, the trade-off +is: + +| Device order | Socket-local collective | Crosses sockets | +| --- | --- | --- | +| `0,4,1,5,2,6,3,7` | Ulysses all-to-all | TP all-reduce | +| `0,1,2,3,4,5,6,7` | TP all-reduce | Ulysses all-to-all | + +The first order assumes physical GPUs `0-3` are on NUMA node 0 and `4-7` on +node 1. Reproduce that relationship on the target host rather than copying +the IDs. The reported measurements use the default order and no NUMA +binding; the two orders were not compared on this host. + +### Headroom variant: TP4 with Ulysses2 + +Raising the tensor-parallel degree to 4 halves per-GPU weight residency and +leaves room for long Ref2VA references, larger output shapes, or +concurrency greater than one. It also adds a four-way all-reduce per DiT +block over a link without NVLink. Use it when the TP2 profile runs out of +headroom, not as the default. Replace the parallel flags above with: + +```bash + --tensor-parallel-size 4 \ + --usp 2 \ +``` + +### Two independent four-GPU servers + +An eight-GPU host can instead run one FL2VA server and one Ref2VA server +side by side, each pinned to its own NUMA node with the four-GPU +configuration. This removes every cross-socket collective and lifts the +one-server-at-a-time restriction, at the cost of doubling the requirements +to 400 GiB available system RAM and 270 GiB of model storage. + +Use `CUDA_VISIBLE_DEVICES=0,2,1,3`, `--cpunodebind=0 --membind=0`, and +`--port 8091` for the FL2VA server. Use `CUDA_VISIBLE_DEVICES=4,6,5,7`, +`--cpunodebind=1 --membind=1`, and `--port 8092` for the Ref2VA server. +This layout is described for completeness and was not part of the measured +runs. + +## Shared serving notes + +Do not add `--enforce-eager` for a performance run. Warm the server before +measuring so regional compilation is outside the measured request. On this +host the first request after startup ran 19% slower than the steady state +(107.58 s versus 90.48 s on eight GPUs); two warmup requests were enough to +converge. + +For Ref2VA on a single-server layout, stop the FL2VA server and restart the +same command with `MODEL="${MODEL_ROOT}/Ref2VA"`. + +## Attention backend + +RTX PRO 6000 Blackwell is an SM120 part, so the datacenter-Blackwell +`TRTLLM_ATTN` default does not apply here. Every configuration above selects +cuDNN BF16 attention explicitly. This keeps the recipe independent of +platform-default backend changes and does not depend on the experimental +SM120 kernel. + +## Optional: online FP8 + +`--quantization fp8` quantizes eligible DiT linears at load time and is +compatible with tensor parallelism and VAE tiling. Use it when the resident +BF16 peak leaves too little headroom for long Ref2VA references, a larger +output shape, or concurrency greater than one. It cannot be combined with +layerwise offload. + +## Target-hardware validation + +T2VA, 1344×768, `duration=5.0`, `fps=24`, 50 steps, `flow_shift=12`, +`seed=1101`, BF16, `CUDNN_ATTN`, tiled VAE, one request at a time. Servers +were started with the commands above — default device order, no NUMA +binding — plus `--enable-diffusion-pipeline-profiler`. Two warmup requests +preceded each measured request. + +| Measurement | Two GPUs | Four GPUs | Eight GPUs | +| --- | ---: | ---: | ---: | +| Topology | TP2 × USP1 | TP2 × USP2 | TP2 × USP4 | +| Text encode | 0.040 s | 0.044 s | 0.035 s | +| Denoise, 50 steps | 278.55 s | 168.73 s | 87.90 s | +| Per step | 5.571 s | 3.375 s | 1.758 s | +| VAE decode | 5.396 s | 2.791 s | 1.798 s | +| Client E2E | 284.76 s | 172.32 s | 90.48 s | +| Peak HBM per GPU | 77.49 GiB | 66.44 GiB | 61.07 GiB | +| Headroom below 96 GiB | 18.5 GiB | 29.6 GiB | 34.9 GiB | + +Stage times are read from the `X-Stage-Durations` response header of +`/v1/videos/sync`. Peak memory is the maximum of `nvidia-smi +--query-gpu=memory.used` sampled at 1 Hz across every device for the +duration of the measured request. Per step is denoise wall time divided by +the 50 requested steps. Stage times sum to roughly 0.75 s less than +end-to-end in all three configurations; queueing, result transfer, and MP4 +muxing sit outside the profiled stages. + +### Scaling behaviour + +Denoise speedup is not uniform across the sweep: 1.65× from two to four +GPUs (83% efficiency) but 1.92× from four to eight (96%). Two GPUs run +Ulysses1, so there is no sequence parallelism at all; going to four GPUs +pays the one-time cost of introducing the all-to-all, while going from four +to eight only widens an all-to-all that already exists. Expect the second +doubling to pay off better than the first on this topology. + +VAE decode behaves in the opposite direction — 1.93× from two to four GPUs +but only 1.55× from four to eight — as patch parallelism produces smaller +tiles and tile-boundary overhead grows. + +Text encoding is 0.04 s in every configuration and is not worth optimising; +denoise is 96-98% of end-to-end throughout. + +### Per-GPU memory model + +Peak memory falls by 11.05 GiB from two to four GPUs and by a further +5.37 GiB from four to eight — halving each time the Ulysses degree doubles, +because all three configurations shard the DiT with TP2 and hold identical +weights per GPU. Fitting the first two points gives: + +``` +peak HBM per GPU ≈ 55.4 GiB + 22.1 GiB / ulysses_degree +``` + +That fit predicts 60.9 GiB at Ulysses4 against 61.07 GiB measured, a 0.25% +error. The practical consequence is a floor near 55 GiB per GPU: adding +GPUs past eight will not reduce the per-GPU requirement further. To serve +on smaller cards, raise the tensor-parallel degree instead. + +Re-measure memory for longer reference inputs, concurrency greater than +one, or a different output shape. + +### Not yet measured + +Ref2VA latency and memory have not been measured on this host. Ref2VA +drives the Qwen3-VL presentation from tens of tokens for text-only prompts +to several thousand with image or video references, so T2VA numbers do not +transfer to it. The eight-GPU device-order comparison in the section above +was also not run. + +## T2VA request example + +`t2va` requires an explicit `aspect_ratio` even when `width` and `height` +are supplied. + +```bash +export API_URL="http://127.0.0.1:${PORT}/v1/videos/sync" + +curl -sS --max-time 1800 -X POST "${API_URL}" \ + -F 'prompt=At night, three cats march into a bedroom playing tiny brass instruments, then abruptly file out, with synchronized room ambience.' \ + -F 'width=1344' \ + -F 'height=768' \ + -F 'aspect_ratio=16:9' \ + -F 'fps=24' \ + -F 'num_inference_steps=50' \ + -F 'flow_shift=12' \ + -F 'seed=1101' \ + -F 'extra_params={"task":"t2va","duration":5.0,"audio_flow_shift":3.0}' \ + -o t2va.mp4 +``` + +To collect the stage breakdown and peak memory for a request, add +`--enable-diffusion-pipeline-profiler` to the server command and `-D +headers.txt` to the curl invocation, then read `X-Stage-Durations` and +`X-Peak-Memory-MB` from the saved headers. + +## Ref2VA request example + +`ref2va` defaults to a 16:9 aspect ratio, so `aspect_ratio` is optional +here. + +```bash +curl -X POST "http://127.0.0.1:${PORT}/v1/videos/sync" \ + --fail-with-body -w '\nHTTP %{http_code}\n' \ + --max-time 1200 \ + -F "input_reference=@/root/hand.jpg;type=image/jpeg" \ + -F "audio_reference=