diff --git a/docs/models/supported_models.md b/docs/models/supported_models.md index d1d4e051d..70f7662ba 100644 --- a/docs/models/supported_models.md +++ b/docs/models/supported_models.md @@ -97,3 +97,10 @@ th { |`HiDreamImagePipeline` | HiDream-I1-Full | `HiDream-ai/HiDream-I1-Full` | ✅︎ | ✅︎ | | | ✅︎ indicates the model is supported on that backend. Empty cells mean not listed as supported on that backend. + +H3 MiniMax H3 AMD GPU support is validated on gfx942 (MI300X) and +gfx950 (MI350) in BF16, using the AITER `FLASH_ATTN` diffusion attention backend. +Other AMD SKUs (e.g. MI325X) are not listed until their +own validation evidence is added. See the +[MiniMax H3 recipe](https://recipes.vllm.ai/MiniMaxAI/MiniMax-H3) for +per-architecture serving commands. diff --git a/recipes/MiniMaxAI/MiniMax-H3.md b/recipes/MiniMaxAI/MiniMax-H3.md index 526a745ae..01785fbbc 100644 --- a/recipes/MiniMaxAI/MiniMax-H3.md +++ b/recipes/MiniMaxAI/MiniMax-H3.md @@ -58,6 +58,10 @@ FlashAttention-4 extra: uv pip install -e '.[fa4]' ``` +On AMD ROCm, install without the `[fa4]` extra (FA4 is CUDA-only) and use +`--diffusion-attention-backend FLASH_ATTN`; see +[AMD ROCm (gfx942 / gfx950)](#amd-rocm-gfx942--gfx950). + `ffmpeg` and `ffprobe` must be available on `PATH`. They are used for reference-video preparation and MP4 output. @@ -387,6 +391,101 @@ incompatible with H3 layerwise offload because the offload path produces a weight stride rejected by the Cutlass FP8 kernel. Use resident FP8 with tensor parallelism and VAE tiling instead. +## AMD ROCm (gfx942 / gfx950) + +The sections above describe the NVIDIA CUDA path. This section covers the AMD +ROCm path; use it instead of the CUDA commands when running on AMD Instinct +GPUs. + +MiniMax H3 runs on AMD Instinct GPUs (gfx942 / gfx950) in BF16. Use +`--diffusion-attention-backend FLASH_ATTN`, which resolves to AITER packed varlen +attention on both architectures. + +Select the device with `HIP_VISIBLE_DEVICES` (not `CUDA_VISIBLE_DEVICES`), drop the +CUDA-only `FLASHINFER_DISABLE_VERSION_CHECK`, and install without the `[fa4]` extra +(FA4 is CUDA-only). The VAE uses AITER GroupNorm on ROCm. + +Install (ROCm wheel + source vLLM-Omni): + +```bash +pip install "vllm==0.26.0+rocm723" \ + --extra-index-url https://wheels.vllm.ai/rocm/0.26.0/rocm723 +VLLM_OMNI_TARGET_DEVICE=rocm pip install -e . --no-build-isolation +``` + +Prebuilt image: `vllm/vllm-omni-rocm:minimax-h3`. All tasks work out of the box: +the image bundles TorchCodec (for image+audio Ref2VA) and `ffmpeg` (for +video-reference Ref2VA). + +### ROCm single GPU + +Single GPU with model-level CPU offload keeps the Qwen3-VL encoder and DiT from +being co-resident: + +```bash +export MODEL="${MODEL_ROOT}/FL2VA" +export PORT=8091 + +HIP_VISIBLE_DEVICES=0 \ +VLLM_WORKER_MULTIPROC_METHOD=spawn \ +VLLM_OMNI_VIDEO_SYNC_TIMEOUT=1800 \ +vllm serve "${MODEL}" \ + --omni --host 0.0.0.0 --port "${PORT}" --trust-remote-code \ + --num-gpus 1 --enable-cpu-offload \ + --diffusion-attention-backend FLASH_ATTN +``` + +### ROCm four GPUs + +The best-practice CUDA four-GPU configuration works on ROCm with the changes above. +It mirrors the CUDA command including Ulysses sequence parallelism, VAE patch +parallelism, and text-encoder tensor parallelism, with no CPU offload when the model +shards across the GPUs: + +```bash +export MODEL="${MODEL_ROOT}/FL2VA" +export PORT=8091 + +HIP_VISIBLE_DEVICES=0,1,2,3 \ +VLLM_WORKER_MULTIPROC_METHOD=spawn \ +VLLM_OMNI_VIDEO_SYNC_TIMEOUT=1800 \ +vllm serve "${MODEL}" \ + --omni \ + --host 0.0.0.0 \ + --port "${PORT}" \ + --trust-remote-code \ + --num-gpus 4 \ + --usp 4 \ + --ring 1 \ + --text-encoder-tp-size 4 \ + --vae-patch-parallel-size 4 \ + --vae-parallel-mode tile \ + --vae-use-tiling \ + --diffusion-attention-backend FLASH_ATTN +``` + +As on CUDA, H3 is CFG-distilled, so keep `--cfg-parallel-size` at 1, and the H3 VAE +supports only its native `tile` mode. `--text-encoder-tp-size` is validated on +gfx942; on gfx950 it was not exercised. + +### Validated ROCm evidence + +vLLM-Omni with MiniMax H3 support, BF16. gfx942 rows measured with the +`vllm/vllm-omni-rocm:minimax-h3` image; gfx950 rows measured with the +`0.26.0+rocm723` wheel (HIP 7.2). + +| Workload | Configuration | Observed result | +|----------|---------------|-----------------| +| T2VA, 1344x768, 209 frames, 50 steps | 4x gfx942 (MI300X), FLASH_ATTN, USP4, text-enc TP4, VAE PP4 tile | encode 0.09 s, denoise 244.04 s, decode 4.15 s, 267.42 s client E2E; H.264 24 FPS + 32 kHz stereo AAC | +| FL2VA, 1344x768, 209 frames, 50 steps | 4x gfx942 (MI300X), FLASH_ATTN, USP4, text-enc TP4, VAE PP4 tile | encode 13.98 s, denoise 257.58 s, decode 4.11 s, 287.07 s client E2E; H.264 24 FPS + 32 kHz stereo AAC | +| T2VA, 832x480, ~4 s, 40 steps | 1x gfx950 (MI350), FLASH_ATTN, CPU offload | valid MP4 (H.264 + synced audio); ~0.73 s/denoise-step (~1.37 it/s), ~55 s client E2E incl. warmup | + +gfx942 figures are the mean of three requests after one excluded warmup +(external evidence: vllm-project/recipes#732). gfx950 figures are +functional-correctness validations, not tuned throughput; the first request +includes lazy regional compilation. MI325X (gfx942) and other MI355X SKUs are not +listed until their own evidence is added. + ## HTTP API examples The following requests use the synchronous endpoint so the returned body can @@ -658,12 +757,22 @@ reduction. - H3 currently executes one generation request per diffusion batch. - The first regional-compile request is a warmup and should not be included in steady-state performance measurements. +- The serving path accepts fewer references than the model supports. H3 documents up + to 9 images, 3 video clips, and 3 audio clips (12 files) per Omni Reference + request; the current vLLM-Omni path takes exactly one image plus one audio + reference, or one or more videos with no separate `audio_reference` (it uses the + source soundtracks). +- The 768 px short-edge mode is available for T2VA and FL2VA; 1344x768 is the + documented 16:9 request shape. +- `--cfg-parallel-size > 1` is rejected by design (CFG-distilled, no negative branch). +- VAE patch parallelism requires size 1 or the full DiT group size and supports the + H3 native `tile` mode only. +- A U2 x Ring2 hybrid currently fails with an attention-mask length mismatch; use + pure Ulysses. - Online FP8 is not compatible with layerwise offload. -- Image+audio Ref2VA accepts exactly one image and one audio reference. -- Video Ref2VA accepts one or more video files, but not an additional standalone - audio reference. -- VAE patch parallelism requires size 1 or the full DiT group size and supports - the H3 native `tile` mode only. +- Pure Ulysses still replicates the full DiT on every rank, so smaller-memory GPUs + cannot use `--usp N --tp 1` as a resident capacity path. Use DiT tensor parallelism + or model-level CPU offload; text-encoder TP alone is not sufficient. ## Additional resources