vllm-omni/recipes/MiniMaxAI/MiniMax-H3-NPU.md

MiniMax H3 on Ascend NPU

Joint video and audio generation with text, first/last keyframes, and mixed image/video/audio references — Ascend NPU deployment guide

Summary

• Vendor: MiniMaxAI
• Model: MiniMaxAI/MiniMax-H3
• Tasks: T2VA, FL2VA, and Ref2VA
• Mode: OpenAI-compatible /v1/videos HTTP serving
• Hardware: Atlas 800I A3
• Maintainer: Community

This recipe adapts MiniMax-H3.md for Ascend NPU environments. Differences from the GPU path:

• Runs on a CPU-only (aarch64) PyTorch build with torch_npu; no CUDA runtime is present.
• Audio loading does not require TorchCodec (whose aarch64 wheels are built against CUDA torch and fail to load on CPU-only builds). vLLM-Omni automatically falls back to soundfile / ffmpeg for wav/mp3/m4a/mp4 audio inputs.

Prerequisites

Checkpoint

Same as the GPU recipe — Hugging Face access approval is required. Authenticate once; vllm serve downloads the required nested components automatically:

hf auth login
export MODEL=MiniMaxAI/MiniMax-H3

Environment

• Ascend driver & firmware: 25.5.1
• CANN toolkit: 9.0.1
• Python: 3.12
• PyTorch: 2.10.0+cpu
• torch_npu: 2.10.0.post2
• Install vLLM-Omni from a checkout with MiniMax H3 support:

uv venv
source .venv/bin/activate
uv pip install -e .

Install the mindie-sd operator library to enable Ascend-optimized fused operators (adalayernorm, etc.) and the RainFusion rf_v2 block-sparse attention kernel:

git clone https://gitcode.com/Ascend/MindIE-SD.git && cd MindIE-SD

# Comment out the tik_ops build step (not needed for this use case)
sed -i 's|^\(\s*\)source ${current_script_dir}/build_tik_ops.sh|\1# source ${current_script_dir}/build_tik_ops.sh|' build/build_ops.sh

python setup.py bdist_wheel
cd dist
pip install mindiesd-*.whl

• ffmpeg and ffprobe must be available on PATH. They are used for reference-video preparation and MP4 output.
• Reference-video decoding uses decord when available and falls back to PyAV otherwise.
• Audio inputs do not require TorchCodec: wav/mp3/m4a/mp4 files are loaded at native sample rate through the soundfile / ffmpeg fallback.

Start a server

Pass the repository ID directly. The pipeline loads the two nested DiTs while sharing the tokenizer, processor, text encoder, and VAEs from FL2VA.

Multi-NPU: 768P combined-service configuration

Use Ulysses sequence parallelism degree 8, text-encoder tensor parallelism degree 8, native tiled VAE patch parallelism degree 8, and layerwise offload:

export ASCEND_RT_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
export PORT=9098
export MODEL=MiniMaxAI/MiniMax-H3
export VLLM_WORKER_MULTIPROC_METHOD=spawn
export VLLM_OMNI_VIDEO_SYNC_TIMEOUT=1800
export PYTHONDONTWRITEBYTECODE=1

vllm serve "${MODEL}" \
  --omni \
  --host 0.0.0.0 \
  --port "${PORT}" \
  --trust-remote-code \
  --num-gpus 8 \
  --usp 8 \
  --ring 1 \
  --text-encoder-tp-size 8 \
  --enable-layerwise-offload \
  --vae-parallel-mode tile \
  --vae-use-tiling \
  --vae-patch-parallel-size 8 \
  --diffusion-attention-backend FLASH_ATTN

Do not add --enforce-eager. The first request includes regional compilation; warm the server once before measuring steady-state latency. H3 is CFG-distilled, so --cfg-parallel-size must remain 1.

The same endpoint accepts task=t2va, task=fl2va, and task=ref2va; no partition restart is required. Layerwise offload applies to both DiTs.

Optional optimizations

Two independent optimizations may be enabled on top of the configuration above. Both are validated for T2VA only.

RainFusion block-sparse attention — switch the attention backend, keeping every other flag unchanged:

  --diffusion-attention-backend RAINFUSION_ATTN

INT8 online quantization — add one flag to the server command above:

  --quantization int8

Keep --ring 1 when using RainFusion: the rf_v2 kernel ranks key blocks over the whole sequence, so ring parallelism would split away the keys it needs. Scale with --usp instead.

HTTP API examples

The request format is identical to the GPU recipe; see MiniMax-H3.md § HTTP API examples. Use the validated 768P shapes (e.g. width=1344 height=768) on NPU.

Key parameters

Same as the GPU recipe; see MiniMax-H3.md § Key parameters. The validated resolution on NPU is 768P (e.g. 1344x768).

Validated NPU evidence

Measured on an Atlas 800I A3 server (8x NPU) with CANN 9.0.1, PyTorch 2.10.0+cpu, and torch_npu 2.10.0.post2, using the multi-NPU configuration above:

Workload	Configuration
T2VA, 209 frames, 1344x768	TE TP8, layerwise offload, Ulysses 8, VPP8 tile, regional compile
Ref2VA (prompt + video), 124 frames, 1344x768	TE TP8, layerwise offload, Ulysses 8, VPP8 tile, regional compile

These measurements describe the validated shapes rather than a general throughput guarantee.

Known limitations

• Combined serving requires sibling FL2VA and Ref2VA directories, loads both task-specific DiTs, and loads one copy of every shared component.
• H3 currently executes one generation request per diffusion batch.
• The first regional-compile request is a warmup and should not be included in steady-state performance measurements.
• The official H3 input matrix and media limits are documented in the GPU recipe; this NPU path uses the same HTTP request contract.
• VAE patch parallelism requires size 1 or the full DiT group size and supports the H3 native tile mode only.
• RainFusion block-sparse attention and INT8 quantization are validated for T2VA only; use the BF16 dense configuration for FL2VA and Ref2VA.
• Online quantization cannot be combined with distributed layerwise offload while AllGather is enabled; pass --dlo-no-use-allgather in that case.

Additional resources

• MiniMax-H3.md — full GPU guide
• Attention backends § RAINFUSION_ATTN — RainFusion knobs and tuning
• Int8 quantization
• Supported models
• Video API