From 817f5d0e4d12751020a2ed8d7886b821eb605467 Mon Sep 17 00:00:00 2001 From: kinnara Date: Mon, 28 Sep 2026 01:34:59 +0800 Subject: [PATCH] =?UTF-8?q?[Hardware][Ascend]=20Add=20FastH3=20four-step?= =?UTF-8?q?=20deployment=20to=20MiniMax-H3=20NPU=20=E2=80=A6=20(#7149)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: kinnara --- recipes/MiniMaxAI/MiniMax-H3-NPU.md | 180 ++++++++++++++++++++++++++++ 1 file changed, 180 insertions(+) diff --git a/recipes/MiniMaxAI/MiniMax-H3-NPU.md b/recipes/MiniMaxAI/MiniMax-H3-NPU.md index 971be278d..23263c9fb 100755 --- a/recipes/MiniMaxAI/MiniMax-H3-NPU.md +++ b/recipes/MiniMaxAI/MiniMax-H3-NPU.md @@ -281,6 +281,186 @@ curl -sS -X POST "${API_URL}" \ -o t2va.mp4 ``` +This requires the FLASH_ATTN backend and MindIE-SD. H3 automatically applies +exact power-of-two input pre-scaling (`laser_input_scale=256`) so the +kernel's fp16 workspace cannot overflow on outlier activations. Measured on +the FastH3 four-step Dense configuration below, T2VA 15 s at 1344x768, this +kernel reduced end-to-end latency from ~73 s to ~57 s (about 22%). + + +## FastH3 four-step on A3 + +[FastH3](https://haoailab.com/blogs/fasth3-preview/) is FastVideo's four-step +DMD2 student of H3-Base; see +[MiniMax-H3.md ยง FastH3 adapter](MiniMax-H3.md#fasth3-adapter) for the adapter +contract. On A3 the adapter cannot be fused the GPU way. The GPU recipe fuses +it at load time from `--lora-path`, but that path replicates the full model per +rank and does not fit in 64 GB HBM. A3 needs distributed layerwise offload, and +offload is refused with `--lora-path` because it streams weights in without +going through the fusion (see the GPU recipe's note on why +`--enable-distributed-layerwise-offload` fails fast with a FastH3 adapter). + +The A3 path therefore fuses the adapter **offline**, once, into a native-layout +checkpoint that distributed layerwise offload can memory-map directly. This +also lets the offloaded server start without `--lora-path`, so the fusion +contract check that rejects offload never fires. + +### Prepare the fused checkpoint (one-time, offline) + +The fusion runs in the vLLM-Omni native namespace +(`blocks.N.attn.qkv_proj.weight`), matching the base H3 `FL2VA` transformer +layout. It reads the base transformer and the Dense adapter, adds the adapter's +low-rank and full-rank deltas per shard, symlinks the unchanged components +(text encoder, VAEs, tokenizer, processor), and writes the four-step sigma +ladder into `model_index.json` so the pipeline samples on the release's rungs +rather than a uniform schedule. + +Download the base checkpoint's `FL2VA` partition (native layout) and the Dense +adapter: + +```bash +export BASE_DIR=/path/to/MiniMax-H3 +hf download MiniMaxAI/MiniMax-H3 --include "FL2VA/*" --local-dir "${BASE_DIR}" + +export FASTH3_DIR=/path/to/fasth3 +hf download FastVideo/FastVideo-FastH3-4-step-Preview-v1-LoRA \ + dense-datafree/adapter_model.safetensors --local-dir "${FASTH3_DIR}" +``` + +> The published full checkpoint +> `FastVideo/FastVideo-FastH3-4-step-Preview-v1-Dense-DataFree` is stored in the +> diffusers layout (`transformer_blocks.N.attn.to_q/to_k/to_v`, split QKV, +> value-first MLP). Distributed layerwise offload memory-maps by exact runtime +> parameter name and has no diffusers-to-native remap for H3, so that artifact +> cannot be served directly. Fuse from the base `FL2VA` partition instead โ€” its +> keys already match the runtime. + +Run the fusion: + +```python +# prepare_fasth3_dense.py โ€” fuse base H3 FL2VA + FastH3 Dense adapter (native layout) +import json, os, shutil, sys, time +import torch +from safetensors.torch import load_file, save_file +from vllm_omni.diffusion.models.minimax_h3.fasth3 import FastH3WeightFusion + +BASE = os.environ["BASE_DIR"] + "/FL2VA" # base H3 FL2VA partition (native layout) +ADAPTER = os.environ["FASTH3_DIR"] + "/dense-datafree/adapter_model.safetensors" +OUT = os.environ.get("FUSED_DIR", "/path/to/FastH3-Dense-Fused/FL2VA") + +# MiniMax-H3 architecture +HEAD_DIM, NUM_BLOCKS, NUM_REFINER_BLOCKS = 128, 50, 2 +# FastH3 four-step sigma positions; the pipeline adds per-modality shift on top. +BASE_SCHEDULE = [0.999, 0.749, 0.5, 0.25, 0.0] +COMPONENTS = ["audio_vae", "video_vae", "text_encoder", "tokenizer", "processor"] + +fusion = FastH3WeightFusion.from_path( + ADAPTER, head_dim=HEAD_DIM, num_blocks=NUM_BLOCKS, + num_refiner_blocks=NUM_REFINER_BLOCKS) +assert fusion is not None, "adapter not recognized as FastH3" + +os.makedirs(OUT + "/transformer", exist_ok=True) +src_t = BASE + "/transformer" +for shard in sorted(f for f in os.listdir(src_t) + if f.endswith(".safetensors") and "index" not in f): + data = load_file(os.path.join(src_t, shard), device="cpu") + fused = {} + for k, v in data.items(): + fv = fusion.fuse(k, v) + fused[k] = fv.to("cpu").to(torch.bfloat16) if fv is not v else v + save_file(fused, os.path.join(OUT, "transformer", shard)) +fusion.validate_fully_applied() # every delta must have met its parameter + +for j in os.listdir(src_t): # copy transformer index/config + if j.endswith(".json"): + shutil.copy2(os.path.join(src_t, j), os.path.join(OUT, "transformer", j)) + +for c in COMPONENTS: # symlink unchanged components + s = os.path.join(BASE, c) + if os.path.exists(s): + os.symlink(os.path.realpath(s), os.path.join(OUT, c)) + +idx = json.loads(open(BASE + "/model_index.json").read()) # inject sigma ladder +idx.setdefault("_minimax_h3", {})["base_schedule"] = BASE_SCHEDULE +open(OUT + "/model_index.json", "w").write(json.dumps(idx, indent=4) + "\n") +print("fused ->", OUT) +``` + +```bash +export FUSED_DIR=/path/to/FastH3-Dense-Fused/FL2VA +python prepare_fasth3_dense.py +``` + +### Serve the fused checkpoint + +Serve the fused directory with the same multi-NPU flags as the base recipe, but +**without** `--lora-path` โ€” the adapter is already in the weights. Keep +distributed layerwise offload and, optionally, LaserAttention: + +```bash +export ASCEND_RT_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 +export PORT=9098 +export MODEL=/path/to/FastH3-Dense-Fused/FL2VA +export VLLM_WORKER_MULTIPROC_METHOD=spawn +export VLLM_OMNI_VIDEO_SYNC_TIMEOUT=1800 +export MINDIE_SD_FA_TYPE="ascend_laser_attention" # optional + +vllm serve "${MODEL}" \ + --omni \ + --host 0.0.0.0 \ + --port "${PORT}" \ + --trust-remote-code \ + --task-type t2va \ + --init-timeout 1800 \ + --stage-init-timeout 1800 \ + --num-gpus 8 \ + --usp 8 \ + --ring 1 \ + --text-encoder-tp-size 8 \ + --enable-distributed-layerwise-offload \ + --vae-parallel-mode tile \ + --vae-use-tiling \ + --vae-patch-parallel-size 8 \ + --diffusion-attention-backend FLASH_ATTN +``` + +Requests must ask for `num_inference_steps=4` and `task=t2va`; FastH3 preview v1 +distills the T2VA path only. The first request includes regional compilation +(~50 s warmup); exclude it from steady-state numbers. + +```bash +curl -s -X POST "http://localhost:${PORT}/v1/videos/sync" \ + -F 'prompt=A golden retriever running through a sunflower field at sunset' \ + -F 'seconds=5' \ + -F 'aspect_ratio=16:9' \ + -F 'num_inference_steps=4' \ + -o out.mp4 +``` + +### Measured FastH3 four-step evidence + +Measured on an Atlas 800I A3 server (8x NPU) with CANN 9.0.1, PyTorch +2.10.0+cpu, torch_npu 2.10.0.post2, the multi-NPU configuration above, T2VA at +1344x768, one warmup excluded: + +| Duration | LaserAttention | End-to-end | +| ---: | --- | ---: | +| 4 s | on | ~12 s (768x768), ~15 s (1344x768) | +| 8 s | on | ~26 s | +| 15 s | on | ~57 s | +| 5 s | off | ~19 s | +| 10 s | off | ~41 s | +| 15 s | off | ~73 s | + +These describe the validated shapes rather than a general throughput guarantee. + +> A native `--lora-path + --enable-distributed-layerwise-offload` path that +> fuses during the offload memory-map (avoiding the offline step) is possible +> through the offload backend's per-tensor transform hook, but is left as future +> work pending upstream design discussion. + + +## HTTP API examples Switch tasks via `extra_params.task` (`t2va` / `fl2va` / `ref2va`); FL2VA and Ref2VA additionally pass `input_references` (first-frame image, reference video/audio). The full parameter table lives in the GPU recipe.