vllm-omni/recipes/SandAI/MAGI-2-preview-L20X.md

MAGI-2 Preview — NVIDIA L20X

Native text/image-to-video-and-audio generation with the released Preview stage

Summary

• Vendor: SandAI
• Model: sand-ai/MAGI-2-preview
• Runtime: Native vLLM-Omni pipeline; no SandAI runtime import
• Modes: Offline shared examples and OpenAI-compatible video serving
• Hardware: NVIDIA L20X 140 GiB
• Recommended deployment: Four-GPU resident SP4 (TP=1, SP=4)
• Maintainer: Community

Supported model contract

Tasks and inputs

Task	Text prompt	Media condition	Shared entrypoint
T2VA	Required	None	text_to_video.py
I2VA	Required	Exactly one still image	image_to_video.py

The I2VA path preserves the source aspect ratio and applies MAGI-2's native resize/pad transform. Video or audio conditions, multiple images, and fused per-rank request batches are not supported.

Output

Category	Supported Preview specification
Duration	Exactly 10 seconds
Video	125 frames at 12.5 fps
Native tiers	272p (448x256) and 540p (896x512)
Optional final resize	output_width and output_height must be supplied together
Audio	44.1 kHz stereo, generated with the video
Refiner	The separate 1080p refiner is not included in this integration

Profiles provided by this recipe

Profile	Devices	Purpose	Qualification
Resident SP4	4	Recommended fidelity/default deployment	Released smoke and reference parity
Rank-local DLO SP4	4	Full-quality lower-HBM transformer streaming	Released checkpoint, 100-step T2VA and I2VA
DLO DP4 / DP2SP2	4	Concurrent request throughput	Bounded one- and four-step coverage
Ordinary layerwise	1	Memory-constrained single request	Released checkpoint, one and four steps
Online resident SP4	4	OpenAI-compatible video API	Serving path covered

References

• Official model card: https://huggingface.co/sand-ai/MAGI-2-preview
• Architecture blog: https://sand.ai/blog/magi-2-preview
• Supported models: docs/models/supported_models.md
• Diffusion feature matrix: docs/user_guide/diffusion_features.md

Prepare the checkpoint

The native Preview path uses approximately 274 GiB on disk. The separate refiner is not required.

export MAGI2_CKPT_ROOT=/path/to/MAGI-2-preview

hf download sand-ai/MAGI-2-preview \
  preview/ text_encoder/ vae/ turbo_vae/ stable-audio-open-1.0/ \
  --revision 2dea51b64db47ee5b4402d36fd90829a0c58913b \
  --local-dir "$MAGI2_CKPT_ROOT"

Install vLLM-Omni and ensure ffmpeg is on PATH. No second model runtime is required.

Hardware support

Locally qualified NVIDIA configuration

Item	Value
Accelerator	NVIDIA L20X
Per-device memory	143,771 MiB (about 140.4 GiB)
Device count	4; the one-device profile uses one of the same GPUs
Device interconnect	All-to-all NV18 links (nvidia-smi topo -m)
Host memory	2.0 TiB

The official SandAI runtime requires eight NVIDIA Hopper GPUs. That official eight-GPU system was unavailable for this PR; compatible eight-worker topologies pass configuration validation but are not locally runtime-qualified. No H100, H200, B200, or B300 runtime claim is made here.

Software environment

Item	Value
OS	Ubuntu 22.04.5 LTS, Linux 5.10.134
Python	3.12.13
NVIDIA driver / CUDA runtime	570.133.20 / 13.0
PyTorch	2.11.0+cu130
vLLM	0.26.0
vLLM-Omni	This PR branch; evidence commits a5af2a8c and 42bbfb57
Precision	BF16

Set MAGI2_DETERMINISTIC=1 before worker startup when deterministic kernels are required. The setting is fixed for the worker lifetime.

Each worker process supports one MAGI-2 pipeline instance. Pipeline startup sets process-wide deterministic state, so multi-pipeline tests and deployments must use separate worker processes rather than constructing multiple instances inside one process.

MAGI-2 uses vLLM's bundled FlashAttention 3 kernels by default on Hopper (compute capability 9.x), FlashAttention 4 on supported Blackwell devices with FA3 as the fallback, and FlashAttention 2 on earlier architectures. Set MAGI2_FLASH_ATTN_VERSION=2, 3, or 4 before worker startup to force an available kernel for diagnostics or output comparisons.

Offline generation

Recommended four-GPU T2VA

export CUDA_VISIBLE_DEVICES=0,1,2,3

python examples/offline_inference/text_to_video/text_to_video.py \
  --model "$MAGI2_CKPT_ROOT" \
  --model-class-name Magi2Pipeline \
  --prompt "A red fox walks through fresh snow while wind moves the pine branches." \
  --height 512 --width 896 --num-frames 125 \
  --num-inference-steps 100 --fps 12.5 \
  --tensor-parallel-size 1 --ulysses-degree 4 \
  --extra-body '{"seconds":10,"resolution":"540p"}' \
  --output magi2_540p_t2va.mp4

For a load-and-kernel smoke, use one inference step. A one-step output is not a quality evaluation.

Recommended four-GPU I2VA

Use the same geometry and parallelism with the shared I2V entrypoint:

python examples/offline_inference/image_to_video/image_to_video.py \
  --model "$MAGI2_CKPT_ROOT" \
  --model-class-name Magi2Pipeline \
  --image /path/to/first_frame.png \
  --prompt "The fox looks up, then walks forward as snow falls around it." \
  --height 512 --width 896 --num-frames 125 \
  --num-inference-steps 100 --fps 12.5 \
  --tensor-parallel-size 1 --ulysses-degree 4 \
  --extra-body '{"seconds":10,"resolution":"540p"}' \
  --output magi2_540p_i2va.mp4

One-device layerwise offload

MAGI-2 already stages Qwen, the image VAE, TurboVAE, and Oobleck from pinned CPU memory. Ordinary layerwise offload streams the Preview DiT blocks so the complete transformer does not need to fit in HBM at once.

export CUDA_VISIBLE_DEVICES=0

python examples/offline_inference/text_to_video/text_to_video.py \
  --model "$MAGI2_CKPT_ROOT" \
  --model-class-name Magi2Pipeline \
  --prompt "A red fox walks through fresh snow while wind moves the pine branches." \
  --height 256 --width 448 --num-frames 125 \
  --num-inference-steps 4 --fps 12.5 \
  --enable-cpu-offload --enable-layerwise-offload \
  --extra-body '{"seconds":10,"resolution":"272p"}' \
  --output magi2_272p_cpu_layerwise.mp4

Standalone --enable-cpu-offload is rejected because staging whole modules does not make the complete Preview DiT fit on one qualified GPU. When both flags are supplied, the shared offloader selects layerwise mode and MAGI's native auxiliary staging remains active.

Online serving

The OpenAI-compatible serving defaults are the full 540p preset and 100 denoising steps. The model always generates a 10-second, 125-frame clip at 12.5 fps; pass resolution=272p and num_inference_steps=1 for a low-cost load smoke. Requests with another duration are rejected before generation.

export CUDA_VISIBLE_DEVICES=0,1,2,3

vllm serve "$MAGI2_CKPT_ROOT" --omni \
  --model-class-name Magi2Pipeline \
  --num-gpus 4 \
  --tensor-parallel-size 1 \
  --ulysses-degree 4 \
  --port 8091

curl -X POST http://localhost:8091/v1/videos/sync \
  -F 'prompt=A red fox walks through fresh snow.' \
  -F 'seconds=10' \
  -F 'size=896x512' \
  -F 'num_frames=125' \
  -F 'fps=12.5' \
  -F 'num_inference_steps=100' \
  -F 'seed=42' \
  -F 'extra_params={"resolution":"540p"}' \
  -o magi2_online.mp4

For I2VA, add -F 'input_reference=@first_frame.png;type=image/png'.

Supported features

Use the shared guides for launch syntax and feature semantics. The flags below are the MAGI-2-specific qualified combinations.

Feature	MAGI-2 status / topology	Guide
Sequence parallel	Resident SP4 default; compatible SP8 configuration	Sequence parallel
Tensor parallel	TP4 or TP2SP2 on four workers	Tensor parallel
DLO	DP4/DP2SP2 AllGather; SP4 rank-local requires --dlo-no-use-allgather	Distributed layerwise offload
HSDP	HSDP4+SP4; alternative to TP and DLO	HSDP
CFG parallel	CFG2xSP2 on four workers; CFG2xSP4 requires eight	CFG parallel
VAE patch parallel	TurboVAE tile decode across the complete worker group	VAE parallelism
Cache-DiT	Repeated Preview transformer layers; approximate	Cache-DiT
One-device offload	Ordinary layerwise DiT plus native auxiliary CPU staging	CPU offload
Quantization	Not supported	Feature matrix
Ring / pipeline parallel	Not supported	Feature matrix

DP request waves must contain exactly data_parallel_size requests with the same explicit step count. Data-parallel replicas require TP=1. SP-only DLO cannot AllGather because SP ranks own different MoE-head shards.

Request fields

Common geometry and sampling values use the shared CLI flags. The native Preview pipeline accepts these model-specific --extra-body fields:

Field	Meaning
seconds	Must be 10.
resolution	272p or 540p; default 540p.
output_width, output_height	Optional final resize; supply both.
deterministic	Must match MAGI2_DETERMINISTIC at worker startup.

Use --image in the shared I2V entrypoint rather than the lower-level image_path field.

Verification

ffprobe -v error \
  -show_entries stream=codec_type,width,height,avg_frame_rate,nb_frames,sample_rate,channels \
  -of json magi2_540p_t2va.mp4

A 540p result contains 125 896x512 frames at 12.5 fps and stereo 44.1 kHz audio. A 272p result contains 125 448x256 frames with the same duration, rate, and audio contract.

Local qualification evidence

Released-checkpoint E2E

Profile	Workload	Steps	E2E	Peak HBM	Peak host PSS	Output
DLO SP4	540p T2VA	100	563.855 s	See PR evidence	See PR evidence	3437d078e...f16fdea
DLO SP4	540p I2VA	100	556.533 s	See PR evidence	See PR evidence	b8c2a4ac...ff2de8
Layerwise 1 GPU	272p T2VA	1	8.86 s	49.29 GiB	Not sampled	Valid video + audio
Layerwise 1 GPU	272p T2VA	4	42.55 s	49.29 GiB	371.97 GiB	Valid video + audio

The one-device four-step host-PSS sampler scans the full process tree and adds observable CPU overhead, so its stage timing is qualification evidence rather than a latency benchmark.

Distributed correctness

• Resident TP4, TP2SP2, and SP4 match a single-rank native oracle within the documented BF16 tolerance; resident SP4 is the reference-aligned default.
• Deterministic rank-local DLO SP4 output matched resident SP4 exactly.
• HSDP4+SP4, HSDP4+CFG2xSP2, Cache-DiT forced hits, and TurboVAE PP4 have focused four-rank coverage. TurboVAE PP4 matched resident decode exactly.
• Full 100-step 540p quality generation is qualified only for four-device DLO SP4. Eight-worker layouts remain configuration-only on this host.

These are bounded qualification runs, not committed benchmarks. Generated artifacts stay outside the repository, and this integration adds no model-specific example or benchmark script.