vllm-omni/examples/offline_inference/magi_human

MagiHuman Generation

MagiHuman is an advanced, omni-modality model that generates both high-quality video and lip-synced audio from a text prompt.

Because MagiHuman is a very large model featuring a powerful DiT MoE backbone and a ~9B parameter T5Gemma text encoder, it natively supports Tensor Parallelism (TP) in vLLM-Omni to run efficiently across multi-GPU setups, reducing device memory bottlenecks.

Setup

Install MagiCompiler (recommended)

MagiHuman relies on MagiCompiler for custom-op registration used by the DiT attention kernels. While the pipeline can fall back to stub implementations, installing MagiCompiler is strongly recommended for correct behaviour.

# Clone the repo
git clone https://github.com/SandAI-org/MagiCompiler.git
cd MagiCompiler

# System dependencies (optional, for FX graph visualization; Debian/Ubuntu)
sudo apt update && sudo apt install -y graphviz

# Python dependencies
pip install -r requirements.txt

# Install MagiCompiler
pip install .          # end users (recommended)
# pip install -e .     # developers (editable install)

Hardware requirements

Ensure your hardware has enough VRAM. For a standard node with 80GB GPUs, running with --tensor-parallel-size 4 is recommended to shard both the MoE weights and the T5Gemma text encoder across 4 GPUs, reducing the per-GPU peak VRAM overhead significantly (by roughly ~13.5GB per GPU compared to single-device inference).

Please refer to the stage configuration documentation for further details on allocating memory.

Run Examples

Get into the example folder:

cd examples/offline_inference/magi_human

End-to-End Generation (Text to Video+Audio)

Generate a video with synchronized speech natively generated by the model.

python end2end.py \
    --model /proj-tango-pvc/users/zhipeng.wang/workspace/models/daVinci-MagiHuman \
    --prompt "A young woman with long, wavy golden blonde hair..." \
    --tensor-parallel-size 4 \
    --output output_magihuman.mp4

Common Parameters

Parameter	Default	Description
--model	(Required)	Local model path or HuggingFace ID
--prompt	(built-in demo prompt)	Highly detailed text prompt dictating visual look and dialogue text
--tensor-parallel-size	4	Tensor parallelism size (Number of GPUs)
--height	256	Initial resolution height
--width	448	Initial resolution width
--num-inference-steps	8	Denoising steps
--seed	52	Random seed
--output	output_magihuman.mp4	Output video with audio path

Example materials

??? abstract "end2end.py" py --8<-- "examples/offline_inference/magi_human/end2end.py"