[Model] Add native SANA-Video 2B T2V and I2V support (#5508)

Signed-off-by: leo <92263211+napleon-liu@users.noreply.github.com>
Signed-off-by: leo <92263211+liuyao0322@users.noreply.github.com>
Signed-off-by: liuyao0322 <yaoliu548926@gmail.com>
Co-authored-by: leo <92263211+napleon-liu@users.noreply.github.com>
This commit is contained in:
leoandleo authored and GitHub committed 2026-08-25 09:46:02 +08:00
1 parent 4af0ddc1c2
commit 50b8a150f9
42 files changed
+6512 -18

No files matched your search

@@ -4,8 +4,8 @@ Source <https://github.com/vllm-project/vllm-omni/tree/main/examples/offline_inf
This example demonstrates how to generate videos from images using Wan2.2,
LTX-2, and HunyuanVideo-1.5 Image-to-Video models with vLLM-Omni's
offline inference API.
LTX-2, HunyuanVideo-1.5, and SANA-Video Image-to-Video models with
vLLM-Omni's offline inference API.
## Local CLI Usage
@@ -66,9 +66,49 @@ python image_to_video.py \
See the [LTX-2 recipe](../../../../recipes/LTX/LTX-2.md) for all checkpoints,
pipeline selection, T2V, defaults, and advanced options.
### SANA-Video-2B
SANA checkpoints identify their upstream T2V pipeline in `model_index.json`.
Select the native I2V implementation explicitly:
```bash
python image_to_video.py \
--model Efficient-Large-Model/SANA-Video_2B_480p_diffusers \
--model-class-name SanaImageToVideoPipeline \
--image cherry_blossom.jpg \
--prompt "Cherry blossoms sway in the breeze as petals drift past the camera." \
--negative-prompt "blurry, low quality, temporal artifacts" \
--height 480 \
--width 832 \
--num-frames 81 \
--num-inference-steps 50 \
--guidance-scale 6.0 \
--extra-body '{"motion_score": 30}' \
--fps 16 \
--seed 42 \
--output sana_video_i2v_480p.mp4
```
For the 720p checkpoint, use
`Efficient-Large-Model/SANA-Video_2B_720p_diffusers` with
`--height 704 --width 1280`. The native I2V path supports both checkpoint
variants: the 480p checkpoint uses the Wan VAE, while the 720p checkpoint uses
the LTX-2 Video VAE.
For SANA-Video, 81 frames at 16 FPS is the standard checkpoint request
(approximately five seconds), not minute-scale long-video generation.
Minute-scale SANA generation requires the separate LongSANA/LongLive
autoregressive workflow, which this pipeline does not implement.
See the [SANA-Video recipe](../../../../recipes/NVIDIA/SANA-Video-2B.md) for
online serving, backend boundaries, and the tested hardware profile.
Key arguments:
- `--model`: Model ID (I2V-A14B for MoE, TI2V-5B for unified T2V+I2V, or LTX-2).
- `--model`: Model ID or local path (for example Wan I2V/TI2V, LTX-2, or
SANA-Video).
- `--model-class-name`: Explicit pipeline override. SANA-Video I2V requires
`SanaImageToVideoPipeline`.
- `--image`: Path to input image (required).
- `--prompt`: Text description of desired motion/animation.
- `--height/--width`: Output resolution (auto-calculated from image if not set).
@@ -16,6 +16,8 @@ For backend selection and SageAttention usage, see the [Diffusion Attention Back
| `Lightricks/LTX-2` | 512x768 | 121 | 40 | video 3.0 / audio 7.0 | Model-dependent |
| `hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-480p_t2v` | 480x832 | 121 | 50 | 6.0 | 1×A100 80GB |
| `hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-720p_t2v` | 720x1280 | 121 | 50 | 6.0 | FP8 + VAE tiling required |
| `Efficient-Large-Model/SANA-Video_2B_480p_diffusers` | 480x832 | 81 | 50 | 6.0 | BF16 DiT + FP32 Wan VAE |
| `Efficient-Large-Model/SANA-Video_2B_720p_diffusers` | 704x1280 | 81 | 50 | 6.0 | ~23.6 GiB |
## Local CLI Usage
@@ -122,6 +124,31 @@ python text_to_video.py \
--output quick_test.mp4
```
### SANA-Video-2B
```bash
python text_to_video.py \
--model Efficient-Large-Model/SANA-Video_2B_480p_diffusers \
--model-class-name SanaVideoPipeline \
--prompt "A cinematic tracking shot of a sailboat crossing the ocean at sunset." \
--height 480 \
--width 832 \
--num-frames 81 \
--num-inference-steps 50 \
--guidance-scale 6.0 \
--extra-body '{"motion_score": 30}' \
--fps 16 \
--output sana_video_480p.mp4
```
For the 720p checkpoint, switch the model to
`Efficient-Large-Model/SANA-Video_2B_720p_diffusers`, use
`--height 704 --width 1280`, and write to a different output path. SANA-Video
accepts `motion_score`, `clean_caption`, and `use_resolution_binning` through
`--extra-body`. For image-to-video, use the shared
`examples/offline_inference/image_to_video/image_to_video.py` example with
`--model-class-name SanaImageToVideoPipeline` and `--image <path>`.
## Key Arguments
### Common