[Model] Add native SANA-Video 2B T2V and I2V support (#5508)
Signed-off-by: leo <92263211+napleon-liu@users.noreply.github.com> Signed-off-by: leo <92263211+liuyao0322@users.noreply.github.com> Signed-off-by: liuyao0322 <yaoliu548926@gmail.com> Co-authored-by: leo <92263211+napleon-liu@users.noreply.github.com>
This commit is contained in:
1 parent
4af0ddc1c2
commit
50b8a150f9
42 files changed
+6512
-18
No files matched your search
@@ -4,8 +4,8 @@ Source <https://github.com/vllm-project/vllm-omni/tree/main/examples/offline_inf
|
||||
|
||||
|
||||
This example demonstrates how to generate videos from images using Wan2.2,
|
||||
LTX-2, and HunyuanVideo-1.5 Image-to-Video models with vLLM-Omni's
|
||||
offline inference API.
|
||||
LTX-2, HunyuanVideo-1.5, and SANA-Video Image-to-Video models with
|
||||
vLLM-Omni's offline inference API.
|
||||
|
||||
## Local CLI Usage
|
||||
|
||||
@@ -66,9 +66,49 @@ python image_to_video.py \
|
||||
See the [LTX-2 recipe](../../../../recipes/LTX/LTX-2.md) for all checkpoints,
|
||||
pipeline selection, T2V, defaults, and advanced options.
|
||||
|
||||
### SANA-Video-2B
|
||||
|
||||
SANA checkpoints identify their upstream T2V pipeline in `model_index.json`.
|
||||
Select the native I2V implementation explicitly:
|
||||
|
||||
```bash
|
||||
python image_to_video.py \
|
||||
--model Efficient-Large-Model/SANA-Video_2B_480p_diffusers \
|
||||
--model-class-name SanaImageToVideoPipeline \
|
||||
--image cherry_blossom.jpg \
|
||||
--prompt "Cherry blossoms sway in the breeze as petals drift past the camera." \
|
||||
--negative-prompt "blurry, low quality, temporal artifacts" \
|
||||
--height 480 \
|
||||
--width 832 \
|
||||
--num-frames 81 \
|
||||
--num-inference-steps 50 \
|
||||
--guidance-scale 6.0 \
|
||||
--extra-body '{"motion_score": 30}' \
|
||||
--fps 16 \
|
||||
--seed 42 \
|
||||
--output sana_video_i2v_480p.mp4
|
||||
```
|
||||
|
||||
For the 720p checkpoint, use
|
||||
`Efficient-Large-Model/SANA-Video_2B_720p_diffusers` with
|
||||
`--height 704 --width 1280`. The native I2V path supports both checkpoint
|
||||
variants: the 480p checkpoint uses the Wan VAE, while the 720p checkpoint uses
|
||||
the LTX-2 Video VAE.
|
||||
|
||||
For SANA-Video, 81 frames at 16 FPS is the standard checkpoint request
|
||||
(approximately five seconds), not minute-scale long-video generation.
|
||||
Minute-scale SANA generation requires the separate LongSANA/LongLive
|
||||
autoregressive workflow, which this pipeline does not implement.
|
||||
|
||||
See the [SANA-Video recipe](../../../../recipes/NVIDIA/SANA-Video-2B.md) for
|
||||
online serving, backend boundaries, and the tested hardware profile.
|
||||
|
||||
Key arguments:
|
||||
|
||||
- `--model`: Model ID (I2V-A14B for MoE, TI2V-5B for unified T2V+I2V, or LTX-2).
|
||||
- `--model`: Model ID or local path (for example Wan I2V/TI2V, LTX-2, or
|
||||
SANA-Video).
|
||||
- `--model-class-name`: Explicit pipeline override. SANA-Video I2V requires
|
||||
`SanaImageToVideoPipeline`.
|
||||
- `--image`: Path to input image (required).
|
||||
- `--prompt`: Text description of desired motion/animation.
|
||||
- `--height/--width`: Output resolution (auto-calculated from image if not set).
|
||||
|
||||
@@ -16,6 +16,8 @@ For backend selection and SageAttention usage, see the [Diffusion Attention Back
|
||||
| `Lightricks/LTX-2` | 512x768 | 121 | 40 | video 3.0 / audio 7.0 | Model-dependent |
|
||||
| `hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-480p_t2v` | 480x832 | 121 | 50 | 6.0 | 1×A100 80GB |
|
||||
| `hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-720p_t2v` | 720x1280 | 121 | 50 | 6.0 | FP8 + VAE tiling required |
|
||||
| `Efficient-Large-Model/SANA-Video_2B_480p_diffusers` | 480x832 | 81 | 50 | 6.0 | BF16 DiT + FP32 Wan VAE |
|
||||
| `Efficient-Large-Model/SANA-Video_2B_720p_diffusers` | 704x1280 | 81 | 50 | 6.0 | ~23.6 GiB |
|
||||
|
||||
## Local CLI Usage
|
||||
|
||||
@@ -122,6 +124,31 @@ python text_to_video.py \
|
||||
--output quick_test.mp4
|
||||
```
|
||||
|
||||
### SANA-Video-2B
|
||||
|
||||
```bash
|
||||
python text_to_video.py \
|
||||
--model Efficient-Large-Model/SANA-Video_2B_480p_diffusers \
|
||||
--model-class-name SanaVideoPipeline \
|
||||
--prompt "A cinematic tracking shot of a sailboat crossing the ocean at sunset." \
|
||||
--height 480 \
|
||||
--width 832 \
|
||||
--num-frames 81 \
|
||||
--num-inference-steps 50 \
|
||||
--guidance-scale 6.0 \
|
||||
--extra-body '{"motion_score": 30}' \
|
||||
--fps 16 \
|
||||
--output sana_video_480p.mp4
|
||||
```
|
||||
|
||||
For the 720p checkpoint, switch the model to
|
||||
`Efficient-Large-Model/SANA-Video_2B_720p_diffusers`, use
|
||||
`--height 704 --width 1280`, and write to a different output path. SANA-Video
|
||||
accepts `motion_score`, `clean_caption`, and `use_resolution_binning` through
|
||||
`--extra-body`. For image-to-video, use the shared
|
||||
`examples/offline_inference/image_to_video/image_to_video.py` example with
|
||||
`--model-class-name SanaImageToVideoPipeline` and `--image <path>`.
|
||||
|
||||
## Key Arguments
|
||||
|
||||
### Common
|
||||
|
||||
Reference in new issue
Block a user