[Model] Add native SANA-Video 2B T2V and I2V support (#5508)

Signed-off-by: leo <92263211+napleon-liu@users.noreply.github.com>
Signed-off-by: leo <92263211+liuyao0322@users.noreply.github.com>
Signed-off-by: liuyao0322 <yaoliu548926@gmail.com>
Co-authored-by: leo <92263211+napleon-liu@users.noreply.github.com>

examples/offline_inference/image_to_video/README.md

@@ -1,7 +1,7 @@
 # Image-To-Video
 
 This shared example generates videos from images with VACE, Wan2.2, LTX-2,
-HunyuanVideo-1.5, Cosmos3, and other compatible pipelines.
+HunyuanVideo-1.5, SANA-Video, Cosmos3, and other compatible pipelines.
 
 - `image_to_video.py`: command-line script for single video generation with advanced options.
 
@@ -27,6 +27,8 @@ This folder provides a unified CLI script for image-to-video generation using vL
 | `Wan-AI/Wan2.2-TI2V-5B-Diffusers` | 480 x 832 | 81 | 50 | 4.0 | Around 20–25 GiB BF16, smallest I2V model |
 | `hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-480p_i2v` | 480 x 832 | 121 | 50 | 6.0 | Around 100 GiB at default settings; the example enables `--enable-cpu-offload` + VAE tiling/slicing to fit an 80 GiB card |
 | `Lightricks/LTX-2` | 512 x 768 | 121 | 40 | video 3.0 / audio 7.0 | Memory use depends on frame count and tensor parallelism |
+| `Efficient-Large-Model/SANA-Video_2B_480p_diffusers` | 480 x 832 | 81 | 50 | 6.0 | Native `SanaImageToVideoPipeline`; Wan VAE |
+| `Efficient-Large-Model/SANA-Video_2B_720p_diffusers` | 704 x 1280 | 81 | 50 | 6.0 | Native `SanaImageToVideoPipeline`; LTX-2 Video VAE |
 
 !!! info
     Peak VRAM: based on basic single-card usage, batch size = 1, without any acceleration/optimization features. Some model weights cannot fit into one card with 80 GiB VRAM, which may need to use CPU offloading.
@@ -227,6 +229,48 @@ python image_to_video.py \
 See the [LTX-2 recipe](../../../recipes/LTX/LTX-2.md) for all checkpoints,
 pipeline selection, T2V, defaults, and advanced options.
 
+### SANA-Video-2B
+
+The checkpoint metadata selects the upstream T2V pipeline, so pass
+`SanaImageToVideoPipeline` explicitly for I2V:
+
+```bash
+python image_to_video.py \
+  --model Efficient-Large-Model/SANA-Video_2B_480p_diffusers \
+  --model-class-name SanaImageToVideoPipeline \
+  --image cherry_blossom.jpg \
+  --prompt "Cherry blossoms sway in the breeze as petals drift past the camera." \
+  --negative-prompt "blurry, low quality, temporal artifacts" \
+  --height 480 \
+  --width 832 \
+  --num-frames 81 \
+  --num-inference-steps 50 \
+  --guidance-scale 6.0 \
+  --extra-body '{"motion_score": 30}' \
+  --fps 16 \
+  --seed 42 \
+  --output sana_video_i2v_480p.mp4
+```
+
+Use the 720p checkpoint with `--height 704 --width 1280`. In this example,
+81 frames at 16 FPS means the standard approximately five-second SANA-Video
+checkpoint request. It does not enable the separate LongSANA/LongLive
+minute-scale autoregressive workflow.
+
+The native `SanaImageToVideoPipeline` supports both checkpoint variants. The
+480p path uses vLLM-Omni's `DistributedAutoencoderKLWan` wrapper and the 720p
+path uses `DistributedAutoencoderKLLTX2Video`; both wrap their corresponding
+Diffusers VAE implementations. The native denoising loop intentionally keeps
+Diffusers' checkpoint-compatible `DPMSolverMultistepScheduler`.
+
+The Diffusers adapter has completed end-to-end I2V serving validation at both
+480p and 720p. For online adapter serving, select the matching checkpoint and
+use `WIDTH=1280 HEIGHT=704` for the 720p request. Adapter T2V is also validated
+at both resolutions; see the recipe for the exact backend matrix.
+
+See the [SANA-Video recipe](../../../recipes/NVIDIA/SANA-Video-2B.md) for
+online serving, native/adapter validation boundaries, and hardware notes.
+
 ## Advanced Features
 
 ### CFG Parallel

examples/offline_inference/image_to_video/image_to_video.py

@@ -3,13 +3,14 @@
 
 """
 Image-to-Video generation example using Wan2.2 I2V/TI2V models, LTX2/LTX-2.3,
-HunyuanVideo-1.5, Cosmos3, or Wan2.1 VACE.
+HunyuanVideo-1.5, SANA-Video, Cosmos3, or Wan2.1 VACE.
 
 Supports:
 - Wan2.2-I2V-A14B-Diffusers: MoE model with CLIP image encoder
 - Wan2.2-TI2V-5B-Diffusers: Unified T2V+I2V model (dense 5B)
 - LTX2 image-to-video pipeline
 - HunyuanVideo-1.5 I2V: SigLIP + VAE dual image conditioning
+- SANA-Video 2B: first-frame latent conditioning at 480p or 720p
 - Wan2.1 VACE: first/last-frame, inpainting, and reference conditioning
 
 Usage:
@@ -38,6 +39,13 @@ Usage:
         --image input.jpg --prompt "A cat playing with yarn" \
         --flow-shift 5.0 --guidance-scale 6.0
 
+    # SANA-Video 2B I2V (480p)
+    python image_to_video.py --model Efficient-Large-Model/SANA-Video_2B_480p_diffusers \
+        --model-class-name SanaImageToVideoPipeline \
+        --image input.jpg --prompt "A cat turns toward the camera." \
+        --height 480 --width 832 --num-frames 81 --num-inference-steps 50 \
+        --guidance-scale 6.0
+
     # Cosmos3 I2V (image conditioning)
     python image_to_video.py --model nvidia/Cosmos3-Nano \
         --image input.jpg --prompt "The scene comes to life with smooth, natural motion." \
@@ -107,13 +115,16 @@ def parse_args() -> argparse.Namespace:
     parser = argparse.ArgumentParser(
         description=(
             "Generate a video from one or more images "
-            "(Wan2.2, LTX2/LTX-2.3, HunyuanVideo-1.5, Cosmos3, or Wan2.1 VACE)."
+            "(Wan2.2, LTX2/LTX-2.3, HunyuanVideo-1.5, SANA-Video, Cosmos3, or Wan2.1 VACE)."
         )
     )
     parser.add_argument(
         "--model",
         default="Wan-AI/Wan2.2-I2V-A14B-Diffusers",
-        help="Diffusers I2V model ID or local path (Wan2.2, LTX2/LTX-2.3, HunyuanVideo-1.5, Cosmos3, or Wan2.1 VACE).",
+        help=(
+            "Diffusers I2V model ID or local path "
+            "(Wan2.2, LTX2/LTX-2.3, HunyuanVideo-1.5, SANA-Video, Cosmos3, or Wan2.1 VACE)."
+        ),
     )
     parser.add_argument(
         "--model-class-name",
@@ -377,6 +388,7 @@ def main():
     is_ltx2_distilled = "distilled" in model_class_name_lower or "distilled" in model_name
     is_ltx23 = "ltx23" in model_class_name_lower or "ltx-2.3" in model_name
     is_ltx2 = is_ltx2_distilled or is_ltx23 or "ltx2" in model_class_name_lower or "ltx-2" in model_name
+    is_sana = model_class_name == "SanaImageToVideoPipeline" or "sana-video" in model_name
     is_cosmos = "cosmos" in model_name or (model_class_name is not None and "cosmos" in model_class_name.lower())
     is_cosmos_edge = is_cosmos and ("edge" in model_name or "edge" in model_class_name_lower)
 
@@ -434,6 +446,17 @@ def main():
             512 * 768,
             32,
         )
+    elif is_sana:
+        is_720p = "720p" in model_name
+        d_fps, d_guidance, d_num_frames, d_steps, d_flow_shift, d_max_area, d_mod = (
+            16,
+            6.0,
+            81,
+            50,
+            5.0,
+            (704 * 1280) if is_720p else (480 * 832),
+            32 if is_720p else 16,
+        )
     else:  # Wan2.2 / HunyuanVideo-1.5
         d_fps, d_guidance, d_num_frames, d_steps, d_flow_shift, d_max_area, d_mod = 16, 5.0, 81, 50, 5.0, 480 * 832, 16
 

examples/offline_inference/text_to_video/text_to_video.md

@@ -14,6 +14,8 @@ A unified script for text-to-video generation. Supports multiple models with mod
 | `hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-720p_t2v` | 720x1280 | 121 | 50 | 6.0 | FP8 + VAE tiling required |
 | `nvidia/Cosmos3-Nano` | 720x1280 | 189 | 35 | 6.0 | ~46 GiB (peak, 720p) |
 | `BestWishYsh/Helios-Base` / `Helios-Mid` / `Helios-Distilled` | 384x640 | 99 | 50 | 5.0 / 5.0 / 1.0 | — |
+| `Efficient-Large-Model/SANA-Video_2B_480p_diffusers` | 480x832 | 81 | 50 | 6.0 | BF16 DiT + FP32 Wan VAE wrapper |
+| `Efficient-Large-Model/SANA-Video_2B_720p_diffusers` | 704x1280 | 81 | 50 | 6.0 | BF16 DiT + LTX-2 Video VAE wrapper |
 
 ## Local CLI Usage
 
@@ -151,6 +153,40 @@ python text_to_video.py \
   --output quick_test.mp4
 ```
 
+### SANA-Video-2B
+
+Select the native T2V pipeline explicitly:
+
+```bash
+python text_to_video.py \
+  --model Efficient-Large-Model/SANA-Video_2B_480p_diffusers \
+  --model-class-name SanaVideoPipeline \
+  --prompt "A cinematic tracking shot of a sailboat crossing the ocean at sunset." \
+  --height 480 \
+  --width 832 \
+  --num-frames 81 \
+  --num-inference-steps 50 \
+  --guidance-scale 6.0 \
+  --extra-body '{"motion_score": 30}' \
+  --fps 16 \
+  --output sana_video_480p.mp4
+```
+
+For the 720p checkpoint, switch the model to
+`Efficient-Large-Model/SANA-Video_2B_720p_diffusers`, use
+`--height 704 --width 1280`, and write to a different output path. Both native
+checkpoint variants are supported. The Diffusers adapter T2V compatibility
+path is also validated at both resolutions through online serving.
+
+The native 480p path uses vLLM-Omni's `DistributedAutoencoderKLWan` wrapper;
+the native 720p path uses `DistributedAutoencoderKLLTX2Video`. Both wrappers
+retain their underlying Diffusers autoencoders. The pipeline intentionally
+uses Diffusers' checkpoint-compatible `DPMSolverMultistepScheduler`.
+
+For image-to-video, use the shared
+[`image_to_video.py`](../image_to_video/README.md#sana-video-2b) example with
+`--model-class-name SanaImageToVideoPipeline` and `--image <path>`.
+
 ### Cosmos3
 
 ```bash

examples/offline_inference/text_to_video/text_to_video.py

@@ -114,12 +114,32 @@ _MODEL_PRESETS = {
         "fps": 24,
         "output": "ltx23_output.mp4",
     },
+    "sana_480p": {
+        "height": 480,
+        "width": 832,
+        "num_frames": 81,
+        "num_inference_steps": 50,
+        "guidance_scale": 6.0,
+        "fps": 16,
+        "output": "sana_video_480p.mp4",
+    },
+    "sana_720p": {
+        "height": 704,
+        "width": 1280,
+        "num_frames": 81,
+        "num_inference_steps": 50,
+        "guidance_scale": 6.0,
+        "fps": 16,
+        "output": "sana_video_720p.mp4",
+    },
 }
 
 
 def _detect_preset(model: str, model_class_name: str | None = None) -> dict:
     model_lower = model.lower()
     class_lower = (model_class_name or "").lower()
+    if "sana-video" in model_lower or "sana_video" in model_lower or "sanavideo" in class_lower:
+        return _MODEL_PRESETS["sana_720p" if "720p" in model_lower else "sana_480p"]
     if "lingbot" in model_lower or "lingbotvideo" in class_lower:
         return _MODEL_PRESETS["lingbot"]
     if "ltx" in class_lower or "ltx" in model_lower:

examples/online_serving/image_to_video/README.md

@@ -1,5 +1,19 @@
 # Image-To-Video
 
+This example demonstrates how to deploy image-to-video models for online
+video generation using vLLM-Omni.
+
+## Supported Models
+
+| Model | Model ID |
+|-------|----------|
+| Wan2.2 I2V | `Wan-AI/Wan2.2-I2V-A14B-Diffusers` |
+| SANA-Video 2B (480p) | `Efficient-Large-Model/SANA-Video_2B_480p_diffusers` |
+| SANA-Video 2B (720p) | `Efficient-Large-Model/SANA-Video_2B_720p_diffusers` |
+
+For SANA-Video 2B image-to-video online serving, see the
+[SANA-Video 2B recipe](../../../recipes/NVIDIA/SANA-Video-2B.md).
+
 This example demonstrates online image-to-video generation with vLLM-Omni.
 The startup script defaults to Wan2.2, but `MODEL` can select any supported
 image-to-video model. The existing curl helper remains a concrete Wan2.2

examples/online_serving/image_to_video/run_curl_sana_video.sh

@@ -0,0 +1,67 @@
+#!/bin/bash
+# Submit and download a standard SANA-Video-2B image-to-video request.
+
+set -euo pipefail
+
+INPUT_IMAGE="${INPUT_IMAGE:-}"
+BASE_URL="${BASE_URL:-http://localhost:8099}"
+OUTPUT_PATH="${OUTPUT_PATH:-sana_video_i2v.mp4}"
+WIDTH="${WIDTH:-832}"
+HEIGHT="${HEIGHT:-480}"
+POLL_INTERVAL="${POLL_INTERVAL:-2}"
+
+if [ -z "$INPUT_IMAGE" ] || [ ! -f "$INPUT_IMAGE" ]; then
+    echo "Set INPUT_IMAGE to an existing image file."
+    exit 1
+fi
+
+create_response="$(
+    curl -sS -X POST "${BASE_URL}/v1/videos" \
+        -H "Accept: application/json" \
+        -F "prompt=A cat turns toward the camera with smooth, natural motion." \
+        -F "negative_prompt=blurry, low quality, temporal artifacts" \
+        -F "input_reference=@${INPUT_IMAGE}" \
+        -F "width=${WIDTH}" \
+        -F "height=${HEIGHT}" \
+        -F "num_frames=81" \
+        -F "fps=16" \
+        -F "num_inference_steps=50" \
+        -F "guidance_scale=6.0" \
+        -F "seed=42" \
+        -F 'extra_params={"motion_score":30}'
+)"
+
+video_id="$(echo "${create_response}" | jq -r '.id')"
+if [ -z "${video_id}" ] || [ "${video_id}" = "null" ]; then
+    echo "Failed to create SANA-Video job:"
+    echo "${create_response}" | jq .
+    exit 1
+fi
+
+while true; do
+    status_response="$(curl -sS "${BASE_URL}/v1/videos/${video_id}")"
+    status="$(echo "${status_response}" | jq -r '.status')"
+
+    case "${status}" in
+        queued|in_progress)
+            echo "Video job ${video_id} status: ${status}"
+            sleep "${POLL_INTERVAL}"
+            ;;
+        completed)
+            break
+            ;;
+        failed)
+            echo "SANA-Video generation failed:"
+            echo "${status_response}" | jq .
+            exit 1
+            ;;
+        *)
+            echo "Unexpected status response:"
+            echo "${status_response}" | jq .
+            exit 1
+            ;;
+    esac
+done
+
+curl -sS -L "${BASE_URL}/v1/videos/${video_id}/content" -o "${OUTPUT_PATH}"
+echo "Saved SANA-Video to ${OUTPUT_PATH}"

examples/online_serving/image_to_video/run_server_sana_video.sh

@@ -0,0 +1,13 @@
+#!/bin/bash
+# SANA-Video-2B native image-to-video serving.
+
+set -euo pipefail
+
+MODEL="${MODEL:-Efficient-Large-Model/SANA-Video_2B_480p_diffusers}"
+PORT="${PORT:-8099}"
+
+vllm serve "$MODEL" \
+    --omni \
+    --model-class-name SanaImageToVideoPipeline \
+    --dtype bfloat16 \
+    --port "$PORT"

examples/online_serving/image_to_video/run_server_sana_video_diffusers.sh

@@ -0,0 +1,18 @@
+#!/bin/bash
+# SANA-Video-2B image-to-video serving through the Diffusers adapter.
+# Both 480p and 720p checkpoints are validated. For 720p:
+# MODEL=Efficient-Large-Model/SANA-Video_2B_720p_diffusers bash run_server_sana_video_diffusers.sh
+# INPUT_IMAGE=input.jpg WIDTH=1280 HEIGHT=704 bash run_curl_sana_video.sh
+
+set -euo pipefail
+
+MODEL="${MODEL:-Efficient-Large-Model/SANA-Video_2B_480p_diffusers}"
+PORT="${PORT:-8099}"
+
+vllm serve "$MODEL" \
+    --omni \
+    --model-class-name SanaImageToVideoPipeline \
+    --diffusion-load-format diffusers \
+    --diffusion-attention-backend TORCH_SDPA \
+    --dtype bfloat16 \
+    --port "$PORT"

examples/online_serving/text_to_video/README.md

@@ -11,6 +11,11 @@ This example demonstrates how to deploy text-to-video models for online video ge
 | Wan2.2 T2V | `Wan-AI/Wan2.2-T2V-A14B-Diffusers` |
 | LTX-2 | `Lightricks/LTX-2` |
 | Helios (Base / Mid / Distilled) | `BestWishYsh/Helios-Base`, `Helios-Mid`, `Helios-Distilled` |
+| SANA-Video 2B (480p) | `Efficient-Large-Model/SANA-Video_2B_480p_diffusers` |
+| SANA-Video 2B (720p) | `Efficient-Large-Model/SANA-Video_2B_720p_diffusers` |
+
+For SANA-Video 2B text-to-video online serving, see the
+[SANA-Video 2B recipe](../../../recipes/NVIDIA/SANA-Video-2B.md).
 
 ## Wan2.2 T2V
 

examples/online_serving/text_to_video/run_curl_sana_video.sh

@@ -0,0 +1,16 @@
+#!/bin/bash
+
+BASE_URL="${BASE_URL:-http://localhost:8091}"
+OUTPUT_PATH="${OUTPUT_PATH:-sana_video.mp4}"
+
+curl -sS -X POST "$BASE_URL/v1/videos/sync" \
+    -F "prompt=A cinematic tracking shot of a sailboat crossing the ocean at sunset" \
+    -F "height=480" \
+    -F "width=832" \
+    -F "num_frames=81" \
+    -F "fps=16" \
+    -F "num_inference_steps=50" \
+    -F "guidance_scale=6.0" \
+    -F "seed=42" \
+    -F 'extra_params={"motion_score":30}' \
+    --output "$OUTPUT_PATH"

examples/online_serving/text_to_video/run_server_sana_video.sh

@@ -0,0 +1,11 @@
+#!/bin/bash
+# SANA-Video-2B text-to-video serving.
+
+MODEL="${MODEL:-Efficient-Large-Model/SANA-Video_2B_480p_diffusers}"
+PORT="${PORT:-8091}"
+
+vllm serve "$MODEL" \
+    --omni \
+    --model-class-name SanaVideoPipeline \
+    --dtype bfloat16 \
+    --port "$PORT"

examples/online_serving/text_to_video/run_server_sana_video_diffusers.sh

@@ -0,0 +1,12 @@
+#!/bin/bash
+# SANA-Video through the black-box Diffusers backend adapter.
+
+MODEL="${MODEL:-Efficient-Large-Model/SANA-Video_2B_480p_diffusers}"
+PORT="${PORT:-8091}"
+
+vllm serve "$MODEL" \
+    --omni \
+    --diffusion-load-format diffusers \
+    --diffusion-attention-backend TORCH_SDPA \
+    --dtype bfloat16 \
+    --port "$PORT"