vllm-omni/examples/online_serving/text_to_video/README.md

Text-To-Video

This example demonstrates how to deploy text-to-video models for online video generation using vLLM-Omni.

Supported Models

Model	Model ID
Wan2.1 T2V (1.3B)	Wan-AI/Wan2.1-T2V-1.3B-Diffusers
Wan2.1 T2V (14B)	Wan-AI/Wan2.1-T2V-14B-Diffusers
Wan2.2 T2V	Wan-AI/Wan2.2-T2V-A14B-Diffusers
LTX-2	Lightricks/LTX-2
Helios (Base / Mid / Distilled)	BestWishYsh/Helios-Base, Helios-Mid, Helios-Distilled
SANA-Video 2B (480p)	Efficient-Large-Model/SANA-Video_2B_480p_diffusers
SANA-Video 2B (720p)	Efficient-Large-Model/SANA-Video_2B_720p_diffusers

For SANA-Video 2B text-to-video online serving, see the SANA-Video 2B recipe.

Wan2.2 T2V

Start Server

Basic Start

vllm serve Wan-AI/Wan2.2-T2V-A14B-Diffusers --omni --port 8091

Start with Parameters

Or use the startup script:

bash run_server.sh

The script allows overriding:

• MODEL (default: Wan-AI/Wan2.2-T2V-A14B-Diffusers)
• PORT (default: 8091)
• BOUNDARY_RATIO (default: 0.875)
• FLOW_SHIFT (default: 5.0)
• CACHE_BACKEND (default: none)
• ENABLE_CACHE_DIT_SUMMARY (default: 0)

Async Job Behavior

POST /v1/videos is asynchronous. It creates a video job and immediately returns metadata like the job ID and initial queued status. To get the final artifact, poll the job status and then download the completed file from the content endpoint.

The main endpoints are:

• POST /v1/videos: create a video generation job (async)
• POST /v1/videos/sync: generate a video and return raw bytes (sync, for benchmarks)
• GET /v1/videos/{video_id}: retrieve the current job status and metadata
• GET /v1/videos: list stored video jobs
• GET /v1/videos/{video_id}/content: download the generated video file
• DELETE /v1/videos/{video_id}: delete the job and any stored output

Sync API (Benchmark / Testing)

POST /v1/videos/sync is a synchronous alternative that blocks until generation completes and returns the raw video bytes (video/mp4) directly in the response body. It is designed for benchmark and testing scenarios where one-shot request/response latency measurement is needed.

The sync endpoint accepts the same form parameters as POST /v1/videos. It does not create any stored job record — the response is purely the generated video file. Metadata is returned via response headers:

• X-Request-Id: unique identifier for this generation request
• X-Model: model name used for generation
• X-Inference-Time-S: wall-clock inference time in seconds

curl -X POST http://localhost:8091/v1/videos/sync \
  -F "prompt=Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage." \
  -F "size=832x480" \
  -F "num_frames=33" \
  -F "fps=16" \
  -F "num_inference_steps=40" \
  -F "guidance_scale=4.0" \
  -F "guidance_scale_2=4.0" \
  -F "boundary_ratio=0.875" \
  -F "flow_shift=5.0" \
  -F "seed=42" \
  -o sync_t2v_output.mp4

Storage

Generated video files are stored on local disk by the async video API. Local file storage behavior can be controlled via the following environment variables:

• VLLM_OMNI_SERVER_STORAGE__PATH: directory used for generated files (default: /tmp/storage)
• VLLM_OMNI_SERVER_STORAGE__FILE_CONCURRENCY: max concurrent save/delete operations (default: 4)

VLLM_OMNI_STORAGE_PATH and VLLM_OMNI_STORAGE_MAX_CONCURRENCY are deprecated and will be removed in a future release; use the names above instead.

Example:

export VLLM_OMNI_SERVER_STORAGE__PATH=/var/tmp/vllm-omni-videos
export VLLM_OMNI_SERVER_STORAGE__FILE_CONCURRENCY=8

API Calls

Method 1: Using curl

# Basic text-to-video generation
bash run_curl_text_to_video.sh

# Or execute directly (OpenAI-style multipart)
create_response=$(curl -s http://localhost:8091/v1/videos \
  -H "Accept: application/json" \
  -F "prompt=Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage." \
  -F "width=832" \
  -F "height=480" \
  -F "num_frames=33" \
  -F "negative_prompt=色调艳丽 ，过曝，静态，细节模糊不清，字幕，风格，作品，画作，画面，静止，整体发灰，最差质量，低质量，JPEG压缩残留，丑陋的，残缺的，多余的手指，画得不好的手部，画得不好的脸部，畸形的，毁容的，形态畸形的肢体，手指融合，静止不动的画面，杂乱的背景，三条腿，背景人很多，倒着走" \
  -F "fps=16" \
  -F "num_inference_steps=40" \
  -F "guidance_scale=4.0" \
  -F "guidance_scale_2=4.0" \
  -F "boundary_ratio=0.875" \
  -F "flow_shift=5.0" \
  -F "seed=42")

video_id=$(echo "$create_response" | jq -r '.id')
while true; do
  status=$(curl -s "http://localhost:8091/v1/videos/${video_id}" | jq -r '.status')
  if [ "$status" = "completed" ]; then
    break
  fi
  if [ "$status" = "failed" ]; then
    echo "Video generation failed"
    exit 1
  fi
  sleep 2
done

curl -s "http://localhost:8091/v1/videos/${video_id}" | jq .
curl -L "http://localhost:8091/v1/videos/${video_id}/content" -o wan22_output.mp4

Request Format

Simple Text-to-Video Generation

curl -X POST http://localhost:8091/v1/videos \
  -F "prompt=A cinematic view of a futuristic city at sunset"

Generation with Parameters

curl -X POST http://localhost:8091/v1/videos \
  -F "prompt=A cinematic view of a futuristic city at sunset" \
  -F "width=832" \
  -F "height=480" \
  -F "num_frames=33" \
  -F "negative_prompt=low quality, blurry, static" \
  -F "fps=16" \
  -F "num_inference_steps=40" \
  -F "guidance_scale=4.0" \
  -F "guidance_scale_2=4.0" \
  -F "boundary_ratio=0.875" \
  -F "flow_shift=5.0" \
  -F "seed=42"

Generation Parameters

Parameter	Type	Default	Description
prompt	str	-	Text description of the desired video
seconds	str	None	Clip duration in seconds
size	str	None	Output size in WIDTHxHEIGHT format
negative_prompt	str	None	Negative prompt
width	int	None	Video width in pixels
height	int	None	Video height in pixels
num_frames	int	None	Number of frames to generate
fps	int	None	Frames per second for output video
num_inference_steps	int	None	Number of denoising steps
guidance_scale	float	None	CFG guidance scale (low-noise stage)
guidance_scale_2	float	None	CFG guidance scale (high-noise stage, Wan2.2)
boundary_ratio	float	None	Boundary split ratio for low/high DiT (Wan2.2)
flow_shift	float	None	Scheduler flow shift (Wan2.2)
seed	int	None	Random seed (reproducible)
lora	object	None	LoRA configuration

Create Response Format

POST /v1/videos returns a job record, not inline base64 video data.

{
  "id": "video_gen_123",
  "object": "video",
  "status": "queued",
  "model": "Wan-AI/Wan2.2-T2V-A14B-Diffusers",
  "prompt": "A cinematic view of a futuristic city at sunset",
  "created_at": 1234567890
}

Retrieve, List, Download, and Delete

Retrieve a job

curl -s http://localhost:8091/v1/videos/${video_id} | jq .

List jobs

curl -s http://localhost:8091/v1/videos | jq .

Download the completed video

curl -L http://localhost:8091/v1/videos/${video_id}/content -o wan22_output.mp4

Delete a job and its stored file

curl -X DELETE http://localhost:8091/v1/videos/${video_id} | jq .

Poll Until Complete

while true; do
  status=$(curl -s http://localhost:8091/v1/videos/${video_id} | jq -r '.status')
  if [ "$status" = "completed" ]; then
    break
  fi
  if [ "$status" = "failed" ]; then
    echo "Video generation failed"
    exit 1
  fi
  sleep 2
done

LTX-2

vllm serve Lightricks/LTX-2 --omni --port 8098

See the LTX-2 recipe for all checkpoints, pipeline selection, requests, defaults, and advanced options.

Helios

Helios ships three variants (Helios-Base, Helios-Mid, Helios-Distilled) that share the same server launch. Variant-specific knobs (declared in vllm_omni/model_extras/helios.py) are sent per request through the generic extra_params JSON form field — no per-model server flags required.

Start Server

vllm serve BestWishYsh/Helios-Base --omni --port 8098
# or: MODEL=BestWishYsh/Helios-Mid bash run_server_helios.sh

Send Requests (curl)

# Helios-Base (Stage 1 only)
bash run_curl_helios.sh

# Helios-Mid (Stage 2 pyramid + CFG-Zero*)
PRESET=mid-stage2 MODEL=BestWishYsh/Helios-Mid bash run_curl_helios.sh

# Helios-Distilled (Stage 2 pyramid + DMD, few-step)
PRESET=distilled MODEL=BestWishYsh/Helios-Distilled bash run_curl_helios.sh

The mid-stage2 and distilled presets attach an extra_params field, e.g. for Helios-Distilled:

curl -sS -X POST http://localhost:8098/v1/videos \
  -H "Accept: application/json" \
  -F "prompt=A dynamic time-lapse of scenery rushing past the window of a speeding train." \
  -F "model=BestWishYsh/Helios-Distilled" \
  -F "size=640x384" \
  -F "num_frames=99" \
  -F "fps=16" \
  -F "guidance_scale=1.0" \
  -F "seed=42" \
  -F 'extra_params={"is_enable_stage2": true, "pyramid_num_inference_steps_list": [2, 2, 2], "is_amplify_first_chunk": true}'