[Doc] Add LTX-2 online serving deployment recipes with optimization benchmarks (#1971)
Signed-off-by: samithuang <285365963@qq.com> Signed-off-by: Samit <285365963@qq.com>
This commit is contained in:
1 parent
001f2e32e2
commit
d1fef41266
4 files changed
+348
-8
No files matched your search
@@ -3,17 +3,28 @@
|
||||
Source <https://github.com/vllm-project/vllm-omni/tree/main/examples/online_serving/text_to_video>.
|
||||
|
||||
|
||||
This example demonstrates how to deploy the Wan2.2 text-to-video model for online video generation using vLLM-Omni.
|
||||
This example demonstrates how to deploy text-to-video models for online video generation using vLLM-Omni.
|
||||
|
||||
## Start Server
|
||||
## Supported Models
|
||||
|
||||
### Basic Start
|
||||
| Model | Model ID |
|
||||
|-------|----------|
|
||||
| Wan2.1 T2V (1.3B) | `Wan-AI/Wan2.1-T2V-1.3B-Diffusers` |
|
||||
| Wan2.1 T2V (14B) | `Wan-AI/Wan2.1-T2V-14B-Diffusers` |
|
||||
| Wan2.2 T2V | `Wan-AI/Wan2.2-T2V-A14B-Diffusers` |
|
||||
| LTX-2 | `Lightricks/LTX-2` |
|
||||
|
||||
## Wan2.2 T2V
|
||||
|
||||
### Start Server
|
||||
|
||||
#### Basic Start
|
||||
|
||||
```bash
|
||||
vllm serve Wan-AI/Wan2.2-T2V-A14B-Diffusers --omni --port 8091
|
||||
```
|
||||
|
||||
### Start with Parameters
|
||||
#### Start with Parameters
|
||||
|
||||
Or use the startup script:
|
||||
|
||||
@@ -234,8 +245,94 @@ while true; do
|
||||
done
|
||||
```
|
||||
|
||||
## LTX-2
|
||||
|
||||
### Start Server
|
||||
|
||||
#### Basic Start
|
||||
|
||||
```bash
|
||||
vllm serve Lightricks/LTX-2 --omni --port 8098 \
|
||||
--enforce-eager --flow-shift 1.0 --boundary-ratio 1.0
|
||||
```
|
||||
|
||||
#### Start with Optimization Presets
|
||||
|
||||
Use the LTX-2 startup script with built-in optimization presets:
|
||||
|
||||
```bash
|
||||
# Baseline (1 GPU, eager)
|
||||
bash run_server_ltx2.sh baseline
|
||||
|
||||
# 4-GPU Ulysses sequence parallelism (lossless)
|
||||
bash run_server_ltx2.sh ulysses4
|
||||
|
||||
# Cache-DiT lossy acceleration (1 GPU, ~1.4× speedup)
|
||||
bash run_server_ltx2.sh cache-dit
|
||||
|
||||
# Best combo: 4-GPU Ulysses SP + Cache-DiT (~2.2× speedup)
|
||||
bash run_server_ltx2.sh best-combo
|
||||
```
|
||||
|
||||
#### Optimization Benchmarks
|
||||
|
||||
Benchmarked on H800, online serving (480×768, 41 frames, 20 steps, `seed=42`).
|
||||
"Inference" is the server-reported inference time; excludes HTTP/poll overhead.
|
||||
|
||||
| Preset | Server Command | Inference (s) | Speedup | Type |
|
||||
|--------|---------------|---------------|---------|------|
|
||||
| `baseline` | `--enforce-eager` | 10.3 | 1.00× | — |
|
||||
| `compile` | *(default, no --enforce-eager)* | ~10.3 (warm) | ~1.00× | Lossless |
|
||||
| `ulysses4` | `--enforce-eager --usp 4` | ~10.3 | ~1.00× | Lossless |
|
||||
| `cache-dit` | `--enforce-eager --cache-backend cache_dit` | 7.4 avg | ~1.4× | Lossy |
|
||||
| `best-combo` | `--enforce-eager --usp 4 --cache-backend cache_dit` | 4.7 avg | **~2.2×** | Lossless + Lossy |
|
||||
|
||||
**Observations**:
|
||||
- **torch.compile**: On H800, warm-request inference time matches the eager baseline (~10.3s).
|
||||
The first request pays ~6s compilation overhead. Benefit depends on model architecture and GPU.
|
||||
- **Ulysses SP (4 GPU)**: No measurable speedup alone for 41-frame generation at this resolution.
|
||||
Communication overhead outweighs gains at this sequence length.
|
||||
- **Cache-DiT**: Inference varies per request (6–10s) due to dynamic caching decisions.
|
||||
Average is ~7.4s (~1.4× speedup) with slight quality tradeoff.
|
||||
- **Best combo**: 4-GPU Ulysses SP + Cache-DiT synergize well — Cache-DiT reduces per-step
|
||||
computation, making the communication overhead of Ulysses SP worthwhile. Average ~4.7s
|
||||
(~2.2× speedup).
|
||||
- **FP8 quantization**: Reduces VRAM but does not speed up LTX-2 on H800 (compute-bound).
|
||||
|
||||
**Deployment Recommendations**:
|
||||
- For **production with quality priority**: use `baseline` with `--enforce-eager`
|
||||
- For **maximum throughput** (4 GPUs, quality tradeoff): use `best-combo` (~2.2× speedup)
|
||||
- For **single-GPU throughput**: use `cache-dit` (~1.4× speedup)
|
||||
- `--enforce-eager` is recommended to avoid torch.compile warmup latency on first request
|
||||
|
||||
### Send Requests (curl)
|
||||
|
||||
```bash
|
||||
# Using the provided script
|
||||
bash run_curl_ltx2.sh
|
||||
|
||||
# Or directly
|
||||
curl -sS -X POST http://localhost:8098/v1/videos \
|
||||
-H "Accept: application/json" \
|
||||
-F "prompt=A serene lakeside sunrise with mist over the water." \
|
||||
-F "width=768" \
|
||||
-F "height=480" \
|
||||
-F "num_frames=41" \
|
||||
-F "fps=24" \
|
||||
-F "num_inference_steps=20" \
|
||||
-F "guidance_scale=3.0" \
|
||||
-F "seed=42"
|
||||
```
|
||||
|
||||
## Example materials
|
||||
|
||||
??? abstract "response.json"
|
||||
``````json
|
||||
--8<-- "examples/online_serving/text_to_video/response.json"
|
||||
``````
|
||||
??? abstract "run_curl_ltx2.sh"
|
||||
``````sh
|
||||
--8<-- "examples/online_serving/text_to_video/run_curl_ltx2.sh"
|
||||
??? abstract "run_curl_hunyuan_video_15.sh"
|
||||
``````sh
|
||||
--8<-- "examples/online_serving/text_to_video/run_curl_hunyuan_video_15.sh"
|
||||
@@ -248,6 +345,9 @@ done
|
||||
``````sh
|
||||
--8<-- "examples/online_serving/text_to_video/run_server.sh"
|
||||
``````
|
||||
??? abstract "run_server_ltx2.sh"
|
||||
``````sh
|
||||
--8<-- "examples/online_serving/text_to_video/run_server_ltx2.sh"
|
||||
??? abstract "run_server_hunyuan_video_15.sh"
|
||||
``````sh
|
||||
--8<-- "examples/online_serving/text_to_video/run_server_hunyuan_video_15.sh"
|
||||
|
||||
Reference in new issue
Block a user