[Docs] Add LTX-2-T2V and LTX-2-I2V recipes (#3294)
Signed-off-by: hanzheli <hanzheli@kuaishou.com> Signed-off-by: fywc <hanzheli@kuaishou.com>
This commit is contained in:
2 files changed
+123
No files matched your search
@@ -0,0 +1,121 @@
|
|||||||
|
# LTX-2 T2V and I2V
|
||||||
|
|
||||||
|
> LTX-2 for text-to-video & image-to-video on 1x H200 141GB
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
- Vendor: Lightricks
|
||||||
|
- Model: `Lightricks/LTX-2`
|
||||||
|
- Task: Text-to-video generation and image-to-video generation
|
||||||
|
- Mode: Online serving with the OpenAI-compatible video API
|
||||||
|
- Maintainer: Community
|
||||||
|
|
||||||
|
## When to use this recipe
|
||||||
|
|
||||||
|
Use this recipe when you want to deploy the base `Lightricks/LTX-2` model
|
||||||
|
for both text-to-video and image-to-video generation. The same model ID is used for
|
||||||
|
both tasks; switch the pipeline class with `--model-class-name` depending on
|
||||||
|
whether you want T2V or I2V.
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- Model: <https://huggingface.co/Lightricks/LTX-2>
|
||||||
|
- User guide:
|
||||||
|
[`docs/user_guide/examples/online_serving/text_to_video.md`](../../docs/user_guide/examples/online_serving/text_to_video.md)
|
||||||
|
- Example guide:
|
||||||
|
[`examples/online_serving/text_to_video/README.md`](../../examples/online_serving/text_to_video/README.md)
|
||||||
|
[`examples/online_serving/image_to_video/README.md`](../../examples/online_serving/image_to_video/README.md)
|
||||||
|
|
||||||
|
## Hardware Support
|
||||||
|
|
||||||
|
## GPU
|
||||||
|
|
||||||
|
### 1x H200 141GB (Text-to-Video)
|
||||||
|
|
||||||
|
#### Environment
|
||||||
|
|
||||||
|
- OS: Ubuntu 22.04.5 LTS
|
||||||
|
- Python: 3.12+
|
||||||
|
- Driver / runtime: NVIDIA CUDA environment with H200 141GB GPU
|
||||||
|
- vLLM version: Match the repository requirements from your current checkout
|
||||||
|
- vLLM-Omni version or commit: Use the commit you are deploying from
|
||||||
|
|
||||||
|
#### Command
|
||||||
|
|
||||||
|
```bash
|
||||||
|
vllm serve Lightricks/LTX-2 \
|
||||||
|
--omni \
|
||||||
|
--model-class-name LTX2Pipeline
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Verification
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://localhost:8000/health
|
||||||
|
|
||||||
|
curl -X POST http://localhost:8000/v1/videos \
|
||||||
|
-F "prompt=A cinematic close-up of ocean waves at golden hour." \
|
||||||
|
-F "negative_prompt=worst quality, inconsistent motion, blurry, jittery, distorted" \
|
||||||
|
-F "width=512" \
|
||||||
|
-F "height=768" \
|
||||||
|
-F "num_frames=121" \
|
||||||
|
-F "fps=24" \
|
||||||
|
-F "num_inference_steps=40" \
|
||||||
|
-F "guidance_scale=4.0" \
|
||||||
|
-o ltx2_t2v_h200.mp4
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Notes
|
||||||
|
|
||||||
|
- Memory usage: Model loads at ~73.5GB, peaks at ~73.5 GiB during inference.
|
||||||
|
- Key flags:
|
||||||
|
- `--model-class-name LTX2Pipeline` selects the text-to-video pipeline explicitly.
|
||||||
|
- Known limitations:
|
||||||
|
- This recipe is tested on 1x H200 validation; add measured latency and VRAM after running it on your target machine.
|
||||||
|
|
||||||
|
### 1x H200 141GB (Image-to-Video)
|
||||||
|
|
||||||
|
#### Environment
|
||||||
|
|
||||||
|
- OS: Ubuntu 22.04.5 LTS
|
||||||
|
- Python: 3.12+
|
||||||
|
- Driver / runtime: NVIDIA CUDA environment with H200 141GB GPU
|
||||||
|
- vLLM version: Match the repository requirements from your current checkout
|
||||||
|
- vLLM-Omni version or commit: Use the commit you are deploying from
|
||||||
|
|
||||||
|
#### Command
|
||||||
|
|
||||||
|
```bash
|
||||||
|
vllm serve Lightricks/LTX-2 \
|
||||||
|
--omni \
|
||||||
|
--model-class-name LTX2ImageToVideoPipeline
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Verification
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://localhost:8000/health
|
||||||
|
|
||||||
|
curl -X POST http://localhost:8000/v1/videos/sync \
|
||||||
|
-F "prompt=A plush toy astronaut gently waving while the camera slowly pushes in." \
|
||||||
|
-F "input_reference=@/absolute/path/to/reference.png" \
|
||||||
|
-F "width=512" \
|
||||||
|
-F "height=768" \
|
||||||
|
-F "num_frames=121" \
|
||||||
|
-F "fps=24" \
|
||||||
|
-F "num_inference_steps=40" \
|
||||||
|
-F "guidance_scale=4.0" \
|
||||||
|
-o ltx2_i2v_h200.mp4
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Notes
|
||||||
|
|
||||||
|
- Memory usage: Model loads at ~73.5GB, peaks at ~73.5 GiB during inference.
|
||||||
|
- Key flags:
|
||||||
|
- `--model-class-name LTX2ImageToVideoPipeline` switches the same `Lightricks/LTX-2` weights into image-to-video mode.
|
||||||
|
- `input_reference=@...` uploads the reference image directly
|
||||||
|
- Known limitations:
|
||||||
|
- Use `image_reference` when you want to pass a URL or JSON-safe image reference
|
||||||
|
instead of uploading a file.
|
||||||
|
- Do not send `input_reference` and `image_reference` in the same request.
|
||||||
|
- This recipe is tested on 1x H200 validation; add measured latency and VRAM after running it on your target machine.
|
||||||
@@ -29,6 +29,8 @@ recipes/
|
|||||||
Qwen-Image on `1x A100 80GB`, including optional step-wise continuous batching replay
|
Qwen-Image on `1x A100 80GB`, including optional step-wise continuous batching replay
|
||||||
- [`Qwen/Qwen3-Omni.md`](./Qwen/Qwen3-Omni.md): online serving recipe for
|
- [`Qwen/Qwen3-Omni.md`](./Qwen/Qwen3-Omni.md): online serving recipe for
|
||||||
multimodal chat on `1x A100 80GB`
|
multimodal chat on `1x A100 80GB`
|
||||||
|
- [`LTX/LTX-2.md`](./LTX/LTX-2.md): text-to-video and image-to-video serving
|
||||||
|
recipe for LTX-2 on `1x H200 141GB`
|
||||||
- [`Wan-AI/Wan2.2-I2V.md`](./Wan-AI/Wan2.2-I2V.md): image-to-video serving
|
- [`Wan-AI/Wan2.2-I2V.md`](./Wan-AI/Wan2.2-I2V.md): image-to-video serving
|
||||||
recipe for Wan2.2 14B on `8x Ascend NPU (A2/A3)`
|
recipe for Wan2.2 14B on `8x Ascend NPU (A2/A3)`
|
||||||
- [`Tencent-Hunyuan/HunyuanImage-3.0-Instruct.md`](./Tencent-Hunyuan/HunyuanImage-3.0-Instruct.md):
|
- [`Tencent-Hunyuan/HunyuanImage-3.0-Instruct.md`](./Tencent-Hunyuan/HunyuanImage-3.0-Instruct.md):
|
||||||
|
|||||||
Reference in new issue
Block a user