[Model] Add LTX-2.5 Diffusers backend support

Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
Hongsheng Liu committed 2026-08-12 08:58:20 +08:00
1 parent 8b0a6779d4
commit 4b541e4d56
5 files changed
+480 -4

No files matched your search

@@ -29,6 +29,10 @@ However, as we strive to ensure output similarity between vLLM-Omni's diffuser b
- Tongyi-MAI/Z-Image-Turbo
- Wan2.2-I2V-A14B-Diffusers
LTX-2.5 is also supported through a source-preview dependency path described
below. The adapter behavior is unit tested, but the gated full checkpoint is
not part of the regular CI matrix.
If you find that a model not listed above also produces different outputs from running diffusers model directly.
Please consider file an issue.
@@ -116,6 +120,69 @@ The model loading and inference strictly follows the diffusers library, and they
Users are encouraged to double-check the model pipeline's interface in [diffusers' official documentation](https://huggingface.co/docs/diffusers/api/pipelines/overview).
Some particular examples are below.
#### LTX-2.5 (source preview)
The repository linked from the LTX-2.5 release,
[`Lightricks/LTX-2.5`](https://huggingface.co/Lightricks/LTX-2.5), contains
component weights and does not have the `model_index.json` required by
`DiffusionPipeline.from_pretrained()`. Use Lightricks' official converted
repository, [`Lightricks/LTX-2.5-Diffusers`](https://huggingface.co/Lightricks/LTX-2.5-Diffusers),
with this backend. Both repositories are gated; accept the model terms on
Hugging Face and authenticate with a read token before starting the server.
LTX-2.5 support landed after Diffusers 0.39.0 and is not in a stable Diffusers
release yet. After installing vLLM-Omni, install the reviewed upstream commit
and Gemma 4-capable Transformers version:
```bash
python -m pip install --upgrade 'transformers>=5.10.1' \
'diffusers @ git+https://github.com/huggingface/diffusers.git@7564fb016dabda0c943416190fc92398c50b1b20'
hf auth login
```
Serve the default distilled text-to-video pipeline as follows. The model is
large; CPU offload and VAE tiling reduce GPU memory pressure at the cost of
latency and host memory.
```bash
vllm serve "Lightricks/LTX-2.5-Diffusers" \
--omni \
--port 8091 \
--diffusion-load-format diffusers \
--dtype bfloat16 \
--enable-cpu-offload \
--vae-use-tiling \
--default-sampling-params \
'{"0":{"height":512,"width":768,"num_frames":121,"fps":24,"frame_rate":24.0,"output_type":"np"}}'
```
Then request an MP4 with synchronized audio:
```bash
curl -X POST http://localhost:8091/v1/videos/sync \
-F 'prompt=A red fox walks through a snowy forest at dawn while birds sing in the distance.' \
-F 'size=768x512' \
-F 'num_frames=121' \
-F 'fps=24' \
-F 'seed=42' \
-o ltx-2.5.mp4
```
The converted repository selects the distilled transformer by default. When a
request supplies neither `sigmas`, `timesteps`, nor `num_inference_steps`, the
adapter uses the official eight-sigma distilled schedule and disables CFG,
spatio-temporal guidance, and modality guidance. An explicit schedule or step
count takes precedence; a step count without `sigmas` uses Diffusers' generic
schedule and may reduce quality. The adapter preserves both `frames` and the
pipeline's singular `audio` output and reads the audio sample rate from the
loaded vocoder (48 kHz for this checkpoint).
This path currently covers the default one-stage distilled T2V pipeline.
Loading `transformer_full`, two-stage upsampling, image/video conditioning, and
the diffusion video decoder require additional pipeline assembly and are not
enabled by this example. Native vLLM-Omni acceleration features listed under
[Limitations](#limitations) remain unavailable with the Diffusers backend.
#### Wan Series
The Wan series video generation models takes `boundary_ratio` and `flow_shift` during model initialization ([ref](https://huggingface.co/docs/diffusers/api/pipelines/wan)), not during inference.