[Feature] Add official LTX multimodal guidance and precision guard (#5148)
Signed-off-by: mglyn <1203789601@qq.com>
This commit is contained in:
Mu GuanLin authored and GitHub committed
1 parent
66a9b0c84f
commit
12d75cf63a
51 files changed
+4547
-3062
No files matched your search
@@ -1,7 +1,7 @@
|
||||
# Image-To-Video
|
||||
|
||||
This shared example generates videos from images with VACE, Wan2.2,
|
||||
LTX-2/LTX-2.3, HunyuanVideo-1.5, Cosmos3, and other compatible pipelines.
|
||||
This shared example generates videos from images with VACE, Wan2.2, LTX-2,
|
||||
HunyuanVideo-1.5, Cosmos3, and other compatible pipelines.
|
||||
|
||||
- `image_to_video.py`: command-line script for single video generation with advanced options.
|
||||
|
||||
@@ -26,7 +26,7 @@ This folder provides a unified CLI script for image-to-video generation using vL
|
||||
| `Wan-AI/Wan2.2-I2V-A14B-Diffusers` | 480 x 832 | 81 | 50 | 5.0 | Around 60 GiB BF16 for basic single-card usage |
|
||||
| `Wan-AI/Wan2.2-TI2V-5B-Diffusers` | 480 x 832 | 81 | 50 | 4.0 | Around 20–25 GiB BF16, smallest I2V model |
|
||||
| `hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-480p_i2v` | 480 x 832 | 121 | 50 | 6.0 | Around 100 GiB at default settings; the example enables `--enable-cpu-offload` + VAE tiling/slicing to fit an 80 GiB card |
|
||||
| LTX2 (local path + `--model-class-name LTX2ImageToVideoPipeline`) | 512 x 768 | 121 | 40 | 4.0 | Memory use depends on frame count and tensor parallelism |
|
||||
| `Lightricks/LTX-2` | 512 x 768 | 121 | 40 | video 3.0 / audio 7.0 | Memory use depends on frame count and tensor parallelism |
|
||||
|
||||
!!! info
|
||||
Peak VRAM: based on basic single-card usage, batch size = 1, without any acceleration/optimization features. Some model weights cannot fit into one card with 80 GiB VRAM, which may need to use CPU offloading.
|
||||
@@ -105,7 +105,7 @@ python image_to_video.py \
|
||||
| Argument | Type | Default | Description |
|
||||
| -------- | ---- | ------- | ----------- |
|
||||
| `--model` | str | `Wan-AI/Wan2.2-I2V-A14B-Diffusers` | Diffusers I2V model ID or local path |
|
||||
| `--model-class-name` | str | `None` | Override model class name (e.g., `LTX2ImageToVideoPipeline`) |
|
||||
| `--model-class-name` | str | `None` | Optional pipeline override |
|
||||
| `--image` | str | (required) | Path to input image |
|
||||
| `--prompt` | str | `""` | Text description of desired motion/animation |
|
||||
| `--negative-prompt` | str | `""` | Optional list of artifacts to suppress |
|
||||
@@ -195,24 +195,19 @@ python image_to_video.py \
|
||||
--output hunyuan_i2v.mp4
|
||||
```
|
||||
|
||||
### LTX2 Image-to-Video
|
||||
### LTX-2
|
||||
|
||||
```bash
|
||||
python image_to_video.py \
|
||||
--model /path/to/LTX-2 \
|
||||
--model-class-name LTX2ImageToVideoPipeline \
|
||||
--model Lightricks/LTX-2 \
|
||||
--image cherry_blossom.jpg \
|
||||
--prompt "A cinematic dolly shot of cherry blossoms" \
|
||||
--height 512 \
|
||||
--width 768 \
|
||||
--num-frames 121 \
|
||||
--num-inference-steps 40 \
|
||||
--guidance-scale 4.0 \
|
||||
--frame-rate 24 \
|
||||
--fps 24 \
|
||||
--output ltx2_i2v.mp4
|
||||
```
|
||||
|
||||
See the [LTX-2 recipe](../../../recipes/LTX/LTX-2.md) for all checkpoints,
|
||||
pipeline selection, T2V, defaults, and advanced options.
|
||||
|
||||
## Advanced Features
|
||||
|
||||
### CFG Parallel
|
||||
@@ -233,30 +228,6 @@ python image_to_video.py \
|
||||
--output i2v_cached.mp4
|
||||
```
|
||||
|
||||
### LTX-2.3
|
||||
|
||||
```bash
|
||||
python image_to_video.py \
|
||||
--model diffusers/LTX-2.3-Diffusers \
|
||||
--model-class-name LTX23ImageToVideoPipeline \
|
||||
--image cherry_blossom.jpg \
|
||||
--prompt "Cherry blossoms swaying gently in the breeze with synchronized ambient sound" \
|
||||
--negative-prompt "worst quality, inconsistent motion, blurry, jittery, distorted" \
|
||||
--height 384 \
|
||||
--width 512 \
|
||||
--num-frames 25 \
|
||||
--guidance-scale 4.0 \
|
||||
--num-inference-steps 20 \
|
||||
--frame-rate 24 \
|
||||
--fps 24 \
|
||||
--output ltx23_i2v_output.mp4
|
||||
```
|
||||
|
||||
Use the Diffusers-format checkpoint `diffusers/LTX-2.3-Diffusers`; the
|
||||
upstream `Lightricks/LTX-2.3` raw safetensors repo is not directly loadable by
|
||||
this pipeline. Pass `--model-class-name LTX23ImageToVideoPipeline` to select
|
||||
the LTX-2.3 image-to-video pipeline.
|
||||
|
||||
### Cosmos3
|
||||
|
||||
```bash
|
||||
@@ -274,8 +245,8 @@ python image_to_video.py \
|
||||
|
||||
Key arguments:
|
||||
|
||||
- `--model`: Model ID (I2V-A14B for MoE, TI2V-5B for unified T2V+I2V,
|
||||
LTX-2/LTX-2.3, Cosmos3, or VACE).
|
||||
- `--model`: Model ID (I2V-A14B for MoE, TI2V-5B for unified T2V+I2V, LTX-2,
|
||||
Cosmos3, or VACE).
|
||||
- `--image`: Path to the first-frame or source image.
|
||||
- `--last-image`: Optional last-frame condition for models such as VACE.
|
||||
- `--mask-image`: Optional inpainting mask. White pixels are regenerated and black pixels are preserved.
|
||||
@@ -290,7 +261,7 @@ Key arguments:
|
||||
- `--guidance-scale` and `--guidance-scale-high`: CFG scale (applied to low/high-noise stages for MoE).
|
||||
- `--negative-prompt`: Optional list of artifacts to suppress.
|
||||
- `--boundary-ratio`: Boundary split ratio for two-stage MoE models.
|
||||
- `--flow-shift`: Scheduler flow shift (default: model-specific — Wan/LTX2 5.0, Cosmos3 10.0).
|
||||
- `--flow-shift`: Scheduler flow shift (default: Wan 5.0, Cosmos3 10.0). LTX does not consume this option.
|
||||
- `--sample-solver`: Wan2.2 sampling solver. Use `unipc` for the default multistep solver, or `euler` for Lightning/Distill checkpoints.
|
||||
- `--num-inference-steps`: Number of denoising steps (default: model-specific — Wan 50, LTX2 40, Cosmos3 35).
|
||||
- `--fps`: Frames per second for the saved MP4 (requires `diffusers` export_to_video).
|
||||
|
||||
@@ -23,16 +23,14 @@ Usage:
|
||||
|
||||
# LTX2 image-to-video
|
||||
python image_to_video.py --model /path/to/LTX-2 \
|
||||
--model-class-name LTX2ImageToVideoPipeline \
|
||||
--image input.jpg --prompt "A cinematic dolly shot of a boat" \
|
||||
--num-frames 121 --num-inference-steps 40 --guidance-scale 4.0 \
|
||||
--num-frames 121 --num-inference-steps 40 \
|
||||
--frame-rate 24 --fps 24 --output ltx2_i2v.mp4
|
||||
|
||||
# LTX-2.3 image-to-video
|
||||
python image_to_video.py --model diffusers/LTX-2.3-Diffusers \
|
||||
--model-class-name LTX23ImageToVideoPipeline \
|
||||
--image input.jpg --prompt "A cinematic dolly shot of a boat" \
|
||||
--height 384 --width 512 --num-frames 25 --num-inference-steps 20 \
|
||||
--height 512 --width 768 --num-frames 121 --num-inference-steps 30 \
|
||||
--frame-rate 24 --fps 24 --output ltx23_i2v.mp4
|
||||
|
||||
# HunyuanVideo-1.5 I2V (480p)
|
||||
@@ -100,7 +98,7 @@ def parse_args() -> argparse.Namespace:
|
||||
parser.add_argument(
|
||||
"--model-class-name",
|
||||
default=None,
|
||||
help="Override model class name (e.g., LTX2ImageToVideoPipeline or LTX23ImageToVideoPipeline).",
|
||||
help="Override model class name (LTX checkpoints default to LTX2Pipeline).",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--deploy-config",
|
||||
@@ -117,7 +115,7 @@ def parse_args() -> argparse.Namespace:
|
||||
help="Path to a reference image. Repeat to provide multiple references.",
|
||||
)
|
||||
parser.add_argument("--prompt", default="", help="Text prompt describing the desired motion.")
|
||||
parser.add_argument("--negative-prompt", default="", help="Negative prompt.")
|
||||
parser.add_argument("--negative-prompt", default=None, help="Negative prompt. Default: model-specific.")
|
||||
parser.add_argument("--seed", type=int, default=42, help="Random seed.")
|
||||
parser.add_argument("--guidance-scale", type=float, default=None, help="CFG scale. Default: model-specific.")
|
||||
parser.add_argument(
|
||||
@@ -331,7 +329,10 @@ def main():
|
||||
generator = torch.Generator(device=current_omni_platform.device_type).manual_seed(args.seed)
|
||||
model_name = str(args.model).lower() if args.model is not None else ""
|
||||
model_class_name = args.model_class_name
|
||||
is_ltx2 = model_class_name in {"LTX2ImageToVideoPipeline", "LTX23ImageToVideoPipeline"}
|
||||
model_class_name_lower = (model_class_name or "").lower()
|
||||
is_ltx2_distilled = "distilled" in model_class_name_lower or "distilled" in model_name
|
||||
is_ltx23 = "ltx23" in model_class_name_lower or "ltx-2.3" in model_name
|
||||
is_ltx2 = is_ltx2_distilled or is_ltx23 or "ltx2" in model_class_name_lower or "ltx-2" in model_name
|
||||
is_cosmos = "cosmos" in model_name or (model_class_name is not None and "cosmos" in model_class_name.lower())
|
||||
|
||||
image = PIL.Image.open(args.image).convert("RGB") if args.image else None
|
||||
@@ -356,13 +357,23 @@ def main():
|
||||
1280 * 720,
|
||||
16,
|
||||
)
|
||||
elif is_ltx2_distilled:
|
||||
d_fps, d_guidance, d_num_frames, d_steps, d_flow_shift, d_max_area, d_mod = (
|
||||
24,
|
||||
None,
|
||||
121,
|
||||
8,
|
||||
None,
|
||||
1024 * 1536,
|
||||
64,
|
||||
)
|
||||
elif is_ltx2:
|
||||
d_fps, d_guidance, d_num_frames, d_steps, d_flow_shift, d_max_area, d_mod = (
|
||||
24,
|
||||
4.0,
|
||||
None,
|
||||
121,
|
||||
40,
|
||||
5.0,
|
||||
30 if is_ltx23 else 40,
|
||||
None,
|
||||
512 * 768,
|
||||
32,
|
||||
)
|
||||
@@ -445,7 +456,6 @@ def main():
|
||||
vae_use_slicing=args.vae_use_slicing,
|
||||
vae_use_tiling=args.vae_use_tiling,
|
||||
boundary_ratio=args.boundary_ratio,
|
||||
flow_shift=flow_shift,
|
||||
diffusion_kv_cache_dtype=args.diffusion_kv_cache_dtype,
|
||||
diffusion_kv_cache_skip_steps=args.diffusion_kv_cache_skip_steps,
|
||||
diffusion_kv_cache_skip_layers=args.diffusion_kv_cache_skip_layers,
|
||||
@@ -460,6 +470,8 @@ def main():
|
||||
)
|
||||
if args.deploy_config:
|
||||
omni_kwargs["deploy_config"] = args.deploy_config
|
||||
if flow_shift is not None:
|
||||
omni_kwargs["flow_shift"] = flow_shift
|
||||
if args.quantization is not None:
|
||||
omni_kwargs["quantization"] = args.quantization
|
||||
# Cosmos3 loads its (gated) guardrail models at build time, so the guardrails
|
||||
@@ -492,10 +504,14 @@ def main():
|
||||
print(f" Video size: {width}x{height}")
|
||||
print(f"{'=' * 60}\n")
|
||||
|
||||
negative_prompt = args.negative_prompt
|
||||
if negative_prompt is None and not is_ltx2:
|
||||
# Preserve the historical empty-prompt behavior for non-LTX examples.
|
||||
negative_prompt = ""
|
||||
prompt_dict = build_image_to_video_prompt(
|
||||
model_class_name=model_class_name,
|
||||
prompt=args.prompt,
|
||||
negative_prompt=args.negative_prompt,
|
||||
negative_prompt=negative_prompt,
|
||||
media_inputs=media_inputs,
|
||||
height=height,
|
||||
width=width,
|
||||
@@ -511,11 +527,10 @@ def main():
|
||||
num_inference_steps=num_inference_steps,
|
||||
num_frames=num_frames,
|
||||
frame_rate=frame_rate,
|
||||
extra_args={
|
||||
"sample_solver": args.sample_solver,
|
||||
"flow_shift": flow_shift,
|
||||
},
|
||||
extra_args={"sample_solver": args.sample_solver},
|
||||
)
|
||||
if flow_shift is not None:
|
||||
sampling_params.extra_args["flow_shift"] = flow_shift
|
||||
|
||||
# Route model-specific knobs through extra_body, filtered against the model's
|
||||
# declared extra_body_params. Models without a declaration only forward explicit
|
||||
|
||||
@@ -8,7 +8,7 @@ A unified script for text-to-video generation. Supports multiple models with mod
|
||||
|---|---|---|---|---|---|
|
||||
| `Wan-AI/Wan2.1-VACE-1.3B-diffusers` | 480x832 | 81 | 30 | 5.0 | ~20 GiB (RTX 5090, VAE tiling) |
|
||||
| `Wan-AI/Wan2.2-T2V-A14B-Diffusers` | 720x1280 | 81 | 40 | 4.0 | ~60 GiB |
|
||||
| `diffusers/LTX-2.3-Diffusers` | 384x512 | 25 | 20 | 4.0 | 96GB-class GPU |
|
||||
| `Lightricks/LTX-2` | 512x768 | 121 | 40 | video 3.0 / audio 7.0 | Model-dependent |
|
||||
| `hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-480p_t2v` | 480x832 | 121 | 50 | 6.0 | 1×A100 80GB |
|
||||
| `hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-720p_t2v` | 720x1280 | 121 | 50 | 6.0 | FP8 + VAE tiling required |
|
||||
| `nvidia/Cosmos3-Nano` | 720x1280 | 189 | 35 | 6.0 | ~46 GiB (peak, 720p) |
|
||||
@@ -55,45 +55,17 @@ python text_to_video.py \
|
||||
--output vace_t2v_output.mp4
|
||||
```
|
||||
|
||||
LTX2 example:
|
||||
### LTX-2
|
||||
|
||||
```bash
|
||||
python text_to_video.py \
|
||||
--model "Lightricks/LTX-2" \
|
||||
--prompt "A cinematic close-up of ocean waves at golden hour." \
|
||||
--negative-prompt "worst quality, inconsistent motion, blurry, jittery, distorted" \
|
||||
--height 512 \
|
||||
--width 768 \
|
||||
--num-frames 121 \
|
||||
--num-inference-steps 40 \
|
||||
--guidance-scale 4.0 \
|
||||
--frame-rate 24 \
|
||||
--output ltx2_out.mp4
|
||||
```
|
||||
|
||||
### LTX-2.3
|
||||
|
||||
```bash
|
||||
python text_to_video.py \
|
||||
--model diffusers/LTX-2.3-Diffusers \
|
||||
--model-class-name LTX23Pipeline \
|
||||
--model Lightricks/LTX-2 \
|
||||
--prompt "Cherry blossoms swaying gently in the breeze with synchronized ambient sound" \
|
||||
--negative-prompt "worst quality, inconsistent motion, blurry, jittery, distorted" \
|
||||
--height 384 \
|
||||
--width 512 \
|
||||
--num-frames 25 \
|
||||
--num-inference-steps 20 \
|
||||
--guidance-scale 4.0 \
|
||||
--frame-rate 24 \
|
||||
--fps 24 \
|
||||
--audio-sample-rate 48000 \
|
||||
--output ltx23_t2v_output.mp4
|
||||
--output ltx2_output.mp4
|
||||
```
|
||||
|
||||
Use the Diffusers-format checkpoint `diffusers/LTX-2.3-Diffusers`; the
|
||||
upstream `Lightricks/LTX-2.3` raw safetensors repo is not directly loadable by
|
||||
this pipeline. Pass `--model-class-name LTX23Pipeline` to select the LTX-2.3
|
||||
text-to-video pipeline explicitly.
|
||||
See the [LTX-2 recipe](../../../recipes/LTX/LTX-2.md) for all checkpoints,
|
||||
pipeline selection, I2V, defaults, and advanced options.
|
||||
|
||||
### HunyuanVideo-1.5 (480p)
|
||||
|
||||
@@ -212,8 +184,7 @@ python text_to_video.py \
|
||||
### Common
|
||||
|
||||
- `--model`: Diffusers model ID or local path.
|
||||
- `--model-class-name`: Optional explicit pipeline class. Use `LTX23Pipeline`
|
||||
for LTX-2.3 text-to-video.
|
||||
- `--model-class-name`: Optional explicit pipeline override.
|
||||
- `--prompt`: text description (string).
|
||||
- `--height/--width`: output resolution. Default depends on model.
|
||||
- `--num-frames`: number of frames. Default depends on model.
|
||||
@@ -229,8 +200,7 @@ python text_to_video.py \
|
||||
- `--enable-cpu-offload`: enable CPU offloading for diffusion models.
|
||||
- `--enable-layerwise-offload`: enable layerwise offloading on DiT modules.
|
||||
- `--frame-rate`: generation FPS for pipelines that require it (e.g., LTX2).
|
||||
- `--audio-sample-rate`: audio sample rate for embedded audio (when the
|
||||
pipeline returns audio; LTX-2.3 outputs 48kHz audio).
|
||||
- `--audio-sample-rate`: fallback audio sample rate when the pipeline returns audio.
|
||||
- `--quantization`: quantization method (such as `fp8` for FP8).
|
||||
- `--flow-shift`: scheduler flow_shift parameter.
|
||||
- `--extra-body`: JSON object of model-specific generation params, filtered against the model's declared `extra_body_params` (see [`vllm_omni/model_extras`](../../../vllm_omni/model_extras)). Used by Cosmos3 (see above).
|
||||
|
||||
@@ -65,11 +65,42 @@ _MODEL_PRESETS = {
|
||||
"fps": 16,
|
||||
"output": "helios_output.mp4",
|
||||
},
|
||||
"ltx2": {
|
||||
"height": 512,
|
||||
"width": 768,
|
||||
"num_frames": 121,
|
||||
"num_inference_steps": 40,
|
||||
"fps": 24,
|
||||
"output": "ltx2_output.mp4",
|
||||
},
|
||||
"ltx2_distilled": {
|
||||
"height": 1024,
|
||||
"width": 1536,
|
||||
"num_frames": 121,
|
||||
"num_inference_steps": 8,
|
||||
"fps": 24,
|
||||
"output": "ltx2_distilled_output.mp4",
|
||||
},
|
||||
"ltx23": {
|
||||
"height": 512,
|
||||
"width": 768,
|
||||
"num_frames": 121,
|
||||
"num_inference_steps": 30,
|
||||
"fps": 24,
|
||||
"output": "ltx23_output.mp4",
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def _detect_preset(model: str) -> dict:
|
||||
def _detect_preset(model: str, model_class_name: str | None = None) -> dict:
|
||||
model_lower = model.lower()
|
||||
class_lower = (model_class_name or "").lower()
|
||||
if "distilled" in class_lower or "distilled" in model_lower:
|
||||
return _MODEL_PRESETS["ltx2_distilled"]
|
||||
if "ltx23" in class_lower or "ltx-2.3" in model_lower or "ltx_2.3" in model_lower:
|
||||
return _MODEL_PRESETS["ltx23"]
|
||||
if "ltx2" in class_lower or "ltx-2" in model_lower or "ltx_2" in model_lower:
|
||||
return _MODEL_PRESETS["ltx2"]
|
||||
if "vace" in model_lower:
|
||||
return _MODEL_PRESETS["vace"]
|
||||
if "cosmos" in model_lower:
|
||||
@@ -122,7 +153,7 @@ def parse_args() -> argparse.Namespace:
|
||||
parser.add_argument(
|
||||
"--model-class-name",
|
||||
default=None,
|
||||
help="Override model class name (e.g., LTX2TwoStagesVideoPipeline).",
|
||||
help="Override model class name (e.g., LTX2Pipeline).",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--deploy-config",
|
||||
@@ -130,7 +161,7 @@ def parse_args() -> argparse.Namespace:
|
||||
help="Optional deploy config YAML to use for pipeline-backed runs.",
|
||||
)
|
||||
parser.add_argument("--prompt", default="A serene lakeside sunrise with mist over the water.", help="Text prompt.")
|
||||
parser.add_argument("--negative-prompt", default="", help="Negative prompt.")
|
||||
parser.add_argument("--negative-prompt", default=None, help="Negative prompt. Default: model-specific.")
|
||||
parser.add_argument(
|
||||
"--extra-body",
|
||||
type=parse_extra_body,
|
||||
@@ -331,7 +362,7 @@ def main():
|
||||
args = parse_args()
|
||||
model_class_name = args.model_class_name
|
||||
|
||||
preset = _detect_preset(args.model)
|
||||
preset = _detect_preset(args.model, model_class_name)
|
||||
for key, default_val in preset.items():
|
||||
if getattr(args, key.replace("-", "_"), None) is None:
|
||||
setattr(args, key.replace("-", "_"), default_val)
|
||||
@@ -422,8 +453,11 @@ def main():
|
||||
print(f"{'=' * 60}\n")
|
||||
|
||||
prompt_dict = {"prompt": args.prompt}
|
||||
if args.negative_prompt:
|
||||
if args.negative_prompt is not None:
|
||||
prompt_dict["negative_prompt"] = args.negative_prompt
|
||||
elif preset not in (_MODEL_PRESETS["ltx2"], _MODEL_PRESETS["ltx23"]):
|
||||
# Preserve the historical empty-prompt behavior for non-LTX examples.
|
||||
prompt_dict["negative_prompt"] = ""
|
||||
|
||||
sampling_kwargs = dict(
|
||||
height=args.height,
|
||||
|
||||
@@ -248,82 +248,12 @@ done
|
||||
|
||||
## LTX-2
|
||||
|
||||
### Start Server
|
||||
|
||||
#### Basic Start
|
||||
|
||||
```bash
|
||||
vllm serve Lightricks/LTX-2 --omni --port 8098 \
|
||||
--enforce-eager --boundary-ratio 1.0
|
||||
vllm serve Lightricks/LTX-2 --omni --port 8098
|
||||
```
|
||||
|
||||
#### Start with Optimization Presets
|
||||
|
||||
Use the LTX-2 startup script with built-in optimization presets:
|
||||
|
||||
```bash
|
||||
# Baseline (1 GPU, eager)
|
||||
bash run_server_ltx2.sh baseline
|
||||
|
||||
# 4-GPU Ulysses sequence parallelism (lossless)
|
||||
bash run_server_ltx2.sh ulysses4
|
||||
|
||||
# Cache-DiT lossy acceleration (1 GPU, ~1.4× speedup)
|
||||
bash run_server_ltx2.sh cache-dit
|
||||
|
||||
# Best combo: 4-GPU Ulysses SP + Cache-DiT (~2.2× speedup)
|
||||
bash run_server_ltx2.sh best-combo
|
||||
```
|
||||
|
||||
#### Optimization Benchmarks
|
||||
|
||||
Benchmarked on H800, online serving (480×768, 41 frames, 20 steps, `seed=42`).
|
||||
"Inference" is the server-reported inference time; excludes HTTP/poll overhead.
|
||||
|
||||
| Preset | Server Command | Inference (s) | Speedup | Type |
|
||||
|--------|---------------|---------------|---------|------|
|
||||
| `baseline` | `--enforce-eager` | 10.3 | 1.00× | — |
|
||||
| `compile` | *(default, no --enforce-eager)* | ~10.3 (warm) | ~1.00× | Lossless |
|
||||
| `ulysses4` | `--enforce-eager --usp 4` | ~10.3 | ~1.00× | Lossless |
|
||||
| `cache-dit` | `--enforce-eager --cache-backend cache_dit` | 7.4 avg | ~1.4× | Lossy |
|
||||
| `best-combo` | `--enforce-eager --usp 4 --cache-backend cache_dit` | 4.7 avg | **~2.2×** | Lossless + Lossy |
|
||||
|
||||
**Observations**:
|
||||
- **torch.compile**: On H800, warm-request inference time matches the eager baseline (~10.3s).
|
||||
The first request pays ~6s compilation overhead. Benefit depends on model architecture and GPU.
|
||||
- **Ulysses SP (4 GPU)**: No measurable speedup alone for 41-frame generation at this resolution.
|
||||
Communication overhead outweighs gains at this sequence length.
|
||||
- **Cache-DiT**: Inference varies per request (6–10s) due to dynamic caching decisions.
|
||||
Average is ~7.4s (~1.4× speedup) with slight quality tradeoff.
|
||||
- **Best combo**: 4-GPU Ulysses SP + Cache-DiT synergize well — Cache-DiT reduces per-step
|
||||
computation, making the communication overhead of Ulysses SP worthwhile. Average ~4.7s
|
||||
(~2.2× speedup).
|
||||
- **FP8 quantization**: Reduces VRAM but does not speed up LTX-2 on H800 (compute-bound).
|
||||
|
||||
**Deployment Recommendations**:
|
||||
- For **production with quality priority**: use `baseline` with `--enforce-eager`
|
||||
- For **maximum throughput** (4 GPUs, quality tradeoff): use `best-combo` (~2.2× speedup)
|
||||
- For **single-GPU throughput**: use `cache-dit` (~1.4× speedup)
|
||||
- `--enforce-eager` is recommended to avoid torch.compile warmup latency on first request
|
||||
|
||||
### Send Requests (curl)
|
||||
|
||||
```bash
|
||||
# Using the provided script
|
||||
bash run_curl_ltx2.sh
|
||||
|
||||
# Or directly
|
||||
curl -sS -X POST http://localhost:8098/v1/videos \
|
||||
-H "Accept: application/json" \
|
||||
-F "prompt=A serene lakeside sunrise with mist over the water." \
|
||||
-F "width=768" \
|
||||
-F "height=480" \
|
||||
-F "num_frames=41" \
|
||||
-F "fps=24" \
|
||||
-F "num_inference_steps=20" \
|
||||
-F "guidance_scale=3.0" \
|
||||
-F "seed=42"
|
||||
```
|
||||
See the [LTX-2 recipe](../../../recipes/LTX/LTX-2.md) for all checkpoints,
|
||||
pipeline selection, requests, defaults, and advanced options.
|
||||
|
||||
## Helios
|
||||
|
||||
|
||||
@@ -22,7 +22,6 @@ create_response=$(
|
||||
-F "num_frames=41" \
|
||||
-F "fps=24" \
|
||||
-F "num_inference_steps=20" \
|
||||
-F "guidance_scale=3.0" \
|
||||
-F "seed=42"
|
||||
)
|
||||
|
||||
|
||||
@@ -21,8 +21,6 @@ set -euo pipefail
|
||||
|
||||
MODEL="${MODEL:-Lightricks/LTX-2}"
|
||||
PORT="${PORT:-8098}"
|
||||
FLOW_SHIFT="${FLOW_SHIFT:-1.0}"
|
||||
BOUNDARY_RATIO="${BOUNDARY_RATIO:-1.0}"
|
||||
|
||||
PRESET="${1:-baseline}"
|
||||
|
||||
@@ -66,19 +64,13 @@ case "$PRESET" in
|
||||
echo "Environment variables:"
|
||||
echo " MODEL - Model path (default: Lightricks/LTX-2)"
|
||||
echo " PORT - Server port (default: 8098)"
|
||||
echo " FLOW_SHIFT - Scheduler flow shift (default: 1.0)"
|
||||
echo " BOUNDARY_RATIO - Boundary ratio (default: 1.0)"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
echo "Model: $MODEL"
|
||||
echo "Port: $PORT"
|
||||
echo "Flow shift: $FLOW_SHIFT"
|
||||
echo "Boundary ratio: $BOUNDARY_RATIO"
|
||||
|
||||
vllm serve "$MODEL" --omni \
|
||||
--port "$PORT" \
|
||||
--flow-shift "$FLOW_SHIFT" \
|
||||
--boundary-ratio "$BOUNDARY_RATIO" \
|
||||
"${EXTRA_ARGS[@]}"
|
||||
Reference in new issue
Block a user