Add comprehensive LTX-2 integration documentation

Co-authored-by: hsliuustc0106 <222337142+hsliuustc0106@users.noreply.github.com>

docs/LTX2_INTEGRATION.md

@@ -0,0 +1,173 @@
+# LTX-2 Model Integration
+
+## Overview
+
+This integration adds support for the Lightricks LTX-2 text-to-video generation model to vllm-omni. LTX-2 is a state-of-the-art open-source model capable of generating high-quality videos (up to 4K resolution) from text prompts or images.
+
+## Model Information
+
+- **Model ID**: `Lightricks/LTX-2`
+- **Model Type**: Text-to-Video (T2V) and Image-to-Video (I2V)
+- **Architecture**: Diffusion Transformer (DiT) with 19B parameters
+- **HuggingFace**: https://huggingface.co/Lightricks/LTX-2
+- **GitHub**: https://github.com/Lightricks/LTX-2
+
+## Quick Start
+
+### Basic Usage
+
+```python
+from vllm_omni.entrypoints.omni import Omni
+
+# Initialize the model
+omni = Omni(model="Lightricks/LTX-2")
+
+# Generate a video
+output = omni.generate(
+    prompt="A panda riding a bicycle through a forest, cinematic lighting",
+    height=512,
+    width=768,
+    num_frames=121,
+    num_inference_steps=40,
+    guidance_scale=4.0,
+)
+```
+
+### Using the Example Script
+
+```bash
+cd examples/offline_inference/ltx2
+
+python text_to_video.py \
+    --prompt "A serene lakeside sunrise with mist over the water" \
+    --height 512 \
+    --width 768 \
+    --num_frames 121 \
+    --output my_video.mp4
+```
+
+See the [example documentation](examples/offline_inference/ltx2/text_to_video.md) for more details.
+
+## Parameters
+
+### Video Dimensions
+- **height**: Video height in pixels (must be divisible by 32)
+- **width**: Video width in pixels (must be divisible by 32)
+- Default: 512x768
+
+### Frame Count
+- **num_frames**: Number of frames (should follow pattern 8*n+1)
+- Valid examples: 25, 41, 81, 121, 161
+- Default: 121 frames (approximately 5 seconds at 24fps)
+
+### Generation Quality
+- **num_inference_steps**: Number of denoising steps (default: 40)
+  - More steps = higher quality but slower generation
+  - Range: 20-50 steps typically
+  
+- **guidance_scale**: Classifier-free guidance strength (default: 4.0)
+  - Higher values follow prompt more strictly
+  - Range: 3.0-5.0 recommended
+
+### Other Options
+- **negative_prompt**: Text describing what to avoid in the video
+- **seed**: Random seed for reproducibility
+- **fps**: Output video frame rate (default: 24)
+
+## Implementation Details
+
+### Architecture
+
+The integration follows vllm-omni's diffusion model patterns:
+
+```
+vllm_omni/diffusion/models/ltx2/
+├── __init__.py                  # Module exports
+└── pipeline_ltx2.py             # Main pipeline implementation
+```
+
+### Key Components
+
+1. **LTX2Pipeline**: Wraps diffusers' `LTXVideoPipeline`
+   - Handles parameter validation
+   - Ensures dimension constraints (divisible by 32)
+   - Manages frame count convention (8*n+1)
+   
+2. **Pre-processing**: `get_ltx2_pre_process_func`
+   - Loads and validates image inputs for I2V mode
+   
+3. **Post-processing**: `get_ltx2_post_process_func`
+   - Converts video tensors to numpy arrays
+   - Handles different output formats
+
+### Registry Integration
+
+The model is registered in `vllm_omni/diffusion/registry.py`:
+- Model class: `LTX2Pipeline`
+- Pre-process function: `get_ltx2_pre_process_func`
+- Post-process function: `get_ltx2_post_process_func`
+
+## Requirements
+
+- **Python**: >= 3.10
+- **GPU**: CUDA-capable GPU with >= 24GB VRAM recommended
+- **Dependencies**: diffusers >= 0.36.0 (already included in vllm-omni)
+
+### Model Download
+
+On first use, the model will be automatically downloaded from HuggingFace:
+- Size: Approximately 20-30GB
+- Location: `~/.cache/huggingface/hub/`
+
+## Performance Tips
+
+1. **Start Small**: Test with lower resolutions (512x768) and fewer frames (25-81)
+2. **Increase Gradually**: Scale up resolution and frame count based on your GPU
+3. **Optimize Steps**: 40 steps provide good quality; adjust based on speed/quality needs
+4. **Batch Processing**: For multiple videos, reuse the same Omni instance to avoid reloading
+
+## Troubleshooting
+
+### Out of Memory (OOM)
+
+If you encounter OOM errors:
+- Reduce video dimensions (e.g., 480x640 instead of 512x768)
+- Reduce frame count (e.g., 25 or 41 instead of 121)
+- Enable VAE tiling/slicing if supported
+
+### Slow Generation
+
+For faster generation:
+- Reduce `num_inference_steps` (e.g., 30 instead of 40)
+- Use smaller dimensions
+- Generate fewer frames
+
+### Quality Issues
+
+For better quality:
+- Increase `num_inference_steps` (e.g., 50)
+- Adjust `guidance_scale` (try 3.5-5.0)
+- Use more descriptive prompts with style keywords
+
+## Example Prompts
+
+```python
+# Cinematic nature scene
+"A serene lakeside sunrise with mist over the water, cinematic, high quality"
+
+# Action scene
+"A futuristic cityscape with flying cars, neon lights, cyberpunk style, 4k"
+
+# Animated style
+"A cartoon panda eating bamboo in a forest, Pixar style animation"
+
+# Close-up shot
+"Close-up of raindrops falling on a window, soft focus, moody lighting"
+```
+
+## References
+
+- [LTX-2 on HuggingFace](https://huggingface.co/Lightricks/LTX-2)
+- [LTX-2 GitHub Repository](https://github.com/Lightricks/LTX-2)
+- [Diffusers Documentation](https://huggingface.co/docs/diffusers/main/en/api/pipelines/ltx_video)
+- [Example Usage](examples/offline_inference/ltx2/text_to_video.md)