2026-03-30 14:28:04 +08:00
# Diffusion Advanced Features
## Table of Contents
- [Overview ](#overview )
- [Supported Features ](#supported-features )
- [Supported Models ](#supported-models )
- [Feature Compatibility ](#feature-compatibility )
- [Learn More ](#learn-more )
## Overview
vLLM-Omni supports various advanced features for diffusion models:
2026-04-09 15:54:23 +08:00
- Acceleration: **cache methods** , **parallelism methods** , **startup optimizations**
2026-03-30 14:28:04 +08:00
- Memory optimization: **cpu offloading** , **quantization**
2026-04-15 11:52:20 +08:00
- Extensions: **LoRA inference** , **frame interpolation**
2026-05-08 09:28:52 +08:00
- Execution modes: **step execution**
2026-03-30 14:28:04 +08:00
## Supported Features
### Acceleration
#### Lossy Acceleration
Cache methods trade minimal quality for significant speedup. Quality loss is typically imperceptible with proper tuning.
| Method | Description | Best For |
|--------|-------------|----------|
| ** [TeaCache ](diffusion/cache_acceleration/teacache.md )** | Adaptive caching using modulated inputs | Quick setup, balanced quality/speed on single GPU |
| ** [Cache-DiT ](diffusion/cache_acceleration/cache_dit.md )** | Multiple caching techniques: DBCache, TaylorSeer, SCM | Fine-grained control, tunable quality-speed tradeoff |
#### Lossless Acceleration
Parallelism methods distribute computation across GPUs without quality loss (mathematically equivalent to single-GPU).
2026-05-19 09:42:17 +08:00
| Method | Description | Best For |
|------------------------------------------------------------------------|--------------------------------------------------------------------------|-------------------------------------------------------------------|
| ** [Ulysses-SP ](diffusion/parallelism/sequence_parallel.md )** | Sequence parallelism via all-to-all communication | High-resolution images (>1536px) or long videos with 2-8 GPUs |
| ** [Ring-Attention ](diffusion/parallelism/sequence_parallel.md )** | Sequence parallelism via ring-based communication | Videos, very long sequences, memory-constrained, with 2-8 GPUs |
| ** [CFG-Parallel ](diffusion/parallelism/cfg_parallel.md )** | Splits CFG positive/negative branches across devices | Image editing with CFG guidance (true_cfg_scale > 1) on 2 GPUs |
| ** [Tensor Parallelism ](diffusion/parallelism/tensor_parallel.md )** | Shards model weights across devices | Large models that don't fit in single GPU, with 2+ GPUs |
| ** [Pipeline Parallelism ](diffusion/parallelism/pipeline_parallel.md )** | Splits the denoising transformer block-wise across sequential GPU stages | Large diffusion transformers that need lower per-GPU model memory |
| ** [HSDP ](diffusion/parallelism/hsdp.md )** | Weight sharding via FSDP2, redistributed on-demand at runtime | Very large models (14B+) on limited VRAM, combinable with SP |
| ** [Expert Parallelism ](diffusion/parallelism/expert_parallel.md )** | Shards MoE expert MLP blocks across devices | MoE diffusion models (e.g., HunyuanImage3.0) |
2026-03-30 14:28:04 +08:00
2026-04-09 15:54:23 +08:00
#### Startup Optimization
| Method | Description | Best For |
|--------|-------------|----------|
| ** [Multi-Thread Weight Loading ](#multi-thread-weight-loading )** | Loads safetensors shards in parallel using a thread pool | All diffusion models; reduces startup from minutes to seconds |
2026-03-30 14:28:04 +08:00
**Note:** Some acceleration methods can be combined together for optimized performance. See [Feature Compatibility Table ](#feature-compatibility ) and [Feature Compatibility Tutorial ](feature_compatibility.md ) for detailed configuration examples.
### Memory Optimization
Memory optimization methods help reduce GPU memory usage, enabling inference on resource-constrained hardware or larger models.
| Method | Description | Best For |
|--------|-------------|----------|
2026-08-06 23:21:43 +08:00
| ** [CPU Offload ](diffusion/cpu_offload.md )** | Offloads model components to CPU memory | Limited VRAM, large models on consumer GPUs |
2026-04-28 19:33:24 +08:00
| ** [Quantization ](quantization/overview.md )** | Reduces transformer stages from BF16 to FP8/INT8/etc. | Limited VRAM, minimal accuracy loss |
2026-06-27 03:01:25 +02:00
| ** [VAE Parallelism ](diffusion/parallelism/vae_parallelism.md )** | Distributes VAE decode work across GPUs | High-resolution generation with reduced VAE memory peak |
2026-03-30 14:28:04 +08:00
### Extensions
Extension methods add specialized capabilities to diffusion models beyond standard inference.
| Method | Description | Best For |
|--------|-------------|----------|
| ** [LoRA Inference ](diffusion/lora.md )** | Enables inference with Low-Rank Adaptation (LoRA) adapters weights | Reinforcement learning extensions |
2026-04-15 11:52:20 +08:00
| ** [Frame Interpolation ](diffusion/frame_interpolation.md )** | Inserts intermediate video frames after generation for smoother motion | Video generation pipelines that need higher temporal smoothness |
2026-03-30 14:28:04 +08:00
2026-04-01 17:25:02 +08:00
### Execution Modes
2026-06-30 00:09:19 +08:00
Execution modes control how the diffusion pipeline processes requests and
denoise steps.
2026-04-01 17:25:02 +08:00
| Method | Description | Best For |
|--------|-------------|----------|
2026-08-05 22:41:18 +08:00
| ** [Diffusion Execution Modes ](diffusion/execution_modes.md )** | Configures serial requests, request batching, step execution, continuous batching, and streaming output | Matching latency, throughput, cancellation, and output-delivery requirements |
2026-04-01 17:25:02 +08:00
2026-08-05 22:41:18 +08:00
**Note:** Request-level batching and step execution are capability-based.
Consult the execution guide and the selected pipeline's documentation for
current support.
2026-04-01 17:25:02 +08:00
2026-03-30 14:28:04 +08:00
### Quantization Methods
| Method | Configuration | Description | Best For |
|--------|--------------|-------------|----------|
2026-04-28 19:33:24 +08:00
| ** [FP8 ](quantization/fp8.md )** | `quantization="fp8"` | FP8 W8A8 on validated transformer stages | Memory reduction, inference speedup |
| ** [INT8 ](quantization/int8.md )** | `quantization="int8"` | INT8 W8A8 on validated transformer stages | Memory reduction, broad GPU compatibility |
| ** [GGUF ](quantization/gguf.md )** | `quantization="gguf"` | Native GGUF transformer-only weights (Q4, Q8, etc.) | Memory reduction on consumer GPUs |
2026-03-30 14:28:04 +08:00
## Supported Models
The following tables show which models support each feature:
- **🔀SP (Ulysses & Ring)**: Includes both Ulysses-SP and Ring-Attention methods
- ✅ = Fully supported
- ❌ = Not supported
> Notes:
2026-08-06 23:21:43 +08:00
> 1. CPU Offload has two methods: Module-wise (default for models with DiT + text encoder) and Layerwise. The tables below show **Layerwise support** only. Split models like Cosmos3 (no separate text encoder) swap their reasoner/generator components for module-wise offload; see the [CPU Offload Guide](diffusion/cpu_offload.md).
> 2. The **💾Quantization** column is collapsed for readability. See [Quantization](quantization/overview.md) for per-method and per-model support details.
2026-03-30 14:28:04 +08:00
### ImageGen
2026-05-19 09:42:17 +08:00
| Model | ⚡TeaCache | ⚡Cache-DiT | 🔀SP (Ulysses & Ring) | 🔀CFG-Parallel | 🔀Tensor-Parallel | 🔀Pipeline-Parallel | 🔀HSDP | 💾CPU Offload (Layerwise) | 💾VAE-Patch-Parallel | 💾Quantization | 🔄Step Execution |
|--------------------------|:---------:|:----------:|:---------------------:|:--------------:|:-----------------:|:-------------------:|:------:|:-------------------------:|:--------------------:|:--------------:|:----------------:|
| **Bagel** | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
| **FLUX.1-dev** | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ | ❌ | ❌ | ✅ | ❌ |
| **FLUX.1-schnell** | ❌ | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ | ❌ | ❌ | ✅ | ❌ |
| **FLUX.2-klein** | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ❌ | ❌ | ✅ | ❌ |
2026-07-22 14:06:12 +08:00
| **FLUX.1-Kontext-dev** | ❌ | ✅ | ❌ | ❌ | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ |
2026-07-24 00:00:57 -07:00
| **FLUX.2-dev** | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ❌ | ❌ |
2026-05-19 09:42:17 +08:00
| **GLM-Image** | ❌ | ❌ | ❌ | ✅ | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ |
2026-05-22 15:23:03 +08:00
| **Hidream-I1-Full** | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
2026-05-19 09:42:17 +08:00
| **HunyuanImage3** | ❌ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ |
2026-07-12 04:55:41 +05:30
| **Krea 2** | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ (decode) | ❌ | ❌ |
2026-05-19 09:42:17 +08:00
| **LongCat-Image** | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ |
| **LongCat-Image-Edit** | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ |
| **MagiHuman** | ❌ | ❌ | ❌ | ❓ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ |
| **MammothModa2(T2I)** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
| **Nextstep_1(T2I)** | ❓ | ❓ | ❌ | ✅ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ |
2026-07-08 10:29:41 +08:00
| **OmniGen2** | ❌ | ✅ | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
2026-05-19 09:42:17 +08:00
| **Ovis-Image** | ❌ | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ |
| **Qwen-Image** | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ (decode) | ✅ | ✅ |
| **Qwen-Image-2512** | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ (decode) | ✅ | ✅ |
| **Qwen-Image-Edit** | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ (decode) | ❌ | ❌ |
| **Qwen-Image-Edit-2509** | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ (decode) | ✅ | ❌ | ❌ |
| **Qwen-Image-Layered** | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ (decode) | ❌ | ❌ |
2026-06-30 16:03:26 +08:00
| **SenseNova-U1** | ❌ | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ |
2026-06-16 09:51:11 -05:00
| **Stable-Diffusion-XL** | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ (decode) | ❌ | ❌ |
2026-05-19 09:42:17 +08:00
| **Stable-Diffusion3.5** | ❌ | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ | ✅ | ✅ (decode) | ❌ | ❌ |
| **Z-Image** | ✅ | ✅ | ✅ | ❓ | ✅ (TP=2 only) | ❌ | ✅ | ❌ | ✅ (decode) | ✅ | ❌ |
| **ERNIE-Image** | ❌ | ✅ | ✅ | ❓ | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
2026-06-01 22:04:31 +02:00
| **Cosmos3** | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ (decode) | ✅ | ❌ |
2026-03-30 14:28:04 +08:00
> Notes:
> 1. Nextstep_1(T2I) does not support cache acceleration methods such as TeaCache or Cache-DiT.
2026-04-07 17:16:12 +08:00
> 2. `Tongyi-MAI/Z-Image-Turbo` and `SII-GAIR/daVinci-MagiHuman-Base-1080p` are distilled models with minimal NFEs; CFG-Parallel is not necessary.
2026-07-13 11:48:53 +02:00
> 3. Cosmos3 T2I uses `Cosmos3OmniDiffusersPipeline` with `modalities=["image"]`. Model-level CPU offload swaps the nested UND reasoner and GEN generator pathways; layerwise offload remains available for blockwise GEN/UND offload.
2026-07-12 04:55:41 +05:30
> 4. Krea 2 currently supports single-GPU inference plus LoRA, Cache-DiT, HSDP, CPU/layerwise offload, and VAE-patch-parallel (decode). TP/SP/CFG-Parallel are not yet wired. The few-step distilled (Turbo) checkpoint uses `is_distilled=true` (fixed timestep shift `mu=1.15`); generate at 2048x2048 by default with `num_inference_steps≈8` and `guidance_scale=0`. The Raw checkpoint uses 1024x1024, `num_inference_steps=28`, and `guidance_scale=4.5`.
2026-03-30 14:28:04 +08:00
### VideoGen
2026-05-19 09:42:17 +08:00
| Model | ⚡TeaCache | ⚡Cache-DiT | 🔀SP (Ulysses & Ring) | 🔀CFG-Parallel | 🔀Tensor-Parallel | Pipeline-Parallel | 🔀HSDP | 💾CPU Offload (Layerwise) | 💾VAE-Patch-Parallel | 💾Quantization | 🔄Step Execution |
|------------------------------|:---------:|:----------:|:---------------------:|:--------------:|:-----------------:|:-----------------:|:------:|:-------------------------:|:--------------------:|:--------------:|:----------------:|
| **Wan2.2** | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ (encode/decode) | ❌ | ❌ |
2026-06-27 23:27:58 -05:00
| **Wan2.2-S2V** | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ (encode/decode) | ❌ | ❌ |
2026-05-19 09:42:17 +08:00
| **Wan2.1-VACE** | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ (decode) | ❌ | ❌ |
2026-07-20 20:02:30 +08:00
| **LTX-2** | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ (decode) | ❌ | ❌ |
| **LTX-2.3** | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ (decode) | ❌ | ❌ |
2026-05-19 09:42:17 +08:00
| **Helios** | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
2026-05-30 14:40:42 +08:00
| **HunyuanVideo-1.5 T2V I2V** | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ (encode/decode) | ✅ | ❌ |
2026-05-19 09:42:17 +08:00
| **DreamID-Omni** | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
2026-06-01 22:04:31 +02:00
| **Cosmos3** | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ (encode/decode) | ✅ | ❌ |
2026-08-05 15:12:01 +08:00
| **MiniMax-H3** | ❌ | ✅ | ✅ | ❌ | ✅ (DiT/TE) | ❌ | ✅ | ✅ | ✅ (tile) | ✅ (DiT) | ❌ |
2026-03-30 14:28:04 +08:00
2026-04-15 11:52:20 +08:00
**Frame Interpolation Support**
- **Supported**: Wan2.2 text-to-video, image-to-video, and TI2V pipelines
2026-04-23 10:51:51 +02:00
- **Not supported**: Wan2.1-VACE, LTX-2, LTX-2.3, Helios, HunyuanVideo-1.5, DreamID-Omni
2026-04-15 11:52:20 +08:00
2026-03-30 14:28:04 +08:00
### AudioGen
2026-05-19 09:42:17 +08:00
| Model | ⚡TeaCache | ⚡Cache-DiT | 🔀SP (Ulysses & Ring) | 🔀CFG-Parallel | 🔀Tensor-Parallel | 🔀Pipeline-Parallel | 🔀HSDP | 💾CPU Offload (Layerwise) | 💾VAE-Patch-Parallel | 💾Quantization | 🔄Step Execution |
|-----------------------|:---------:|:----------:|:---------------------:|:--------------:|:-----------------:|:-------------------:|:------:|:-------------------------:|:--------------------:|:--------------:|:----------------:|
| **Stable-Audio-Open** | ✅ | ❌ | ❓ | ❓ | ❌ | ❌ | ✅ | ✅ | ❌ | ✅ | ❌ |
2026-03-30 14:28:04 +08:00
## Feature Compatibility
**Legend:**
- ✅: Functionality is supported
- ❌: No support plan
- ❓: Not verified yet and Not Recommended
2026-04-01 17:25:02 +08:00
| | ⚡TeaCache | ⚡Cache-DiT | 🔀Ulysses-SP | 🔀Ring-Attn | 🔀CFG-Parallel | 🔀Tensor Parallel | 🔀HSDP | 🔀Expert Parallel | 💾CPU Offloading (Layerwise) | 💾CPU Offloading (Module-wise) | 💾VAE Patch Parallel | 💾FP8 Quant | 🔧LoRA Inference | 🔄Step Execution |
|---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
| ** ⚡TeaCache** | | | | | | | | | | | | | | |
| ** ⚡Cache-DiT** | ❌ | | | | | | | | | | | | | |
| ** 🔀Ulysses-SP** | ✅ | ✅ | | | | | | | | | | | | |
| ** 🔀Ring-Attn** | ✅ | ✅ | ✅ | | | | | | | | | | | |
| ** 🔀CFG-Parallel** | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | |
| ** 🔀Tensor Parallel** | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | |
| ** 🔀HSDP** | ❓ | ❓ | ❓ | ❓ | ❓ | ❌ | | | | | | | | |
| ** 🔀Expert Parallel** | ❓ | ❓ | ❓ | ❓ | ❓ | ❓ | ❓ | | | | | | | |
| ** 💾CPU Offloading (Layerwise)** | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | | | | | | |
| ** 💾CPU Offloading (Module-wise)** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❓ | ❓ | ❌ | | | | | |
| ** 💾VAE Patch Parallel** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | | | | |
| ** 💾FP8 Quant** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❓ | ❓ | ✅ | ✅ | ✅ | | | |
| ** 🔧LoRA Inference** | ❓ | ❓ | ❓ | ❓ | ❓ | ❓ | ❓ | ❓ | ❓ | ❓ | ❓ | ❓ | | |
2026-05-20 22:26:18 +08:00
| ** 🔄Step Execution** | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ❓ | ❓ | ✅ | ❓ | ✅ | ✅ | ✅ | |
2026-03-30 14:28:04 +08:00
!!! info
1. Tensor Parallel and HSDP are not compatible.
2. TeaCache and Cache-DiT are not compatible.
3. CPU Offloading (Layerwise) and CPU Offloading (Module-wise) are not compatible.
4. CPU Offloading (Layerwise) supports single-card for now.
5. Using FP8-Quant as an example of qunatization methods.
2026-08-05 22:41:18 +08:00
6. Step Execution is not compatible with any diffusion cache backend. LoRA is supported, but each scheduled batch must use a single adapter (requests with different `lora_request` or `lora_scale` are kept in separate batches).
2026-03-30 14:28:04 +08:00
2026-04-09 15:54:23 +08:00
## Multi-Thread Weight Loading
Large diffusion models can take several minutes to load weights at startup (e.g., ~3 min for Qwen-Image, ~5 min for Wan2.2 I2V 14B). Multi-thread weight loading speeds up this process by loading safetensors shards in parallel using a thread pool instead of sequentially.
This optimization is **enabled by default** with 4 threads. No configuration is needed for the default behavior.
### Configuration
| Parameter | CLI Flag | Default | Description |
|-----------|----------|---------|-------------|
| `enable_multithread_weight_load` | `--disable-multithread-weight-load` | `True` (enabled) | Pass the flag to disable multi-thread loading |
| `num_weight_load_threads` | `--num-weight-load-threads` | `4` | Number of threads for parallel weight loading |
!!! tip
The default of 4 threads balances speed and disk I/O contention. On fast NVMe storage you may benefit from more threads (e.g., 8). On HDD or network storage, the default of 4 avoids saturating I/O bandwidth.
### Online Serving
```bash
# Default (multi-thread enabled, 4 threads)
vllm serve Qwen/Qwen-Image --omni --port 8091
# Custom thread count
vllm serve Wan-AI/Wan2.2-I2V-A14B-Diffusers --omni --num-weight-load-threads 8
# Disable multi-thread loading
vllm serve Qwen/Qwen-Image --omni --disable-multithread-weight-load
```
### Offline Inference
```python
from vllm_omni import Omni
# Default (multi-thread enabled, 4 threads)
omni = Omni ( model = "Qwen/Qwen-Image" )
# Custom thread count
omni = Omni (
model = "Wan-AI/Wan2.2-I2V-A14B-Diffusers" ,
num_weight_load_threads = 8 ,
)
```
### Benchmarks
Measured on NVIDIA H800:
| Model | Before | After | Speedup |
|-------|--------|-------|---------|
| **Qwen/Qwen-Image** (53.7 GiB) | 168s | 27s | **6.2x** |
| **Wan-AI/Wan2.2-I2V-A14B-Diffusers** (64.5 GiB) | 283s | 56s | **5.1x** |
2026-03-30 14:28:04 +08:00
## Learn More
**Cache Acceleration:**
- **[TeaCache Configuration Guide ](diffusion/cache_acceleration/teacache.md )** - Parameter tuning, performance tips, troubleshooting
- **[Cache-DiT Advanced Guide ](diffusion/cache_acceleration/cache_dit.md )** - DBCache, TaylorSeer, SCM techniques and optimization
**Parallelism Methods:**
2026-05-19 09:42:17 +08:00
- **[Parallelism Overview ](diffusion/parallelism/overview.md )** - Tensor Parallelism, Sequence Parallelism, CFG Parallelism, Pipeline Parallelism, HSDP, and Expert Parallelism
2026-03-30 14:28:04 +08:00
**Memory Optimization:**
2026-08-06 23:21:43 +08:00
- **[CPU Offload Guide ](diffusion/cpu_offload.md )** - Offload model components to CPU, reduce GPU memory usage
2026-06-27 03:01:25 +02:00
- **[VAE Parallelism Guide ](diffusion/parallelism/vae_parallelism.md )** - Distribute VAE decode work across GPUs for high-resolution images and videos
2026-08-06 23:21:43 +08:00
- **[Quantization ](quantization/overview.md )** - Quantization methods for diffusion, multi-stage omni/TTS, and multi-stage diffusion models
2026-03-30 14:28:04 +08:00
**Extensions:**
- **[LoRA Inference Guide ](diffusion/lora.md )** - Low-Rank Adaptation for style customization and fine-tuning
2026-04-15 11:52:20 +08:00
- **[Frame Interpolation Guide ](diffusion/frame_interpolation.md )** - Worker-side post-generation video frame interpolation for smoother motion
2026-03-30 14:28:04 +08:00
2026-04-01 17:25:02 +08:00
**Execution Modes:**
2026-08-05 22:41:18 +08:00
- **[Diffusion Execution Modes ](diffusion/execution_modes.md )** - Configure request batching, step execution, continuous batching, and streaming output
2026-04-01 17:25:02 +08:00
2026-04-09 15:54:23 +08:00
**Startup Optimization:**
- **[Multi-Thread Weight Loading ](#multi-thread-weight-loading )** - Speed up model startup by loading safetensors shards in parallel
2026-03-30 14:28:04 +08:00
**Advanced Topics:**
- **[Feature Compatibility ](feature_compatibility.md )** - How to combine multiple features for maximum performance
