anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

[Misc] Fix docs and update image edit example with cache-dit (#318)

Browse Source 
Signed-off-by: samithuang <285365963@qq.com>
This commit is contained in:
Samit authored and GitHub committed 2025-12-15 13:43:28 +00:00
1 parent 5a9e312a2e
commit 06c2b7f6e4
10 files changed
+155 -198
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/.nav.yml
+3 -3
View File
@@ -21,9 +21,9 @@ nav:
- configuration/README.md
- configuration/*
- Diffusion Acceleration:
- Overview: user_guide/diffusion_acceleration.md
- TeaCache: user_guide/teacache.md
- Cache-DiT: user_guide/cache_dit_acceleration.md
- Overview: user_guide/diffusion_acceleration.md
- TeaCache: user_guide/teacache.md
- Cache-DiT: user_guide/cache_dit_acceleration.md
- Models:
- models/supported_models.md
- Developer Guide:
docs/api/README.md
+19 -9
View File
@@ -1,15 +1,8 @@
# Summary
## Entry Points
## Configuration
Configuration classes.
- [vllm_omni.config.model.OmniModelConfig][]
## EntryPoints
Main entrypoints for vLLM-Omni inference and serving.
Main entry points for vLLM-Omni inference and serving.
- [vllm_omni.entrypoints.async_omni.AsyncOmni][]
- [vllm_omni.entrypoints.async_omni.AsyncOmniStageLLM][]
@@ -19,6 +12,7 @@ Main entrypoints for vLLM-Omni inference and serving.
- [vllm_omni.entrypoints.cli.serve.OmniServeCommand][]
- [vllm_omni.entrypoints.log_utils.OrchestratorMetrics][]
- [vllm_omni.entrypoints.omni.Omni][]
- [vllm_omni.entrypoints.omni_diffusion.OmniDiffusion][]
- [vllm_omni.entrypoints.omni_llm.OmniLLM][]
- [vllm_omni.entrypoints.omni_llm.OmniStageLLM][]
- [vllm_omni.entrypoints.omni_stage.OmniStage][]
@@ -97,14 +91,30 @@ Model execution components.
- [vllm_omni.model_executor.models.qwen3_omni.qwen3_omni_moe_thinker.Qwen3_VisionPatchEmbed][]
- [vllm_omni.model_executor.models.qwen3_omni.qwen3_omni_moe_thinker.Qwen3_VisionPatchMerger][]
## Configuration
Configuration classes.
- [vllm_omni.config.model.OmniModelConfig][]
- [vllm_omni.diffusion.cache.teacache.config.TeaCacheConfig][]
- [vllm_omni.distributed.omni_connectors.utils.config.ConnectorSpec][]
- [vllm_omni.distributed.omni_connectors.utils.config.OmniTransferConfig][]
## Workers
Worker classes and model runners for distributed inference.
- [vllm_omni.diffusion.worker.gpu_worker.GPUWorker][]
- [vllm_omni.diffusion.worker.gpu_worker.WorkerProc][]
- [vllm_omni.diffusion.worker.npu.npu_worker.NPUWorker][]
- [vllm_omni.diffusion.worker.npu.npu_worker.NPUWorkerProc][]
- [vllm_omni.worker.gpu_ar_model_runner.GPUARModelRunner][]
- [vllm_omni.worker.gpu_ar_worker.GPUARWorker][]
- [vllm_omni.worker.gpu_generation_model_runner.GPUGenerationModelRunner][]
- [vllm_omni.worker.gpu_generation_worker.GPUGenerationWorker][]
- [vllm_omni.worker.gpu_model_runner.OmniGPUModelRunner][]
- [vllm_omni.worker.npu.npu_ar_model_runner.NPUARModelRunner][]
- [vllm_omni.worker.npu.npu_ar_worker.NPUARWorker][]
- [vllm_omni.worker.npu.npu_generation_model_runner.NPUGenerationModelRunner][]
- [vllm_omni.worker.npu.npu_generation_worker.NPUGenerationWorker][]
- [vllm_omni.worker.npu.npu_model_runner.OmniNPUModelRunner][]
docs/user_guide/cache_dit_acceleration.md
+53 -169
View File
@@ -31,12 +31,7 @@ images = omni.generate(
)
```
**Default Parameters**: When `cache_config` is not provided, cache-dit uses:
- `Fn_compute_blocks=8` (aligned with cache-dit's recommended default)
- `Bn_compute_blocks=0`
- `max_warmup_steps=8`
- `residual_diff_threshold=0.08`
- No TaylorSeer or SCM enabled
**Default Parameters**: When `cache_config` is not provided, cache-dit uses optimized default values. See the [Configuration Reference](#configuration-reference) section for a complete list of all parameters and their default values.
### Custom Configuration
@@ -55,42 +50,6 @@ omni = Omni(
)
```
### Example Script
See `examples/offline_inference/qwen_image/text_to_image.py` for a complete working example with cache-dit acceleration.
```bash
# Enable cache-dit with default parameters
cd examples/offline_inference/qwen_image
python text_to_image.py \
--prompt "a cup of coffee on the table" \
--enable_cache_dit \
--num_inference_steps 50
```
The `--enable_cache_dit` flag enables cache-dit acceleration with these customized parameters:
```python
omni = Omni(
...
cache_backend="cache_dit" if args.enable_cache_dit else None,
cache_config={
# Scheme: Hybrid DBCache + SCM + TaylorSeer
# DBCache
"Fn_compute_blocks": 8,
"Bn_compute_blocks": 0,
"max_warmup_steps": 4,
"residual_diff_threshold": 0.12,
# TaylorSeer
"enable_taylorseer": True,
"taylorseer_order": 1,
# SCM
"scm_steps_mask_policy": "fast",
"scm_steps_policy": "dynamic",
},
)
```
## Acceleration Methods
@@ -104,10 +63,10 @@ DBCache intelligently caches intermediate transformer block outputs when the res
- `Fn_compute_blocks` (int, default: 1): Number of **first n** transformer blocks used to compute stable feature differences. Higher values provide more accurate caching decisions but increase computation.
- `Bn_compute_blocks` (int, default: 0): Number of **last n** transformer blocks used for additional fusion. These blocks act as an auto-scaler for approximate hidden states.
- `max_warmup_steps` (int, default: 8): Number of initial steps where caching is disabled to ensure the model learns sufficient features before caching begins.
- `residual_diff_threshold` (float, default: 0.08): Threshold for residual difference. Higher values lead to faster performance but may reduce precision.
- `max_warmup_steps` (int, default: 4): Number of initial steps where caching is disabled to ensure the model learns sufficient features before caching begins. Optimized for few-step distilled models.
- `residual_diff_threshold` (float, default: 0.24): Threshold for residual difference. Higher values lead to faster performance but may reduce precision. Default uses a relatively higher threshold for more aggressive caching.
- `max_cached_steps` (int, default: -1): Maximum number of cached steps. Set to -1 for unlimited caching.
- `max_continuous_cached_steps` (int, default: -1): Maximum number of consecutive cached steps. Set to -1 for unlimited consecutive caching.
- `max_continuous_cached_steps` (int, default: 3): Maximum number of consecutive cached steps. Limits consecutive caching to prevent precision degradation.
**Example Configuration**:
@@ -122,9 +81,10 @@ cache_config={
```
**Performance Tips**:
- Start with `Fn_compute_blocks=8` and adjust based on your model size
- Increase `residual_diff_threshold` (e.g., 0.12-0.15) for faster inference with slight quality trade-off
- Reduce `max_warmup_steps` (e.g., 4-6) for faster startup, but ensure sufficient warmup
- Default `Fn_compute_blocks=1` works well for most cases. Increase to 8-12 for larger models or when more accuracy is needed
- Increase `residual_diff_threshold` (e.g., 0.12-0.15) for faster inference with slight quality trade-off, or decrease from default 0.24 for higher quality
- Default `max_warmup_steps=4` is optimized for few-step models. Increase to 6-8 for more steps if needed
### 2. TaylorSeer
@@ -145,6 +105,7 @@ cache_config={
```
**Performance Tips**:
- Use `taylorseer_order=1` for most cases (good balance of speed and quality)
- Combine with DBCache for maximum acceleration
- Higher orders (2-3) may improve quality but reduce speed gains
@@ -156,6 +117,7 @@ SCM allows you to specify which steps must be computed and which can use cached
**Key Parameters**:
- `scm_steps_mask_policy` (str | None, default: None): Predefined mask policy. Options:
- `None`: SCM disabled (default)
- `"slow"`: More compute steps, higher quality (18 compute steps out of 28)
- `"medium"`: Balanced (15 compute steps out of 28)
- `"fast"`: More cache steps, faster inference (11 compute steps out of 28)
@@ -174,151 +136,73 @@ cache_config={
```
**Performance Tips**:
- SCM is disabled by default (`scm_steps_mask_policy=None`). Enable it by setting a policy value if you need additional acceleration
- Start with `"medium"` policy and adjust based on quality requirements
- Use `"fast"` or `"ultra"` for maximum speed when quality can be slightly compromised
- `"dynamic"` policy generally provides better quality than `"static"`
- SCM mask is automatically regenerated when `num_inference_steps` changes during inference
## Complete Configuration Examples
### Example 1: DBCache Only (Recommended Starting Point)
```python
omni = Omni(
model="Qwen/Qwen-Image",
cache_backend="cache_dit",
cache_config={
"Fn_compute_blocks": 8,
"Bn_compute_blocks": 0,
"max_warmup_steps": 8,
"residual_diff_threshold": 0.08,
},
)
```
### Example 2: DBCache + TaylorSeer (Balanced Speed/Quality)
```python
omni = Omni(
model="Qwen/Qwen-Image",
cache_backend="cache_dit",
cache_config={
# DBCache
"Fn_compute_blocks": 8,
"Bn_compute_blocks": 0,
"max_warmup_steps": 8,
"residual_diff_threshold": 0.12,
# TaylorSeer
"enable_taylorseer": True,
"taylorseer_order": 1,
},
)
```
### Example 3: DBCache + SCM
```python
omni = Omni(
model="Qwen/Qwen-Image",
cache_backend="cache_dit",
cache_config={
# DBCache
"Fn_compute_blocks": 8,
"Bn_compute_blocks": 0,
"max_warmup_steps": 4,
"residual_diff_threshold": 0.12,
# SCM
"scm_steps_mask_policy": "fast",
"scm_steps_policy": "dynamic",
},
)
```
### Example 4: All Methods Combined
```python
omni = Omni(
model="Qwen/Qwen-Image",
cache_backend="cache_dit",
cache_config={
# DBCache
"Fn_compute_blocks": 8,
"Bn_compute_blocks": 0,
"max_warmup_steps": 4,
"residual_diff_threshold": 0.12,
# TaylorSeer
"enable_taylorseer": True,
"taylorseer_order": 1,
# SCM
"scm_steps_mask_policy": "fast",
"scm_steps_policy": "dynamic",
},
)
```
## Dynamic num_inference_steps
Cache-dit automatically handles changes in `num_inference_steps` during inference. When you call `generate()` with a different `num_inference_steps` than the previous request, cache-dit will:
1. Automatically refresh the cache context
2. Regenerate SCM masks if SCM is enabled
3. Update internal configurations to match the new step count
**Example**:
```python
# First request with 50 steps
images1 = omni.generate("prompt 1", num_inference_steps=50)
# Second request with 28 steps - cache context automatically refreshes
images2 = omni.generate("prompt 2", num_inference_steps=28)
```
No manual intervention is required - the refresh happens automatically and efficiently.
## Configuration Reference
### DiffusionCacheConfig Parameters
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `Fn_compute_blocks` | int | 8 | First n blocks for difference computation (aligned with cache-dit's default) |
| `Fn_compute_blocks` | int | 1 | First n blocks for difference computation (optimized for single-transformer models) |
| `Bn_compute_blocks` | int | 0 | Last n blocks for fusion |
| `max_warmup_steps` | int | 8 | Steps before caching starts |
| `max_warmup_steps` | int | 4 | Steps before caching starts (optimized for few-step distilled models) |
| `max_cached_steps` | int | -1 | Max cached steps (-1 = unlimited) |
| `max_continuous_cached_steps` | int | -1 | Max consecutive cached steps (-1 = unlimited) |
| `residual_diff_threshold` | float | 0.08 | Residual difference threshold |
| `max_continuous_cached_steps` | int | 3 | Max consecutive cached steps (prevents precision degradation) |
| `residual_diff_threshold` | float | 0.24 | Residual difference threshold (higher for more aggressive caching) |
| `num_inference_steps` | int \| None | None | Initial inference steps for SCM mask generation (optional, auto-refreshed during inference) |
| `enable_taylorseer` | bool | False | Enable TaylorSeer acceleration |
| `enable_taylorseer` | bool | False | Enable TaylorSeer acceleration (not suitable for few-step distilled models) |
| `taylorseer_order` | int | 1 | Taylor expansion order |
| `scm_steps_mask_policy` | str \| None | None | SCM mask policy ("slow", "medium", "fast", "ultra") |
| `scm_steps_mask_policy` | str \| None | None | SCM mask policy (None, "slow", "medium", "fast", "ultra") |
| `scm_steps_policy` | str | "dynamic" | SCM computation policy ("dynamic" or "static") |
## Tips and Best Practices
## Example: Accelerate Text-to-Image Generation with CacheDiT
### 1. Start Simple
The easiest way to get started is to enable cache-dit without any configuration:
See `examples/offline_inference/text_to_image/text_to_image.py` for a complete working example with cache-dit acceleration.
```python
omni = Omni(model="Qwen/Qwen-Image", cache_backend="cache_dit")
```bash
# Enable cache-dit with hybrid acceleration
cd examples/offline_inference/text_to_image
python text_to_image.py \
--model Qwen/Qwen-Image \
--prompt "a cup of coffee on the table" \
--cache_backend cache_dit \
--num_inference_steps 50
```
This uses cache-dit's recommended defaults (`Fn_compute_blocks=8`). Once you understand the baseline performance, you can customize the configuration to add other acceleration methods or fine-tune parameters.
### 2. Quality vs Speed Trade-offs
- **Higher quality**: Lower `residual_diff_threshold` (0.08), more `Fn_compute_blocks` (8-12), SCM `"slow"` or `"medium"`
- **Faster inference**: Higher `residual_diff_threshold` (0.12-0.15), fewer `max_warmup_steps` (4-6), SCM `"fast"` or `"ultra"`
The script uses cache-dit acceleration with a hybrid configuration combining DBCache, SCM, and TaylorSeer:
### 3. Model-Specific Tuning
Different models may benefit from different configurations:
- **Smaller models**: May work well with `Fn_compute_blocks=4-6`
- **Larger models**: Often benefit from `Fn_compute_blocks=8-12`
- **High-resolution images**: May need lower `residual_diff_threshold` for quality
```python
omni = Omni(
model="Qwen/Qwen-Image",
cache_backend="cache_dit",
cache_config={
# Scheme: Hybrid DBCache + SCM + TaylorSeer
# DBCache
"Fn_compute_blocks": 8,
"Bn_compute_blocks": 0,
"max_warmup_steps": 4,
"residual_diff_threshold": 0.12,
# TaylorSeer
"enable_taylorseer": True,
"taylorseer_order": 1,
# SCM
"scm_steps_mask_policy": "fast", # Set to None to disable SCM
"scm_steps_policy": "dynamic",
},
)
```
You can customize the configuration by modifying the `cache_config` dictionary to use only specific methods (e.g., DBCache only, DBCache + SCM, etc.) based on your quality and speed requirements.
To test another model, you can modify `--model` with the target model identifier like `Tongyi-MAI/Z-Image-Turbo` and update `cache_confg` according the model architecture (e.g., number of transformer blocks).
### 4. Combining Methods
- **DBCache + TaylorSeer**: Good balance, works well for most cases
- **DBCache + SCM**: Can try when quality can be slightly compromised
- **All three**: Maximum acceleration if the parameters are carefully tuned
## Additional Resources
docs/user_guide/diffusion_acceleration.md
+10 -9
View File
@@ -29,13 +29,12 @@ The following table shows which models are currently supported by each cache bac
|-------|-----------------|----------|-----------|
| **Qwen-Image** | `Qwen/Qwen-Image` | ✅ | ✅ |
| **Z-Image** | `Tongyi-MAI/Z-Image-Turbo` | ❌ | ✅ |
| **Qwen-Image-Edit** | `Qwen/Qwen-Image-Edit` | ❌ | ✅* |
| **Qwen-Image-Edit** | `Qwen/Qwen-Image-Edit` | ❌ | ✅ |
\* Qwen-Image-Edit support is not fully verified
## Performance Benchmarks
The following benchmarks were measured on **Qwen/Qwen-Image** model generating images with 50 inference steps:
The following benchmarks were measured on **Qwen/Qwen-Image** and **Qwen/Qwen-Image-Edit** models with 50 inference steps:
!!! note "Benchmark Disclaimer"
These benchmarks are provided for **general reference only**. The configurations shown use default or common parameter settings and have not been exhaustively optimized for maximum performance. Actual performance may vary based on:
@@ -47,12 +46,14 @@ The following benchmarks were measured on **Qwen/Qwen-Image** model generating i
For optimal performance in your specific scenario, we recommend experimenting with different parameter configurations as described in the detailed guides below.
| Configuration | Generation Time | Speedup | Notes |
|---------------|----------------|---------|-------|
| **Baseline (diffusers)** | 20.0s | 1.0x | No acceleration |
| **TeaCache** (`rel_l1_thresh=0.2`) | 10.47s | **1.91x** | Recommended default setting |
| **Cache-DiT (DBCache + TaylorSeer)** | 10.8s | **1.85x** | Fn=1, Bn=0, W=8, TaylorSeer order=1 |
| **Cache-DiT (DBCache + TaylorSeer + SCM)** | 14.0s | **1.43x** | Fn=8, Bn=0, W=4, TaylorSeer order=1, SCM fast |
| Model | Cache Backend | Cache Config | Generation Time | Speedup | Notes |
|-------|---------------|--------------|----------------|---------|-------|
| **Qwen/Qwen-Image** | None | None | 20.0s | 1.0x | Baseline (diffusers) |
| **Qwen/Qwen-Image** | TeaCache | `rel_l1_thresh=0.2` | 10.47s | **1.91x** | Recommended default setting |
| **Qwen/Qwen-Image** | Cache-DiT | DBCache + TaylorSeer (Fn=1, Bn=0, W=8, TaylorSeer order=1) | 10.8s | **1.85x** | - |
| **Qwen/Qwen-Image** | Cache-DiT | DBCache + TaylorSeer + SCM (Fn=8, Bn=0, W=4, TaylorSeer order=1, SCM fast) | 14.0s | **1.43x** | - |
| **Qwen/Qwen-Image-Edit** | None | No acceleration | 51.5s | 1.0x | Baseline (diffusers) |
| **Qwen/Qwen-Image-Edit** | Cache-DiT | Default (Fn=1, Bn=0, W=4, TaylorSeer disabled, SCM disabled) | 21.6s | **2.38x** | - |
## Quick Start
docs/user_guide/examples/offline_inference/image_to_image.md
+1 -1
View File
@@ -1,6 +1,6 @@
# Image-To-Image
Source <https://github.com/vllm-project/vllm-omni/tree/main/examples/offline_inference/image_to_image>. Added in PR [#196](https://github.com/vllm-project/vllm-omni/pull/196).
Source <https://github.com/vllm-project/vllm-omni/tree/main/examples/offline_inference/image_to_image>.
This example edits an input image with `Qwen/Qwen-Image-Edit` using the `image_edit.py` CLI.
docs/user_guide/examples/offline_inference/text_to_image.md
+13 -2
View File
@@ -3,11 +3,22 @@
Source <https://github.com/vllm-project/vllm-omni/tree/main/examples/offline_inference/text_to_image>.
This folder provides two simple entrypoints for experimenting with `Qwen/Qwen-Image` `Tongyi-MAI/Z-Image-Turbo` using vLLM-Omni:
This folder provides several entrypoints for experimenting with `Qwen/Qwen-Image` `Tongyi-MAI/Z-Image-Turbo` using vLLM-Omni:
- `text_to_image.py`: command-line script for single image generation.
- `text_to_image.py`: command-line script for single image generation with advanced options.
- `web_demo.py`: lightweight Gradio UI for interactive prompt/seed/CFG exploration.
## Basic Usage
```python
from vllm_omni.entrypoints.omni import Omni
if __name__ == "__main__":
omni = Omni(model="Qwen/Qwen-Image")
prompt = "a cup of coffee on the table"
images = omni.generate(prompt)
images[0].save("coffee.png")
```
## Local CLI Usage
docs/user_guide/examples/offline_inference/text_to_video.md
+2 -2
View File
@@ -1,6 +1,6 @@
# Text-To-Video
Source <https://github.com/vllm-project/vllm-omni/tree/main/examples/offline_inference/text_to_video>. Added in PR [#202](https://github.com/vllm-project/vllm-omni/pull/202).
Source <https://github.com/vllm-project/vllm-omni/tree/main/examples/offline_inference/text_to_video>.
The `Wan-AI/Wan2.2-T2V-A14B-Diffusers` pipeline generates short videos from text prompts.
@@ -36,5 +36,5 @@ Key arguments:
??? abstract "text_to_video.py"
``````py
--8<-- "examples/offline_inference/wan22/text_to_video.py"
--8<-- "examples/offline_inference/text_to_video/text_to_video.py"
``````
examples/offline_inference/image_to_image/image_edit.py
+51
View File
@@ -18,6 +18,7 @@ For more options, run:
import argparse
import os
import time
from pathlib import Path
import torch
@@ -82,6 +83,17 @@ def parse_args() -> argparse.Namespace:
default=50,
help="Number of denoising steps for the diffusion sampler.",
)
parser.add_argument(
"--cache_backend",
type=str,
default=None,
choices=["cache_dit", "tea_cache"],
help=(
"Cache backend to use for acceleration. "
"Options: 'cache_dit' (DBCache + SCM + TaylorSeer), 'tea_cache' (Timestep Embedding Aware Cache). "
"Default: None (no cache acceleration)."
),
)
return parser.parse_args()
@@ -103,15 +115,49 @@ def main():
vae_use_slicing = is_npu()
vae_use_tiling = is_npu()
# Configure cache based on backend type
cache_config = None
if args.cache_backend == "cache_dit":
# cache-dit configuration: Hybrid DBCache + SCM + TaylorSeer
# All parameters marked with [cache-dit only] in DiffusionCacheConfig
cache_config = {
# DBCache parameters [cache-dit only]
"Fn_compute_blocks": 1, # Optimized for single-transformer models
"Bn_compute_blocks": 0, # Number of backward compute blocks
"max_warmup_steps": 4, # Maximum warmup steps (works for few-step models)
"residual_diff_threshold": 0.24, # Higher threshold for more aggressive caching
"max_continuous_cached_steps": 3, # Limit to prevent precision degradation
# TaylorSeer parameters [cache-dit only]
"enable_taylorseer": False, # Disabled by default (not suitable for few-step models)
"taylorseer_order": 1, # TaylorSeer polynomial order
# SCM (Step Computation Masking) parameters [cache-dit only]
"scm_steps_mask_policy": None, # SCM mask policy: None (disabled), "slow", "medium", "fast", "ultra"
"scm_steps_policy": "dynamic", # SCM steps policy: "dynamic" or "static"
}
elif args.cache_backend == "tea_cache":
raise ValueError("TeaCache is not supported for image-to-image generation.")
# Initialize Omni with QwenImageEditPipeline
omni = Omni(
model=args.model,
model_class_name="QwenImageEditPipeline",
vae_use_slicing=vae_use_slicing,
vae_use_tiling=vae_use_tiling,
cache_backend=args.cache_backend,
cache_config=cache_config,
)
print("Pipeline loaded")
# Time profiling for generation
print(f"\n{'=' * 60}")
print("Generation Configuration:")
print(f" Model: {args.model}")
print(f" Inference steps: {args.num_inference_steps}")
print(f" Cache backend: {args.cache_backend if args.cache_backend else 'None (no acceleration)'}")
print(f" Input image size: {input_image.size}")
print(f"{'=' * 60}\n")
generation_start = time.perf_counter()
# Generate edited image
images = omni.generate(
prompt=args.prompt,
@@ -122,6 +168,11 @@ def main():
num_inference_steps=args.num_inference_steps,
num_outputs_per_prompt=args.num_outputs_per_prompt,
)
generation_end = time.perf_counter()
generation_time = generation_end - generation_start
# Print profiling results
print(f"Total generation time: {generation_time:.4f} seconds ({generation_time * 1000:.2f} ms)")
# Save output image(s)
output_path = Path(args.output)
examples/offline_inference/text_to_image/text_to_image.py
+1 -1
View File
@@ -85,7 +85,7 @@ def main():
"enable_taylorseer": False, # Disabled by default (not suitable for few-step models)
"taylorseer_order": 1, # TaylorSeer polynomial order
# SCM (Step Computation Masking) parameters [cache-dit only]
"scm_steps_mask_policy": "fast", # SCM mask policy: "slow", "medium", "fast", "ultra"
"scm_steps_mask_policy": None, # SCM mask policy: None (disabled), "slow", "medium", "fast", "ultra"
"scm_steps_policy": "dynamic", # SCM steps policy: "dynamic" or "static"
}
elif args.cache_backend == "tea_cache":
vllm_omni/diffusion/data.py
+2 -2
View File
@@ -101,8 +101,8 @@ class DiffusionCacheConfig:
enable_taylorseer: bool = False
# Default: 1st order TaylorSeer polynomial
taylorseer_order: int = 1
# Default: "fast" SCM mask policy for good speed/quality balance
scm_steps_mask_policy: str = "fast"
# Default: None SCM mask policy (disabled by default)
scm_steps_mask_policy: str | None = None
# Default: "dynamic" steps policy for adaptive caching
scm_steps_policy: str = "dynamic"
# Used by cache-dit for scm mask generation. If this value changes during inference,