[Feature] teacache integration (#179)
Signed-off-by: Prajwal A <prajwalanagani@gmail.com> Signed-off-by: zjy0516 <riverclouds.zhu@qq.com> Signed-off-by: Jiangyun Zhu <riverclouds.zhu@qq.com> Signed-off-by: Samit <285365963@qq.com> Co-authored-by: zjy0516 <riverclouds.zhu@qq.com> Co-authored-by: Samit <285365963@qq.com>
This commit is contained in:
21 files changed
+1401
-30
No files matched your search
@@ -0,0 +1,99 @@
|
||||
# TeaCache Configuration Guide
|
||||
|
||||
TeaCache speeds up diffusion model inference by caching transformer computations when consecutive timesteps are similar. This typically provides **1.5x-2.0x speedup** with minimal quality loss.
|
||||
|
||||
## Quick Start
|
||||
|
||||
Enable TeaCache by setting `cache_adapter` to `"tea_cache"`:
|
||||
|
||||
```python
|
||||
from vllm_omni import Omni
|
||||
|
||||
# Simple configuration - model_class_name and model_type are auto-detected
|
||||
omni = Omni(
|
||||
model="Qwen/Qwen-Image",
|
||||
cache_adapter="tea_cache",
|
||||
cache_config={
|
||||
"rel_l1_thresh": 0.2 # Optional, defaults to 0.2
|
||||
}
|
||||
)
|
||||
outputs = omni.generate(prompt="A cat sitting on a windowsill", num_inference_steps=50)
|
||||
```
|
||||
|
||||
### Using Environment Variable
|
||||
|
||||
You can also enable TeaCache via environment variable:
|
||||
|
||||
```bash
|
||||
export DIFFUSION_CACHE_ADAPTER=tea_cache
|
||||
```
|
||||
|
||||
Then initialize without explicitly setting `cache_adapter`:
|
||||
|
||||
```python
|
||||
from vllm_omni import Omni
|
||||
|
||||
omni = Omni(
|
||||
model="Qwen/Qwen-Image",
|
||||
cache_config={"rel_l1_thresh": 0.2} # Optional
|
||||
)
|
||||
```
|
||||
|
||||
## Configuration Parameters
|
||||
|
||||
### `rel_l1_thresh` (float, default: `0.2`)
|
||||
|
||||
Controls the balance between speed and quality. Lower values prioritize quality, higher values prioritize speed.
|
||||
|
||||
**Recommended values:**
|
||||
|
||||
- `0.2` - **~1.5x speedup** with minimal quality loss (recommended)
|
||||
- `0.4` - **~1.8x speedup** with slight quality loss
|
||||
- `0.6` - **~2.0x speedup** with noticeable quality loss
|
||||
- `0.8` - **~2.25x speedup** with significant quality loss
|
||||
|
||||
## Examples
|
||||
|
||||
### Python API
|
||||
|
||||
```python
|
||||
from vllm_omni import Omni
|
||||
|
||||
omni = Omni(
|
||||
model="Qwen/Qwen-Image",
|
||||
cache_adapter="tea_cache",
|
||||
cache_config={"rel_l1_thresh": 0.2}
|
||||
)
|
||||
outputs = omni.generate(prompt="A cat sitting on a windowsill", num_inference_steps=50)
|
||||
```
|
||||
|
||||
## Performance Tuning
|
||||
|
||||
Start with the default `rel_l1_thresh=0.2` and adjust based on your needs:
|
||||
|
||||
- **Maximum quality**: Use `0.1-0.2`
|
||||
- **Balanced**: Use `0.2-0.4` (recommended)
|
||||
- **Maximum speed**: Use `0.6-0.8` (may reduce quality)
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Quality Degradation
|
||||
|
||||
If you notice quality issues, lower the threshold:
|
||||
|
||||
```python
|
||||
cache_config={"rel_l1_thresh": 0.1} # More conservative caching
|
||||
```
|
||||
|
||||
## Supported Models
|
||||
|
||||
Currently supported models:
|
||||
|
||||
- **QwenImagePipeline**
|
||||
- Model identifier: `"Qwen/Qwen-Image"`
|
||||
- Example: `model="Qwen/Qwen-Image"`
|
||||
|
||||
Future support (extractors in development):
|
||||
|
||||
- **FluxPipeline** - Coming soon
|
||||
- **CogVideoXPipeline** - Coming soon
|
||||
Reference in new issue
Block a user