1.5 KiB
1.5 KiB
Configuration Options
This section lists the most common options for running vLLM-Omni.
For options within a vLLM Engine. Please refer to vLLM Configuration
Currently, the main options are maintained by stage configs for each model.
For specific example, please refer to Qwen2.5-omni stage config
For introduction, please check Introduction for stage config
Memory Configuration
- GPU Memory Calculation and Configuration - Guide on how to calculate memory requirements and set up
gpu_memory_utilizationfor optimal performance
Multi-Stage Recipes
- Prefill-Decode Disaggregation - How to derive a PD-aware Qwen3-Omni stage config from the default config without introducing another bundled YAML
Optimization Features
- TeaCache Configuration - Enable TeaCache adaptive caching for DiT models to achieve 1.5x-2.0x speedup with minimal quality loss
- Cache-DiT Configuration - Enable Cache-DiT as cache acceleration backends for DiT models
- Parallelism Configuration - Enable parallelism (e.g., sequence parallelism) for for DiT models
- CPU Offloading - Enable CPU offloading (model-level and layerwise) for for DiT models