1.2 KiB
1.2 KiB
Configuration Options
This section lists the most common options for running vLLM-Omni.
For options within a vLLM Engine. Please refer to vLLM Configuration
Currently, the main options are maintained by stage configs for each model.
For specific example, please refer to Qwen2.5-omni stage config
For introduction, please check Introduction for stage config
Memory Configuration
- GPU Memory Calculation and Configuration - Guide on how to calculate memory requirements and set up
gpu_memory_utilizationfor optimal performance
Optimization Features
- TeaCache Configuration - Enable TeaCache adaptive caching for DiT models to achieve 1.5x-2.0x speedup with minimal quality loss
- Cache-DiT Configuration - Enable Cache-DiT as cache acceleration backends for DiT models
- Parallelism Configuration - Enable parallelism (e.g., sequence parallelism) for for DiT models