a3903810fe
Signed-off-by: lishunyang <lishunyang12@163.com> Signed-off-by: reidliu41 <reid201711@gmail.com> Signed-off-by: Alex Brooks <albrooks@redhat.com> Co-authored-by: reidliu41 <reid201711@gmail.com> Co-authored-by: xiaohajiayou <75477391+xiaohajiayou@users.noreply.github.com> Co-authored-by: Alex Brooks <albrooks@redhat.com> Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
1.2 KiB
1.2 KiB
Configuration Options
This section lists the most common options for running vLLM-Omni.
For options within a vLLM Engine. Please refer to vLLM Configuration
Currently, the main options are maintained by stage configs for each model.
For a specific example, see the Qwen2.5-Omni deploy config. The matching frozen pipeline topology lives at vllm_omni/model_executor/models/qwen2_5_omni/pipeline.py.
For introduction, please check Introduction for stage config
Memory Configuration
- GPU Memory Calculation and Configuration - Guide on how to calculate memory requirements and set up
gpu_memory_utilizationfor optimal performance
Multi-Stage Recipes
- Prefill-Decode Disaggregation - How to derive a PD-aware Qwen3-Omni stage config from the default config without introducing another bundled YAML
Optimization Features
- Diffusion Features Overview - Complete overview of all diffusion model features and supported models