tzhouam
80a0d8d20a
Drop the old-base wheel reinstall and dependency repair block. Update source installation and configuration references to vLLM 0.29 while retaining explicitly matched 0.28 prebuilt Omni examples until its next release. Validation: official image and CUDA/ROCm release artifacts checked; 6 documentation tests passed; diff whitespace checks passed. Markdown lint reports existing snippet/anchor issues. Container build not run: no builder available. Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
1.2 KiB
1.2 KiB
Configuration Options
This section lists the most common options for running vLLM-Omni.
For options within a vLLM Engine, please refer to the vLLM 0.29 configuration guide.
Each model defines fixed topology in a registered PipelineConfig and runtime
overrides in a deploy YAML.
For process-level settings shared by the CLI, deploy YAML, workers, and model integrations, see Environment Variables.
For a specific example, see the Qwen2.5-Omni deploy config. The matching frozen pipeline topology lives at vllm_omni/model_executor/models/qwen2_5_omni/pipeline.py.
For an introduction, see Pipeline and deploy configurations.
Memory Configuration
- GPU Memory Calculation and Configuration - Guide on how to calculate memory requirements and set up
gpu_memory_utilizationfor optimal performance
Optimization Features
- Diffusion Features Overview - Complete overview of all diffusion model features and supported models