tzhouam
80a0d8d20a
Drop the old-base wheel reinstall and dependency repair block. Update source installation and configuration references to vLLM 0.29 while retaining explicitly matched 0.28 prebuilt Omni examples until its next release. Validation: official image and CUDA/ROCm release artifacts checked; 6 documentation tests passed; diff whitespace checks passed. Markdown lint reports existing snippet/anchor issues. Container build not run: no builder available. Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
24 lines
1.2 KiB
Markdown
24 lines
1.2 KiB
Markdown
# Configuration Options
|
|
|
|
This section lists the most common options for running vLLM-Omni.
|
|
|
|
For options within a vLLM Engine, please refer to the [vLLM 0.29 configuration guide](https://docs.vllm.ai/en/v0.29.0/configuration/index.html).
|
|
|
|
Each model defines fixed topology in a registered `PipelineConfig` and runtime
|
|
overrides in a deploy YAML.
|
|
|
|
For process-level settings shared by the CLI, deploy YAML, workers, and model
|
|
integrations, see [Environment Variables](./environment_variables.md).
|
|
|
|
For a specific example, see the [Qwen2.5-Omni deploy config](gh-file:vllm_omni/deploy/qwen2_5_omni.yaml). The matching frozen pipeline topology lives at [vllm_omni/model_executor/models/qwen2_5_omni/pipeline.py](gh-file:vllm_omni/model_executor/models/qwen2_5_omni/pipeline.py).
|
|
|
|
For an introduction, see [Pipeline and deploy configurations](./stage_configs.md).
|
|
|
|
## Memory Configuration
|
|
|
|
- **[GPU Memory Calculation and Configuration](./gpu_memory_utilization.md)** - Guide on how to calculate memory requirements and set up `gpu_memory_utilization` for optimal performance
|
|
|
|
## Optimization Features
|
|
|
|
- **[Diffusion Features Overview](../user_guide/diffusion_features.md)** - Complete overview of all diffusion model features and supported models
|