1 parent
1ae0c64fe0
commit
51c0303945
11 files changed
+25
-74
No files matched your search
@@ -13,25 +13,18 @@ vLLM-Omni uses `uv` as the environment manager, to create and manage Python envi
|
||||
uv venv --python 3.12 --seed
|
||||
source .venv/bin/activate
|
||||
```
|
||||
### Development Environment for vLLM
|
||||
|
||||
vLLM-Omni is built on top of the stable release 0.12.0 of vLLM. You may install `vllm==0.12.0` directly or install the library from source if you need to check, modify or debug with source code of vLLM.
|
||||
### Development Environment for vLLM and vLLM-Omni
|
||||
|
||||
The release 0.12.0 of vLLM is based on PyTorch 2.9.0 which requires CUDA 12.9 environment.
|
||||
|
||||
### Development Environment for vLLM-Omni
|
||||
|
||||
Install vLLM-Omni from source with development dependencies:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/vllm-project/vllm-omni.git
|
||||
cd vllm-omni
|
||||
uv pip install -e ".[dev]"
|
||||
```
|
||||
vLLM-Omni is quickly evolving, please see the [installation guide](../getting_started/installation/README.md) for details. It's recommended to build from source to provide the latest development environment.
|
||||
|
||||
!!! tip
|
||||
vLLM-Omni is compatible with Python versions 3.10 to 3.12. However, we recommend developing with Python 3.12 to minimize the chance of your local environment clashing with our CI environment.
|
||||
|
||||
### Adding a new model to vLLM-Omni
|
||||
|
||||
Please check [model implementation](model/README.md) for how to add diffusion and omni-modality models to vLLM-Omni.
|
||||
|
||||
### Linting
|
||||
|
||||
vLLM-Omni uses `pre-commit` to lint and format the codebase. See [pre-commit documentation](https://pre-commit.com/#usage) if `pre-commit` is new to you. Setting up `pre-commit` is as easy as:
|
||||
|
||||
@@ -26,4 +26,4 @@ The guide covers:
|
||||
|
||||
## Quick Start
|
||||
|
||||
For a quick reference, see the [Adding a New Omni Model Guide](adding_multi_stage_model.md) and [Adding a New Diffusion Model Guide](adding_diffusion_model.md).
|
||||
For a quick reference, see the [Adding a New Multi-Stage Model Guide](adding_multi_stage_model.md) and [Adding a New Diffusion Model Guide](adding_diffusion_model.md).
|
||||
@@ -1,4 +1,4 @@
|
||||
# Adding a New Diffusion Model to vLLM-Omni
|
||||
# Adding a Diffusion Model
|
||||
This guide walks through the process of adding a new Diffusion model to vLLM-Omni, using Qwen/Qwen-Image-Edit as a comprehensive example.
|
||||
|
||||
# Table of Contents
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Adding a New Multi-Stage Model to vLLM-Omni
|
||||
# Adding a Multi-Stage Model
|
||||
|
||||
This guide walks through the process of adding a new model to vLLM-Omni, using **Qwen3-Omni** as a comprehensive example. Qwen3-Omni is a multi-stage omni-modality model that demonstrates the full capabilities of vLLM-Omni's architecture.
|
||||
This guide walks through the process of adding a new multi-stage model to vLLM-Omni, using **Qwen3-Omni** as a comprehensive example. Qwen3-Omni is a multi-stage omni-modality model that demonstrates the full capabilities of vLLM-Omni's architecture.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
# Profiling vLLM-Omni (update soon)
|
||||
# Profiling vLLM-Omni
|
||||
|
||||
Profiling is only intended for vLLM-Omni developers and maintainers to understand the proportion of time spent in different parts of the codebase. **vLLM-Omni end-users should never turn on profiling** as it will significantly slow down the inference.
|
||||
Reference in new issue
Block a user