anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

[Doc] Refine the document. (#403)

Browse Source 
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
This commit is contained in:
Alicia authored and GitHub committed 2025-12-22 21:29:40 +08:00
1 parent 89e0836f78
commit f363108425
13 files changed
+59 -130
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/api/README.md
+2 -9
View File
@@ -43,9 +43,9 @@ Engine classes for offline and online inference.
- [vllm_omni.engine.PromptEmbedsPayload][]
- [vllm_omni.engine.arg_utils.AsyncOmniEngineArgs][]
- [vllm_omni.engine.arg_utils.OmniEngineArgs][]
- [vllm_omni.engine.input_processor.OmniInputProcessor][]
- [vllm_omni.engine.output_processor.MultimodalOutputProcessor][]
- [vllm_omni.engine.output_processor.OmniRequestState][]
- [vllm_omni.engine.processor.OmniProcessor][]
## Core
@@ -62,13 +62,8 @@ Model execution components.
- [vllm_omni.model_executor.models.output_templates.OmniOutput][]
- [vllm_omni.model_executor.models.qwen2_5_omni.qwen2_5_omni.Qwen2_5OmniForConditionalGeneration][]
- [vllm_omni.model_executor.models.qwen2_5_omni.qwen2_5_omni_talker.Qwen2_5OmniTalkerForConditionalGeneration][]
- [vllm_omni.model_executor.models.qwen2_5_omni.qwen2_5_omni_thinker.Qwen2_5OmniAudioFeatureInputs][]
- [vllm_omni.model_executor.models.qwen2_5_omni.qwen2_5_omni_thinker.Qwen2_5OmniConditionalGenerationMixin][]
- [vllm_omni.model_executor.models.qwen2_5_omni.qwen2_5_omni_thinker.Qwen2_5OmniThinkerDummyInputsBuilder][]
- [vllm_omni.model_executor.models.qwen2_5_omni.qwen2_5_omni_thinker.Qwen2_5OmniThinkerForConditionalGeneration][]
- [vllm_omni.model_executor.models.qwen2_5_omni.qwen2_5_omni_thinker.Qwen2_5OmniThinkerMultiModalDataParser][]
- [vllm_omni.model_executor.models.qwen2_5_omni.qwen2_5_omni_thinker.Qwen2_5OmniThinkerMultiModalProcessor][]
- [vllm_omni.model_executor.models.qwen2_5_omni.qwen2_5_omni_thinker.Qwen2_5OmniThinkerProcessingInfo][]
- [vllm_omni.model_executor.models.qwen2_5_omni.qwen2_5_omni_token2wav.Qwen2_5OmniToken2WavBigVGANModel][]
- [vllm_omni.model_executor.models.qwen2_5_omni.qwen2_5_omni_token2wav.Qwen2_5OmniToken2WavDiTModel][]
- [vllm_omni.model_executor.models.qwen2_5_omni.qwen2_5_omni_token2wav.Qwen2_5OmniToken2WavForConditionalGenerationVLLM][]
@@ -89,9 +84,6 @@ Model execution components.
- [vllm_omni.model_executor.models.qwen3_omni.qwen3_omni_moe_thinker.Qwen3OmniMoeThinkerForConditionalGeneration][]
- [vllm_omni.model_executor.models.qwen3_omni.qwen3_omni_moe_thinker.Qwen3OmniMoeThinkerMultiModalProcessor][]
- [vllm_omni.model_executor.models.qwen3_omni.qwen3_omni_moe_thinker.Qwen3OmniMoeThinkerProcessingInfo][]
- [vllm_omni.model_executor.models.qwen3_omni.qwen3_omni_moe_thinker.Qwen3Omni_VisionTransformer][]
- [vllm_omni.model_executor.models.qwen3_omni.qwen3_omni_moe_thinker.Qwen3_VisionPatchEmbed][]
- [vllm_omni.model_executor.models.qwen3_omni.qwen3_omni_moe_thinker.Qwen3_VisionPatchMerger][]
## Configuration
@@ -110,6 +102,7 @@ Worker classes and model runners for distributed inference.
- [vllm_omni.diffusion.worker.gpu_worker.WorkerProc][]
- [vllm_omni.diffusion.worker.npu.npu_worker.NPUWorker][]
- [vllm_omni.diffusion.worker.npu.npu_worker.NPUWorkerProc][]
- [vllm_omni.worker.gpu_ar_model_runner.ExecuteModelState][]
- [vllm_omni.worker.gpu_ar_model_runner.GPUARModelRunner][]
- [vllm_omni.worker.gpu_ar_worker.GPUARWorker][]
- [vllm_omni.worker.gpu_generation_model_runner.GPUGenerationModelRunner][]
docs/getting_started/installation/gpu.md
-6
View File
@@ -56,12 +56,6 @@ vLLM-Omni is a Python library that supports the following GPU variants. The libr
--8<-- "docs/getting_started/installation/gpu/rocm.inc.md:build-docker"
### Build wheel from source
=== "AMD ROCm"
--8<-- "docs/getting_started/installation/gpu/rocm.inc.md:build-wheel-from-source-in-docker"
### Pre-built images
=== "NVIDIA CUDA"
docs/getting_started/installation/gpu/cuda.inc.md
+3 -3
View File
@@ -18,9 +18,9 @@ Therefore, it is recommended to install vLLM and vLLM-Omni with a **fresh new**
#### Installation of vLLM
vLLM-Omni is built based on vLLM v0.12.0. Please install it with command below.
vLLM-Omni is built based on vLLM. Please install it with command below.
```bash
uv pip install vllm==0.12.0 --torch-backend=auto
uv pip install vllm==0.11.0 --torch-backend=auto
```
#### Installation of vLLM-Omni
@@ -89,7 +89,7 @@ docker run --runtime nvidia --gpus 2 \
--env "HF_TOKEN=$HF_TOKEN" \
-p 8091:8091 \
--ipc=host \
vllm/vllm-omni:v0.12.0rc1 \
vllm/vllm-omni:v0.11.0rc1 \
--model Qwen/Qwen3-Omni-30B-A3B-Instruct --port 8091
```
docs/getting_started/installation/gpu/rocm.inc.md
-92
View File
@@ -15,98 +15,6 @@ vLLM-Omni current recommends the steps in under setup through Docker Images.
# --8<-- [end:build-wheel-from-source]
# --8<-- [start:build-wheel-from-source-in-docker]
#### Get the correct docker environment
The v0.12.0 releases is on Dec 3, 2025, so we will pull a AMD nightly docker image close to the release period. `rocm/vllm-dev:nightly_main_20251205` is picked as it has a correct `transformers` version that supports qwen3_omni.
`docker pull rocm/vllm-dev:nightly_main_20251005`
#### Launch the docker environment
In this docker environment, we have all the dependencies for vLLM installed, like `flash-attn` and `aiter`
```bash
#!/bin/bash
docker run -it \
--privileged \
--network=host \
--group-add=video \
--ipc=host \
--cap-add=SYS_PTRACE \
--security-opt seccomp=unconfined \
--device /dev/kfd \
--device /dev/dri \
--name vllmomni \
rocm/vllm-dev:nightly_main_20251005 \
bash
```
#### Update docker dependencies
```bash
sudo apt update
sudo apt install ffmpeg -y
```
#### Installation of vLLM
Install the v0.12.0 release of vLLM as vLLM-Omni is currently depend on v0.12.0.
```bash
# uninstall existing vllm which is based on the main branch, not on the releases branch
python3 -m pip uninstall -y vllm
git clone https://github.com/vllm-project/vllm.git vllm0120
cd vllm0120/
# checkout the version tag v0.12.0
git checkout -b v0.12.0 v0.12.0
# you should see 0.12.0
python -c "import setuptools_scm; print(setuptools_scm.get_version())"
# PYTORCH_ROCM_ARCH=<your-gpu-arch> python3 setup.py develop
PYTORCH_ROCM_ARCH=gfx942 python3 setup.py develop
```
#### Installation of vLLM-Omni
There are two approaches to installing `vllm-omni`
1. Install from PyPI
```bash
pip install vllm-omni
# you might want to also download the source code of the same version to run the examples
```
2. Install from source
```bash
# clone and checkout to the release version.
git clone https://github.com/vllm-project/vllm-omni.git
cd vllm-omni
git checkout v0.12.0rc1
pip install -e .
```
#### Setup Environment Variables
After installing `vllm` and `vllm-omni` please setup the environment variable as well before running any of the examples and command.
```bash
export MIOPEN_FIND_MODE=FAST
export VLLM_ROCM_USE_AITER=1
export VLLM_ROCM_USE_AITER_MHA=1
export VLLM_ROCM_USE_AITER_LINEAR=0
export VLLM_ROCM_USE_AITER_RMSNORM=0
```
# --8<-- [end:build-wheel-from-source-in-docker]
# --8<-- [start:build-docker]
#### Build docker image
docs/getting_started/quickstart.md
+9 -1
View File
@@ -12,7 +12,15 @@ This guide will help you quickly get started with vLLM-Omni to perform:
## Installation
Please refer to [installation](installation/README.md)
For installation on GPU using pre-built-wheel:
```bash
uv venv --python 3.12 --seed
source .venv/bin/activate
uv pip install vllm==0.11.0 --torch-backend=auto
uv pip install vllm-omni
```
For additional details—including alternative installation methods, installation on NPU and other platforms — please see the installation guide in [installation](installation/README.md)
## Offline Inference
docs/user_guide/examples/offline_inference/image_to_image.md
+1 -2
View File
@@ -16,7 +16,6 @@ python image_edit.py \
--output output_image_edit.png \
--num_inference_steps 50 \
--cfg_scale 4.0 \
--guidance_scale 1.0
```
### Multiple Image Editing (Qwen-Image-Edit-2509)
@@ -26,7 +25,7 @@ For multiple image inputs, use `Qwen/Qwen-Image-Edit-2509` or later version:
```bash
python image_edit.py \
--model Qwen/Qwen-Image-Edit-2509 \
--image img1.png img2.png img3.png \
--image img1.png img2.png \
--prompt "Combine these images into a single scene" \
--output output_image_edit.png \
--num_inference_steps 50 \
docs/user_guide/examples/offline_inference/qwen2_5_omni.md
-4
View File
@@ -3,10 +3,6 @@
Source <https://github.com/vllm-project/vllm-omni/tree/main/examples/offline_inference/qwen2_5_omni>.
## 🛠️ Installation
Please refer to [README.md](https://github.com/vllm-project/vllm-omni/tree/main/README.md)
## Run examples (Qwen2.5-Omni)
### Multiple Prompts
docs/user_guide/examples/offline_inference/qwen3_omni.md
-4
View File
@@ -3,10 +3,6 @@
Source <https://github.com/vllm-project/vllm-omni/tree/main/examples/offline_inference/qwen3_omni>.
## 🛠️ Installation
Please refer to [README.md](https://github.com/vllm-project/vllm-omni/tree/main/README.md)
## Run examples (Qwen3-Omni)
### Multiple Prompts
docs/user_guide/examples/online_serving/image_to_image.md
+36
View File
@@ -5,6 +5,8 @@ Source <https://github.com/vllm-project/vllm-omni/tree/main/examples/online_serv
This example demonstrates how to deploy Qwen-Image-Edit model for online image editing service using vLLM-Omni.
For **multi-image** input editing, use **Qwen-Image-Edit-2509** (QwenImageEditPlusPipeline) and send multiple images in the user message content.
## Start Server
### Basic Start
@@ -13,6 +15,12 @@ This example demonstrates how to deploy Qwen-Image-Edit model for online image e
vllm serve Qwen/Qwen-Image-Edit --omni --port 8092
```
### Multi-Image Edit (Qwen-Image-Edit-2509)
```bash
vllm serve Qwen/Qwen-Image-Edit-2509 --omni --port 8092
```
### Start with Parameters
@@ -22,6 +30,12 @@ Or use the startup script:
bash run_server.sh
```
To serve Qwen-Image-Edit-2509 with the script:
```bash
MODEL=Qwen/Qwen-Image-Edit-2509 bash run_server.sh
```
## API Calls
### Method 1: Using curl (Image Editing)
@@ -56,6 +70,9 @@ curl -s http://localhost:8092/v1/chat/completions \
```bash
python openai_chat_client.py --input input.png --prompt "Convert to oil painting style" --output output.png
# Multi-image editing (Qwen-Image-Edit-2509 server required)
python openai_chat_client.py --input input1.png input2.png --prompt "Combine these images into a single scene" --output output.png
```
### Method 3: Using Gradio Demo
@@ -124,6 +141,25 @@ Use `extra_body` to pass generation parameters:
}
```
### Multi-Image Editing (Qwen-Image-Edit-2509)
Provide multiple images in `content` (order matters):
```json
{
"messages": [
{
"role": "user",
"content": [
{"type": "text", "text": "Combine these images into a single scene"},
{"type": "image_url", "image_url": {"url": "data:image/png;base64,..."} },
{"type": "image_url", "image_url": {"url": "data:image/png;base64,..."} }
]
}
]
}
```
## Generation Parameters (extra_body)
| Parameter | Type | Default | Description |
docs/user_guide/examples/online_serving/qwen2_5_omni.md
+4
View File
@@ -165,3 +165,7 @@ sudo apt install ffmpeg
``````sh
--8<-- "examples/online_serving/qwen2_5_omni/run_curl_multimodal_generation.sh"
``````
??? abstract "run_gradio_demo.sh"
``````sh
--8<-- "examples/online_serving/qwen2_5_omni/run_gradio_demo.sh"
``````
docs/user_guide/examples/online_serving/qwen3_omni.md
+4 -1
View File
@@ -150,7 +150,6 @@ The gradio script supports the following arguments:
- `--ip`: Host/IP for Gradio server (default: 127.0.0.1)
- `--port`: Port for Gradio server (default: 7861)
- `--share`: Share the Gradio demo publicly (creates a public link)
- `--share`: Share the Gradio demo publicly (creates a public link)
## Example materials
@@ -170,3 +169,7 @@ The gradio script supports the following arguments:
``````sh
--8<-- "examples/online_serving/qwen3_omni/run_curl_multimodal_generation.sh"
``````
??? abstract "run_gradio_demo.sh"
``````sh
--8<-- "examples/online_serving/qwen3_omni/run_gradio_demo.sh"
``````
examples/offline_inference/qwen2_5_omni/README.md
-4
View File
@@ -1,9 +1,5 @@
# Qwen2.5-Omni
## 🛠️ Installation
Please refer to [README.md](../../../README.md)
## Run examples (Qwen2.5-Omni)
### Multiple Prompts
examples/offline_inference/qwen3_omni/README.md
-4
View File
@@ -1,9 +1,5 @@
# Qwen3-Omni
## 🛠️ Installation
Please refer to [README.md](../../../README.md)
## Run examples (Qwen3-Omni)
### Multiple Prompts