anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

Merge branch 'main' into dev/vllm-align

Browse Source
This commit is contained in:
Gao Han authored and GitHub committed 2026-09-10 11:23:08 +08:00
2 parents 80a0d8d20a 4b8dcac681
commit 0ae9bdd8c2
54 files changed
+1659 -500
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/contributing/ci/ci_settings.md
+7 -2
View File
@@ -129,7 +129,7 @@ Canonical layout (prefer these paths for new changes):
=== "NPU"
**Bootstrap:** [`npu/pipeline-npu.yml`](https://github.com/vllm-project/vllm-omni/blob/main/.buildkite/npu/pipeline-npu.yml) + [`npu/bootstrap-upload-steps.yml`](https://github.com/vllm-project/vllm-omni/blob/main/.buildkite/npu/bootstrap-upload-steps.yml)—same split as CUDA; builds A2/B3 and A3 CI images, then uploads child test pipelines.
**Bootstrap:** [`npu/pipeline-npu.yml`](https://github.com/vllm-project/vllm-omni/blob/main/.buildkite/npu/pipeline-npu.yml) + [`npu/bootstrap-upload-steps.yml`](https://github.com/vllm-project/vllm-omni/blob/main/.buildkite/npu/bootstrap-upload-steps.yml)—same split as CUDA; builds A2/B3, A3, A5, and 310P CI images, then uploads child test pipelines.
**Test YAML:** `npu/test-npu-ready.yml` (L2), `test-npu-nightly.yml` (L4).
@@ -137,7 +137,12 @@ Canonical layout (prefer these paths for new changes):
**Upload:** same as CUDA—`upload_pipeline.py --upload`.
**Hardware in YAML:** `mirror_hardwares` preset (string), expanded to `agents`, top-level `image`, and `plugins`. Presets: `a2b3_npu_1`, `a2b3_npu_4`, `a3_npu_2` in `common/ci_mirror_hardwares.yml`.
**Hardware in YAML:** `mirror_hardwares` preset (string), expanded to `agents`, top-level `image`, and `plugins`. NPU presets in `common/ci_mirror_hardwares.yml` are:
- A2: `a2b3_npu_1`, `a2b3_npu_4`, `a2b3_npu_8`
- A3: `a3_npu_2`, `a3_npu_4`, `a3_npu_8`, `a3_npu_16`
- A5: `a5_npu_2`, `a5_npu_4`, `a5_npu_8`, `a5_npu_16`
- 310P: `310p_npu_1`, `310p_npu_2`, `310p_npu_4`
**Conventions**
docs/contributing/ci/test_writing_guide.md
+3 -1
View File
@@ -48,6 +48,8 @@ Defined in `pyproject.toml`:
| `MI325` | Tests that require MI325 GPU (AMD/ROCm) * |
| `A2` | Tests that require A2 NPU * |
| `A3` | Tests that require A3 NPU * |
| `A5` | Tests that require A5 NPU * |
| `310P` | Tests that require 310P NPU * |
| `cards_{n}` | Tests that require *n* accelerator cards (`cards_1` … `cards_8`); auto-added from `num_cards` * |
| `slow` | Slow tests (may skip in quick CI) |
| `benchmark` | Benchmark tests (decorator on runner test functions; perf JSON uses `full_model` + type marker instead) |
@@ -76,7 +78,7 @@ def test_video_to_audio()
This decorator is intended to make hardware-aware, cross-platform test authoring easier and more robust for CI/CD environments. The `hardware_test` decorator in `vllm-omni/tests/helpers/mark.py` performs the following actions:
1. **Applies platform and resource markers**
Adds the appropriate pytest markers for each specified hardware platform (e.g., `cuda`, `rocm`, `xpu`, `npu`) and resource type (e.g., `L4`, `H100`, `H200`, `B200`, `MI325`, `B60`, `A2`, `A3`).
Adds the appropriate pytest markers for each specified hardware platform (e.g., `cuda`, `rocm`, `xpu`, `npu`) and resource type (e.g., `L4`, `H100`, `H200`, `B200`, `MI325`, `B60`, `A2`, `A3`, `A5`, `310P`).
```python
@pytest.mark.cuda
@pytest.mark.L4
docs/getting_started/installation/npu/npu.inc.md
+156 -42
View File
@@ -7,13 +7,34 @@ For detailed hardware and software requirements, please refer to the [vLLM-Ascen
The recommended way to use vLLM-Omni on NPU is through the vLLM-Ascend pre-built Docker images:
=== "A2"
```bash
export IMAGE=quay.io/atlas-ci/vllm-ascend:v0.28.0
```
=== "A3"
```bash
export IMAGE=quay.io/atlas-ci/vllm-ascend:v0.28.0-a3
```
=== "A5"
```bash
export IMAGE=quay.io/atlas-ci/vllm-ascend:v0.28.0-a5
```
=== "310P (Experimental)"
```bash
export IMAGE=quay.io/atlas-ci/vllm-ascend:v0.28.0-310p
```
```bash
# vLLM-Ascend image
# Atlas A2: quay.io/atlas-ci/vllm-ascend:v0.28.0
# Atlas A3: quay.io/atlas-ci/vllm-ascend:v0.28.0-a3
docker run --rm \
--name vllm-omni-npu \
--shm-size=1g \
--shm-size=64g \
--device /dev/davinci0 \
--device /dev/davinci1 \
--device /dev/davinci2 \
@@ -27,8 +48,8 @@ docker run --rm \
-v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
-v /etc/ascend_install.info:/etc/ascend_install.info \
-v /root/.cache:/root/.cache \
-p 8000:8000 \
-it quay.io/atlas-ci/vllm-ascend:v0.28.0-a3 bash
-p 8091:8091 \
-it "$IMAGE" bash
# Inside the container, install vLLM-Omni from source
cd /vllm-workspace
@@ -72,53 +93,146 @@ Supported images as following.
| image name | Hardware | OS |
|-|-|-|
| image-tag | Atlas A2 | Ubuntu |
| image-tag-a3 | Atlas A3 | Ubuntu |
| image-tag | A2 | Ubuntu |
| image-tag-a3 | A3 | Ubuntu |
| image-tag-a5 | A5 | Ubuntu |
| image-tag-310p | 310P (experimental) | Ubuntu |
Here's an example deployment command that has been verified on 4 x NPUs:
```bash
# Atlas A2: quay.io/ascend/vllm-omni:v0.28.0
# Atlas A3: quay.io/ascend/vllm-omni:v0.28.0-a3
docker run --rm \
--name vllm-omni-a3 \
--shm-size=4g \
--device /dev/davinci0 \
--device /dev/davinci1 \
--device /dev/davinci2 \
--device /dev/davinci3 \
--device /dev/davinci_manager \
--device /dev/devmm_svm \
--device /dev/hisi_hdc \
-v /usr/local/dcmi:/usr/local/dcmi \
-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
-v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
-v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
--device /dev/davinci_manager \
--device /dev/devmm_svm \
--device /dev/hisi_hdc \
-v /usr/local/dcmi:/usr/local/dcmi \
-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
-v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
-v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
-v /etc/ascend_install.info:/etc/ascend_install.info \
-v ~/.cache:/root/.cache \
-p 8091:8091 \
-it quay.io/ascend/vllm-omni:v0.28.0-a3 bash
```
=== "A2"
```bash
export IMAGE=quay.io/ascend/vllm-omni:v0.28.0
docker run --rm \
--name vllm-omni-a2 \
--shm-size=64g \
--device /dev/davinci0 \
--device /dev/davinci1 \
--device /dev/davinci2 \
--device /dev/davinci3 \
--device /dev/davinci_manager \
--device /dev/devmm_svm \
--device /dev/hisi_hdc \
-v /usr/local/dcmi:/usr/local/dcmi \
-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
-v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
-v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
-v /etc/ascend_install.info:/etc/ascend_install.info \
-v ~/.cache:/root/.cache \
-p 8091:8091 \
-it "$IMAGE" bash
```
=== "A3"
```bash
export IMAGE=quay.io/ascend/vllm-omni:v0.28.0-a3
docker run --rm \
--name vllm-omni-a3 \
--shm-size=64g \
--device /dev/davinci0 \
--device /dev/davinci1 \
--device /dev/davinci2 \
--device /dev/davinci3 \
--device /dev/davinci_manager \
--device /dev/devmm_svm \
--device /dev/hisi_hdc \
-v /usr/local/dcmi:/usr/local/dcmi \
-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
-v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
-v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
-v /etc/ascend_install.info:/etc/ascend_install.info \
-v ~/.cache:/root/.cache \
-p 8091:8091 \
-it "$IMAGE" bash
```
=== "A5"
```bash
export IMAGE=quay.io/ascend/vllm-omni:v0.28.0-a5
docker run --rm \
--name vllm-omni-a5 \
--shm-size=64g \
--device /dev/davinci0 \
--device /dev/davinci1 \
--device /dev/davinci2 \
--device /dev/davinci3 \
--device /dev/davinci_manager \
--device /dev/devmm_svm \
--device /dev/hisi_hdc \
-v /usr/local/dcmi:/usr/local/dcmi \
-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
-v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
-v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
-v /etc/ascend_install.info:/etc/ascend_install.info \
-v ~/.cache:/root/.cache \
-p 8091:8091 \
-it "$IMAGE" bash
```
=== "310P (Experimental)"
```bash
export IMAGE=quay.io/ascend/vllm-omni:v0.28.0-310p
docker run --rm \
--name vllm-omni-310p \
--shm-size=64g \
--device /dev/davinci0 \
--device /dev/davinci1 \
--device /dev/davinci2 \
--device /dev/davinci3 \
--device /dev/davinci_manager \
--device /dev/devmm_svm \
--device /dev/hisi_hdc \
-v /usr/local/dcmi:/usr/local/dcmi \
-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
-v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
-v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
-v /etc/ascend_install.info:/etc/ascend_install.info \
-v ~/.cache:/root/.cache \
-p 8091:8091 \
-it "$IMAGE" bash
```
!!! tip
You can use this docker image to serve models the same way you would with in vLLM! To do so, make sure you overwrite the default entrypoint (`vllm serve --omni`) which works only for models supported in the vLLM-Omni project.
Or build IMAGE from **source code**:
Or build the image from **source code**:
```bash
git clone https://github.com/vllm-project/vllm-omni.git
cd vllm-omni
# A2
# docker build -t vllm-omni-dev-image:latest -f ./docker/Dockerfile.npu .
# A3
docker build -t vllm-omni-dev-image:latest -f ./docker/Dockerfile.npu.a3 .
```
All hardware variants use `docker/Dockerfile.npu`. Choose the device suffix for your hardware:
=== "A2"
```bash
docker build -t vllm-omni-dev-image:latest -f docker/Dockerfile.npu .
```
=== "A3"
```bash
docker build -t vllm-omni-dev-image:latest -f docker/Dockerfile.npu \
--build-arg VLLM_ASCEND_DEVICE_SUFFIX=-a3 .
```
=== "A5"
```bash
docker build -t vllm-omni-dev-image:latest -f docker/Dockerfile.npu \
--build-arg VLLM_ASCEND_DEVICE_SUFFIX=-a5 .
```
=== "310P (Experimental)"
```bash
docker build -t vllm-omni-dev-image:latest -f docker/Dockerfile.npu \
--build-arg VLLM_ASCEND_DEVICE_SUFFIX=-310p .
```
# --8<-- [end:pre-built-images]
docs/user_guide/examples/online_serving/text_to_video.md
+9
View File
@@ -199,12 +199,21 @@ curl -X POST http://localhost:8091/v1/videos \
| `boundary_ratio` | float | None | Boundary split ratio for low/high DiT (Wan2.2) |
| `flow_shift` | float | None | Scheduler flow shift (Wan2.2) |
| `seed` | int | None | Random seed (reproducible) |
| `num_outputs_per_prompt` | int | 1 | Number of videos to generate (1-10, MiniMax H3); only the first is returned, see below |
| `lora` | object | None | LoRA configuration |
| `enable_frame_interpolation` | bool | false | Enable RIFE frame interpolation before MP4 encoding |
| `frame_interpolation_exp` | int | 1 | Interpolation exponent; 1=2x temporal resolution, 2=4x |
| `frame_interpolation_scale` | float | 1.0 | RIFE inference scale; use 0.5 for high-resolution inputs |
| `frame_interpolation_model_path` | str | None | Local directory or Hugging Face repo ID with `flownet.pkl`; defaults to `elfgum/RIFE-4.22.lite` |
!!! note "Only the first output is returned"
`num_outputs_per_prompt` is forwarded to the pipeline, so the model does
generate that many videos and you pay the generation cost for all of them.
`/v1/videos` and `/v1/videos/sync` then return only the **first** one,
because an async video job stores a single file per video id. The server
logs a warning when it discards the extra outputs. Keep this at `1`
unless you have a specific reason to generate videos you will not receive.
## Frame Interpolation
Frame interpolation is an optional post-processing step for `/v1/videos` and