diff --git a/.claude/skills/add-diffusion-model/SKILL.md b/.claude/skills/add-diffusion-model/SKILL.md index bae8cd1b5..111f8e9bf 100644 --- a/.claude/skills/add-diffusion-model/SKILL.md +++ b/.claude/skills/add-diffusion-model/SKILL.md @@ -610,6 +610,24 @@ See the [Profiling Single-Stage Diffusion](../../../docs/contributing/profiling. --- +## Pre-commit conventions + +New library files must pass the local gates in +[docs/contributing/README.md](../../../docs/contributing/README.md#linting). +That page is the full hook list (SPDX, forbidden imports including Hugging Face +Hub / Triton / pickle, `torch.cuda`, mypy, test marks, markdownlint, Buildkite, +shellcheck). In particular: + +- SPDX copyright is `vLLM-Omni project` (stale `vLLM project` is rewritten). +- Use `import regex as re` and `pybase64` in `vllm_omni/`; do not import stdlib + `re` or `base64`. Hugging Face Hub downloads go through + `vllm.transformers_utils.repo_utils`. +- Do not add `torch.cuda.*` call sites; use `current_omni_platform`. +- New `tests/**/test_*.py` files need a CI level mark and a hardware mark. +- Do not expand `CHECK_IMPORTS[*].allowed_files` or `ALLOWED_FILES` without review. +- GitHub Actions skips SPDX/shellcheck/mypy-3.10/test-marks/markdownlint; run + `pre-commit` locally. + ## Iterative Development Tips 1. **Start minimal**: Basic generation first, no parallelism/caching diff --git a/.claude/skills/add-tts-model/SKILL.md b/.claude/skills/add-tts-model/SKILL.md index 494d6a52b..034ac7cd4 100644 --- a/.claude/skills/add-tts-model/SKILL.md +++ b/.claude/skills/add-tts-model/SKILL.md @@ -563,7 +563,7 @@ Use this checklist when integrating a new TTS model: - [ ] `pre-commit run --files ` passes before every push - [ ] Every commit has `Signed-off-by` matching the author email (`git commit -s`) - [ ] `git config user.email` matches the email registered on your GitHub account -- [ ] Details and failure-recovery commands: [references/precommit-dco.md](references/precommit-dco.md) +- [ ] Hook list, failure table, and DCO recovery: [references/precommit-dco.md](references/precommit-dco.md) ## References diff --git a/.claude/skills/add-tts-model/references/precommit-dco.md b/.claude/skills/add-tts-model/references/precommit-dco.md index 86a1f42ce..160ce71b0 100644 --- a/.claude/skills/add-tts-model/references/precommit-dco.md +++ b/.claude/skills/add-tts-model/references/precommit-dco.md @@ -30,6 +30,18 @@ but the changes are correct — stage the modified files and re-commit. | `ruff F841` | Variable extracted but never forwarded to model call | Remove the extraction or wire it through | | `ruff E402` | Import added below function definitions | Move to top-level import block | | `ruff format` | Line length, spacing, quote style | Accept auto-fix, stage, re-commit | +| `check-spdx-header` | Missing header, or copyright still says `vLLM project` | Two-line Omni header (`vLLM-Omni project`); restage the rewrite | +| `check-forbidden-imports` | Stdlib `re`/`base64`, pickle, Hugging Face Hub API, or direct Triton/TileLang | `import regex as re` and `pybase64`; use `vllm.transformers_utils.repo_utils`. Do **not** add the file to `allowed_files` without review | +| `check-torch-cuda-call` | New `torch.cuda.*` call site | Use `current_omni_platform` / `OmniPlatform`; do not grow `ALLOWED_FILES` without review | +| `check-tts-adapter-migration` | New `self._tts_model_type` branch in `serving_speech.py` | Put per-model logic in `tts_adapters/`. Lower `MAX_MODEL_TYPE_BRANCHES` when removing branches; do not raise it without review | +| `check-test-ci-coverage` | New `tests/**/test_*.py` missing level or hardware mark | Add `core_model`/`advanced_model`/… plus `cpu`/`cuda`/`hardware_test(` | +| `mypy-3.10` | Type error on changed `vllm_omni/` files | Fix the types; model trees are excluded. Extra versions: `pre-commit run --hook-stage manual mypy-3.12` | +| `markdownlint-cli2` | Docs/README markdown lint | Accept auto-fix under `docs/` / `recipes/` / root README | +| `check-buildkite` | Invalid `.buildkite/*.yml` | Fix the YAML; do not grow `SKIP_FILES` without review | +| `shellcheck` | No `shellcheck` on PATH, or script warning | `apt-get`/`dnf`/`brew install shellcheck`, or `shellcheck.exe` on Windows PATH | + +Canonical policy, CI `SKIP` list, and allowlist locations: +[docs/contributing/README.md](../../../docs/contributing/README.md#linting). ## DCO sign-off diff --git a/.claude/skills/precheck-pr/SKILL.md b/.claude/skills/precheck-pr/SKILL.md index 4c8086029..ea3a4ed34 100644 --- a/.claude/skills/precheck-pr/SKILL.md +++ b/.claude/skills/precheck-pr/SKILL.md @@ -67,6 +67,8 @@ Ask: "Quick mode or full mode?" Then walk the checklist for the detected PR type **Also run the Examples-Policy check on every PR** using [references/examples-policy.md](references/examples-policy.md). Inspect only Python paths introduced by the diff. Treat a new model-, checkpoint-, vendor-, or family-specific Python example as blocking; do not report pre-existing example debt. +**Also confirm local pre-commit gates** from [docs/contributing/README.md](../../../docs/contributing/README.md#linting). GitHub Actions `SKIP` does not prove they passed. Full new-hook list: Omni SPDX (`vLLM-Omni project`); forbidden imports (stdlib `re`/`base64`, pickle, Hugging Face Hub API, Triton/TileLang); no new `torch.cuda` call sites; shellcheck (macOS/Windows must install the binary); mypy-3.10; test files have CI level + hardware marks; TTS `_tts_model_type` branches in `serving_speech.py` must not increase; markdownlint on `docs/`; Buildkite YAML schema. ✗ for growing `allowed_files` / `ALLOWED_FILES` or raising `MAX_MODEL_TYPE_BRANCHES` without review. + ### Step 5: Print Report ``` diff --git a/.claude/skills/precheck-pr/references/checklists.md b/.claude/skills/precheck-pr/references/checklists.md index b65a66713..dc7f85ea9 100644 --- a/.claude/skills/precheck-pr/references/checklists.md +++ b/.claude/skills/precheck-pr/references/checklists.md @@ -164,6 +164,7 @@ Diffusion models live under `vllm_omni/diffusion/`. In addition to the New Model - [ ] **PR body matches diff:** description doesn't claim changes not in the diff - [ ] **Branch is rebased:** `git merge-base HEAD origin/main` is recent, no merge conflicts - [ ] **CI gates passing:** DCO, pre-commit, build — check `gh pr view --json statusCheckRollup` if a PR already exists +- [ ] **Local pre-commit gates:** full list in [Linting](../../../../docs/contributing/README.md#linting). Blocking when the diff introduces: stale `vLLM project` SPDX; stdlib `re`/`base64` or pickle in library code; new `torch.cuda` call sites; `tests/**/test_*.py` without level+hardware marks; new `_tts_model_type` branches in `serving_speech.py`; invalid `.buildkite` YAML. macOS/Windows need a native `shellcheck`. ✗ for growing `CHECK_IMPORTS[*].allowed_files`, `ALLOWED_FILES`, or `MAX_MODEL_TYPE_BRANCHES` without review. ### Full (adds) diff --git a/.claude/skills/review-pr/SKILL.md b/.claude/skills/review-pr/SKILL.md index 165fa1d5e..63046190d 100644 --- a/.claude/skills/review-pr/SKILL.md +++ b/.claude/skills/review-pr/SKILL.md @@ -21,7 +21,10 @@ Make every finding: - **Calibrated:** match severity to user and maintainer impact. Do not report unrelated backlog, style already enforced by pre-commit, or a -missing test that would not protect changed behavior. +missing test that would not protect changed behavior. Do report new allowlist +or budget entries in `check_forbidden_imports.py` / `check_torch_cuda.py` / +`check_tts_adapter.py` / `check_buildkite.py` unless the PR justifies them: +those are policy changes, not lint noise. ## Select the input and depth diff --git a/.claude/skills/review-pr/references/process/review-execution.md b/.claude/skills/review-pr/references/process/review-execution.md index f06f27933..7c0305254 100644 --- a/.claude/skills/review-pr/references/process/review-execution.md +++ b/.claude/skills/review-pr/references/process/review-execution.md @@ -111,6 +111,17 @@ Mark early findings as preliminary and continue. This is not a GitHub comment. but do not publish review events without separate authorization. - Record DCO, pre-commit, required CI, and mergeability. Pending or unknown gates do not block source review. +- GitHub Actions `SKIP` omits SPDX, shellcheck, markdownlint, mypy-3.10, and + test-mark coverage. A green GHA pre-commit job does not prove those local + gates passed. New files still need the full Linting list: Omni SPDX + (`vLLM-Omni project`), no stdlib `re`/`base64` in `vllm_omni/`, no new + pickle / Hugging Face Hub API / `torch.cuda` call sites, test marks, TTS + adapter ratchet, Buildkite schema, and native shellcheck on macOS/Windows. + See [Linting](https://vllm-omni.readthedocs.io/en/latest/contributing/#linting). +- Expanding `CHECK_IMPORTS[*].allowed_files`, `ALLOWED_FILES`, + `MAX_MODEL_TYPE_BRANCHES`, or Buildkite `SKIP_FILES` is a policy change. + Do not rubber-stamp allowlist/budget growth; require justification and + prefer fixing the call site. - Treat a failed gate as its own evidence. Do not restate its formatting/lint output as a new code-review finding. - Open CI logs only when the first failing step overlaps the frozen diff or diff --git a/.claude/skills/vllm-omni-test/SKILL.md b/.claude/skills/vllm-omni-test/SKILL.md index 3e4d0e489..1ccdb3324 100644 --- a/.claude/skills/vllm-omni-test/SKILL.md +++ b/.claude/skills/vllm-omni-test/SKILL.md @@ -274,6 +274,7 @@ def test_text_to_video_001(omni_server, openai_client) -> None: **3. Cross-cutting rules** +- New `.py` files need the Omni SPDX header (`Copyright contributors to the vLLM-Omni project`, not `vLLM project`). `tests/` may keep stdlib `re`/`base64`; pickle is still banned unless the file is already on the pickle allowlist. Every collected `tests/**/test_*.py` needs a CI **level** mark and a **hardware** mark/helper (`check-test-ci-coverage` runs locally; GHA skips it). Do not grow `allowed_files` to land a test. Policy: [docs/contributing/README.md](../../../docs/contributing/README.md#linting). - Reuse existing fixtures for the chosen scenario; do not mix “online client” assumptions into offline `OmniRunner` tests without a clear reason. - Avoid external network dependency in assertions unless the scenario is explicitly “online serving” or doc examples that require a model hub (then align with CI secrets/cache). - Keep **one test function = one intent** (one modality combo, one endpoint contract, or one acceleration combo). diff --git a/.github/mergify.yml b/.github/mergify.yml index 4e4c68425..fdba32b17 100644 --- a/.github/mergify.yml +++ b/.github/mergify.yml @@ -45,7 +45,7 @@ pull_request_rules: > # For mypy (substitute "3.10" with the failing version if needed) > pre-commit run --hook-stage manual mypy-3.10 > # For markdownlint - > pre-commit run --hook-stage manual markdownlint + > pre-commit run markdownlint-cli2 > ``` > diff --git a/.github/workflows/pre-commit.yml b/.github/workflows/pre-commit.yml index 2bd0cc133..a08b385f3 100644 --- a/.github/workflows/pre-commit.yml +++ b/.github/workflows/pre-commit.yml @@ -24,9 +24,7 @@ jobs: python-version: "3.12" cache: 'pip' - uses: pre-commit/action@v3.0.1 - # Don't run the pytest mark check against everything in the CI - # for now since a lot of files are still missing them. This check - # is used to help contributors remember to add the marks when they - # commit. + # Don't run historical-debt hooks against the whole tree in CI yet. + # They still run locally on changed files. env: - SKIP: check-test-ci-coverage + SKIP: check-test-ci-coverage,markdownlint-cli2,shellcheck,check-spdx-header,mypy-3.10 diff --git a/.gitignore b/.gitignore index 7502df8a0..9155a85cc 100644 --- a/.gitignore +++ b/.gitignore @@ -278,3 +278,6 @@ tests/.ci_generated/ # Local Lance precision-matching dump scripts (not part of the repo) lance_compare/ + +# Linting binaries downloaded by local pre-commit hooks +shellcheck*/ diff --git a/.markdownlint.yaml b/.markdownlint.yaml new file mode 100644 index 000000000..329597772 --- /dev/null +++ b/.markdownlint.yaml @@ -0,0 +1,17 @@ +# Keep these rules aligned with vLLM so MkDocs Material syntax is tolerated. +MD007: + indent: 4 +MD013: false +# GitHub READMEs start with a centered logo, not an H1. +MD041: false +MD024: + siblings_only: true +MD025: + # Allow front matter title to be different from the first heading in the document. + front_matter_title: "" +MD031: + list_items: false +MD033: false +MD046: false +MD052: false +MD059: false diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 535bc1452..65b4fd27b 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -24,7 +24,7 @@ repos: - id: ruff-format - repo: https://github.com/crate-ci/typos - rev: typos-dict-v0.13.13 + rev: v1.49.0 hooks: - id: typos # only for staged files @@ -38,7 +38,19 @@ repos: - id: actionlint files: ^\.github/workflows/.*\.ya?ml$ + - repo: https://github.com/DavidAnson/markdownlint-cli2 + rev: v0.23.2 + hooks: + - id: markdownlint-cli2 + language_version: lts + args: [--fix] + files: (^docs/|^recipes/|(^|/)README\.md$|^CONTRIBUTING\.md$) + # Agent skill docs and CLAUDE.md are not user-facing docs/READMEs. + exclude: (^|/)CLAUDE\.md$|^\.claude/|^\.cursor/ + # Keep `local` last so `suggestion` stays the final hook. Autofix repos + # (ruff, typos, markdownlint, trailing-whitespace, etc.) must run first; + # otherwise their "files were modified" output displaces the SKIP tip. - repo: local hooks: - id: signoff-commit @@ -54,6 +66,45 @@ repos: verbose: true stages: [commit-msg] + # TODO: Use https://github.com/pre-commit/mirrors-mypy when mypy setup + # is less awkward. Local wrapper groups files so follow_imports=skip on + # tests does not leak into library code. + # mypy-3.10 runs on git commit (changed files). CI SKIPs it because + # --all-files is not clean yet. Extra versions are manual-stage only + # (not in default `pre-commit run --all-files`): + # pre-commit run --hook-stage manual mypy-3.12 + - id: mypy-3.10 + name: Run mypy for Python 3.10 + entry: python tools/pre_commit/mypy.py "3.10" + <<: &mypy_common + language: python + types_or: [python, pyi] + require_serial: true + additional_dependencies: + - mypy==1.11.1 + - regex + - types-cachetools + - types-setuptools + - types-PyYAML + - types-requests + - types-torch + - pydantic + - id: mypy-3.11 + name: Run mypy for Python 3.11 + entry: python tools/pre_commit/mypy.py "3.11" + <<: *mypy_common + stages: [manual] + - id: mypy-3.12 + name: Run mypy for Python 3.12 + entry: python tools/pre_commit/mypy.py "3.12" + <<: *mypy_common + stages: [manual] + - id: mypy-3.13 + name: Run mypy for Python 3.13 + entry: python tools/pre_commit/mypy.py "3.13" + <<: *mypy_common + stages: [manual] + - id: check-test-ci-coverage name: Ensure test files have pytestmarks entry: python tools/pre_commit/check_test_marks.py @@ -71,7 +122,45 @@ repos: # pre-commit hides output from passing hooks unless the hook is verbose. verbose: true - # Keep `suggestion` last + - id: shellcheck + name: Lint shell scripts + # language: script breaks on Windows (shebang path loses slashes). + # Invoke the vLLM-style wrapper with bash so Linux CI and Win/Git Bash work. + entry: bash + args: [tools/pre_commit/shellcheck.sh] + language: system + types: [shell] + + - id: check-spdx-header + name: Check SPDX headers + entry: python tools/pre_commit/check_spdx_header.py + language: python + types_or: [python, pyi, rust, proto, shell] + files: \.(py|pyi|rs|proto|sh)$ + + - id: check-forbidden-imports + name: Check forbidden imports + entry: python tools/pre_commit/check_forbidden_imports.py + language: python + types: [python] + additional_dependencies: [regex] + + - id: check-torch-cuda-call + name: Prevent new torch.cuda API calls + entry: python tools/pre_commit/check_torch_cuda.py + language: python + types: [python] + additional_dependencies: [regex] + + - id: check-buildkite + name: Validate Buildkite Pipelines + entry: python tools/pre_commit/check_buildkite.py + language: python + files: ^\.buildkite/.*\.ya?ml$ + additional_dependencies: [pyyaml, check-jsonschema==0.38.0] + + # Keep `suggestion` as the final hook. Run after autofix hooks so the + # bypass/SKIP tip is not displaced. - id: suggestion name: Suggestion entry: bash -c 'echo "To bypass all the pre-commit hooks, add --no-verify to git commit. To skip a specific hook, prefix the commit command with SKIP=."' @@ -79,10 +168,3 @@ repos: verbose: true pass_filenames: false # Insert new entries above the `suggestion` entry - - - id: check-pickle-imports - name: Prevent new pickle/cloudpickle imports - entry: python tools/pre_commit/check_pickle_imports.py - language: python - types: [python] - additional_dependencies: [regex] diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 4ace89fd6..36cefcf0d 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,5 +1,11 @@ # Contributing to vLLM-Omni -You may find information about contributing to vLLM-Omni on [Contributing](https://vllm-omni.readthedocs.io/en/latest/contributing/) +You may find information about contributing to vLLM-Omni on [Contributing](https://vllm-omni.readthedocs.io/en/latest/contributing/). + +Local `pre-commit` gates (SPDX, forbidden imports, `torch.cuda`, shellcheck, +mypy, test marks, TTS adapter ratchet, markdownlint, Buildkite schema, and +how to extend allowlists) are documented in the +[Linting](https://vllm-omni.readthedocs.io/en/latest/contributing/#linting) +section of that guide. Before submitting a PR, run the [precheck-pr skill](https://github.com/vllm-project/vllm-omni/blob/main/.claude/skills/precheck-pr/SKILL.md) with the code agent for a self-check against project conventions. diff --git a/docs/contributing/README.md b/docs/contributing/README.md index 2224c667e..43afd3e5c 100644 --- a/docs/contributing/README.md +++ b/docs/contributing/README.md @@ -44,6 +44,149 @@ vLLM-Omni's `pre-commit` hooks will now run automatically every time you commit. pre-commit run --show-diff-on-failure --color=always --all-files # runs on all files (short for --all-files) ``` +!!! warning + GitHub Actions `pre-commit` **skips** several local gates so a whole-tree + `--all-files` run stays green while historical debt is cleaned up. The + current `SKIP` list in + [`.github/workflows/pre-commit.yml`](https://github.com/vllm-project/vllm-omni/blob/main/.github/workflows/pre-commit.yml) + is `check-test-ci-coverage`, `markdownlint-cli2`, `shellcheck`, + `check-spdx-header`, and `mypy-3.10`. Those hooks still run on **your + commit** for changed files. A passing GitHub pre-commit check does not mean + they passed locally. Hooks **not** on that list (forbidden imports, + `torch.cuda`, TTS adapter ratchet, Buildkite schema, Ruff, typos, …) run in + both places. + +The hooks below are new relative to the previous Omni config (Ruff, typos, +actionlint, YAML/whitespace, DCO sign-off, and the pickle-only checker were +already there). `check-pickle-imports` is gone: pickle is now one rule inside +`check-forbidden-imports`. + +| Hook id | Enforces | GitHub Actions | +| ------- | -------- | -------------- | +| `markdownlint-cli2` | Markdown in `docs/`, `recipes/`, `README.md`, `CONTRIBUTING.md` (not `.claude/` / `.cursor/` / `CLAUDE.md`) | skipped | +| `mypy-3.10` | Type-check changed `vllm_omni/` files (model trees excluded). `tests/` uses `--follow-imports skip` | skipped | +| `mypy-3.11` / `3.12` / `3.13` | Same checker, extra Python versions | not installed; `pre-commit run --hook-stage manual mypy-3.12` | +| `check-test-ci-coverage` | Every `tests/**/test_*.py` has a CI level mark and a hardware mark/helper | skipped | +| `check-tts-adapter-migration` | `self._tts_model_type` branches in `serving_speech.py` must not increase | runs | +| `shellcheck` | `*.sh` quoting / undefined vars | skipped | +| `check-spdx-header` | Omni SPDX header on `.py` / `.pyi` / `.sh` / `.rs` / `.proto` | skipped | +| `check-forbidden-imports` | pickle, stdlib `re`/`base64`, Triton, TileLang, Hugging Face Hub download/API | runs | +| `check-torch-cuda-call` | Direct `torch.cuda.*` helpers outside platform adapters | runs | +| `check-buildkite` | `.buildkite/*.yml` against the Buildkite schema | runs | + +Related config (not a new hook id): Ruff `TID251` also bans `librosa` and several +`torch.cuda.*` names; typos is pinned and ignores git-describe hashes (`+g` + +hex) and NumPy `writeable`. Keep the `suggestion` hook last in +`.pre-commit-config.yaml` so autofix output does not bury the SKIP tip. + +#### SPDX headers + +Source files must start with: + +```text +# SPDX-License-Identifier: Apache-2.0 +# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project +``` + +Rust and proto files use `//` instead of `#`. Empty files are exempt. A stale +`Copyright contributors to the vLLM project` line is **rewritten in place** to +the Omni copyright; the hook then exits non-zero so you restage the rewrite. + +#### Forbidden imports + +Under `vllm_omni/`, do not use: + +- stdlib `re` → `import regex as re` +- stdlib `base64` → `import pybase64` (or `import pybase64 as base64`) +- `pickle` / `cloudpickle` (also banned in most tests) +- `from huggingface_hub import snapshot_download` / `hf_hub_download` / `HfApi` + / similar → `vllm.transformers_utils.repo_utils` +- direct `triton` / `tilelang` → `vllm.triton_utils` / `vllm.tilelang_utils` + +`examples/`, `tests/`, `benchmarks/`, `apps/`, `docs/`, `tools/`, `scripts/`, +`.buildkite/`, and `.github/` may keep stdlib `re` and `base64` +(`_NON_LIBRARY_DIRS`). Pickle is stricter: tests are not exempt unless the +file is on the pickle allowlist. + +#### `torch.cuda` call sites + +Prefer `current_omni_platform` / `OmniPlatform`. Platform adapters under +`vllm_omni/platforms/` are allowed; other files must not add new +`torch.cuda.*` helpers (`device_count`, `synchronize`, `empty_cache`, +`device()`, `manual_seed`, and similar). `import torch` is fine. + +#### shellcheck + +The hook does **not** download a binary. Install `shellcheck` with a signed +package manager, then re-run: + +- Debian/Ubuntu/WSL: `sudo apt-get install shellcheck` +- Fedora: `sudo dnf install ShellCheck` +- macOS: `brew install shellcheck` +- Git Bash (MINGW): `scoop install shellcheck` so `shellcheck.exe` is on PATH. + WSL is Linux: use `apt-get`, not a Windows `.exe`. + +If no native binary is found, the wrapper prints those commands (and +[the upstream install page](https://github.com/koalaman/shellcheck?tab=readme-ov-file#installing)) +and exits 1. AMD and other vendor scripts are **not** excluded. + +#### mypy + +`mypy-3.10` runs on commit for changed files. `[tool.mypy]` uses Python 3.12, +`ignore_missing_imports = true`, and `follow_imports = "silent"`. Wrapper +`tools/pre_commit/mypy.py` skips `vllm_omni/model_executor/models/` and +`vllm_omni/diffusion/models/`, and type-checks `tests/` with +`--follow-imports skip`. Extra versions: + +```bash +pre-commit run --hook-stage manual mypy-3.12 +``` + +#### Test CI marks + +`check-test-ci-coverage` requires each collected test module under `tests/` to +have at least one CI **level** mark (`core_model`, `advanced_model`, +`full_model`, `local_model`, or `slow`) and a **hardware** mark (`cpu`, `cuda`, +`H100`, …) or helper (`hardware_test(` / `hardware_marks(`). GitHub Actions +skips this hook; local commit does not. See the +[test writing guide](./ci/test_writing_guide.md). + +#### TTS adapter ratchet + +`check-tts-adapter-migration` counts `self._tts_model_type` comparisons in +`vllm_omni/entrypoints/openai/serving_speech.py`. The count must stay at or +below `MAX_MODEL_TYPE_BRANCHES` in +[`tools/pre_commit/check_tts_adapter.py`](https://github.com/vllm-project/vllm-omni/blob/main/tools/pre_commit/check_tts_adapter.py). +New per-model behavior belongs in +`vllm_omni/entrypoints/openai/tts_adapters/`. Removing branches **must** lower +the budget in the same change; raising it is a reviewable policy edit. + +#### Markdownlint and Buildkite + +`markdownlint-cli2` auto-fixes `docs/`, `recipes/`, and the root README / +CONTRIBUTING (rules in `.markdownlint.yaml`). Skill files under `.claude/` and +`.cursor/` are excluded. + +`check-buildkite` validates `.buildkite/*.yml` against the official schema +after the same expansion `upload_pipeline.py` uses. Omni-only keys such as +`mirror_hardwares` are stripped or rendered first. A few files are skipped in +`SKIP_FILES` inside `tools/pre_commit/check_buildkite.py`. + +#### Extending allowlists and budgets + +Prefer fixing the call site. Growing an allowlist or raising a budget is a +policy change and needs review — do not edit these just to make the hook pass. + +| Gate | File | What to edit | +| ---- | ---- | ------------ | +| pickle, `re`, `base64`, Hugging Face Hub, Triton/TileLang | [`tools/pre_commit/check_forbidden_imports.py`](https://github.com/vllm-project/vllm-omni/blob/main/tools/pre_commit/check_forbidden_imports.py) | `CHECK_IMPORTS[].allowed_files`. Use `allowed_dirs` only when a whole tree should stay exempt. | +| `torch.cuda` | [`tools/pre_commit/check_torch_cuda.py`](https://github.com/vllm-project/vllm-omni/blob/main/tools/pre_commit/check_torch_cuda.py) | `ALLOWED_FILES`, or `ALLOWED_PREFIXES` for a platform adapter tree. | +| TTS `_tts_model_type` branches | [`tools/pre_commit/check_tts_adapter.py`](https://github.com/vllm-project/vllm-omni/blob/main/tools/pre_commit/check_tts_adapter.py) | `MAX_MODEL_TYPE_BRANCHES` (down only, unless the PR justifies a raise). | +| Buildkite skip list | [`tools/pre_commit/check_buildkite.py`](https://github.com/vllm-project/vllm-omni/blob/main/tools/pre_commit/check_buildkite.py) | `SKIP_FILES` for pipelines that cannot be expanded. | + +To skip one hook for a single commit (discouraged): `SKIP= git commit`. +To bypass every hook: `git commit --no-verify` (also discouraged). + ### Documentation MkDocs is a fast, simple and downright gorgeous static site generator that's geared towards building project documentation. Documentation source files are written in Markdown, and configured with a single YAML configuration file, `mkdocs.yml`. @@ -162,12 +305,14 @@ Only specific types of PRs will be reviewed. The PR title is prefixed appropriat Before submitting a PR, run the [precheck-pr skill](https://github.com/vllm-project/vllm-omni/blob/main/.claude/skills/precheck-pr/SKILL.md) with the code agent for a self-review against project conventions: The skill offers two modes: + - **Quick (~3 min):** catches showstoppers — PR title format, missing benchmark claims, rebase status - **Full (~10 min):** thorough maintainer-grade review — dead code scan, copy-paste detection, import hygiene The precheck covers five PR types: Bug Fix, Performance, New Model, Diffusion Model, and General. Each type has a tailored checklist that validates evidence quality (repro steps, A/B benchmarks, registry entries, etc.). See the [precheck-pr skill](https://github.com/vllm-project/vllm-omni/blob/main/.claude/skills/precheck-pr/SKILL.md) for the full checklist. ### Local Test + Please run the L1 and L2 test cases locally first and attach the results before contacting us to add the "ready" label. Please refer to the [test instructions](./ci/test_execution_guide.md) for running the test cases. ### Automatic skip-ci (docs and pytest skip marks) diff --git a/docs/contributing/ci/failures.md b/docs/contributing/ci/failures.md index 9743097cf..e4b846cca 100644 --- a/docs/contributing/ci/failures.md +++ b/docs/contributing/ci/failures.md @@ -6,7 +6,7 @@ When you open a PR against vLLM-Omni, several CI checks run automatically: | Check | Platform | What it does | | ----- | -------- | ------------ | -| **pre-commit** | GitHub Actions | Runs linting (Ruff), formatting, spell-checking (typos), and YAML validation. | +| **pre-commit** | GitHub Actions | Runs Ruff, formatting, typos, YAML validation, forbidden imports, `torch.cuda`, TTS adapter ratchet, and Buildkite schema. Local-only gates (SPDX, shellcheck, markdownlint, mypy-3.10, test marks) are in the workflow `SKIP` env — see [Linting](../README.md#linting). | | **Build Wheel** | GitHub Actions | Builds Python wheels for Python 3.11 and 3.12 on Ubuntu. Skipped for docs-only or Markdown-only changes (controlled by `paths-ignore` in the workflow). | | **DCO** | GitHub | Verifies every commit has a `Signed-off-by` line. | | **docs/readthedocs.org:vllm-omni** | Read the Docs | Builds the MkDocs documentation site. | @@ -29,6 +29,13 @@ pre-commit run --all-files Then commit the fixes and push. +A green GitHub pre-commit job can still miss SPDX, shellcheck, mypy-3.10, +markdownlint, and test-mark coverage because CI skips those IDs. If your +**local** commit hook fails, follow [Linting](../README.md#linting) for the +full hook list (forbidden imports, `torch.cuda`, TTS adapter ratchet, and +Buildkite schema still run in GHA). Do not grow allowlists or raise +`MAX_MODEL_TYPE_BRANCHES` without review. + ### DCO failures Every commit must include a `Signed-off-by` line. If you forgot, amend your commits: diff --git a/docs/contributing/ci/test_writing_guide.md b/docs/contributing/ci/test_writing_guide.md index 3b422ec9f..7912fc4a1 100644 --- a/docs/contributing/ci/test_writing_guide.md +++ b/docs/contributing/ci/test_writing_guide.md @@ -24,38 +24,39 @@ Deploy YAMLs: `vllm_omni/deploy/` via `tests.helpers.stage_config.get_deploy_con By adding markers before test functions, tests can later be executed uniformly by simply declaring the corresponding marker type. ### Current Markers + Defined in `pyproject.toml`: -| Marker | Description | -| ------------------ | --------------------------------------------------------- | -| `core_model` | L1&L2 tests (run in each PR) | -| `advanced_model` | L3 tests (run on each merge to main) | -| `full_model` | L4 tests (run nightly) | -| `diffusion` | Diffusion model tests | -| `omni` | Omni multimodal model tests | -| `tts` | TTS model tests | -| `cache` | Cache backend tests | -| `parallel` | Parallelism/distributed tests | -| `cpu` | Tests that run on CPU | -| `gpu` | Tests that run on GPU * | -| `cuda` | Tests that run on CUDA * | -| `rocm` | Tests that run on AMD/ROCm * | -| `xpu` | Tests that run on Intel XPU * | -| `npu` | Tests that run on NPU/Ascend * | -| `H100` | Tests that require H100 GPU * | -| `L4` | Tests that require L4 GPU * | -| `MI325` | Tests that require MI325 GPU (AMD/ROCm) * | -| `A2` | Tests that require A2 NPU * | -| `A3` | Tests that require A3 NPU * | -| `distributed_cuda` | Tests that require multi cards on CUDA platform * | -| `distributed_rocm` | Tests that require multi cards on ROCm platform * | -| `distributed_npu` | Tests that require multi cards on NPU platform * | -| `skipif_cuda` | Skip if the num of CUDA cards is less than the required * | -| `skipif_rocm` | Skip if the num of ROCm cards is less than the required * | -| `skipif_npu` | Skip if the num of NPU cards is less than the required * | -| `slow` | Slow tests (may skip in quick CI) | +| Marker | Description | +| ------------------ | ------------------------------------------------------------------------------------------------------- | +| `core_model` | L1&L2 tests (run in each PR) | +| `advanced_model` | L3 tests (run on each merge to main) | +| `full_model` | L4 tests (run nightly) | +| `diffusion` | Diffusion model tests | +| `omni` | Omni multimodal model tests | +| `tts` | TTS model tests | +| `cache` | Cache backend tests | +| `parallel` | Parallelism/distributed tests | +| `cpu` | Tests that run on CPU | +| `gpu` | Tests that run on GPU * | +| `cuda` | Tests that run on CUDA * | +| `rocm` | Tests that run on AMD/ROCm * | +| `xpu` | Tests that run on Intel XPU * | +| `npu` | Tests that run on NPU/Ascend * | +| `H100` | Tests that require H100 GPU * | +| `L4` | Tests that require L4 GPU * | +| `MI325` | Tests that require MI325 GPU (AMD/ROCm) * | +| `A2` | Tests that require A2 NPU * | +| `A3` | Tests that require A3 NPU * | +| `distributed_cuda` | Tests that require multi cards on CUDA platform * | +| `distributed_rocm` | Tests that require multi cards on ROCm platform * | +| `distributed_npu` | Tests that require multi cards on NPU platform * | +| `skipif_cuda` | Skip if the num of CUDA cards is less than the required * | +| `skipif_rocm` | Skip if the num of ROCm cards is less than the required * | +| `skipif_npu` | Skip if the num of NPU cards is less than the required * | +| `slow` | Slow tests (may skip in quick CI) | | `benchmark` | Benchmark tests (decorator on runner test functions; perf JSON uses `full_model` + type marker instead) | -| `local_model` | Tests requiring local / non-HF-hub model weights | +| `local_model` | Tests requiring local / non-HF-hub model weights | \* Means those markers are auto-added by `@hardware_test` (parametrization decorator) or `hardware_marks` (only returning the list of marks for flexibility). @@ -81,13 +82,13 @@ This decorator is intended to make hardware-aware, cross-platform test authoring 1. **Applies platform and resource markers** Adds the appropriate pytest markers for each specified hardware platform (e.g., `cuda`, `rocm`, `xpu`, `npu`) and resource type (e.g., `L4`, `H100`, `MI325`, `B60`, `A2`, `A3`). - ``` + ```python @pytest.mark.cuda @pytest.mark.L4 ``` 2. **Handles multi-card (distributed) scenarios** For tests requiring multiple cards, it automatically adds distributed markers such as `distributed_cuda`, `distributed_rocm`, or `distributed_npu`. - ``` + ```python @pytest.mark.distributed_cuda(num_cards=num_cards) ``` 3. **Supports flexible card requirements** @@ -97,11 +98,11 @@ This decorator is intended to make hardware-aware, cross-platform test authoring On CUDA, adds a skip marker (`skipif_cuda`) if the system does not have the required number of devices. Support for `skipif_rocm` and `skipif_npu` will be implemented later. - 5. **Works with pytest filtering** Allows tests to be filtered and selected at runtime using standard pytest marker expressions (e.g., `-m "distributed_cuda and L4"`). ##### Example usage for decorator + - Single call for multiple platforms: ```python @hardware_test( @@ -220,6 +221,7 @@ If you want to add support for a new platform (e.g., "tpu" for a new accelerator ``` **Summary**: + - Add pytest markers for your new platform/resources - Implement a marker function (`xxx_marks`) - Plug into `hardware_marks` @@ -237,47 +239,46 @@ L1 and L2 level testing form the foundation of the quality assurance system. L1 L2 level testing builds upon L1 by introducing GPU resources and verifying that the end-to-end (E2E) process of the model in basic deployment scenarios is smooth. For example, it uses dummy models to confirm that core interfaces like the inference pipeline, output format, and streaming response work properly. The common goal of these two levels is to provide developers with rapid feedback, discovering and fixing issues early in the development cycle. - - #### 1.2 Testing Content and Scope -- ***L1 (Unit & Logic Testing)***: -- - ***Scope***: Tests internal functions and methods of core components such as `entrypoints`, `models`. - - ***Focus***: Branch coverage, exception handling, algorithm logic correctness. Does not involve external dependencies or the complete service stack. - - ***Time Cost***: Execution time is controlled within ***15 minutes*** to ensure fast feedback. -- ***L2 (Basic End-to-End Testing)***: -- - ***Scope***: Covers two basic deployment scenarios: `online` (serving) and `offline` (inference). - - ***Focus***: Uses `dummy` weights (via deploy YAML patching at `core_model`) or lightweight real models to verify that the entire chain from request input to result output works normally, including output data structure, streaming (stream) support, and **cheap payload checks** at `--run-level core_model` (see below). Also includes some unit tests that require launching independent service instances. - - ***L2 response validation (`core_model`)***: Implemented in `tests/helpers/assertions.py` and invoked by `OpenAIClientHandler` based on `--run-level`. At L2 we require **request success** plus minimal output sanity—not full accuracy: - - **Speech / TTS** (`assert_audio_speech_response`): decoded audio must be present and non-empty (or exceed `min_audio_bytes` when set in `request_config`); `response_format` must match the returned content-type (e.g. `wav`, `pcm`). Whisper transcript similarity, PCM HNR, and preset-voice gender checks run only at L3+. - - **Diffusion** (`assert_diffusion_response`): at least one non-empty image, video, or audio artifact. Resolution/frame-count parity and other deep checks run only at L3+. - - **Omni multimodal** (`assert_omni_response`): L2 asserts successful completion; keyword, transcript, and cross-modal similarity checks run only at L3+. - - ***Characteristic***: Requires ***GPU*** resources to perform model computations. +- ***L1 (Unit & Logic Testing)***: + - ***Scope***: Tests internal functions and methods of core components such as `entrypoints`, `models`. + - ***Focus***: Branch coverage, exception handling, algorithm logic correctness. Does not involve external dependencies or the complete service stack. + - ***Time Cost***: Execution time is controlled within ***15 minutes*** to ensure fast feedback. +- ***L2 (Basic End-to-End Testing)***: + - ***Scope***: Covers two basic deployment scenarios: `online` (serving) and `offline` (inference). + - ***Focus***: Uses `dummy` weights (via deploy YAML patching at `core_model`) or lightweight real models to verify that the entire chain from request input to result output works normally, including output data structure, streaming (stream) support, and **cheap payload checks** at `--run-level core_model` (see below). Also includes some unit tests that require launching independent service instances. + - ***L2 response validation (`core_model`)***: Implemented in `tests/helpers/assertions.py` and invoked by `OpenAIClientHandler` based on `--run-level`. At L2 we require **request success** plus minimal output sanity—not full accuracy: + - **Speech / TTS** (`assert_audio_speech_response`): decoded audio must be present and non-empty (or exceed `min_audio_bytes` when set in `request_config`); `response_format` must match the returned content-type (e.g. `wav`, `pcm`). Whisper transcript similarity, PCM HNR, and preset-voice gender checks run only at L3+. + - **Diffusion** (`assert_diffusion_response`): at least one non-empty image, video, or audio artifact. Resolution/frame-count parity and other deep checks run only at L3+. + - **Omni multimodal** (`assert_omni_response`): L2 asserts successful completion; keyword, transcript, and cross-modal similarity checks run only at L3+. + - ***Characteristic***: Requires ***GPU*** resources to perform model computations. #### 1.3 Test Directory and Execution Files A clear directory structure is key to managing test cases efficiently. See the **Test Dir** column in [Test System Overview](./test_system_overview.md) for the canonical placement rules. -- ***L1 (component / unit)***: `tests/{component_name}/…` mirroring `vllm_omni/{component_name}/` (e.g. `tests/diffusion/`, `tests/entrypoints/`). Do **not** invent new top-level `tests/` folders unrelated to a component. -- ***L2 (basic model E2E)***: - - Online serving: `tests/e2e/online_serving/test_{model_name}.py` - - Offline inference: `tests/e2e/offline_inference/test_{model_name}.py` -- ***Feature-level integration*** (cross-cutting, not a single-model smoke): `tests/e2e/features//` (e.g. `fullduplex/`, `custom_pipeline/`, `rlhf_test/`). +- ***L1 (component / unit)***: `tests/{component_name}/…` mirroring `vllm_omni/{component_name}/` (e.g. `tests/diffusion/`, `tests/entrypoints/`). Do **not** invent new top-level `tests/` folders unrelated to a component. +- ***L2 (basic model E2E)***: + - Online serving: `tests/e2e/online_serving/test_{model_name}.py` + - Offline inference: `tests/e2e/offline_inference/test_{model_name}.py` +- ***Feature-level integration*** (cross-cutting, not a single-model smoke): `tests/e2e/features//` (e.g. `fullduplex/`, `custom_pipeline/`, `rlhf_test/`). #### 1.4 Execution Method and Example -- ***Trigger Timing***: **`PR with ready label`**. That is, when a developer adds a "ready for review" or similar label to a PR on platforms like GitHub, L1 and L2 tests are automatically triggered. -- ***Diff-aware step skipping***: On L2, **E2E Test** jobs may be omitted at pipeline upload when the PR diff does not touch their [`source_file_dependencies`](ci_settings.md#step-filtering) prefixes; non-E2E groups still always upload. See [CI Settings — Diff-aware CI](ci_settings.md#diff-aware-ci). -- ***Execution Environment***: L1 uses ***CPU*** environment; L2 requires ***GPU*** environment. -- ***Script Example***: +- ***Trigger Timing***: **`PR with ready label`**. That is, when a developer adds a "ready for review" or similar label to a PR on platforms like GitHub, L1 and L2 tests are automatically triggered. +- ***Diff-aware step skipping***: On L2, **E2E Test** jobs may be omitted at pipeline upload when the PR diff does not touch their [`source_file_dependencies`](ci_settings.md#step-filtering) prefixes; non-E2E groups still always upload. See [CI Settings — Diff-aware CI](ci_settings.md#diff-aware-ci). +- ***Execution Environment***: L1 uses ***CPU*** environment; L2 requires ***GPU*** environment. +- ***Script Example***:
L1 Test Examples Examples from `tests/model_executor/models/qwen2_5_omni/test_audio_length.py` + ```python # SPDX-License-Identifier: Apache-2.0 -# SPDX-FileCopyrightText: Copyright contributors to the vLLM project +# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project import pytest @@ -331,6 +332,7 @@ def test_cap_and_align_mel_length_no_mismatch(repeats, code_len, max_mel_frames) if max_mel_frames is not None and int(max_mel_frames) > 0 and int(max_mel_frames) >= repeats: assert target_mel_len <= int(max_mel_frames) ``` +
@@ -338,7 +340,7 @@ def test_cap_and_align_mel_length_no_mismatch(repeats, code_len, max_mel_frames) You can refer to Test Examples in the L3 section to see example test cases that incorporate both L2 and L3 testing logic.
-- - ***Run Command***: +- ***Run Command***: `pytest -s -v /tests/e2e/online_serving/test_{model_name}.py` `pytest -s -v -m 'core_model and cpu' --run-level=core_model` @@ -351,36 +353,34 @@ L3 level testing executes after code is merged into the main branch. Its core pu . It acts as the "quality gatekeeper" for the main branch, ensuring that no merge breaks the core capabilities of the model service. Testing needs to provide clear conclusions within a relatively short time (<30min), balancing test depth with feedback speed. - - #### 2.2 Testing Content and Scope -- ***Deployment Scenarios***: Covers richer `online` and `offline` deployment configurations, which may include different hardware configurations, batch sizes, concurrency levels, etc. -- ***Core Verification***: -- 1. ***Inference Functionality***: Ensures real models can perform forward computation normally and return results. - 2. ***Accuracy Compliance***: Verifies that the model's evaluation metrics (e.g., accuracy) meet the expected baseline, preventing code changes from introducing accuracy issues. - 3. ***Important Performance***: Verifies whether performance (e.g., P99 latency, throughput) in core scenarios meets preset thresholds. -- ***L3 response validation (`advanced_model`)***: At `--run-level advanced_model`, `tests/helpers/assertions.py` adds semantic checks on top of L2 payload gates (see L1 & L2 above): - - **Speech / TTS** (`assert_audio_speech_response`): Whisper transcript of returned audio vs `request_config["input"]` (cosine similarity > 0.9 when input text is set); optional `min_audio_bytes` floor; PCM harmonic-to-noise ratio when `response_format` is `pcm`; **preset voice gender** via `_assert_preset_voice_gender_from_audio` when `voice` matches a known preset in `_PRESET_VOICE_GENDER_MAP` (pitch/F0-based classifier; skipped for unknown voices or `pcm` output). - - **Omni multimodal** (`assert_omni_response`): non-empty text/audio outputs; `key_words` in transcript or text; text–audio similarity / containment; preset **`speaker`** gender check (same helper as TTS). - - **Diffusion** (`assert_*_diffusion_response`): image/video dimension and frame-count parity with request parameters where configured. - - **Accuracy suites**: pixel/video similarity and metric baselines under `/tests/e2e/accuracy/` (separate from inline helper assertions). +- ***Deployment Scenarios***: Covers richer `online` and `offline` deployment configurations, which may include different hardware configurations, batch sizes, concurrency levels, etc. +- ***Core Verification***: + 1. ***Inference Functionality***: Ensures real models can perform forward computation normally and return results. + 2. ***Accuracy Compliance***: Verifies that the model's evaluation metrics (e.g., accuracy) meet the expected baseline, preventing code changes from introducing accuracy issues. + 3. ***Important Performance***: Verifies whether performance (e.g., P99 latency, throughput) in core scenarios meets preset thresholds. +- ***L3 response validation (`advanced_model`)***: At `--run-level advanced_model`, `tests/helpers/assertions.py` adds semantic checks on top of L2 payload gates (see L1 & L2 above): + - **Speech / TTS** (`assert_audio_speech_response`): Whisper transcript of returned audio vs `request_config["input"]` (cosine similarity > 0.9 when input text is set); optional `min_audio_bytes` floor; PCM harmonic-to-noise ratio when `response_format` is `pcm`; **preset voice gender** via `_assert_preset_voice_gender_from_audio` when `voice` matches a known preset in `_PRESET_VOICE_GENDER_MAP` (pitch/F0-based classifier; skipped for unknown voices or `pcm` output). + - **Omni multimodal** (`assert_omni_response`): non-empty text/audio outputs; `key_words` in transcript or text; text–audio similarity / containment; preset **`speaker`** gender check (same helper as TTS). + - **Diffusion** (`assert_*_diffusion_response`): image/video dimension and frame-count parity with request parameters where configured. + - **Accuracy suites**: pixel/video similarity and metric baselines under `/tests/e2e/accuracy/` (separate from inline helper assertions). #### 2.3 Test Directory and Execution Files -- ***Functional / model E2E***: - - Online serving: `tests/e2e/online_serving/test_{model_name}_expansion.py` - - Offline inference: `tests/e2e/offline_inference/test_{model_name}_expansion.py` - - Accuracy / similarity: `tests/e2e/accuracy/` - - (`_expansion.py` holds broader scenario coverage than the L2 smoke file.) -- ***Feature-level integration***: `tests/e2e/features//` (not under a new top-level `tests/` directory). +- ***Functional / model E2E***: + - Online serving: `tests/e2e/online_serving/test_{model_name}_expansion.py` + - Offline inference: `tests/e2e/offline_inference/test_{model_name}_expansion.py` + - Accuracy / similarity: `tests/e2e/accuracy/` + - (`_expansion.py` holds broader scenario coverage than the L2 smoke file.) +- ***Feature-level integration***: `tests/e2e/features//` (not under a new top-level `tests/` directory). #### 2.4 Execution Method and Example -- ***Trigger Timing***: **`PR Merged`**. Automatically triggered after code review is approved and merged into the main branch (typically via `merge-test` label on the PR before merge). -- ***Diff-aware step skipping***: Same [`source_file_dependencies`](ci_settings.md#step-filtering) mechanism as L2—**all E2E Test** leaf steps are diff-gated; other L3 groups always upload. See [CI Settings — Diff-aware CI](ci_settings.md#diff-aware-ci). -- ***Execution Environment***: ***GPU*** servers. -- ***Script Example***: +- ***Trigger Timing***: **`PR Merged`**. Automatically triggered after code review is approved and merged into the main branch (typically via `merge-test` label on the PR before merge). +- ***Diff-aware step skipping***: Same [`source_file_dependencies`](ci_settings.md#step-filtering) mechanism as L2—**all E2E Test** leaf steps are diff-gated; other L3 groups always upload. See [CI Settings — Diff-aware CI](ci_settings.md#diff-aware-ci). +- ***Execution Environment***: ***GPU*** servers. +- ***Script Example***: ???+ example "Test Examples" @@ -496,9 +496,9 @@ L3 level testing executes after code is merged into the main branch. Its core pu **Audio output debugging**: Deep validation may transcribe returned audio via `convert_audio_bytes_to_text` (Whisper). If an audio keyword or text–audio similarity assertion fails, set `VLLM_OMNI_KEEP_REQUEST_MEDIA=1` before running pytest to keep the intermediate WAV files for inspection (see [Test helper environment variables](test_system_overview.md#test-helper-environment-variables)). -- ***Run Command (L3 merge)***: `pytest -s -v /tests/e2e/online_serving/test_{model_name}.py -m advanced_model --run-level=advanced_model` +- ***Run Command (L3 merge)***: `pytest -s -v /tests/e2e/online_serving/test_{model_name}.py -m advanced_model --run-level=advanced_model` -- ***Run Command (L4 nightly expansion)***: `pytest -s -v /tests/e2e/online_serving/test_{model_name}_expansion.py -m full_model --run-level=full_model` +- ***Run Command (L4 nightly expansion)***: `pytest -s -v /tests/e2e/online_serving/test_{model_name}_expansion.py -m full_model --run-level=full_model` ### L4 Level Testing - Full Functionality, Performance, and Documentation Testing @@ -506,27 +506,25 @@ L3 level testing executes after code is merged into the main branch. Its core pu L4 level testing is a comprehensive quality audit before a version release. It expands upon L3, executing ***full*** functional scenarios, conducting systematic ***performance stress tests***, and simultaneously verifying the correctness of accompanying ***example documentation***. Its purpose is to perform deep validation of the system during off-peak nighttime hours, providing quality trend reports for daytime development and data support for release decisions. - - #### 3.2 Testing Content and Scope -- ***Full Functionality Testing***: Executes all test cases defined in `test_{model_name}_expansion.py`, covering all implemented features, positive flows, boundary conditions, and exception handling. -- ***Performance Testing***: Uses `tests/dfx/perf/tests/test_qwen3_omni_*.json` (Omni), `test_tts.json` / `test_voxcpm2.json` / `test_higgs_audio_v3.json` (TTS), and diffusion configs `tests/dfx/perf/tests/test_*_vllm_omni.json` (passed to `run_benchmark.py` or `run_diffusion_benchmark.py` via `--test-config-file` in nightly **Perf Test** steps) to drive throughput, latency, and memory benchmarks. Each JSON **case** may declare an optional top-level **`mark`** array: exactly one ``hardware_marks`` object plus pytest marker name strings (`full_model`, `omni` / `tts` / `diffusion`, …). Runners attach those marks to each parametrized `(server, benchmark index)` pair so **local** bulk runs can filter with `-m` (for example `-m "full_model and H100 and diffusion"`). Nightly CI perf jobs select workloads by **`--test-config-file`**, not `-m`. Details are in the Performance Tests example below. -- ***Documentation Testing***: Verifies whether the example code provided to users is runnable and its results match the description. +- ***Full Functionality Testing***: Executes all test cases defined in `test_{model_name}_expansion.py`, covering all implemented features, positive flows, boundary conditions, and exception handling. +- ***Performance Testing***: Uses `tests/dfx/perf/tests/test_qwen3_omni_*.json` (Omni), `test_tts.json` / `test_voxcpm2.json` / `test_higgs_audio_v3.json` (TTS), and diffusion configs `tests/dfx/perf/tests/test_*_vllm_omni.json` (passed to `run_benchmark.py` or `run_diffusion_benchmark.py` via `--test-config-file` in nightly **Perf Test** steps) to drive throughput, latency, and memory benchmarks. Each JSON **case** may declare an optional top-level **`mark`** array: exactly one ``hardware_marks`` object plus pytest marker name strings (`full_model`, `omni` / `tts` / `diffusion`, …). Runners attach those marks to each parametrized `(server, benchmark index)` pair so **local** bulk runs can filter with `-m` (for example `-m "full_model and H100 and diffusion"`). Nightly CI perf jobs select workloads by **`--test-config-file`**, not `-m`. Details are in the Performance Tests example below. +- ***Documentation Testing***: Verifies whether the example code provided to users is runnable and its results match the description. #### 3.3 Test Directory and Execution Files -- ***Functional Testing***: Same directories as L3. -- ***Performance Test Configuration***: `tests/dfx/perf/tests/test_qwen3_omni_*.json`, `test_tts.json`, `test_voxcpm2.json`, `test_higgs_audio_v3.json`, and diffusion configs `tests/dfx/perf/tests/test_*_vllm_omni.json` (e.g. `test_qwen_image_vllm_omni.json`, `test_cosmos3_vllm_omni.json`). Optional per-case `mark` for local `-m` filtering is documented in Section 3.4 Performance Tests. -- ***Documentation Example Tests***: - - `tests/examples/online_serving/test_{model_name}.py` - - `tests/examples/offline_inference/test_{model_name}.py` +- ***Functional Testing***: Same directories as L3. +- ***Performance Test Configuration***: `tests/dfx/perf/tests/test_qwen3_omni_*.json`, `test_tts.json`, `test_voxcpm2.json`, `test_higgs_audio_v3.json`, and diffusion configs `tests/dfx/perf/tests/test_*_vllm_omni.json` (e.g. `test_qwen_image_vllm_omni.json`, `test_cosmos3_vllm_omni.json`). Optional per-case `mark` for local `-m` filtering is documented in Section 3.4 Performance Tests. +- ***Documentation Example Tests***: + - `tests/examples/online_serving/test_{model_name}.py` + - `tests/examples/offline_inference/test_{model_name}.py` #### 3.4 Execution Method and Example -- ***Trigger Timing***: **`Nightly`**, automatically executed every night. -- ***Execution Environment***: ***GPU*** server clusters to meet the resource demands of performance testing. -- ***Script Example***: +- ***Trigger Timing***: **`Nightly`**, automatically executed every night. +- ***Execution Environment***: ***GPU*** server clusters to meet the resource demands of performance testing. +- ***Script Example***: ??? example "Test Examples: Documentation Example Tests" @@ -540,7 +538,7 @@ L4 level testing is a comprehensive quality audit before a version release. It e --8<-- "docs/contributing/ci/test_examples/l4_functionality_tests.inc.md" -- ***Run Command***: (Specific commands would depend on the performance testing tool and configuration defined in `nightly.json`). +- ***Run Command***: (Specific commands would depend on the performance testing tool and configuration defined in `nightly.json`). ### L5 Level Testing - Stability and Reliability Testing @@ -548,33 +546,36 @@ L4 level testing is a comprehensive quality audit before a version release. It e L5 level testing focuses on the performance of model services under ***long-running*** and ***abnormal fault*** scenarios. It aims to uncover deep-seated issues that only manifest under sustained pressure or extreme conditions, such as memory leaks, resource contention, gradual performance degradation, and lack of fault tolerance mechanisms. This is the final, yet crucial, line of defense for ensuring service high availability and production environment robustness. - - #### 4.2 Testing Content and Scope -- ***Long-term Stability (Stability) Testing***: Uses JSON under `tests/dfx/stability/tests/` (for example `test_qwen3_omni.json` and `test_wan22.json`) to run the service under moderate load for an extended period (e.g., over 12 hours), monitoring whether metrics like memory/VRAM usage, response time, and throughput degrade over time, and whether the service process remains stable. -- ***Reliability Testing***: Uses pytest suites under `tests/dfx/reliability/` to inject controlled faults against a **live** `vllm_omni serve` instance (same **`omni_server` / `omni_server_function`** fixture style as E2E). Current suites emphasize **GPU memory pressure** (CUDA sidecar “memory hog”), **worker / runtime process kill** (`SIGKILL` on `VLLM::Worker` for Qwen3-Omni, `multiprocessing.spawn` for Wan2.2 video workers, or `vLLM-Omni::` for HunyuanImage DiT workers), **large multimodal chat**, **`/v1/videos`**, **`/v1/images/generations`**, or **`/v1/audio/speech`** jobs under OOM, **`/health` → 503** and **fast-fail / non-hanging concurrent** requests after kill, and **OpenAI-style 5xx error contracts** (e.g. text vs text+audio under OOM). **Post-fault recovery** checks exist where enabled (some cases may be `skip` while issues are tracked). See the Reliability `
` block in Section 4.4 for file-level responsibilities and CI markers (`slow`, `hardware_test`, POSIX-only kill). +- ***Long-term Stability (Stability) Testing***: Uses JSON under `tests/dfx/stability/tests/` (for example `test_qwen3_omni.json` and `test_wan22.json`) to run the service under moderate load for an extended period (e.g., over 12 hours), monitoring whether metrics like memory/VRAM usage, response time, and throughput degrade over time, and whether the service process remains stable. +- ***Reliability Testing***: Uses pytest suites under `tests/dfx/reliability/` to inject controlled faults against a **live** `vllm_omni serve` instance (same **`omni_server` / `omni_server_function`** fixture style as E2E). Current suites emphasize **GPU memory pressure** (CUDA sidecar “memory hog”), **worker / runtime process kill** (`SIGKILL` on `VLLM::Worker` for Qwen3-Omni, `multiprocessing.spawn` for Wan2.2 video workers, or `vLLM-Omni::` for HunyuanImage DiT workers), **large multimodal chat**, **`/v1/videos`**, **`/v1/images/generations`**, or **`/v1/audio/speech`** jobs under OOM, **`/health` → 503** and **fast-fail / non-hanging concurrent** requests after kill, and **OpenAI-style 5xx error contracts** (e.g. text vs text+audio under OOM). **Post-fault recovery** checks exist where enabled (some cases may be `skip` while issues are tracked). See Section 4.3 for file-level responsibilities and Section 4.4 Test Examples (Reliability) for CI markers (`slow`, `hardware_test`, POSIX-only kill). #### 4.3 Test Directory and Execution Files -- ***Stability Test Configuration***: `tests/dfx/stability/tests/test_qwen3_omni.json`, `tests/dfx/stability/tests/test_wan22.json` (one JSON per model / runner family) -- ***Reliability Test Suite*** (`tests/dfx/reliability/`): - - `test_reliability_qwen3_omni.py` — Qwen3-Omni chat / multimodal reliability (GPU OOM, process kill, recovery, error contract under `--async-chunk` vs default). - - `test_reliability_wan22.py` — Wan2.2 I2V video API reliability (`/v1/videos` under OOM and process kill, recovery). - - `test_reliability_hunyuan_image.py` — HunyuanImage-3.0-Instruct DiT-only reliability (`/v1/images/generations` under OOM and process kill; deploy `hunyuan_image3_dit.yaml`, H100 × 4). - - `helpers.py` — Shared primitives used by current suites: raw HTTP probes for `/v1/chat/completions` and `/health`, OpenAI-style error parsing, GPU OOM sidecar (`inject_gpu_oom` / `stop_gpu_oom_hogs`), and `pgrep`-based process-kill injector construction (`make_process_kill_fault_injector`). - - `conftest.py` — `fault_injector` and `omni_server_after_fault` / `omni_server_after_fault_function` fixtures to run a callable **after** the server is ready. - - `README.md` — Short local run commands for this directory. +- ***Stability Test Configuration***: `tests/dfx/stability/tests/test_qwen3_omni.json`, `tests/dfx/stability/tests/test_wan22.json` (one JSON per model / runner family) +- ***Reliability Test Suite*** (`tests/dfx/reliability/`): + - `test_reliability_qwen3_omni.py` — Qwen3-Omni chat / multimodal reliability (GPU OOM, process kill, recovery, error contract under `--async-chunk` vs default). + - `test_reliability_wan22.py` — Wan2.2 I2V video API reliability (`/v1/videos` under OOM and process kill, recovery). + - `test_reliability_hunyuan_image.py` — HunyuanImage-3.0-Instruct DiT-only reliability (`/v1/images/generations` under OOM and process kill; deploy `hunyuan_image3_dit.yaml`, H100 × 4). + - `helpers.py` — Shared primitives used by current suites: raw HTTP probes for `/v1/chat/completions` and `/health`, OpenAI-style error parsing, GPU OOM sidecar (`inject_gpu_oom` / `stop_gpu_oom_hogs`), and `pgrep`-based process-kill injector construction (`make_process_kill_fault_injector`). + - `conftest.py` — `fault_injector` and `omni_server_after_fault` / `omni_server_after_fault_function` fixtures to run a callable **after** the server is ready. + - `README.md` — Short local run commands for this directory. #### 4.4 Execution Method and Example -- ***Trigger Timing***: **`Weekly`** (weekly) or **`Days before Release`** (several days before a major release). Due to long execution times, the frequency is lower. -- ***Execution Environment***: ***GPU*** servers, requiring a stable and exclusive testing environment. -- ***Script Example***: +- ***Trigger Timing***: **`Weekly`** (weekly) or **`Days before Release`** (several days before a major release). Due to long execution times, the frequency is lower. +- ***Run Command***: + - ***Stability***: `pytest -s -v tests/dfx/stability/scripts/test_stability_qwen3_omni.py` or `pytest -s -v tests/dfx/stability/scripts/test_stability_wan22.py` (or add `test_stability_.py` alongside a matching JSON config) + - ***Reliability***: `pytest -s -v tests/dfx/reliability/test_reliability_.py -m slow` (current suites: `qwen3_omni`, `wan22`, `hunyuan_image`). Weekly CI (`.buildkite/cuda/test-weekly.yml`) runs one step per suite (`WEEKLY=1` or PR label `weekly-test`) +- ***Script Example***: +
Test Examples -When you want to add L5-level stability test cases, add or extend the appropriate JSON file under `tests/dfx/stability/tests/` (for example `test_qwen3_omni.json` for Omni bench traffic, or `test_wan22.json` for diffusion `/v1/videos` workloads). The following illustrates the Qwen3-Omni shape: +##### Stability + +When you want to add L5-level stability test cases, add or extend the appropriate JSON file under `tests/dfx/stability/tests/` (for example `test_qwen3_omni.json` for Omni bench traffic, or `test_wan22.json` for diffusion `/v1/videos` workloads). Pair the JSON with `tests/dfx/stability/scripts/test_stability_.py`. The following illustrates the Qwen3-Omni shape: ```json { @@ -600,7 +601,7 @@ When you want to add L5-level stability test cases, add or extend the appropriat } ``` -##### Parameter Explanation +###### Parameter Explanation ***Overview*** @@ -610,91 +611,47 @@ When you want to add L5-level stability test cases, add or extend the appropriat | server_params | Yes | Server-side configuration parameters (model, deploy configuration, etc.) | | benchmark_params | Yes | Stability benchmark running parameters (supports multiple configurations) | -##### server_params Configuration +###### server_params Configuration -###### Basic Parameters +Basic parameters: | Parameter | Required | Example | Description | | ----------------- | -------- | ---------------------------------- | ----------------------------------- | | model | Yes | "Qwen/Qwen3-Omni-30B-A3B-Instruct" | Model name or path | | stage_config_name | Yes | "qwen3_omni_moe.yaml" | Deploy configuration file name | -###### Dynamic Configuration (update/delete) - -Supports incremental modifications based on the basic configuration: +Dynamic configuration (`update` / `delete`) supports incremental modifications based on the basic configuration: | Operation | Description | | --------- | ------------------------------------ | | update | Update or add configuration items | | delete | Delete specified configuration items | -***Example***: -You can refer to Test Examples in L4 §3.4 +***Example***: You can refer to Test Examples in L4 §3.4. -##### benchmark_params Configuration +###### benchmark_params Configuration For stability testing, the key parameters are: -- **duration_sec**: Total duration (in seconds) during which benchmark traffic is sent. The stability benchmark will keep sending batches until this duration is reached. -- **request_rate** / **max_concurrency**: Exactly one of them must be specified. They control how the traffic is generated for each batch: - - `request_rate`: Number of requests per second. The benchmark will send `num_prompts_per_batch` requests at the given rate. - - `max_concurrency`: Maximum number of concurrent requests. When this is used, `request_rate` is set to `inf` internally. -- **num_prompts_per_batch**: Number of prompts sent in each batch. Multiple batches will be executed sequentially within `duration_sec`. +- **duration_sec**: Total duration (in seconds) during which benchmark traffic is sent. The stability benchmark will keep sending batches until this duration is reached. +- **request_rate** / **max_concurrency**: Exactly one of them must be specified. They control how the traffic is generated for each batch: + - `request_rate`: Number of requests per second. The benchmark will send `num_prompts_per_batch` requests at the given rate. + - `max_concurrency`: Maximum number of concurrent requests. When this is used, `request_rate` is set to `inf` internally. +- **num_prompts_per_batch**: Number of prompts sent in each batch. Multiple batches will be executed sequentially within `duration_sec`. All other optional parameters follow the same rules as in L4 §3.4. -
+##### Reliability -
- Reliability test suite (tests/dfx/reliability) +Add pytest modules under `tests/dfx/reliability/` (for example `test_reliability_qwen3_omni.py`, `test_reliability_wan22.py`, `test_reliability_hunyuan_image.py`). Reuse `helpers.py` and `conftest.py` rather than duplicating OOM / process-kill / raw HTTP plumbing. -##### Purpose and relationship to stability - -Reliability tests are **short fault-injection** integration runs (L5 **(b)** in `tests/dfx/reliability/README.md`). They complement **stability** JSON-driven long runs: instead of hours of steady traffic, they **perturb** the server (GPU OOM sidecar, fatal signals on selected processes) and check **failure mode** and **latency bounds** (e.g. chat or `/v1/videos` must not hang under concurrent fault-time load). - -##### Directory layout - -| Path | Responsibility | -| ---- | -------------- | -| `helpers.py` | Shared helpers used by current reliability suites: raw `POST`/`GET` probes (`/v1/chat/completions`, `/health`), OpenAI error parsing (`extract_openai_error_contract_from_bytes`), GPU OOM sidecar lifecycle (`inject_gpu_oom`, `stop_gpu_oom_hogs`), and process-kill injector builder (`make_process_kill_fault_injector`). | -| `conftest.py` | Pytest fixtures: indirect `fault_injector`, `omni_server_after_fault` / `omni_server_after_fault_function` (run injector after server is ready, then yield server). | -| `test_reliability_qwen3_omni.py` | Qwen3-Omni: OOM vs **text vs text+audio** error contract, large multimodal chat under OOM, concurrent pressure, **SIGKILL** on `VLLM::Worker`, `/health` → 503 + fast-fail + concurrent chat; optional OOM recovery scenario (may be skipped while tracked in issues). | -| `test_reliability_wan22.py` | Wan2.2 I2V: large `/v1/videos` under OOM, **SIGKILL** on `multiprocessing.spawn` chain, health / fast-fail / concurrent video requests; optional recovery test (may be skipped). | -| `test_reliability_hunyuan_image.py` | HunyuanImage DiT-only: large `/v1/images/generations` under OOM, **SIGKILL** on `vLLM-Omni::` workers and serve/tree targets, health / fast-fail / concurrent image requests; some OOM/recovery cases may be skipped while tracked in issues. | -| `README.md` | Minimal run / collect examples. | - -##### Parametrization and markers +###### Parametrization and markers - Each test module defines a **`RELIABILITY_SCENARIOS`** list (`test_name`, `server_params`: model, `stage_config_name` or diffusion `server_args`, etc.). **`create_reliability_omni_server_params()`** in `tests/dfx/conftest.py` resolves stage paths (including XPU substitutions where applicable) and builds **`OmniServerParams`** lists consumed by **`@pytest.mark.parametrize(..., indirect=True)`** on `omni_server` or `omni_server_function`. - Cases are tagged **`@pytest.mark.slow`** for weekly / selective CI. GPU-heavy suites use **`@hardware_test(res={"cuda": "H100"}, num_cards=...)`** (Qwen3-Omni **2**× H100; Wan2.2 **1**× H100; HunyuanImage DiT **4**× H100). - **Process-kill** tests use **`@pytest.mark.skipif(os.name == "nt", ...)`** because injection uses POSIX **`pgrep` / `kill`**. -##### CI trigger - -Weekly Buildkite (`.buildkite/cuda/test-weekly.yml`) runs one step per model suite (trigger: `WEEKLY=1` or PR label `weekly-test`), for example: - -| Buildkite step | Test file | CI hardware | -| -------------- | --------- | ----------- | -| Reliability Test - qwen3-omni | `test_reliability_qwen3_omni.py` | H100 × 2 (`mithril-h100-pool`) | -| Reliability Test - wan22 | `test_reliability_wan22.py` | H100 × 2 (`mithril-h100-pool`) | -| Reliability Test - hunyuan-image | `test_reliability_hunyuan_image.py` | H100 × 4 (`mithril-h100-pool`) | - -```bash -pytest -s -v tests/dfx/reliability/test_reliability_qwen3_omni.py -m "slow" -pytest -s -v tests/dfx/reliability/test_reliability_wan22.py -m "slow" -pytest -s -v tests/dfx/reliability/test_reliability_hunyuan_image.py -m "slow" -``` - -##### Local commands - -```bash -pytest --collect-only tests/dfx/reliability -pytest -s -v tests/dfx/reliability/test_reliability_qwen3_omni.py -m slow -pytest -s -v tests/dfx/reliability/test_reliability_wan22.py -m slow -pytest -s -v tests/dfx/reliability/test_reliability_hunyuan_image.py -m slow -``` - -##### Adding a new model suite +###### Adding a new model suite 1. Add `test_reliability_.py` under `tests/dfx/reliability/`. 2. Define **`RELIABILITY_SCENARIOS`** and pass them through **`create_reliability_omni_server_params()`** with the correct deploy or e2e deploy-config directory (same pattern as existing files). @@ -702,6 +659,3 @@ pytest -s -v tests/dfx/reliability/test_reliability_hunyuan_image.py -m slow 4. Register **`slow`** (and **`hardware_test`** if needed); extend **`.buildkite/cuda/test-weekly.yml`** when the suite should run in weekly L5.
- -- - ***Stability***: `pytest -s -v tests/dfx/stability/scripts/test_stability_qwen3_omni.py` or `pytest -s -v tests/dfx/stability/scripts/test_stability_wan22.py` (or add `test_stability_.py` alongside a matching JSON config) - - ***Reliability***: `pytest -s -v tests/dfx/reliability/test_reliability_.py -m slow` (current suites: `qwen3_omni`, `wan22`, `hunyuan_image`; add `test_reliability_.py` and a matching step in `.buildkite/cuda/test-weekly.yml` for new models) diff --git a/docs/contributing/model/adding_tts_model.md b/docs/contributing/model/adding_tts_model.md index 76aab4291..0b8e07c9c 100644 --- a/docs/contributing/model/adding_tts_model.md +++ b/docs/contributing/model/adding_tts_model.md @@ -78,7 +78,7 @@ Profile before optimizing. new TTS model must pass all three levels: | Layer | Catches | Tool | -|-------|---------|------| +| ------- | --------- | ------ | | Offline RTF / duration | Throughput regressions, missing audio, wrong sample rate | `end2end.py`, pytest e2e | | Browser streaming playback | Delta-vs-cumulative bugs, chunk boundary glitches, TTFP regressions | Gradio demo over `/v1/audio/speech?stream=true` | | Concurrent requests | Per-request state leaks, codec window round-robin gaps | `max_num_seqs>1` smoke with 4+ parallel prompts | @@ -94,10 +94,10 @@ the symptom is crosstalk or truncation under load, nothing in single-request tes vLLM-Omni supports TTS models as multi-stage pipelines where each stage runs independently and can be placed on different devices. Qwen3-TTS has two stages: -| Stage | Name | Input | Output | -|-------|------|-------|--------| -| 0 | Code Predictor (AR) | Text tokens | Discrete RVQ codec codes | -| 1 | Code2Wav (Decoder) | RVQ codec codes | Audio waveform | +| Stage | Name | Input | Output | +| ----- | ------------------- | --------------- | ------------------------ | +| 0 | Code Predictor (AR) | Text tokens | Discrete RVQ codec codes | +| 1 | Code2Wav (Decoder) | RVQ codec codes | Audio waveform | Each stage is a separate model class configured independently via YAML. The two stages are connected by the `async_chunk` framework, which enables inter-stage streaming for @@ -170,7 +170,7 @@ and Qwen3-Omni). When adding a new TTS model, create the following structure: -``` +```text vllm_omni/model_executor/models/ your_model_name/ __init__.py @@ -201,7 +201,7 @@ it would not be picked up — the hub README is the documented surface. **Qwen3-TTS reference files:** | File | Purpose | -|------|---------| +| ------ | --------- | | `models/qwen3_tts/qwen3_tts.py` | Unified model class | | `models/qwen3_tts/qwen3_tts_code_predictor_vllm.py` | Stage 0 - optimized AR | | `models/qwen3_tts/qwen3_tts_code2wav.py` | Stage 1 - decoder | @@ -528,9 +528,9 @@ Understanding what's available in stage outputs: - `stage_list[source_id].engine_outputs` - list of `EngineCoreOutput` objects - Each `EngineCoreOutput` has `outputs` - list of `RequestOutput` objects - Each `RequestOutput` has: - - `token_ids` - generated token IDs - - `multimodal_output` - dict with keys matching your model's `OmniOutput.multimodal_outputs` - - `prompt_token_ids` - original prompt token IDs + - `token_ids` - generated token IDs + - `multimodal_output` - dict with keys matching your model's `OmniOutput.multimodal_outputs` + - `prompt_token_ids` - original prompt token IDs ### Batch mode (non-streaming) @@ -610,6 +610,7 @@ def ar2decoder_async_chunk( ``` Key points: + - `transfer_manager` is the `OmniChunkTransferAdapter` that owns the chunk lifecycle - Each call appends one AR decode step's output; a chunk is emitted every `chunk_size` steps - The final (possibly partial) chunk is flushed when `is_finished` is true @@ -664,6 +665,7 @@ Create `vllm_omni/entrypoints/openai/tts_adapters/your_model.py`: ```python # SPDX-License-Identifier: Apache-2.0 +# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project """YourModel serving adapter.""" from typing import TYPE_CHECKING @@ -802,7 +804,7 @@ directly from the model's own generator. ### Directory structure -``` +```text vllm_omni/model_executor/models/your_model_name/ __init__.py modeling_your_model_name.py # unified class: load_weights + forward + streaming @@ -950,10 +952,17 @@ is correct. Stage the modified files and commit again — do not revert the chan Common failures and fixes: | Check | Cause | Fix | -|-------|-------|-----| +| ------- | ------- | ----- | | `ruff F841` | Local variable assigned but never used | Remove the extraction or forward it to the model call | | `ruff E402` | Module-level import not at top of file | Move import to the top-level import block | | `ruff format` | Line length, spacing, or quote style | Accept the auto-fix, stage, and re-commit | +| `check-spdx-header` | Missing header, or copyright still says `vLLM project` | Use `Copyright contributors to the vLLM-Omni project`; restage the rewrite | +| `check-forbidden-imports` | Stdlib `re`/`base64`, pickle, Hugging Face Hub API, or direct Triton/TileLang | `import regex as re` and `pybase64`; do not grow the allowlist without review | +| `check-torch-cuda-call` | New `torch.cuda.*` call site | Use `current_omni_platform`; do not grow `ALLOWED_FILES` without review | +| `check-tts-adapter-migration` | New `_tts_model_type` branch in `serving_speech.py` | Put logic in `tts_adapters/`; lower `MAX_MODEL_TYPE_BRANCHES` when removing branches | +| `check-test-ci-coverage` | Test file missing level or hardware mark | Add `core_model`/… plus `cpu`/`cuda`/`hardware_test(` | +| `shellcheck` | Native `shellcheck` missing, or script warning | Install via apt/dnf/brew (or `shellcheck.exe` on Windows). See [Linting](../README.md#linting) | +| `mypy-3.10` / `markdownlint-cli2` / `check-buildkite` | Types, docs markdown, or Buildkite YAML | See [Linting](../README.md#linting) | ### DCO sign-off @@ -1008,7 +1017,7 @@ Adding a TTS model to vLLM-Omni involves: ### Qwen3-TTS Reference Files | File | Purpose | -|------|---------| +| ------ | --------- | | `models/qwen3_tts/qwen3_tts.py` | Unified model class | | `models/qwen3_tts/qwen3_tts_code_predictor_vllm.py` | AR stage with vLLM fused ops | | `models/qwen3_tts/qwen3_tts_code2wav.py` | Decoder stage with `chunked_decode_streaming()` | diff --git a/pyproject.toml b/pyproject.toml index 160c6bc2f..e9bdd3b29 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -225,19 +225,14 @@ ignore = [ "torch.cuda.synchronize".msg = "Use torch.accelerator.synchronize" [tool.mypy] -python_version = "3.12, 3.13" -warn_return_any = true -warn_unused_configs = true -disallow_untyped_defs = true -disallow_incomplete_defs = true +python_version = "3.12" +# Align with vLLM: silent follow for library code; missing third-party stubs +# are ignored everywhere (including tests). tests/ still get +# --follow-imports skip from tools/pre_commit/mypy.py. +plugins = ["pydantic.mypy"] +ignore_missing_imports = true check_untyped_defs = true -disallow_untyped_decorators = true -no_implicit_optional = true -warn_redundant_casts = true -warn_unused_ignores = true -warn_no_return = true -warn_unreachable = true -strict_equality = true +follow_imports = "silent" [tool.pytest.ini_options] asyncio_mode = "auto" @@ -343,6 +338,8 @@ extend-ignore-identifiers-re = [ ] extend-ignore-re = [ "\\.\\.\\.n,d->\\.\\.\\.nd", + # setuptools-scm / git-describe local versions, e.g. 0.26.1.dev14+gae6577ea + "\\+g[0-9a-f]{7,}", ] [tool.typos.default.extend-words] ue = "ue" @@ -367,3 +364,5 @@ Leary = "Leary" fo = "fo" ba = "ba" SER = "SER" +# NumPy ndarray.flags.writeable (canonical flag name, not "writable"). +writeable = "writeable" diff --git a/tests/diffusion/test_diffusion_ipc.py b/tests/diffusion/test_diffusion_ipc.py index 546aa11e7..f24767e18 100644 --- a/tests/diffusion/test_diffusion_ipc.py +++ b/tests/diffusion/test_diffusion_ipc.py @@ -1,5 +1,5 @@ # SPDX-License-Identifier: Apache-2.0 -# SPDX-FileCopyrightText: Copyright contributors to the vLLM project +# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project import contextlib import multiprocessing as mp @@ -7,6 +7,7 @@ import queue import threading import time from multiprocessing import shared_memory +from multiprocessing.connection import Connection import numpy as np import pytest @@ -29,7 +30,7 @@ pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, pytest.mark.cpu] def _large_numel(dtype: torch.dtype) -> int: - return (_SHM_TENSOR_THRESHOLD // torch.empty((), dtype=dtype).element_size()) + 1 + return int(_SHM_TENSOR_THRESHOLD // torch.empty((), dtype=dtype).element_size()) + 1 def _cleanup_shm_handle(value: object) -> None: @@ -38,7 +39,7 @@ def _cleanup_shm_handle(value: object) -> None: _unpack_if_shm_handle(value) -def _result_queue_worker(connection, rank: int) -> None: +def _result_queue_worker(connection: Connection, rank: int) -> None: """Send one nested NumPy result through a worker-owned MessageQueue.""" result_mq = MessageQueue(n_reader=1, n_local_reader=1, local_reader_ranks=[0]) try: @@ -309,11 +310,9 @@ def test_per_request_views_of_a_batch_tensor_do_not_ship_whole_storage() -> None """Regression: per-request slices used to pickle the whole batch storage. Each view is under the packing threshold, but pickle writes the entire - shared storage per view, so a batch's message grew by the full batch - tensor once per request and overflowed the ring buffer. + shared storage per view. Packing must key off storage nbytes so those + views become SHM handles instead of going on the wire. """ - import pickle - num_requests = 16 # Each row stays under the threshold; the shared storage is well over it. row_numel = _SHM_TENSOR_THRESHOLD // (2 * torch.empty((), dtype=torch.float32).element_size()) @@ -322,6 +321,13 @@ def test_per_request_views_of_a_batch_tensor_do_not_ship_whole_storage() -> None assert per_request.nelement() * per_request.element_size() <= _SHM_TENSOR_THRESHOLD assert per_request.untyped_storage().nbytes() > _SHM_TENSOR_THRESHOLD + packed_view = _pack_value_if_large(per_request) + try: + assert isinstance(packed_view, dict) + assert packed_view["__tensor_shm__"] is True + finally: + _cleanup_shm_handle(packed_view) + output = BatchRunnerOutput.from_list( [ RunnerOutput( @@ -332,16 +338,10 @@ def test_per_request_views_of_a_batch_tensor_do_not_ship_whole_storage() -> None for i in range(num_requests) ] ) - unpacked_size = len(pickle.dumps(output, protocol=pickle.HIGHEST_PROTOCOL)) pack_diffusion_output_shm(output) - packed_size = len(pickle.dumps(output, protocol=pickle.HIGHEST_PROTOCOL)) assert output.runner_outputs[0].result.trajectory_latents["__tensor_shm__"] is True - # Whole storage per request before; handles only after. - storage_bytes = per_request.untyped_storage().nbytes() - assert unpacked_size > num_requests * storage_bytes / 2 - assert packed_size < _SHM_TENSOR_THRESHOLD unpack_diffusion_output_shm(output) for i in range(num_requests): @@ -380,31 +380,37 @@ def test_pack_value_recurses_nested_dicts_and_lists_without_mutating_inline_valu large = torch.arange(_large_numel(torch.float32), dtype=torch.float32) small = torch.arange(8, dtype=torch.float32) list_tensor = torch.arange(_large_numel(torch.float32), dtype=torch.float32) + original_list = [list_tensor] payload = { "media": { "large": large, "small": small, }, - "list_value": [list_tensor], + "list_value": original_list, "metadata": {"prompt": "keep inline"}, } packed = _pack_value_if_large(payload) try: + assert isinstance(packed, dict) assert packed is not payload - assert packed["media"] is not payload["media"] - assert packed["media"]["large"]["__tensor_shm__"] is True - assert packed["media"]["small"] is small + media = packed["media"] + list_value = packed["list_value"] + assert isinstance(media, dict) + assert isinstance(list_value, list) + assert media is not payload["media"] + assert media["large"]["__tensor_shm__"] is True + assert media["small"] is small # Lists are recursed too: the large tensor inside is packed and a new # list is returned, while the input payload is left untouched. - assert packed["list_value"] is not payload["list_value"] - assert packed["list_value"][0]["__tensor_shm__"] is True - assert payload["list_value"][0] is list_tensor + assert list_value is not original_list + assert list_value[0]["__tensor_shm__"] is True + assert original_list[0] is list_tensor assert packed["metadata"] == {"prompt": "keep inline"} - torch.testing.assert_close(_unpack_if_shm_handle(packed["media"]["large"]), large) - torch.testing.assert_close(_unpack_if_shm_handle(packed["list_value"][0]), list_tensor) + torch.testing.assert_close(_unpack_if_shm_handle(media["large"]), large) + torch.testing.assert_close(_unpack_if_shm_handle(list_value[0]), list_tensor) finally: if isinstance(packed, dict): _cleanup_shm_handle(packed.get("media", {}).get("large")) diff --git a/tools/pre_commit/check_buildkite.py b/tools/pre_commit/check_buildkite.py new file mode 100644 index 000000000..362253494 --- /dev/null +++ b/tools/pre_commit/check_buildkite.py @@ -0,0 +1,118 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: Apache-2.0 +# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project +"""Validate .buildkite YAML against the official Buildkite schema. + +Test pipelines (test-ready.yml, etc.) use Omni-only keys such as +``mirror_hardwares`` and ``source_file_dependencies``. Those are expanded or +stripped the same way ``upload_pipeline.py`` does before upload, then the +rendered YAML is checked with check-jsonschema. +""" + +from __future__ import annotations + +import subprocess +import sys +import tempfile +from pathlib import Path +from typing import Any + +import yaml + +ROOT = Path(__file__).resolve().parents[2] +SCRIPTS = ROOT / ".buildkite" / "common" / "scripts" +sys.path.insert(0, str(SCRIPTS)) + +from upload_pipeline import _render_test_pipeline # noqa: E402 + +SKIP_FILES = { + ".buildkite/common/ci_mirror_hardwares.yml", + ".buildkite/npu/pipeline-npu-a3.yml", +} + +# AMD pipelines are uploaded by a different path and use list-valued +# mirror_hardwares plus agent_pool/grade, which CUDA's expander rejects. +AMD_STRIP_KEYS = frozenset({"mirror_hardwares", "agent_pool", "grade", "source_file_dependencies"}) + + +def _normalize(path: str) -> str: + return path.replace("\\", "/") + + +def _strip_keys(obj: Any, keys: frozenset[str]) -> Any: + if isinstance(obj, dict): + return {k: _strip_keys(v, keys) for k, v in obj.items() if k not in keys} + if isinstance(obj, list): + return [_strip_keys(item, keys) for item in obj] + return obj + + +def _prepare_pipeline(path: Path) -> dict[str, Any]: + rel = _normalize(str(path.relative_to(ROOT) if path.is_absolute() else path)) + doc = yaml.safe_load(path.read_text(encoding="utf-8")) + if not isinstance(doc, dict): + raise ValueError(f"{rel}: expected a mapping (pipeline YAML), got {type(doc).__name__}") + + if "/amd/" in f"/{rel}": + return _strip_keys(doc, AMD_STRIP_KEYS) + + return _render_test_pipeline(doc, changed_files=None) + + +def _validate(path: Path, doc: dict[str, Any]) -> int: + with tempfile.NamedTemporaryFile( + mode="w", + suffix=".yml", + encoding="utf-8", + delete=False, + ) as handle: + yaml.safe_dump(doc, handle, sort_keys=False) + tmp = Path(handle.name) + + try: + result = subprocess.run( + [ + sys.executable, + "-m", + "check_jsonschema", + "--builtin-schema", + "vendor.buildkite", + str(tmp), + ], + capture_output=True, + text=True, + ) + finally: + tmp.unlink(missing_ok=True) + + if result.returncode == 0: + return 0 + + rel = _normalize(str(path)) + sys.stderr.write(f"{rel}: Buildkite schema validation failed\n") + sys.stderr.write(result.stdout or "") + sys.stderr.write(result.stderr or "") + return 1 + + +def main(argv: list[str]) -> int: + rc = 0 + for raw in argv: + path = Path(raw) + rel = _normalize(str(path)) + if rel in SKIP_FILES: + continue + if not path.is_file(): + path = ROOT / path + try: + doc = _prepare_pipeline(path) + except Exception as exc: # noqa: BLE001 - surface expander errors as hook failures + sys.stderr.write(f"{rel}: {exc}\n") + rc = 1 + continue + rc |= _validate(path, doc) + return rc + + +if __name__ == "__main__": + raise SystemExit(main(sys.argv[1:])) diff --git a/tools/pre_commit/check_forbidden_imports.py b/tools/pre_commit/check_forbidden_imports.py new file mode 100644 index 000000000..a3f5e320a --- /dev/null +++ b/tools/pre_commit/check_forbidden_imports.py @@ -0,0 +1,368 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: Apache-2.0 +# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project +"""Ban dangerous / policy-violating imports unless a file is allowlisted. + +Ported from vllm/tools/pre_commit/check_forbidden_imports.py. Existing +vllm_omni/ usages are grandfathered; add a path to the matching +allowed_files set only after a security/policy review. +""" + +import sys +from dataclasses import dataclass, field + +import regex as re + +# Hub entry points that must go through the vLLM-tagged repo_utils helpers. +_HF_NAMES = ( + r"HfApi|HfFileSystem|hf_hub_download|snapshot_download" + r"|list_repo_files|file_exists|try_to_load_from_cache" + r"|list_repo_refs|repo_exists" +) + +# Non-library trees: examples, tests, and tooling may keep stdlib imports. +_NON_LIBRARY_DIRS = { + "examples/", + "tests/", + "benchmarks/", + "apps/", + "docs/", + "tools/", + "scripts/", + ".buildkite/", + ".github/", +} + + +@dataclass +class ForbiddenImport: + pattern: str + tip: str + allowed_pattern: re.Pattern = re.compile(r"^$") # matches nothing by default + allowed_files: set[str] = field(default_factory=set) + allowed_dirs: set[str] = field(default_factory=set) + + +CHECK_IMPORTS = { + "pickle/cloudpickle": ForbiddenImport( + pattern=( + r"^\s*(import\s+(?:[\w.]+\s*(?:as\s+\w+)?\s*,\s*)*(pickle|cloudpickle)\b" + r"|from\s+(pickle|cloudpickle)\s+import\b)" + ), + tip=("Avoid using pickle or cloudpickle or add this file to tools/pre_commit/check_forbidden_imports.py."), + allowed_files={ + # STOP AND READ BEFORE YOU ADD ANYTHING ELSE TO THIS LIST: + # pickle/cloudpickle are unsafe when deserializing untrusted data. + "tests/diffusion/attention/test_attention_sp.py", + "tests/helpers/process.py", + "vllm_omni/diffusion/distributed/group_coordinator.py", + }, + ), + "base64": ForbiddenImport( + pattern=r"^\s*(?:import\s+base64(?:$|\s|,)|from\s+base64\s+import)", + tip=("Replace 'import base64' with 'import pybase64' or 'import pybase64 as base64'."), + allowed_pattern=re.compile(r"^\s*import\s+pybase64(\s*|\s+as\s+base64\s*)$"), + allowed_dirs=_NON_LIBRARY_DIRS, + allowed_files={ + "vllm_omni/benchmarks/data_modules/daily_omni_dataset.py", + "vllm_omni/benchmarks/data_modules/random_multi_modal_dataset.py", + "vllm_omni/benchmarks/data_modules/seed_tts_dataset.py", + "vllm_omni/entrypoints/openai/api_server.py", + "vllm_omni/entrypoints/openai/audio_utils_mixin.py", + "vllm_omni/entrypoints/openai/image_api_utils.py", + "vllm_omni/entrypoints/openai/protocol/images.py", + "vllm_omni/entrypoints/openai/realtime_connection.py", + "vllm_omni/entrypoints/openai/serving_chat.py", + "vllm_omni/entrypoints/openai/serving_speech.py", + "vllm_omni/entrypoints/openai/serving_speech_stream.py", + "vllm_omni/entrypoints/openai/video_api_utils.py", + "vllm_omni/entrypoints/openai/video_stream_base.py", + "vllm_omni/entrypoints/openai/video_stream_session.py", + "vllm_omni/experimental/fullduplex/client.py", + "vllm_omni/experimental/fullduplex/engine/contracts.py", + "vllm_omni/experimental/fullduplex/joyvl/memory/memory.py", + "vllm_omni/experimental/fullduplex/minicpmo45/adapter.py", + "vllm_omni/experimental/fullduplex/minicpmo45/input.py", + "vllm_omni/experimental/fullduplex/minicpmo45/runtime.py", + "vllm_omni/experimental/fullduplex/minicpmo45/stage0.py", + "vllm_omni/experimental/fullduplex/openai/audio.py", + "vllm_omni/experimental/fullduplex/openai/realtime_input.py", + "vllm_omni/experimental/fullduplex/openai/runtime_bridge.py", + "vllm_omni/experimental/fullduplex/openai/serving.py", + "vllm_omni/experimental/fullduplex/personaplex/input.py", + "vllm_omni/experimental/fullduplex/personaplex/runtime_extension.py", + "vllm_omni/experimental/fullduplex/personaplex/stage0.py", + "vllm_omni/model_executor/models/indextts2/tokenizer_v2_5.py", + "vllm_omni/model_executor/models/qwen3_tts/prompt_embeds_builder.py", + "vllm_omni/model_executor/models/qwen3_tts/qwen3_tts_tokenizer.py", + "vllm_omni/tokenizers/mammoth_moda2_tokenizer.py", + "vllm_omni/transformers_utils/processors/audiox.py", + }, + ), + "re": ForbiddenImport( + pattern=r"^\s*(?:import\s+re(?:$|\s|,)|from\s+re\s+import)", + tip="Replace 'import re' with 'import regex as re' or 'import regex'.", + allowed_pattern=re.compile(r"^\s*import\s+regex(\s*|\s+as\s+re\s*)$"), + allowed_dirs=_NON_LIBRARY_DIRS, + allowed_files={ + "setup.py", + "vllm_omni/benchmarks/data_modules/daily_omni_eval.py", + "vllm_omni/config/stage_config.py", + "vllm_omni/diffusion/layers/mot/ops/mot_gemm.py", + "vllm_omni/diffusion/model_loader/diffusers_loader.py", + "vllm_omni/diffusion/models/diffusers_adapter/pipeline_diffusers_adapter.py", + "vllm_omni/diffusion/models/dreamid_omni/fusion.py", + "vllm_omni/diffusion/models/dreamzero/pipeline_dreamzero.py", + "vllm_omni/diffusion/models/glm_image/pipeline_glm_image.py", + "vllm_omni/diffusion/models/gr00t/modeling/processing_gr00t_n1d7.py", + "vllm_omni/diffusion/models/hunyuan_video/pipeline_hunyuan_video_1_5.py", + "vllm_omni/diffusion/models/longcat_image/pipeline_longcat_image.py", + "vllm_omni/diffusion/models/longcat_image/pipeline_longcat_image_edit.py", + "vllm_omni/diffusion/models/longcat_video/pipeline_longcat_video_avatar.py", + "vllm_omni/diffusion/models/minimax_h3/encoder.py", + "vllm_omni/diffusion/models/nextstep_1_1/pipeline_nextstep_1_1.py", + "vllm_omni/diffusion/models/omnivoice/pipeline_omnivoice.py", + "vllm_omni/diffusion/models/soulx_singer/modules/preprocess/asr.py", + "vllm_omni/diffusion/models/soulx_singer/preprocess/g2p.py", + "vllm_omni/diffusion/profiler/diffusion_pipeline_profiler.py", + "vllm_omni/distributed/omni_connectors/utils/env.py", + "vllm_omni/entrypoints/cli/logo.py", + "vllm_omni/entrypoints/openai/serving_speech.py", + "vllm_omni/experimental/fullduplex/joyvl/memory/memory.py", + "vllm_omni/model_executor/models/common/whisper_vq.py", + "vllm_omni/model_executor/models/fish_speech/prompt_utils.py", + "vllm_omni/model_executor/models/glm_tts/text_frontend.py", + "vllm_omni/model_executor/models/higgs_audio_v2/higgs_audio_v2_tokenizer.py", + "vllm_omni/model_executor/models/indextts2/indextts2_talker.py", + "vllm_omni/model_executor/models/indextts2/text_processing_v2_5.py", + "vllm_omni/model_executor/models/indextts2/utils/common.py", + "vllm_omni/model_executor/models/ming_flash_omni/text_processing.py", + "vllm_omni/model_executor/models/ming_flash_omni/vision_encoder.py", + "vllm_omni/model_executor/models/ming_tts/prompt_assembly.py", + "vllm_omni/model_executor/models/nemotron_voicechat/nemo_vendored/ear_tts_commons.py", + "vllm_omni/model_executor/stage_input_processors/ming_flash_omni.py", + "vllm_omni/outputs/output_modality.py", + "vllm_omni/quantization/mxfp4_config.py", + "vllm_omni/quantization/tools/merge_mxfp4_dualscale_checkpoint.py", + "vllm_omni/utils/custom_voice_io.py", + }, + ), + "triton": ForbiddenImport( + pattern=r"^(from|import)\s+triton(\s|\.|$)", + tip="Use 'from vllm.triton_utils import triton' instead.", + allowed_pattern=re.compile(r"from vllm.triton_utils import (triton|tl|tl, triton)"), + ), + "tilelang": ForbiddenImport( + pattern=r"^(from|import)\s+tilelang(\s|\.|$)", + tip="Use 'from vllm.tilelang_utils import tilelang, T' instead.", + allowed_pattern=re.compile( + r"from\s+vllm\.tilelang_utils\s+import\s+" + r"(tilelang|T|T, tilelang|tilelang, T)\b" + ), + ), + "huggingface_hub repo API": ForbiddenImport( + # Catch `from huggingface_hub import `, including parenthesized, + # multi-line imports. + pattern=( + r"^\s*from\s+huggingface_hub\s+import\s*\([^)]*\b(?:" + _HF_NAMES + r")\b" + r"|" + r"^\s*from\s+huggingface_hub\s+import\b[^\n]*\b(?:" + _HF_NAMES + r")\b" + ), + tip=( + "Use the shared, vLLM-tagged helpers from " + "vllm.transformers_utils.repo_utils (e.g. hf_api(), hf_fs(), " + "list_repo_files, file_exists) instead of calling " + "huggingface_hub directly." + ), + allowed_dirs=_NON_LIBRARY_DIRS, + allowed_files={ + "vllm_omni/benchmarks/data_modules/daily_omni_dataset.py", + "vllm_omni/benchmarks/data_modules/seed_tts_dataset.py", + "vllm_omni/benchmarks/data_modules/seed_tts_eval.py", + "vllm_omni/diffusion/lora/loader.py", + "vllm_omni/diffusion/model_loader/hub_prefetch.py", + "vllm_omni/diffusion/models/cosmos3/sound_tokenizer.py", + "vllm_omni/diffusion/models/dreamzero/pipeline_dreamzero.py", + "vllm_omni/diffusion/models/helios/pipeline_helios.py", + "vllm_omni/diffusion/models/longcat_video/pipeline_longcat_video_avatar.py", + "vllm_omni/diffusion/models/ltx2/ltx2_components.py", + "vllm_omni/diffusion/models/magi_human/pipeline_magi_human.py", + "vllm_omni/diffusion/models/omnivoice/pipeline_omnivoice.py", + "vllm_omni/diffusion/models/pi0/pipeline_pi0.py", + "vllm_omni/diffusion/models/sensenova_u1/pipeline_sensenova_u1.py", + "vllm_omni/diffusion/models/soulx_singer/utils.py", + "vllm_omni/diffusion/models/utils.py", + "vllm_omni/diffusion/models/wan2_2/pipeline_wan2_2.py", + "vllm_omni/engine/arg_utils.py", + "vllm_omni/engine/stage_init_utils.py", + "vllm_omni/entrypoints/openai/api_server.py", + "vllm_omni/entrypoints/openai/serving_speech.py", + "vllm_omni/entrypoints/openai/tts_adapters/cosyvoice3.py", + "vllm_omni/experimental/fullduplex/personaplex/runtime.py", + "vllm_omni/experimental/fullduplex/personaplex/serving/server.py", + "vllm_omni/model_executor/model_loader/weight_utils.py", + "vllm_omni/model_executor/models/audex/checkpoint.py", + "vllm_omni/model_executor/models/bagel/bagel.py", + "vllm_omni/model_executor/models/cosyvoice3/cosyvoice3.py", + "vllm_omni/model_executor/models/covo_audio/covo_audio_code2wav.py", + "vllm_omni/model_executor/models/dots_tts/dots_tts_talker.py", + "vllm_omni/model_executor/models/dynin_omni/dynin_omni_common.py", + "vllm_omni/model_executor/models/dynin_omni/dynin_omni_token2audio.py", + "vllm_omni/model_executor/models/glm_tts/glm_tts.py", + "vllm_omni/model_executor/models/higgs_audio_v2/higgs_audio_v2_code2wav.py", + "vllm_omni/model_executor/models/higgs_audio_v2/higgs_audio_v2_tokenizer.py", + "vllm_omni/model_executor/models/higgs_audio_v3/higgs_audio_v3_code2wav.py", + "vllm_omni/model_executor/models/indextts2/preprocess_utils.py", + "vllm_omni/model_executor/models/indextts2/s2mel/modules/bigvgan.py", + "vllm_omni/model_executor/models/indextts2/tokenizer_v2_5.py", + "vllm_omni/model_executor/models/ming_tts/speaker_extractor.py", + "vllm_omni/model_executor/models/minicpmo_4_5/minicpmo_4_5_code2wav.py", + "vllm_omni/model_executor/models/omnivoice/omnivoice.py", + "vllm_omni/model_executor/models/personaplex/personaplex_mimi.py", + "vllm_omni/model_executor/models/step_audio2/step_audio2_token2wav.py", + "vllm_omni/model_executor/models/voxtral_tts/voxtral_tts.py", + "vllm_omni/transformers_utils/configs/higgs_audio_v3.py", + }, + ), +} + + +def _normalize_path(path: str) -> str: + return path.replace("\\", "/") + + +def check_file(path: str) -> int: + path = _normalize_path(path) + with open(path, encoding="utf-8") as f: + content = f.read() + return_code = 0 + # Check all patterns in the whole file + for import_name, forbidden_import in CHECK_IMPORTS.items(): + # Skip files that are allowed for this import + if path in forbidden_import.allowed_files: + continue + # Skip directories that are allowed for this import + if any(path.startswith(prefix) for prefix in forbidden_import.allowed_dirs): + continue + # Search for forbidden imports + for match in re.finditer(forbidden_import.pattern, content, re.MULTILINE): + # Check if it's allowed + if forbidden_import.allowed_pattern.match(match.group()): + continue + # Skip matches inside a comment + line_start = content.rfind("\n", 0, match.start()) + 1 + if "#" in content[line_start : match.start()]: + continue + # Calculate line number from match position + line_num = content[: match.start() + 1].count("\n") + 1 + print( + f"{path}:{line_num}: " + "\033[91merror:\033[0m " # red color + f"Found forbidden import: {import_name}. {forbidden_import.tip}" + ) + return_code = 1 + return return_code + + +def main(): + returncode = 0 + for path in sys.argv[1:]: + returncode |= check_file(path) + return returncode + + +def test_regex(): + def matches(rule: str, content: str) -> bool: + return bool(re.search(CHECK_IMPORTS[rule].pattern, content, re.MULTILINE)) + + pickle_cases = [ + # Should match + ("import pickle", True), + ("import cloudpickle", True), + ("import pickle as pkl", True), + ("import cloudpickle as cpkl", True), + ("from pickle import *", True), + ("from cloudpickle import dumps", True), + ("from pickle import dumps, loads", True), + ("from cloudpickle import (dumps, loads)", True), + (" import pickle", True), + ("\timport cloudpickle", True), + ("from pickle import loads", True), + ("import pickle, os", True), + ("import os, pickle", True), + ("import os, pickle as pkl", True), + ("import pickle as pkl, os", True), + ("import os.path, cloudpickle", True), + ("import cloudpickle, json", True), + # Should not match + ("import somethingelse", False), + ("from somethingelse import pickle", False), + ("# import pickle", False), + ("print('import pickle')", False), + ("import pickleas as asdf", False), + ] + for i, (content, should_match) in enumerate(pickle_cases): + result = matches("pickle/cloudpickle", content) + assert result == should_match, f"pickle case {i} failed: {content!r} (expected {should_match}, got {result})" + + tilelang_cases = [ + # Should match + ("import tilelang", True), + ("import tilelang.language as T", True), + ("from tilelang.jit import JITImpl", True), + ("from tilelang.jit.kernel import JITKernel", True), + # Should not match: indented (local) imports are allowed, mirroring + # the "triton" rule, so mocked-module test imports are not flagged. + (" import tilelang", False), + (" from tilelang.jit import JITImpl", False), + ("from vllm.tilelang_utils import tilelang", False), + ("from vllm.tilelang_utils import T", False), + ("from vllm.tilelang_utils import T, tilelang", False), + ("from vllm.tilelang_utils import tilelang, T", False), + ("import tilelang_kernels", False), + ] + for i, (content, should_match) in enumerate(tilelang_cases): + result = matches("tilelang", content) + assert result == should_match, f"tilelang case {i} failed: {content!r} (expected {should_match}, got {result})" + + hf_cases = [ + # Should match + ("from huggingface_hub import snapshot_download", True), + ("from huggingface_hub import hf_hub_download", True), + ("from huggingface_hub import HfApi", True), + ("from huggingface_hub import HfFileSystem", True), + ("from huggingface_hub import list_repo_files", True), + ("from huggingface_hub import try_to_load_from_cache", True), + (" from huggingface_hub import snapshot_download", True), + ("from huggingface_hub import PyTorchModelHubMixin, hf_hub_download", True), + ("from huggingface_hub import (snapshot_download)", True), + # Parenthesized multi-line import must not bypass the hook + ("from huggingface_hub import (\n snapshot_download,\n)", True), + ( + "from huggingface_hub import (\n PyTorchModelHubMixin,\n HfApi,\n)", + True, + ), + # Should not match + ("import huggingface_hub", False), + ("import huggingface_hub as hf", False), + ("from huggingface_hub import PyTorchModelHubMixin", False), + ("from huggingface_hub.constants import HF_HUB_CACHE", False), + ("from huggingface_hub.utils import EntryNotFoundError", False), + ("from vllm.transformers_utils.repo_utils import hf_api", False), + ("from huggingface_hub import (\n PyTorchModelHubMixin,\n)", False), + ("# from huggingface_hub import snapshot_download", False), + ] + for i, (content, should_match) in enumerate(hf_cases): + result = matches("huggingface_hub repo API", content) + assert result == should_match, ( + f"huggingface_hub case {i} failed: {content!r} (expected {should_match}, got {result})" + ) + + print("All regex tests passed.") + + +if __name__ == "__main__": + if "--test-regex" in sys.argv: + test_regex() + else: + sys.exit(main()) diff --git a/tools/pre_commit/check_pickle_imports.py b/tools/pre_commit/check_pickle_imports.py deleted file mode 100644 index f8f0ef8f2..000000000 --- a/tools/pre_commit/check_pickle_imports.py +++ /dev/null @@ -1,87 +0,0 @@ -#!/usr/bin/env python3 -# SPDX-License-Identifier: Apache-2.0 -# SPDX-FileCopyrightText: Copyright contributors to the vLLM project -import sys - -import regex as re - -# List of files (relative to repo root) that are allowed to import pickle or -# cloudpickle -# -# STOP AND READ BEFORE YOU ADD ANYTHING ELSE TO THIS LIST: -# The pickle and cloudpickle modules are known to be unsafe when deserializing -# data from potentially untrusted parties. They have resulted in multiple CVEs -# for vLLM and numerous vulnerabilities in the Python ecosystem more broadly. -# Before adding new uses of pickle/cloudpickle, please consider safer -# alternatives like msgpack or pydantic that are already in use in vLLM. Only -# add to this list if absolutely necessary and after careful security review. -ALLOWED_FILES = { - "tests/helpers/process.py", - "vllm_omni/diffusion/distributed/group_coordinator.py", - "tests/diffusion/attention/test_attention_sp.py", - # Measures the wire size of diffusion outputs. The transport pickles them, - # and only pickle reproduces the per-view storage duplication under test - # (torch.save deduplicates shared storages). No untrusted data is loaded. - "tests/diffusion/test_diffusion_ipc.py", -} - -PICKLE_RE = re.compile( - r"^\s*(import\s+(pickle|cloudpickle)(\s|$|\sas)" - r"|from\s+(pickle|cloudpickle)\s+import\b)" -) - - -def scan_file(path: str) -> int: - with open(path, encoding="utf-8") as f: - for i, line in enumerate(f, 1): - if PICKLE_RE.match(line): - print( - f"{path}:{i}: " - "\033[91merror:\033[0m " # red color - "Found pickle/cloudpickle import" - ) - return 1 - return 0 - - -def main(): - returncode = 0 - for filename in sys.argv[1:]: - if filename in ALLOWED_FILES: - continue - returncode |= scan_file(filename) - return returncode - - -def test_regex(): - test_cases = [ - # Should match - ("import pickle", True), - ("import cloudpickle", True), - ("import pickle as pkl", True), - ("import cloudpickle as cpkl", True), - ("from pickle import *", True), - ("from cloudpickle import dumps", True), - ("from pickle import dumps, loads", True), - ("from cloudpickle import (dumps, loads)", True), - (" import pickle", True), - ("\timport cloudpickle", True), - ("from pickle import loads", True), - # Should not match - ("import somethingelse", False), - ("from somethingelse import pickle", False), - ("# import pickle", False), - ("print('import pickle')", False), - ("import pickleas as asdf", False), - ] - for i, (line, should_match) in enumerate(test_cases): - result = bool(PICKLE_RE.match(line)) - assert result == should_match, f"Test case {i} failed: '{line}' (expected {should_match}, got {result})" - print("All regex tests passed.") - - -if __name__ == "__main__": - if "--test-regex" in sys.argv: - test_regex() - else: - sys.exit(main()) diff --git a/tools/pre_commit/check_spdx_header.py b/tools/pre_commit/check_spdx_header.py new file mode 100644 index 000000000..5bfb26034 --- /dev/null +++ b/tools/pre_commit/check_spdx_header.py @@ -0,0 +1,206 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: Apache-2.0 +# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project +"""Require SPDX license headers on source files. + +Ported from vllm/tools/pre_commit/check_spdx_header.py. The copyright line +uses the vLLM-Omni project name rather than upstream vLLM. Shell scripts +(``.sh``) are included so wrappers such as ``shellcheck.sh`` cannot keep a +stale upstream copyright that Python/Rust/proto files would have rewritten. +``.pyi`` stubs use the same ``#`` header style as ``.py``. +""" + +import sys +from dataclasses import dataclass +from enum import Enum +from pathlib import Path + +# Older files still carry the upstream vLLM copyright line. Treat it as stale +# and rewrite it to the Omni line instead of inserting a duplicate header. +LEGACY_COPYRIGHT_TEXT = "SPDX-FileCopyrightText: Copyright contributors to the vLLM project" + + +@dataclass(frozen=True) +class HeaderStyle: + """Comment syntax and preamble handling for an SPDX header.""" + + comment_prefix: str + preserve_shebang: bool = False + + +class SPDXStatus(Enum): + """SPDX header status enumeration""" + + EMPTY = "empty" # empty __init__.py + COMPLETE = "complete" + MISSING_LICENSE = "missing_license" # Only has copyright line + MISSING_COPYRIGHT = "missing_copyright" # Only has license line + MISSING_BOTH = "missing_both" # Completely missing + + +LICENSE_TEXT = "SPDX-License-Identifier: Apache-2.0" +COPYRIGHT_TEXT = "SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project" +FILE_STYLES = { + ".py": HeaderStyle("#", preserve_shebang=True), + ".pyi": HeaderStyle("#"), + ".sh": HeaderStyle("#", preserve_shebang=True), + ".rs": HeaderStyle("//"), + ".proto": HeaderStyle("//"), +} + + +def file_style(file_path): + """Return the declared header style for a file.""" + suffix = Path(file_path).suffix + try: + return FILE_STYLES[suffix] + except KeyError: + raise ValueError(f"Unsupported file type: {file_path}") from None + + +def spdx_header(style): + """Return the SPDX header for a file style.""" + license_line = f"{style.comment_prefix} {LICENSE_TEXT}" + copyright_line = f"{style.comment_prefix} {COPYRIGHT_TEXT}" + return license_line, copyright_line + + +def legacy_copyright_line(style): + """Return the stale upstream vLLM copyright line for a file style.""" + return f"{style.comment_prefix} {LEGACY_COPYRIGHT_TEXT}" + + +def header_insertion_index(style, lines): + """Return the line index where a missing header should be inserted.""" + if style.preserve_shebang and lines and lines[0].startswith("#!"): + return 1 + return 0 + + +def check_spdx_header_status(file_path): + """Check SPDX header status of the file""" + style = file_style(file_path) + license_line, copyright_line = spdx_header(style) + with open(file_path, encoding="UTF-8") as file: + lines = file.readlines() + if not lines: + # Empty file + return SPDXStatus.EMPTY + + has_license = False + has_copyright = False + + # Check all lines for SPDX headers (not just the first two) + for raw_line in lines: + line = raw_line.strip() + if line == license_line: + has_license = True + elif line == copyright_line: + has_copyright = True + + # Determine status based on what we found + if has_license and has_copyright: + return SPDXStatus.COMPLETE + elif has_license and not has_copyright: + # Only has license line (or a stale vLLM copyright line) + return SPDXStatus.MISSING_COPYRIGHT + elif not has_license and has_copyright: + # Only has copyright line + return SPDXStatus.MISSING_LICENSE + else: + # Completely missing both lines + return SPDXStatus.MISSING_BOTH + + +def add_header(file_path, status): + """Add or supplement SPDX header based on status""" + style = file_style(file_path) + license_line, copyright_line = spdx_header(style) + stale_copyright_line = legacy_copyright_line(style) + full_spdx_header = f"{license_line}\n{copyright_line}" + with open(file_path, "r+", encoding="UTF-8") as file: + lines = file.readlines() + file.seek(0, 0) + file.truncate() + + if status == SPDXStatus.MISSING_BOTH: + # Completely missing, add complete header + insertion_index = header_insertion_index(style, lines) + file.writelines(lines[:insertion_index]) + file.write(full_spdx_header + "\n") + remaining_lines = lines[insertion_index:] + if remaining_lines and remaining_lines[0].strip(): + file.write("\n") + file.writelines(remaining_lines) + + elif status == SPDXStatus.MISSING_COPYRIGHT: + # Only has license line, need to add copyright line. + # Rewrite a stale upstream vLLM copyright line in place. + replaced = False + for i, line in enumerate(lines): + if line.strip() == stale_copyright_line: + lines[i] = f"{copyright_line}\n" + replaced = True + break + if not replaced: + for i, line in enumerate(lines): + if line.strip() == license_line: + lines.insert(i + 1, f"{copyright_line}\n") + break + + file.writelines(lines) + + elif status == SPDXStatus.MISSING_LICENSE: + # Only has copyright line, need to add license line + # Find the copyright line and add license line before it + for i, line in enumerate(lines): + if line.strip() == copyright_line: + # Insert license line before copyright line + lines.insert(i, f"{license_line}\n") + break + file.writelines(lines) + + +def main(): + """Main function""" + files_missing_both = [] + files_missing_copyright = [] + files_missing_license = [] + + for file_path in sys.argv[1:]: + if Path(file_path).suffix not in FILE_STYLES: + continue + status = check_spdx_header_status(file_path) + + if status == SPDXStatus.MISSING_BOTH: + files_missing_both.append(file_path) + elif status == SPDXStatus.MISSING_COPYRIGHT: + files_missing_copyright.append(file_path) + elif status == SPDXStatus.MISSING_LICENSE: + files_missing_license.append(file_path) + else: + continue + + # Collect all files that need fixing + all_files_to_fix = files_missing_both + files_missing_copyright + files_missing_license + if all_files_to_fix: + print("The following files are missing the SPDX header:") + if files_missing_both: + for file_path in files_missing_both: + print(f" {file_path}") + add_header(file_path, SPDXStatus.MISSING_BOTH) + + if files_missing_copyright: + for file_path in files_missing_copyright: + print(f" {file_path}") + add_header(file_path, SPDXStatus.MISSING_COPYRIGHT) + if files_missing_license: + for file_path in files_missing_license: + print(f" {file_path}") + add_header(file_path, SPDXStatus.MISSING_LICENSE) + + sys.exit(1 if all_files_to_fix else 0) + + +if __name__ == "__main__": + main() diff --git a/tools/pre_commit/check_torch_cuda.py b/tools/pre_commit/check_torch_cuda.py new file mode 100644 index 000000000..a298baae5 --- /dev/null +++ b/tools/pre_commit/check_torch_cuda.py @@ -0,0 +1,109 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: Apache-2.0 +# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project +"""Ban direct torch.cuda.* usage outside an allowlist. + +Modeled on vllm/tools/pre_commit/check_torch_cuda.py. Call sites should use +current_omni_platform / OmniPlatform instead of CUDA-only torch.cuda APIs. +""" + +import sys + +import regex as re + +# Match `torch.cuda.xxx` device/memory/seed helpers that have portable Omni +# platform equivalents. CUDAGraph / is_available / stream capture are not in +# this list (same split as upstream vLLM). +_TORCH_CUDA_PATTERNS = [ + r"\btorch\.cuda\.(empty_cache|synchronize|device_count|current_device|memory_reserved|memory_allocated|max_memory_allocated|max_memory_reserved|reset_peak_memory_stats|memory_stats|mem_get_info|set_device)\b", + # Match torch.cuda.device(...) including args. `device\()\b` misses + # `device()` (no word char after `)`) and `device(0)`. + r"\btorch\.cuda\.device\s*\(", + r"\btorch\.cuda\.(manual_seed|manual_seed_all)\b", + r"\bwith\storch\.cuda\.device\b", + # Calls torch.cuda.{_is_compiled/_device_count_amdsmi/_device_count_nvml} internally + r"\bcuda_device_count_stateless\(\)\b", +] + +# Platform adapters must call native torch.cuda / torch.npu / torch.xpu APIs. +ALLOWED_PREFIXES = { + "vllm_omni/platforms/", +} + +# Grandfathered call sites. Remove a path after migrating it to +# current_omni_platform / OmniPlatform. Do not add new files without review. +ALLOWED_FILES = { + "benchmarks/kernels/mot_linear_benchmarks.py", + "examples/online_serving/qwen2_5_omni/gradio_demo.py", + "examples/online_serving/qwen3_omni/gradio_demo.py", + "tests/dfx/reliability/helpers.py", + "tests/diffusion/distributed/test_cfg_parallel.py", + "tests/diffusion/distributed/test_pipeline_parallel.py", + "tests/e2e/accuracy/wan22_i2v/run_wan22_i2v_diffusers_cp.py", + "tests/e2e/features/custom_pipeline/test_async_omni_collective_rpc.py", + "tests/e2e/features/fullduplex/test_personaplex_elastic.py", + "tests/entrypoints/test_omni_sleep_mode.py", + "tools/configure_stage_memory.py", + "vllm_omni/diffusion/models/bagel/pipeline_bagel.py", + "vllm_omni/diffusion/models/lance/pipeline_lance.py", + "vllm_omni/diffusion/models/magi_human/pipeline_magi_human.py", + "vllm_omni/diffusion/models/nextstep_1_1/pipeline_nextstep_1_1.py", + "vllm_omni/distributed/omni_connectors/connectors/mooncake_transfer_engine_connector.py", + "vllm_omni/distributed/omni_connectors/connectors/mori_transfer_engine_connector.py", + "vllm_omni/experimental/ar_diffusion/runner.py", + "vllm_omni/model_executor/models/moss_tts/cuda_graph_streaming_decoder_wrapper.py", + "vllm_omni/model_executor/models/moss_tts_nano/modeling_moss_tts_nano.py", + "vllm_omni/model_executor/models/qwen3_tts/cuda_graph_decoder_wrapper.py", + "vllm_omni/worker/gpu_ar_model_runner.py", +} + +_GENERIC_TIP = ( + "Found torch.cuda API call. Use current_omni_platform / OmniPlatform " + "(empty_cache, synchronize, get_device_count, current_device, " + "get_device_memory, set_device) instead." +) +_SEED_TIP = ( + "Found CUDA manual-seed API call. Use " + "torch.Generator(device=current_omni_platform.device_type).manual_seed(...) " + "instead." +) + + +def _normalize_path(path: str) -> str: + return path.replace("\\", "/") + + +def _is_allowed(path: str) -> bool: + if any(path.startswith(prefix) for prefix in ALLOWED_PREFIXES): + return True + return path in ALLOWED_FILES + + +def scan_file(path: str) -> int: + with open(path, encoding="utf-8") as f: + content = f.read() + for pattern in _TORCH_CUDA_PATTERNS: + for match in re.finditer(pattern, content, re.MULTILINE): + line_start = content.rfind("\n", 0, match.start()) + 1 + if "#" in content[line_start : match.start()]: + continue + line_num = content[: match.start() + 1].count("\n") + 1 + matched_text = match.group(0) + tip = _SEED_TIP if "manual_seed" in matched_text else _GENERIC_TIP + print(f"{path}:{line_num}: \033[91merror:\033[0m {tip}") + return 1 + return 0 + + +def main(): + returncode = 0 + for filename in sys.argv[1:]: + path = _normalize_path(filename) + if _is_allowed(path): + continue + returncode |= scan_file(path) + return returncode + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tools/pre_commit/mypy.py b/tools/pre_commit/mypy.py new file mode 100644 index 000000000..e24f62444 --- /dev/null +++ b/tools/pre_commit/mypy.py @@ -0,0 +1,97 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: Apache-2.0 +# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project +"""Run mypy on changed files. + +Ported from vllm/tools/pre_commit/mypy.py. Group files so tests can use +``follow_imports=skip`` without pulling that setting into library code. + +Usage: + python tools/pre_commit/mypy.py + +Args: + python_version: Python version to use (e.g. "3.10") or "local" to use + the interpreter running this script. + changed_files: List of changed files to check. +""" + +from __future__ import annotations + +import subprocess +import sys + +import regex as re + +# tests/ uses follow_imports=skip so third-party and library errors do not +# leak into test type-checking. After a directory is clean under skip, it +# can join the default group (follow_imports from pyproject.toml). +SEPARATE_GROUPS = [ + "tests", +] + +# Model trees are large third-party ports. Drop a prefix from this list when +# that directory is ready to be type-checked (same ratchet as upstream vLLM). +EXCLUDE = [ + r"vllm_omni/model_executor/models/", + r"vllm_omni/diffusion/models/", +] + + +def _normalize_path(path: str) -> str: + return path.replace("\\", "/") + + +def group_files(changed_files: list[str]) -> dict[str, list[str]]: + """Group changed files into different mypy calls.""" + exclude_pattern = re.compile(f"^{'|'.join(EXCLUDE)}.*") + file_groups: dict[str, list[str]] = {"": []} + file_groups.update({k: [] for k in SEPARATE_GROUPS}) + # Longest path first so a sub-directory is not shadowed by its parent. + separate_groups = sorted(SEPARATE_GROUPS, key=len, reverse=True) + for changed_file in changed_files: + changed_file = _normalize_path(changed_file) + if exclude_pattern.match(changed_file): + continue + for directory in separate_groups: + if re.match(f"^{directory}.*", changed_file): + file_groups[directory].append(changed_file) + break + else: + if changed_file.startswith(("vllm_omni/", "tests/")): + file_groups[""].append(changed_file) + return file_groups + + +def mypy( + targets: list[str], + python_version: str | None, + follow_imports: str | None, + file_group: str, +) -> int: + """Run mypy on the given targets.""" + args = ["mypy"] + if python_version is not None: + args += ["--python-version", python_version] + if follow_imports is not None: + args += ["--follow-imports", follow_imports] + print(f"$ {' '.join(args)} {file_group}") + return subprocess.run(args + targets, check=False).returncode + + +def main() -> int: + python_version = sys.argv[1] + file_groups = group_files(sys.argv[2:]) + + if python_version == "local": + python_version = f"{sys.version_info.major}.{sys.version_info.minor}" + + returncode = 0 + for file_group, changed_files in file_groups.items(): + follow_imports = None if file_group == "" else "skip" + if changed_files: + returncode |= mypy(changed_files, python_version, follow_imports, file_group) + return returncode + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tools/pre_commit/shellcheck.sh b/tools/pre_commit/shellcheck.sh new file mode 100755 index 000000000..7794ffe51 --- /dev/null +++ b/tools/pre_commit/shellcheck.sh @@ -0,0 +1,79 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: Apache-2.0 +# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project +# +# Lint bash scripts for undefined vars, quoting, and similar bugs. +# Do not download a binary: use a distro/package-manager install so the +# artifact is signed (apt/dnf/brew) or at least an explicit local install. +set -euo pipefail + +is_windows_exe() { + [[ "$1" == *.exe ]] +} + +is_git_bash() { + case "$(uname -s)" in + MINGW*|MSYS*|CYGWIN*) return 0 ;; + *) return 1 ;; + esac +} + +find_native_shellcheck() { + local cand + cand="$(command -v shellcheck 2>/dev/null || true)" + if [ -n "$cand" ] && ! is_windows_exe "$cand"; then + echo "$cand" + return 0 + fi + return 1 +} + +install_hint() { + echo "Please install shellcheck with your package manager, then re-run:" + echo " Debian/Ubuntu/WSL: sudo apt-get install shellcheck" + echo " Fedora: sudo dnf install ShellCheck" + echo " macOS: brew install shellcheck" + echo " Git Bash: scoop install shellcheck" + echo " https://github.com/koalaman/shellcheck?tab=readme-ov-file#installing" +} + +SHELLCHECK_BIN="" +if SHELLCHECK_BIN="$(find_native_shellcheck)"; then + : +elif is_git_bash && SHELLCHECK_BIN="$(command -v shellcheck.exe 2>/dev/null || true)" && [ -n "$SHELLCHECK_BIN" ]; then + : +else + install_hint + exit 1 +fi + +should_lint() { + local f="${1//\\//}" + f="${f#./}" + case "$f" in + *.sh) ;; + *) return 1 ;; + esac + git check-ignore -q "$f" && return 1 + return 0 +} + +run_shellcheck() { + local f + for f in "$@"; do + if should_lint "$f"; then + "$SHELLCHECK_BIN" -s bash "$f" + fi + done +} + +if [ "$#" -gt 0 ]; then + run_shellcheck "$@" + exit 0 +fi + +# Direct invocation with no args: lint every tracked *.sh. +while IFS= read -r -d '' f || [ -n "$f" ]; do + git check-ignore -q "$f" && continue + "$SHELLCHECK_BIN" -s bash "$f" +done < <(find . -path ./.git -prune -o -name "*.sh" -print0) diff --git a/vllm_omni/diffusion/attention/ops/minimax_h3_modulation.py b/vllm_omni/diffusion/attention/ops/minimax_h3_modulation.py index 53d3eb9b9..08e9e0e4a 100644 --- a/vllm_omni/diffusion/attention/ops/minimax_h3_modulation.py +++ b/vllm_omni/diffusion/attention/ops/minimax_h3_modulation.py @@ -1,11 +1,11 @@ # SPDX-License-Identifier: Apache-2.0 +# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project """Fused MiniMax H3 modulation with FP32 accumulation.""" from __future__ import annotations import torch -import triton -import triton.language as tl +from vllm.triton_utils import tl, triton @triton.jit diff --git a/vllm_omni/model_executor/models/indextts2/gpt/conformer_encoder.py b/vllm_omni/model_executor/models/indextts2/gpt/conformer_encoder.py index 67eebbab3..6ba4ff00b 100644 --- a/vllm_omni/model_executor/models/indextts2/gpt/conformer_encoder.py +++ b/vllm_omni/model_executor/models/indextts2/gpt/conformer_encoder.py @@ -1,5 +1,5 @@ # SPDX-License-Identifier: Apache-2.0 -# SPDX-FileCopyrightText: Copyright contributors to the vLLM project +# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project import torch import torch.nn as nn @@ -175,7 +175,7 @@ class ConformerEncoderLayer(nn.Module): instance. `PositionwiseFeedForward` instance can be used as the argument. conv_module (torch.nn.Module): Convolution module instance. - `ConvlutionModule` instance can be used as the argument. + `ConvolutionModule` instance can be used as the argument. dropout_rate (float): Dropout rate. normalize_before (bool): True: use layer_norm before each sub-block. diff --git a/vllm_omni/model_executor/models/minimax_music3/prompt.py b/vllm_omni/model_executor/models/minimax_music3/prompt.py index b9093eae9..25e18fd7e 100644 --- a/vllm_omni/model_executor/models/minimax_music3/prompt.py +++ b/vllm_omni/model_executor/models/minimax_music3/prompt.py @@ -1,5 +1,5 @@ # SPDX-License-Identifier: Apache-2.0 -# SPDX-FileCopyrightText: Copyright contributors to the vLLM project +# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project # # Prompt normalization follows the MiniMaxAI/MiniMax-Music3 input contract. # The transformations are reproduced byte-for-byte because any divergence @@ -19,9 +19,10 @@ Both rows therefore have identical length and identical audio-start position. from __future__ import annotations -import re from typing import Any +import regex as re + # Pinned by ``validate_tokenizer_ids``: a checkpoint whose tokenizer disagrees # is not this model, and silently producing noise is worse than failing. SPECIAL_TOKEN_IDS: dict[str, int] = {