diff --git a/.claude/skills/precheck-pr/SKILL.md b/.claude/skills/precheck-pr/SKILL.md index cd86ec4aa..4c8086029 100644 --- a/.claude/skills/precheck-pr/SKILL.md +++ b/.claude/skills/precheck-pr/SKILL.md @@ -1,6 +1,6 @@ --- name: precheck-pr -description: Self-check your branch before creating a PR — catch dead code, verify accuracy/perf claims, validate PR title format, and confirm merge readiness. Use when the user says "precheck", "self review", "pre-submit check", or "check my PR before I open it." Never posts to GitHub. +description: Self-check your branch before creating a PR — catch dead code, prevent new model-specific Python examples, verify accuracy/perf claims, validate PR title format, and confirm merge readiness. Use when the user says "precheck", "self review", "pre-submit check", or "check my PR before I open it." Never posts to GitHub. --- # PR Pre-Check @@ -65,6 +65,8 @@ Ask: "Quick mode or full mode?" Then walk the checklist for the detected PR type **Also run the Code-Quality sweep on every PR**, regardless of type or mode: the five diff-scoped checks in [references/code-quality.md](references/code-quality.md) — kwargs fragility, broad-except swallow, `Any`/wrong type hints, hot-path `.clone()`/`deepcopy`, and event-loop blocking — plus the advisory conventions (log level, structured logging, synchronization, cleanup, dependencies, naming) in [checklists.md](references/checklists.md). These count only lines the PR *adds* — the pre-existing backlog across the repo is out of scope. +**Also run the Examples-Policy check on every PR** using [references/examples-policy.md](references/examples-policy.md). Inspect only Python paths introduced by the diff. Treat a new model-, checkpoint-, vendor-, or family-specific Python example as blocking; do not report pre-existing example debt. + ### Step 5: Print Report ``` @@ -77,6 +79,7 @@ Pre-check report for ───────────────── ────── PR title format ✓ Code quality ⚠ 1 broad except, 2 Any hints + Examples policy ✗ new model-specific Python example PR desc integrity ✓ Registry/config ✓ Dead code ⚠ 2 warnings diff --git a/.claude/skills/precheck-pr/references/checklists.md b/.claude/skills/precheck-pr/references/checklists.md index a232403a7..b65a66713 100644 --- a/.claude/skills/precheck-pr/references/checklists.md +++ b/.claude/skills/precheck-pr/references/checklists.md @@ -47,6 +47,19 @@ Diff-scoped sweep for five fragility patterns that are pervasive in the existing Roll the ⚠/✗ counts into the report as a single **Code quality** dimension row. +--- + +## All PRs: Examples Policy (run on every PR, quick & full) + +Inspect Python files introduced under `examples/` using +[examples-policy.md](examples-policy.md). This is a diff-scoped ratchet: existing +model-specific examples may be edited or removed without triggering the check. + +- [ ] **No new model-specific Python examples:** the diff adds, copies, or renames no Python file whose path or behavior is specific to one model, checkpoint, vendor, or model family. ✗ for any violation. Route inference through a shared task/protocol entrypoint, model request behavior through `vllm_omni/model_extras`, and commands through task docs or `recipes/`. +- [ ] **New generic example code is justified:** any new task-neutral Python file under `examples/` is an actual user-facing entrypoint rather than reusable production logic, a benchmark, an app, a setup tool, or a reproducer. ⚠ when the placement is questionable. + +Roll the result into the report as a separate **Examples policy** dimension. + ### Conventions (eyeball — no grep) Judgment calls that don't grep cleanly — apply them while reading the diff, not as a mechanical check: diff --git a/.claude/skills/precheck-pr/references/examples-policy.md b/.claude/skills/precheck-pr/references/examples-policy.md new file mode 100644 index 000000000..b1b855ea0 --- /dev/null +++ b/.claude/skills/precheck-pr/references/examples-policy.md @@ -0,0 +1,67 @@ +# Examples Python Policy + +Prevent new model-specific Python examples without forcing cleanup of the +existing backlog. Run this check on every PR in quick and full modes. + +## Scope + +Set the merge base as in the main workflow, then list Python paths introduced +by the PR, including additions, copies, and renames: + +```bash +BASE="$(git merge-base HEAD origin/main)" +git diff --name-status --diff-filter=ACR --find-renames --find-copies \ + "${BASE}"...HEAD -- 'examples/**/*.py' +``` + +Inspect destination paths for copies and renames. Existing model-specific files +that are only modified or deleted are grandfathered and must not be reported. + +## Blocking classification + +Mark ✗ when a newly introduced Python file does any of the following: + +- Names or scopes a path to one model, checkpoint, vendor, or model family. +- Hard-codes a model identifier, model-specific prompt, request shape, output + conversion, or launch configuration behind an otherwise generic filename. +- Duplicates an existing task/protocol runner to demonstrate one model. +- Adds a per-model helper below a generic task hub such as `text_to_speech/`. + +Do not infer safety from a generic filename alone. For example, a file named +`text_to_audio.py` is still model-specific if it only implements one model's +contract. + +## Acceptable additions + +A new Python example may pass when it is a genuinely model-neutral, +user-facing task or protocol entrypoint. It must accept model selection through +configuration and keep model-specific behavior out of the script. + +Use ⚠ when the file is model-neutral but reusable logic appears misplaced under +`examples/`. Route code by purpose: + +| Code purpose | Preferred location | +|---|---| +| Model prompt, defaults, request/output adaptation | `vllm_omni/model_extras/` or production model modules | +| Runnable model command and validation evidence | Task documentation or `recipes/` | +| UI or interactive application | `apps/` | +| Benchmark or evaluator | `benchmarks/` | +| Regression reproducer | `tests/` | +| Download, conversion, or setup utility | `tools/` | + +If a model cannot use an existing shared runner, report the missing generic +capability. Do not solve the gap by adding a model-specific Python example. + +## Report format + +Add one row to the pre-check report: + +```text +Examples policy ✓ no new Python example paths +Examples policy ⚠ generic helper belongs in tools/ +Examples policy ✗ examples/offline_inference/new_model/end2end.py is model-specific +``` + +For a blocker, name the path, the model-specific behavior, and the appropriate +shared runner or destination. Do not propose deleting unrelated existing +examples as part of the PR.