docs(review): enforce configuration provenance

Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
[Perf][MiniCPM-O-4.5]Add CUDA Graph For HiFTGenerator (#5869 )
2026-08-09 22:39:57 +08:00 · 2026-08-09 22:29:49 +08:00 · 2026-08-09 22:18:24 +08:00 · 2026-08-09 14:08:43 +00:00 · 2026-08-09 21:30:05 +08:00 · 2026-08-09 20:53:17 +08:00
83 changed files with 4732 additions and 515 deletions
@@ -220,65 +220,70 @@ See `plan/voxcpm2_native_ar_design.md`.

 ### Steps

-1. **Register in `serving_speech.py`** — add all 5 points in a **single commit**;
-   partial integration causes hard-to-debug failures. This file is modified by every
-   model PR and is the most common source of rebase conflicts — see conflict note below.
+1. **Write one adapter** under `vllm_omni/entrypoints/openai/tts_adapters/`.
+   `serving_speech.py` should not need an edit — detection, stage discovery and
+   dispatch are all derived from what the adapter declares.

-   **Point 1** — stage constant (near the top, alongside the other `_*_TTS_MODEL_STAGES` sets):
+   Create `vllm_omni/entrypoints/openai/tts_adapters/your_model.py`:
   ```python
-   _YOUR_MODEL_TTS_MODEL_STAGES = {"your_stage_key"}
+   @register_tts_adapter
+   class YourModelAdapter(ARTTSAdapter):
+       name = "your_model"                              # registry key + log label
+       stage_keys = frozenset({"your_stage_key"})       # the deploy yaml's model_stage
+
+       def validate(self, request) -> str | None:
+           if not request.input or not request.input.strip():
+               return "Input text cannot be empty"
+           return None
+
+       async def build(self, request, sampling_params_list, has_inline_ref_audio):
+           params = {"text": [request.input]}
+           if request.voice is not None:
+               params["voice"] = [request.voice]
+           return PreparedRequest(
+               prompt={"prompt": request.input},
+               tts_params=params,
+               model_type=self.name,
+           )
   ```

-   **Point 2** — union into `_TTS_MODEL_STAGES`:
-   ```python
-   _TTS_MODEL_STAGES: set[str] = (
-       ...
-       | _YOUR_MODEL_TTS_MODEL_STAGES
-   )
-   ```
+   Then add the module to the import block at the bottom of
+   `tts_adapters/__init__.py` so it registers. That import line is the only shared
+   file a new model touches — which is also why the old rebase-conflict hotspot is
+   gone.

-   **Point 3** — model type detection in `_detect_tts_model_type()`:
-   ```python
-   if model_stage in _YOUR_MODEL_TTS_MODEL_STAGES:
-       return "your_model"
-   ```
+   > **Pure-diffusion TTS does not go through adapters yet.** Under
+   > `for_diffusion()`, `create_speech()` routes straight to
+   > `_create_diffusion_speech()` and never calls `validate()`/`build()`.
+   > `DiffusionTTSAdapter` is scaffolding with no production subclass, so logic
+   > placed in one would silently never run. Diffusion-engine models follow the
+   > existing diffusion path; wiring it through adapters is open work (#4855).

-   **Point 4** — validation dispatch in `_validate_tts_request()`:
-   ```python
-   if self._tts_model_type == "your_model":
-       return self._validate_your_model_request(request)
-   ```
+   **If a stage key alone cannot identify the model**, declare
+   `model_archs = frozenset({"YourModelForConditionalGeneration"})`; add
+   `arch_identifies_entry_stage = True` when the model owns no stage key at all
+   (Ming dense). For a rule that is not set membership, override `matches()` (see
+   `covo_audio.py`). For a genuine overlap with another adapter, give one an
+   explicit `detect_priority` — `test_tts_detection.py` fails on an unordered
+   overlap. Models that only serve speech in some topologies override
+   `stage_serves_speech()` (see `audex.py`).

-   **Point 5** — validation + parameter-builder methods:
-   ```python
-   def _validate_your_model_request(self, request) -> str | None:
-       if not request.input or not request.input.strip():
-           return "Input text cannot be empty"
-       return None
+   **Reuse shared helpers via `self.ctx.server`** rather than reimplementing them:
+   `_resolve_ref_audio`, `_apply_uploaded_speaker`, `_validate_ref_audio_format`,
+   `_max_instructions_length`. Read a comparable adapter first — `fish_speech.py`
+   (voice cloning), `higgs_audio_v3.py` (parameter-heavy), `moss_tts.py` (family
+   sharing a base class).

-   def _build_your_model_params(self, request) -> dict:
-       params = {"text": [request.input]}
-       if request.voice is not None:
-           params["voice"] = [request.voice]
-       return params
-   ```
-   Wire `_build_your_model_params` into `_create_tts_request()` alongside the other
-   model-specific param builders.
+   > **Do not add `self._tts_model_type == ...` branches to `serving_speech.py`.**
+   > Older models predate the adapter framework and still have them; they are being
+   > migrated out (RFC #4327, #4855). `tools/pre_commit/check_tts_adapter.py` is a
+   > ratchet on the remaining count and fails the commit if it grows. Behaviour that
+   > no adapter hook can express is a missing hook — propose it on the RFC.

-   > **Two dispatch patterns coexist**: Fish Speech uses a `self._is_fish_speech` boolean
-   > instance attribute checked before `elif self._is_tts`, while all newer models
-   > (CosyVoice3, MOSS-TTS-Nano) use the `_tts_model_type` string returned by
-   > `_detect_tts_model_type()`. For new models, always use the `_tts_model_type` string
-   > pattern — do not add new `_is_*` flags.
-
-   > **Unused variable rule**: only extract fields in `_build_your_model_params` that
-   > are actually forwarded to the model. Unused extractions fail `ruff F841`.
-   > For voice-cloning fields (`ref_audio` → `prompt_audio_path`, `ref_text` →
-   > `prompt_text`), add them to the param builder and verify they reach the model call.
-
-   **Rebase conflict note**: when rebasing onto `main` after another model was merged,
-   `serving_speech.py` will conflict. Resolution: always keep *both* the upstream
-   model's additions and your own — never discard either side.
+   > **Unused variable rule**: only extract fields in `build()` that are actually
+   > forwarded to the model. Unused extractions fail `ruff F841`. For voice-cloning
+   > fields (`ref_audio` -> `prompt_audio_path`, `ref_text` -> `prompt_text`), add
+   > them to the params and verify they reach the model call.

 2. **Handle model-specific parameters**:
   - Voice cloning: `ref_audio` encoding and prompt injection
@@ -295,7 +300,7 @@ import base64
 from pathlib import Path

 def build_voice_clone_prompt(ref_audio_path: str, text: str, codec) -> list:
-    """Build prompt with reference audio for voice cloning in serving_speech.py."""
+    """Build prompt with reference audio for voice cloning, called from the adapter."""
    audio_bytes = Path(ref_audio_path).read_bytes()
    codes = codec.encode(audio_bytes)  # Encode on CPU using model's codec (e.g., DAC)
    token_ids = [code + codec.vocab_offset for code in codes.flatten().tolist()]
@@ -321,7 +326,7 @@ Classify the model's **CI priority** first (high / medium / low). High-priority

 | Level | Location | Marker | CI pipeline | Notes |
 |-------|----------|--------|-------------|-------|
-| **L1** | `tests/model_executor/…`, `tests/entrypoints/openai_api/…`, stage-processor tests | `core_model` + `cpu` | `test-ready.yml` | Prompt assembly, async_chunk helpers, `serving_speech` validation — no GPU |
+| **L1** | `tests/model_executor/…`, `tests/entrypoints/openai_api/…`, stage-processor tests | `core_model` + `cpu` | `test-ready.yml` | Prompt assembly, async_chunk helpers, adapter validation — no GPU |
 | **L2** | `tests/e2e/online_serving/test_{slug}.py` | **`core_model` + `advanced_model`** (both on baseline smoke) + `tts` + `@hardware_test(...)` | `test-ready.yml` (`ready` label) | Default deploy smoke: single `/v1/audio/speech` or offline `OmniRunner` path |
 | **L3** | `tests/e2e/online_serving/test_{slug}.py` **and** `tests/e2e/offline_inference/test_{slug}.py` | Baseline smoke: **`core_model` + `advanced_model`**; heavier cases: `advanced_model` only (+ `tts`) | `test-merge.yml` **or** merged into nightly TTS function job | Streaming, voice clone, batch/queue, async_chunk |
 | **L4** | `tests/e2e/online_serving/test_{slug}_expansion.py`, optional offline expansion | `full_model` + `tts` | `test-nightly.yml` (`:full_moon: TTS · Function Test with L4`) | Feature matrix; perf → `tests/dfx/perf/tests/test_tts.json` |
@@ -366,7 +371,7 @@ See [vllm-omni-test skill](../vllm-omni-test/SKILL.md) § **Runtime send helpers

 ### Deliverables

- Updated `serving_speech.py` with all 5 integration points (single commit)
+- One adapter file under `tts_adapters/` plus its line in the package import block
 - Client scripts and server launcher under `examples/online_serving/text_to_speech/<model>/`
 - Gradio demo with streaming and voice cloning UI in the same dir
 - E2E tests per **Test Case Writing (CI Levels)** above (priority tier determines L1–L4 scope)
@@ -514,8 +519,8 @@ Use this checklist when integrating a new TTS model:
 - [ ] README.md written

 ### Phase 3: Online Serving
- [ ] All 5 `serving_speech.py` integration points added in one commit
- [ ] Only extract params in `_build_*_params` that are forwarded to the model call (ruff F841)
+- [ ] Adapter written under `tts_adapters/` and registered in the import block
+- [ ] Only extract params in `build()` that are forwarded to the model call (ruff F841)
 - [ ] Prompt builder handles text input correctly
 - [ ] Voice cloning works (if supported)
 - [ ] All response formats work (wav, mp3, flac, pcm)
@@ -0,0 +1,238 @@
+---
+name: review-pr
+description: Review pull requests and local branches for vllm-project/vllm-omni with a frozen snapshot, module-design ownership, feature-design overlays, targeted validation, and concise evidence-backed findings. Use for default, detailed, or repeat maintainer reviews; checking correctness, compatibility, tests, benchmarks, model additions, distributed changes, or breaking behavior; and identifying or explicitly requesting the most relevant code-owner reviewers. Use precheck-pr instead for an author's pre-submit self-check.
+---
+
+# Review vLLM-Omni Pull Requests
+
+Review like a maintainer: direct, selective, and focused on issues that CI does
+not prove. Prefer a few high-confidence findings over exhaustive commentary.
+Zero findings is a valid result.
+
+## Quality contract
+
+Make every finding:
+
+- **Correct:** prove a reachable failure, not a suspicion.
+- **Prioritized:** lead with merge blockers and high-impact defects.
+- **Actionable:** identify the smallest safe fix direction.
+- **Evidence-based:** cite code, tests, docs, CI, or measurements.
+- **Concise:** avoid review templates and repeated summaries.
+- **Calibrated:** match severity to user and maintainer impact.
+
+Do not report unrelated backlog, style already enforced by pre-commit, or a
+missing test that would not protect changed behavior.
+
+## Select the input and depth
+
+Use `vllm-project/vllm-omni` as the base repository. Accept its forks and local
+checkouts; use another skill for unrelated repositories.
+
+| Input | Review surface |
+| --- | --- |
+| PR number or URL | Frozen PR metadata, full diff, and relevant threads. |
+| Local branch/worktree | Frozen target-base SHA through committed, staged, unstaged, and in-scope untracked changes. |
+| Pre-filled context | Reuse supplied metadata; fetch only missing facts and the full diff. |
+
+Default to maintainer brevity. A detailed or audit request expands coverage and
+lists `path:line` findings, but keeps the same confidence and severity bar.
+
+## Reference guide
+
+Load references in review order: process, one primary module contract, matching
+feature designs, evidence checks, then delivery. Every file is linked directly
+below; do not load unrelated module references.
+
+Each concise reference links to the maintained
+[vLLM-Omni documentation](https://docs.vllm.ai/projects/vllm-omni/en/latest/).
+For branch-specific behavior, inspect the matching `docs/` file in the reviewed
+checkout first; use the published latest docs for current guidance and discovery.
+If docs and live code disagree, verify the code/tests and report the drift.
+
+### Review process
+
+| Reference | Read when |
+| --- | --- |
+| [review-execution.md](references/process/review-execution.md) | Every review; freeze inputs, inspect safely, and deliver against the same snapshot. |
+| [general-checks.md](references/process/general-checks.md) | Every review; apply repository-wide correctness and evidence rules. |
+| [design-contracts.md](references/process/design-contracts.md) | Every production review; resolve branch-local module and feature design status. |
+| [review-routing.md](references/process/review-routing.md) | After the diff census; select one primary module and conditional overlays. |
+
+### Primary module contract
+
+| Reference | Read when |
+| --- | --- |
+| [entrypoints.md](references/modules/entrypoints.md) | Offline/CLI/API ingress, validation, rendering, streaming, or sessions change. |
+| [configuration.md](references/modules/configuration.md) | Config construction, deploy/stage schema, defaults, registry, or topology changes. |
+| [input-output-modality.md](references/modules/input-output-modality.md) | Requests, messages, serialization, output types, accumulation, or completion change. |
+| [error-contracts.md](references/modules/error-contracts.md) | Error classification, fatality, propagation, sanitization, or rendering changes. |
+| [engine-orchestration.md](references/modules/engine-orchestration.md) | Cross-stage routing, request state, output ordering, RPC correlation, or terminal convergence changes. |
+| [stage-runtime.md](references/modules/stage-runtime.md) | Placement, startup, readiness, replica identity, affinity, membership, or shutdown changes. |
+| [omni-connector.md](references/modules/omni-connector.md) | Cross-stage/process/device/node transport or synchronization changes. |
+| [model-integration.md](references/modules/model-integration.md) | Registration, preprocessing, loading, runners, or model-specific execution changes. |
+| [ar-runtime.md](references/modules/ar-runtime.md) | AR scheduling, request/cache state, adapters, workers, or upstream vLLM semantics change. |
+| [diffusion.md](references/modules/diffusion.md) | Diffusion runtime, models, batching, parallelism, or offload changes. |
+| [execution-platforms.md](references/modules/execution-platforms.md) | Hardware selection, capabilities, vendor workers, kernels, or patches change. |
+| [cache-management.md](references/modules/cache-management.md) | Cache identity, reuse, validity, reset, eviction, or teardown changes. |
+| [quantization.md](references/modules/quantization.md) | Quantization selection, checkpoint metadata, layer mapping, precision, or constraints change. |
+| [observability.md](references/modules/observability.md) | Metrics, logs, units, labels, correlation, or lifecycle changes. |
+| [profiling.md](references/modules/profiling.md) | Profiling instrumentation, traces, start/stop lifecycle, or overhead changes. |
+| [benchmarking.md](references/modules/benchmarking.md) | Benchmark workload, metric calculation, CLI, or result metadata changes. |
+
+### Feature-design overlays
+
+| Reference | Read when |
+| --- | --- |
+| [runtime-stage-execution.md](references/features/runtime-stage-execution.md) | Disaggregated inference, async chunk/output/materialization, or prefix caching changes. |
+| [communication.md](references/features/communication.md) | A concrete OmniConnector backend or its deployment contract changes. |
+| [diffusion-acceleration.md](references/features/diffusion-acceleration.md) | Diffusion parallelism, attention, quantization, cache, batching, or offload changes. |
+| [infrastructure-performance.md](references/features/infrastructure-performance.md) | Metrics infrastructure or documented speech optimization stacks change. |
+
+### Evidence and quality checks
+
+| Reference | Read when |
+| --- | --- |
+| [model-addition-checklist.md](references/checks/model-addition-checklist.md) | A model, architecture, loader, processor, registry, or stage config is added. |
+| [perf-verification.md](references/checks/perf-verification.md) | The PR makes a latency, throughput, memory, or quality claim. |
+| [config-provenance.md](references/checks/config-provenance.md) | Changed or newly load-bearing behavior reads config, CLI, environment, legacy, or default values. |
+| [test-quality-evaluation.md](references/checks/test-quality-evaluation.md) | Tests change, are absent for risky code, or may not exercise production behavior. |
+| [tests-docs-checklist.md](references/checks/tests-docs-checklist.md) | Coverage, CI markers, examples, user docs, or PR evidence need review. |
+| [verification.md](references/checks/verification.md) | Hardware, a server, or a runnable affected path is available for active verification. |
+
+### Delivery and reviewer coordination
+
+| Reference | Read when |
+| --- | --- |
+| [maintainer-style-study.md](references/delivery/maintainer-style-study.md) | Findings are ready for concise maintainer-style delivery. |
+| [review-requests.md](references/delivery/review-requests.md) | The user asks to identify, suggest, request, or ping code-owner reviewers. |
+
+## Workflow
+
+### 1. Freeze and report the snapshot
+
+Pin the base and head before reading source or running validation. Within 60 seconds,
+report the pinned head, CI, mergeability, and preliminary findings in the host
+conversation. Do not wait for CI or post this update to GitHub.
+
+If the target changes while fetching, discard the evidence and retry once. If
+it changes again, report the churn and wait for a stable target.
+
+For a trusted PR head, materialize the pinned head in an isolated detached
+worktree. A worktree freezes identity but is not a security sandbox. Treat fork
+heads as untrusted unless the user and environment policy explicitly establish
+otherwise: execute them only in a disposable, secret-free sandbox with restricted
+filesystem, network, and resources; without one, use static SHA-addressed reads
+and CI evidence only. For a local review, freeze the committed, index, worktree,
+and NUL-safe in-scope untracked contents. Follow
+[review-execution.md](references/process/review-execution.md) for trust gates,
+state fingerprints, and byte-for-byte staleness checks.
+
+### 2. Build the diff census
+
+Group files into production code, tests, docs, configuration, build/CI, and
+generated artifacts. Map each changed production file and test group to the PR
+goal. Compare the title/body claims with the actual diff; use linked issues only
+when they define the contract or reproduction.
+
+Mark unrelated scope and unexplained generated artifacts. Do not infer behavior
+from the PR description without tracing the live code.
+
+### 3. Route from the live behavior
+
+Trace each claimed behavior through the changed producer to its live consumer,
+then use [design-contracts.md](references/process/design-contracts.md) and
+[review-routing.md](references/process/review-routing.md) to select one primary
+module contract, a second only for a real documented cross-boundary call path,
+and every matching feature-design and evidence overlay. Treat titles and paths
+as hints; live behavior and the frozen head's current design metadata are
+authoritative. For docs-, tests-, or CI-only changes, route to the production
+contract they protect or use only the applicable evidence checks.
+
+### 4. Run the blocker scan
+
+Apply every category in [general-checks.md](references/process/general-checks.md) before
+lower-priority comments.
+
+For each changed or newly load-bearing value or behavior, trace:
+
+```text
+public ingress -> validation/defaulting -> producer -> transformations
+  -> stage/worker/connector boundary -> final consumer -> terminal cleanup
+```
+
+Treat unchanged context as in scope when the diff makes it authoritative for a
+new consumer. When config, CLI, environment, legacy, or default sources can
+control that value, apply
+[config-provenance.md](references/checks/config-provenance.md) and prove the
+documented source at the final sink with distinct sentinel values.
+
+Cover every applicable offline/online, streaming/non-streaming, sync/async,
+feature-on/off, topology, and compatibility path. Search bounded callers and
+sibling implementations rather than assuming the changed hunk is the only path.
+
+### 5. Apply module and feature contracts
+
+Apply the reference set selected in step 3 and any matching repo-local skill.
+Read the exact module and feature pages in the frozen head, including status,
+ownership boundary, dependencies, candidate invariants, safe-change guide, and
+promotion gate. Candidate or draft rules are questions, not blockers, unless
+current code, tests, or policy enforce them. Inspect both sides of any config,
+registry, serialization, connector, cache, or stage boundary.
+
+When a diff adds or expands a helper, class, fallback, compatibility branch, or
+public behavior, run a subtraction pass: remove out-of-scope behavior and check
+whether each new abstraction can be deleted, merged, moved, or inlined.
+
+### 6. Verify the changed path
+
+Before each validation group, verify the frozen SHA plus the tracked, index,
+untracked, and ignored-file fingerprint, or recreate a pristine snapshot. On a
+trusted head or inside the required sandbox, run an import/version preflight,
+then the narrowest relevant tests and low-cost static checks. Bind every result
+to the head SHA, snapshot fingerprint, and environment fingerprint. Never run
+imports, tests, builds, hooks, or repo-configurable tooling from an untrusted
+head on the reviewer host.
+
+- Treat CI as status evidence; inspect only the first overlapping failure.
+- For docs-only changes, use diff hygiene, links/build checks, and bounded live
+  contract verification instead of dependency setup or pytest.
+- For hardware-dependent paths, run available static/CPU checks and name the
+  exact GPU/NPU gap. Never simulate device evidence.
+- For performance or accuracy claims, require comparable base/head runs with
+  the same environment, workload, warmup, repetitions, and quality criteria.
+
+Stop when each changed semantic path has a supported finding or an explicit
+no-issue conclusion. Do not search further only to increase confidence.
+
+### 7. Consolidate and deliver
+
+Verify each finding against the current diff, deduplicate by root cause, and
+order by severity.
+
+Re-read the remote head and reverify or recreate the pristine validation
+snapshot immediately before delivery. If either changed, mark the review stale
+and restart from the new snapshot.
+
+Return findings first. Use
+[maintainer-style-study.md](references/delivery/maintainer-style-study.md) to keep them
+direct and brief. Each finding must include an exact `path:line`, trigger or
+call path, current behavior, impact, and smallest fix direction. If there are
+no findings, say so briefly and name material validation gaps.
+
+Keep the review read-only unless the user explicitly authorizes posting. Do not
+submit `APPROVE`, `COMMENT`, or `REQUEST_CHANGES`, add labels, edit code, or push
+commits as an implied part of review.
+
+### 8. Optionally request focused owner reviews
+
+Only when the user asks to identify or request reviewers, read
+[review-requests.md](references/delivery/review-requests.md). Rank path-matched
+CODEOWNERS with the frozen module page's owners or required reviewers and
+documented governance expertise; propose one to three focused reviewers with an
+explicit contract rationale.
+
+Identifying or suggesting reviewers is read-only. Requesting reviewers or
+posting `@mention` comments changes external state and requires explicit user
+authorization. When authorized, recheck the head, deduplicate existing
+requests, and post at most one consolidated comment. Do not infer this
+permission from a request to review the code.
@@ -0,0 +1,4 @@
+interface:
+  display_name: "vLLM-Omni PR Review"
+  short_description: "Review vLLM-Omni PRs and route owner requests"
+  default_prompt: "Use $review-pr to review this vLLM-Omni pull request with owner-specific rules and evidence-backed findings."
@@ -0,0 +1,49 @@
+# Configuration Provenance
+
+Read this reference when changed or newly load-bearing behavior reads config,
+CLI, environment, legacy, alias, or default values. Always use it for endpoint,
+host, port, address, connector metadata, and `*_extra_config` consumers, even
+when the schema or documentation is unchanged.
+
+## Trace the source of truth
+
+For each semantic value, record one compact ledger:
+
+```text
+documented input -> parsed field -> competing sources and precedence
+  -> runtime projection/transport -> final consumer
+```
+
+- Search the exact field, environment variable, default literal, and aliases in
+  code, tests, docs, examples, and deployment files.
+- Treat unchanged producers and context lines as review scope when the diff
+  starts relying on them or makes their output authoritative.
+- Resolve the intended precedence from active branch-local docs, enforced
+  code/tests, and compatibility policy. Report a documented input that is
+  silently ignored or reinterpreted at a live sink.
+- Check every supported construction path and feature-off behavior without
+  inventing parity for unsupported paths.
+
+## Require discriminating evidence
+
+Give every competing source a distinct value so the test reveals which source
+wins. Prefer non-default sentinels; for example:
+
+```text
+documented config port = 25301
+environment/default port = 8998
+expected final endpoint = http://10.0.0.8:25301
+```
+
+Assert the final URL, connection target, runtime argument, or other live sink,
+not only an intermediate object. A test that changes the environment value to
+the expected configured value proves formatting, not configuration propagation.
+For a bug fix, show the discriminating test fails on the frozen base and passes
+on the reviewed head when the environment permits execution.
+
+## Finding bar
+
+A supported non-default value reaching the wrong endpoint, device, topology, or
+runtime behavior is a correctness or compatibility finding. A missing
+discriminating test alone is a validation gap unless repository policy requires
+that regression coverage for the change.
@@ -0,0 +1,121 @@
+# Model Addition Checklist
+
+Read this reference when a PR adds or expands a model architecture, pipeline,
+loader, processor, registry entry, or stage configuration.
+
+Official docs: [model contribution guides](https://docs.vllm.ai/projects/vllm-omni/en/latest/contributing/model/),
+[adding an omni model](https://docs.vllm.ai/projects/vllm-omni/en/latest/contributing/model/adding_omni_model/),
+[adding a diffusion model](https://docs.vllm.ai/projects/vllm-omni/en/latest/contributing/model/adding_diffusion_model/),
+[adding a TTS model](https://docs.vllm.ai/projects/vllm-omni/en/latest/contributing/model/adding_tts_model/),
+[supported models](https://docs.vllm.ai/projects/vllm-omni/en/latest/models/supported_models/),
+[diffusion features](https://docs.vllm.ai/projects/vllm-omni/en/latest/user_guide/diffusion_features/),
+and [feature compatibility](https://docs.vllm.ai/projects/vllm-omni/en/latest/user_guide/feature_compatibility/).
+
+## Match claims to the diff
+
+- Reconcile every claimed architecture, modality, variant, feature, and example
+  with changed files and registry entries.
+- Resolve config model types, architecture names, callable paths, exports, and
+  dependency extras against live symbols.
+- Check sibling generations sharing an architecture name are disambiguated by
+  a stable config predicate rather than import order or an accidental default.
+
+## Close the integration chain
+
+Trace:
+
+```text
+public model id -> config/pipeline selection -> registry -> loader/processor
+  -> stage inputs -> model execution -> stage/public output
+```
+
+Verify optional dependencies fail with an actionable message, weight names and
+dtypes map correctly, and each advertised serving mode reaches the production
+dispatcher. Confirm outputs are non-empty and valid for their modality, shape,
+sample rate, or response schema.
+
+## Keep support documentation truthful
+
+- Build an evidence-backed inventory of the new model's exact architecture and
+  checkpoint identifiers, modalities and tasks, hardware backends, serving
+  modes, and supported acceleration, parallelism, cache, offload, and
+  quantization features. Treat untested support as unknown, not supported.
+- Update `docs/models/supported_models.md` with the architecture, model family,
+  example checkpoints, and only the backends validated by the PR.
+- For a diffusion model, add it to the applicable ImageGen, VideoGen, or
+  AudioGen table in `docs/user_guide/diffusion_features.md`. Fill every feature
+  column according to its legend and document partial support or constraints.
+- Update `docs/user_guide/feature_compatibility.md` when the model changes valid
+  feature combinations, configuration constraints, examples, or limitations,
+  and update any other branch-local support table affected by the claims.
+- Cross-check every documented support mark against the registry, configuration,
+  production-path tests, and PR evidence; do not copy a sibling model's support
+  claims without validation.
+
+## Ship the model recipe
+
+- Require every new model PR to add or update one model-family recipe under
+  `recipes/<vendor>/` and its row in the `recipes/README.md` Available Recipes
+  table. Follow [`recipes/TEMPLATE.md`](../../../../../recipes/TEMPLATE.md) and
+  the layout rules in
+  [`recipes/README.md`](../../../../../recipes/README.md); the
+  external recipes repository is a structural reference, not a substitute.
+- Include only tested task, serving-mode, and hardware configurations. Keep the
+  exact model identifiers, flags, commands, verification, feature support,
+  constraints, and limitations consistent with code, tests, examples, and the
+  support documentation.
+- Cover each validated platform in its own hardware section. Use one recipe per
+  model family by default; justify a split when configurations cannot be kept
+  clear in one file, and do not add placeholder sections for untested platforms.
+
+## Compare the reference implementation
+
+Require every new model PR to include a head-versus-reference accuracy or
+quality and performance comparison:
+
+- Name the canonical implementation and pin its repository revision, package
+  version, configuration, and checkpoint. If it cannot run, explain why and use
+  the closest runnable reference; do not silently omit the comparison.
+- Match inputs or dataset, seed, output settings, precision or quantization,
+  hardware, software, warmup, repetitions, synchronization, and timing and
+  memory scope. Include exact commands and environment details for both sides.
+- Compare accuracy or output quality with a task-appropriate metric and explicit
+  tolerance. When no reliable metric exists, provide paired fixed-input outputs,
+  artifacts, and evaluation criteria, and explain expected numerical or
+  qualitative differences.
+- Report end-to-end latency, task-appropriate throughput, peak device memory,
+  run-to-run variability, and head/reference ratios at equivalent output quality.
+- Prefer a timing split for preprocessing or encoding, each model stage,
+  inter-stage transfer or communication, decoding or postprocessing, and output
+  assembly. Define the boundaries, account for overlap, reconcile the split with
+  end-to-end time, and explain any unavailable stage measurement.
+
+Use [perf-verification.md](perf-verification.md) for the comparison protocol and
+repository benchmark and profiling links.
+
+## Remove accidental surface
+
+Search bounded call sites for:
+
+- inference-dead training `forward()` paths, unused factories/wrappers, never-set
+  branch keys, and immediately discarded parameters;
+- duplicate payload strings, validation, or shape coercion across stages;
+- compatibility aliases in brand-new code without an existing caller;
+- private symbols re-exported as public API or module-level side effects.
+
+Keep code only when it has a distinct live caller, invariant, or compatibility
+contract. Prefer one typed producer-consumer schema to repeated string keys.
+
+## Require proportionate evidence
+
+- Run a representative production-path inference and assert output content, not
+  only process survival.
+- Require profiling or A/B tables when the PR makes performance, memory,
+  precision, or quality claims, or when a suspected hot-path/device bug needs
+  that evidence. Do not impose fixed utilization or regression thresholds that
+  the repository does not define.
+- Require focused registry/config/loading tests, a representative E2E path, and
+  user-facing model/usage documentation.
+
+Use [diffusion.md](../modules/diffusion.md) for diffusion models and
+[perf-verification.md](perf-verification.md) for quantitative claims.
@@ -0,0 +1,51 @@
+# Performance and Accuracy Verification
+
+Activate this reference when the PR adds a model or claims or intentionally
+changes latency, throughput, memory, scaling, precision, or output quality.
+
+Official docs: [profiling](https://docs.vllm.ai/projects/vllm-omni/en/latest/contributing/profiling/),
+[serving benchmarks](https://docs.vllm.ai/projects/vllm-omni/en/latest/cli/bench/serve/),
+and [metrics](https://docs.vllm.ai/projects/vllm-omni/en/latest/contributing/metrics/).
+
+## Comparable A/B contract
+
+Compare frozen base and head for an existing path. For a new model absent from
+base, compare head with a pinned canonical reference implementation. Use the same:
+
+- hardware, driver, software versions, model/checkpoint, and dependencies;
+- input or dataset, seed, precision, batch/concurrency, topology, and feature
+  flags;
+- warmup, measured repetitions, synchronization, timing scope, and memory
+  collection method.
+
+Keep exact commands and report variability rather than a cherry-picked run.
+Use the repository's benchmark when it covers the claim; otherwise use the
+smallest reproducible workload that reaches the changed production path.
+
+## Evidence table
+
+| Dimension | Typical evidence |
+| --- | --- |
+| Latency | End-to-end latency and, when available, TTFT plus a defined per-stage and transfer-time split. |
+| Throughput/scaling | Requests, tokens, frames, or audio duration per second across the claimed concurrency/topology. |
+| Memory | Peak allocated/reserved device memory and OOM boundary if relevant. |
+| Quality/accuracy | Repository metric or known-good output comparison with an explicit tolerance; paired samples when no metric exists. |
+
+Every optimization needs correctness or quality evidence. A faster result on a
+different workload, precision, seed, or topology is not a valid comparison.
+
+## Reviewer verification ladder
+
+1. Run base/head A/B on suitable hardware when affordable.
+2. If only one side can run, verify the script and request a comparable pair.
+3. If hardware or assets are unavailable, audit methodology, code path, and
+   contributor evidence; name the exact unverified claim.
+
+Classify discrepancies before reporting: implementation regression, benchmark
+bug, environmental drift, noise, or unsupported claim. Explain material
+regressions against the PR's stated goal or repository contract; do not invent
+universal percentage thresholds.
+
+Report claimed and measured values together, bind them to the frozen SHAs and
+environment, and keep unavailable hardware as a validation gap rather than a
+fabricated pass.
@@ -0,0 +1,42 @@
+# Test Quality Evaluation
+
+Read this reference when tests change, a risky production path has no test, or
+a test-only PR may not prove the intended behavior.
+
+Official docs: [test system overview](https://docs.vllm.ai/projects/vllm-omni/en/latest/contributing/ci/test_system_overview/)
+and [test writing guide](https://docs.vllm.ai/projects/vllm-omni/en/latest/contributing/ci/test_writing_guide/).
+
+## Static proof check
+
+For each changed semantic path, ask whether the test would fail if that behavior
+were reverted or broken. Check:
+
+- assertions pin the contract, not only non-null output, logs, or “no crash”;
+- the production dispatcher, registry, connector, or scheduler is reached
+  instead of the changed behavior being mocked away;
+- fakes preserve relevant types, MRO, shapes, devices, async behavior, and
+  lifecycle transitions;
+- configuration tests give competing config, CLI, environment, legacy, and
+  default sources distinct non-default values and assert the final consumer;
+  setting only the source read by the implementation does not prove precedence;
+- seeds, ordering, timing, synchronization, external services, and numeric
+  tolerances are deterministic or explicitly controlled;
+- normal, invalid, boundary, feature-off, failure/cancellation, and regression
+  paths are covered where the diff changes them;
+- run-level and domain markers place hardware/model tests in the intended CI
+  lane without silently skipping the contract.
+
+Map source symbols to tests with bounded `rg` searches; do not assume the test
+tree mirrors production paths.
+
+## Runtime check
+
+Run the narrowest affected tests supported by the environment. Record skipped
+hardware/model cases as gaps. Classify failures as code, test, infrastructure,
+or flaky before turning them into findings; one passing rerun does not erase a
+flakiness signal.
+
+Keep grades or matrices internal. Report only concrete code bugs or the one or
+two test defects that leave changed behavior materially unprotected. Use
+[tests-docs-checklist.md](tests-docs-checklist.md) for CI and documentation
+coverage.
@@ -0,0 +1,48 @@
+# Tests and Documentation Checklist
+
+Use this reference to connect changed behavior to coverage, CI selection,
+examples, documentation, and reproducible PR evidence.
+
+Official docs: [test writing guide](https://docs.vllm.ai/projects/vllm-omni/en/latest/contributing/ci/test_writing_guide/),
+[test execution guide](https://docs.vllm.ai/projects/vllm-omni/en/latest/contributing/ci/test_execution_guide/),
+and [documentation guide](https://docs.vllm.ai/projects/vllm-omni/en/latest/contributing/DOCS_GUIDE/).
+
+## Coverage packet
+
+For each high-risk change, record a compact internal packet:
+
+```text
+change and failure risk -> existing unit/E2E coverage -> uncovered boundary
+  -> smallest test and stable assertion that closes the gap
+```
+
+Prefer a regression test that fails on the frozen base for bug fixes. Check
+numeric tolerances, deterministic inputs, realistic mocks, hardware/domain
+markers, and Buildkite/CI wiring using the documentation present on the reviewed
+branch. A skipped test is not a pass.
+
+## Documentation sync
+
+Require user-facing updates when the diff changes a model, feature, CLI/API,
+config key, default, compatibility behavior, or supported platform. Verify:
+
+- the actual docs navigation and support tables used by the branch;
+- exact identifiers, defaults, constraints, known limitations, and migration;
+- a minimal runnable example for newly exposed behavior;
+- examples and docs agree on commands, parameters, output schema, and supported
+  modes;
+- links and the bounded live contract build or resolve correctly.
+
+Do not request docs for internal behavior with no user-visible contract.
+
+## PR evidence
+
+For contributor-run tests or benchmarks, require enough provenance to evaluate
+the claim: hardware, relevant software versions, exact commands, inputs/model,
+result, and approximate runtime. Performance or accuracy evidence must compare
+base and head under equivalent conditions; use
+[perf-verification.md](perf-verification.md).
+
+Turn a missing test or document into a finding only when it protects or explains
+changed behavior. Name the smallest useful addition rather than requesting a
+broad coverage expansion.
@@ -0,0 +1,36 @@
+# Active Verification
+
+Read this reference when a runnable affected path and suitable local or remote
+hardware are available. Never claim device evidence from static analysis or a
+simulated backend.
+
+Official docs: [installation](https://docs.vllm.ai/projects/vllm-omni/en/latest/getting_started/installation/)
+and [test execution guide](https://docs.vllm.ai/projects/vllm-omni/en/latest/contributing/ci/test_execution_guide/).
+
+## Choose the narrowest level
+
+| Available environment | Verification |
+| --- | --- |
+| Matching accelerator, model/assets, and server | Targeted unit/integration test plus one representative E2E request. |
+| Matching accelerator without serving setup | Targeted model/runtime test and output or metric inspection. |
+| CPU/static environment only | Import/version preflight, focused CPU tests, config/registry resolution, and static contract checks. |
+| Missing dependencies or inaccessible hardware | Record the exact gap and evaluate contributor/CI evidence. |
+
+## Execute safely
+
+1. Bind the run to the frozen head SHA and record Python, platform, dependency,
+   model, device, and relevant environment fingerprints.
+2. Run the smallest affected unit test before an E2E workload.
+3. Exercise the changed public or production-dispatch path with bounded inputs.
+4. Compare actual output, metrics, cleanup, and exit behavior with the PR claim.
+5. For base/head claims, use the equivalent A/B protocol in
+   [perf-verification.md](perf-verification.md).
+
+Avoid disturbing shared servers, ports, models, or accelerators without user
+authorization. Do not install large dependencies, download gated assets, or
+start remote workloads merely because hardware might exist.
+
+Report the command, environment, result, and any skipped path. Classify failures
+as change-induced, pre-existing, infrastructure, environment, or flaky before
+making a finding. A hardware gap lowers confidence; it is not automatic proof
+of a defect.
@@ -0,0 +1,105 @@
+# Maintainer Review Style
+
+Use this reference only when turning proved findings into comments. The baseline
+maintainer samples favor short, direct comments and reserve long explanations
+for architecture or non-obvious failure paths.
+
+Official docs: [contributing and review process](https://docs.vllm.ai/projects/vllm-omni/en/latest/contributing/).
+
+## Evidence basis
+
+This calibration used 928 PRs merged from 2026-05-10 through 2026-08-08.
+Candidates were ranked by substantive human inline-review threads, then by
+distinct reviewers and review rounds. Author self-comments, bots and AI
+reviewers, and bodyless approvals were excluded. Counts selected the sample;
+the practices below came from reading root comments and follow-up replies.
+
+The main set contains the six leaders after filtering; three other high-ranked
+PRs add core-runtime and benchmark coverage. All were merged.
+
+| PR | Review boundary |
+| --- | --- |
+| [#2906](https://github.com/vllm-project/vllm-omni/pull/2906) | New TTS model, shared utilities, recipes, and request state. |
+| [#3907](https://github.com/vllm-project/vllm-omni/pull/3907) | Full-duplex API, session lifecycle, concurrency, and realtime evidence. |
+| [#2162](https://github.com/vllm-project/vllm-omni/pull/2162) | Diffusion world model, OpenPI serving, tests, examples, and docs. |
+| [#3454](https://github.com/vllm-project/vllm-omni/pull/3454) | Diffusion model, generic CLI surface, loaders, and configuration scope. |
+| [#4425](https://github.com/vllm-project/vllm-omni/pull/4425) | Structured configuration, typing, defaults, and compatibility. |
+| [#5397](https://github.com/vllm-project/vllm-omni/pull/5397) | Distributed offload, collectives, failure handling, and load semantics. |
+| [#4281](https://github.com/vllm-project/vllm-omni/pull/4281) | Composable parallelism, configuration contracts, and examples. |
+| [#3628](https://github.com/vllm-project/vllm-omni/pull/3628) | Stage metrics, benchmark definitions, opt-in API behavior, and cleanup. |
+| [#3855](https://github.com/vllm-project/vllm-omni/pull/3855) | Stage runtime, thread safety, startup, and compatibility. |
+
+Historical comments calibrate delivery; they do not create merge policy. Live
+code, tests, repository docs, and the current `CODEOWNERS` file remain
+authoritative. When refreshing this study, promote a practice only when it
+recurs across at least three PRs or is supported by current repository guidance.
+
+## Recurring review behavior
+
+- Start from a changed contract, not a style preference. State the exact
+  request, configuration, topology, or failure path that triggers the issue.
+- Protect repository boundaries. Keep model-specific behavior in its domain,
+  move genuinely reusable logic to a shared module, and reuse existing loaders,
+  protocols, definitions, and config sources instead of bypassing them.
+- Challenge unrelated scope and abstractions. Ask why a new flag, config, class,
+  helper, fallback, or file is needed when an existing path appears sufficient.
+- Trace concurrency and terminal paths: overlapping requests, disconnect,
+  cancellation, partial failure, stale replies, collective schedule mismatch,
+  cleanup, and retry. Prefer an explicit startup error for unsupported
+  combinations over silently running with incomplete state or wrong weights.
+- Preserve user-visible behavior across variants. An accepted CLI/API/config
+  field must be applied, explicitly rejected, or visibly warned about; compare
+  normal and headless, offline and online, and streaming and non-streaming paths.
+- Ask for evidence at the affected boundary: a regression test using the real
+  tensor or protocol shape, correct pytest markers so CI runs it, comparable A/B
+  performance results, or an SLO-oriented latency/quality measurement.
+- Keep interfaces explicit: avoid `Any` and opaque strings where a precise
+  union, enum, or typed envelope exists; remove dead code and duplicated
+  constants; validate requests before expensive execution.
+- Document module scope and non-obvious invariants. Update examples, recipes,
+  supported-model/feature docs, and hardware assumptions when users rely on
+  them.
+- When expertise crosses an ownership boundary, ask a named owner to inspect a
+  specific file or contract and explain why. Do not send a generic `PTAL` list.
+- Keep non-blocking cleanup in a linked follow-up only after the current PR is
+  safe. If a revision fixes only part of a failure chain, continue the existing
+  thread with the remaining trigger instead of opening a duplicate finding.
+
+## Write the finding
+
+- Lead with the issue or decisive question; omit a review preamble.
+- Name the concrete trigger and impact, then the smallest fix direction.
+- Keep obvious fixes to one sentence. Use more detail only when the call path or
+  ownership boundary is otherwise unclear.
+- Prefer one root-cause comment over repeated symptom comments.
+- Hedge only when evidence is genuinely incomplete; mark that item as a
+  question or validation gap rather than overstating severity.
+- Say briefly that no findings were found when the review is clean; zero
+  comments is a valid outcome when posting is authorized.
+
+Useful forms:
+
+```text
+This leaves <state> allocated when <failure/cancellation path>. Release it in
+the shared terminal cleanup path.
+```
+
+```text
+Does this producer also update <consumer>? It still reads the old field here.
+```
+
+```text
+With <configuration>, this bypasses <existing loader/validator>, so <impact>.
+Reuse that path or reject this combination before startup.
+```
+
+```text
+@<owner>, could you check <specific boundary>? This changes <contract> used by
+<consumer>.
+```
+
+Use a priority label only for a proved high-impact defect. Avoid generic praise,
+routine “Nit:” prefixes, decorative severity labels, audit templates, rule IDs,
+comment-count narration, and “I left a few comments.” Keep approvals, review
+events, reviewer requests, mentions, and GitHub replies subject to the user's
+explicit authorization.
@@ -0,0 +1,99 @@
+# Focused Code-Owner Review Requests
+
+Read only when the user asks to identify, suggest, request, add, or ping
+reviewers. Finding candidates is read-only; GitHub review requests and
+`@mention` comments are external writes.
+
+Owner sources, in order:
+
+1. the reviewed head's
+   [CODEOWNERS](https://github.com/vllm-project/vllm-omni/blob/main/.github/CODEOWNERS)
+   for path ownership;
+2. the active module design page's `owners` and `required_reviewers` metadata
+   for contract ownership and mandatory design review; and
+3. documented
+   [governance expertise](https://docs.vllm.ai/projects/vllm-omni/en/latest/community/governance/)
+   for specialization and tie-breaking.
+
+Resolve all three from the reviewed head when available because ownership
+changes over time. Use published docs only to discover a page absent from an
+older checkout, never to override the frozen head, and do not copy a static
+owner list into review output. CODEOWNERS remains the path authority; module
+metadata identifies contract experts and required design reviewers.
+
+## Rank candidates
+
+1. Freeze the PR head and collect changed files, author, requested reviewers,
+   submitted reviews, and existing comments.
+2. Resolve the **last matching CODEOWNERS rule** for each changed file. Do not
+   union an earlier broad rule with a later specific rule.
+3. Resolve the primary module and feature overlays through
+   [design-contracts.md](../process/design-contracts.md). Read the active module
+   page's status, `owners`, and `required_reviewers`; do not treat owners on an
+   archived, superseded, or unrelated page as current authority.
+4. Group files by the selected primary owner and the exact module or feature
+   contract needing review. Weight production owners and live consumers above
+   tests/docs/CI owners and the repository-wide fallback. Include an applicable
+   `required_reviewers` entry even when it is not the path owner, and label why.
+5. Use documented governance specialties only to choose among matching owners:
+   for example serving versus configuration, diffusion model versus cache or
+   parallelism, TTS versus generic model execution, or CI versus tests.
+6. Exclude the PR author, bots, users who already completed the needed review,
+   and duplicate pending requests. Do not infer availability or invent
+   expertise from a username.
+7. Select one reviewer for a single-owner change and at most three for a real
+   cross-module change. If more are plausible, rank them but do not ping a
+   broad group; ask the user before exceeding three.
+
+If every specific owner is excluded, consider one repository-wide fallback
+owner and label that choice as a fallback rather than a path match. Never fan
+out to the full fallback list. Prefer a qualified owner who covers multiple
+critical path groups over separate reviewers for each file. If CODEOWNERS is
+missing or invalid, state that gap and use governance only as a labeled fallback.
+
+Recent file history may break a tie only when CODEOWNERS and governance remain
+ambiguous. Never use commit count alone to override an explicit current owner.
+
+## Preview before writing
+
+Show the user the exact candidate set and rationale:
+
+```text
+@owner — <owned paths or module role>; please verify <specific module or feature contract>.
+```
+
+Draft one consolidated comment tied to the frozen head:
+
+```text
+Requesting focused review for `<short-head-sha>`:
+
+- @owner1: could you check the <module contract> in `<path/group>`?
+- @owner2: could you check the <feature or cross-boundary contract> in `<path/group>`?
+
+Relevant validation: <one concise result or exact remaining gap>.
+```
+
+Mention only owners being requested. Keep the reason specific enough that each
+person knows what decision or invariant needs attention; do not post a generic
+“please review” or a generated ownership audit.
+
+## External-write gate
+
+- “Who should review?”, “suggest reviewers”, or “draft a request” authorizes
+  only the preview.
+- “Request/add these reviewers” authorizes the GitHub review-request mutation.
+- “Ping/comment/tag the owners” authorizes the consolidated comment.
+- When the user explicitly asks to request reviews **with an owner comment**,
+  both writes are authorized. Obey narrower wording when only one action is
+  requested.
+
+Immediately before an authorized write, verify the PR is still open, the head
+SHA is unchanged, and the candidate is not the author or already satisfied.
+Report draft/WIP state and avoid notifications unless the user explicitly wants
+reviewers requested before ready-for-review.
+
+Use the GitHub connector for reviewer requests and the single top-level comment;
+use `gh` only when connector coverage is unavailable. Do not submit an approval,
+request-changes event, label, assignment, or multiple comments as part of this
+action. After writing, reread the PR and report the reviewers requested and the
+comment URL. If the head changed, discard the draft and rerank before posting.
@@ -0,0 +1,24 @@
+# Communication Features
+
+Use with the OmniConnector, stage-runtime, configuration, and I/O module
+contracts. Load the concrete backend design from the reviewed head:
+
+| Backend | Design |
+| --- | --- |
+| Mooncake store | [MooncakeStoreConnector](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/feature/omni_connectors/mooncake_store_connector/) |
+| Mooncake transfer engine | [MooncakeTransferEngineConnector](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/feature/omni_connectors/mooncake_transfer_engine_connector/) |
+| Mori transfer engine | [MoriTransferEngineConnector](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/feature/omni_connectors/mori_transfer_engine_connector/) |
+| Shared memory | [SharedMemoryConnector](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/feature/omni_connectors/shared_memory_connector/) |
+| Yuanrong store | [YuanrongConnector](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/feature/omni_connectors/yuanrong_connector/) |
+| Yuanrong transfer engine | [YuanrongTransferEngineConnector](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/feature/omni_connectors/yuanrong_transfer_engine_connector/) |
+
+## Feature checks
+
+- Verify dependency, service, device/network, port, memory-pool, and environment
+  prerequisites with actionable startup errors.
+- Match key/metadata construction, sender and receiver roles, allocation,
+  ownership, completion, and supported fast/serialized paths.
+- Exercise timeout, retry, stale sender/buffer reclamation, partial transfer,
+  cancellation, and connector close semantics.
+- Validate the smallest supported same-host or cross-node topology and document
+  backend limitations and operational diagnostics.
@@ -0,0 +1,26 @@
+# Diffusion Acceleration Features
+
+Use with the exact diffusion submodule plus platform, cache, quantization, or
+benchmarking contracts selected by the live path.
+
+| Feature family | Designs |
+| --- | --- |
+| Parallelism | [CFG](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/feature/cfg_parallel/), [Expert](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/feature/expert_parallel/), [HSDP](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/feature/hsdp/), [Pipeline](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/feature/pipeline_parallel/), [Sequence](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/feature/sequence_parallel/), [Tensor](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/feature/tensor_parallel/), [VAE patch](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/feature/vae_parallel/) |
+| Attention | [Skip-Softmax](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/feature/skip_softmax/) |
+| Quantization | [Quantization](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/feature/quantization/) |
+| Cache | [Cache-DiT](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/feature/cache_dit/), [TeaCache](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/feature/teacache/) |
+| Batching | [Diffusion Continuous Batching](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/feature/diffusion_continuous_batching/) |
+| Offload | [Distributed Layerwise Offload](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/feature/distributed_layerwise_offload/) |
+
+## Feature checks
+
+- Verify feature selection, supported models/platforms/topologies, defaults,
+  dependencies, and every touched feature combination.
+- Preserve feature-off and single-rank correctness; reject unsupported
+  combinations before collectives, loading, or execution begin.
+- Trace shapes, shards, ranks, collectives, precision, cache validity, residency,
+  and per-request state through the selected runtime hooks.
+- Compare output quality or accuracy against the disabled/reference path before
+  accepting latency or memory improvements.
+- Require exact commands, topology, environment, warmup, repetitions, and
+  measurements for performance claims; update feature compatibility docs.
@@ -0,0 +1,22 @@
+# Infrastructure and Performance Features
+
+Load the matching design from the reviewed head:
+
+- [Prometheus Metrics](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/metrics/)
+  for logger wrapping, gating, lifecycle, stage transfer, and metric definitions.
+- [Speech Generation Performance Optimizations](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/qwen3_omni_tts_performance_optimization/)
+  for stage batching, CUDA graphs, async chunk, streaming, re-prefill, compile,
+  and deployment evidence.
+
+## Feature checks
+
+- Route instrumentation changes through observability or profiling and result
+  methodology through benchmarking and the performance evidence check.
+- Preserve opt-in/default behavior, units, labels, aggregation, timing scope,
+  lifecycle resets, and bounded overhead.
+- For stacked optimizations, measure each increment against the same baseline
+  and explain interactions, quality guards, regressions, and limitations.
+- Require exact model, checkpoint, hardware, software, workload, topology,
+  concurrency, warmup, repetitions, and production commands.
+- Update the design page when metric meaning, supported optimization stack, or
+  deployment guidance changes.
@@ -0,0 +1,24 @@
+# Runtime and Stage Execution Features
+
+Load the matching design from the reviewed head:
+
+| Feature | Design | Primary module intersections |
+| --- | --- | --- |
+| Disaggregated inference | [Design](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/feature/disaggregated_inference/) | Configuration, stage runtime, OmniConnector, I/O contracts |
+| Async chunk | [Design](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/feature/async_chunk/) | Model integration, orchestration, stage runtime, connector, I/O |
+| Async diffusion output | [Design](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/feature/async_diffusion_output/) | Diffusion runtime, I/O contracts, entrypoints |
+| Async Omni output materialization | [Design](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/feature/omni_async_output_materialization/) | AR runtime, model integration, I/O contracts, orchestration |
+| Automatic prefix caching | [Design](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/feature/prefix_caching/) | AR runtime, cache management, I/O contracts |
+
+## Feature checks
+
+- Confirm the feature's supported models, entrypoints, stage types, topology,
+  configuration, default, and compatibility limitations.
+- Trace enablement from public/config ingress through every participating module
+  and verify the advertised production dispatch actually selects it.
+- Preserve a correct disabled or fallback path and make unsupported combinations
+  fail or warn as documented.
+- Test overlap, ordering, identity, backpressure, cancellation, failure, final
+  partial output, and cleanup at each asynchronous boundary.
+- Update the feature page, support matrices, examples, and recipes when behavior
+  or coverage changes.
@@ -0,0 +1,22 @@
+# Autoregressive Runtime
+
+Primary design: [Autoregressive Runtime](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/module/ar_runtime/).
+
+Use for AR request queues, scheduling state, token and memory budgets, KV/input
+readiness, workers, runners, and upstream vLLM request/cache transitions.
+
+## Contract checks
+
+- Preserve upstream request-state, scheduling, cache, and preemption semantics
+  unless an Omni-specific difference is documented and tested.
+- Convert modality-specific data through explicit input or output adapters; do
+  not inject it through unrelated scheduler state.
+- Keep workers and model runners focused on assigned execution rather than
+  admission or cross-stage routing.
+- Synchronize token, sequence, batch, and memory budgets with worker-visible
+  state and prevent starvation, double counting, or stale readiness.
+- Preserve legal transitions and one terminal cleanup path across completion,
+  abort, timeout, preemption, worker failure, and shutdown.
+
+Test request lifecycle, scheduling decisions, abort, cache state, adapters, and
+every affected worker execution mode against the pinned upstream contract.
@@ -0,0 +1,24 @@
+# Benchmarking
+
+Primary design: [Benchmarking](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/module/benchmarking/).
+
+Use when benchmark code, workloads, metric calculations, CLI benchmark
+entrypoints, or result metadata change. Use the performance check separately
+when a PR uses benchmark results as evidence.
+
+## Contract checks
+
+- Record model and revision, code SHA, hardware, software, configuration,
+  workload, precision, topology, concurrency, warmup, repetitions, and
+  measurement window.
+- Validate successful and semantically acceptable outputs before accepting
+  timing, memory, or scaling results.
+- Define every metric's unit, population, aggregation, synchronization, and
+  timing scope.
+- Keep workload parsing and metric computation deterministic and reusable
+  across base and head.
+- Reject comparisons that change material workload or runtime settings without
+  isolating and explaining that difference.
+
+Test workload parsing and metric calculations with deterministic fixtures and
+publish exact commands plus variability for quantitative claims.
@@ -0,0 +1,22 @@
+# Cache Management
+
+Primary design: [Cache Management](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/module/cache_management/).
+
+Use for AR or diffusion cache identity, reusable state, validity, lifecycle,
+reset, eviction, and teardown.
+
+## Contract checks
+
+- Include every model, request, input, timestep, precision, device, topology,
+  and feature property that can change a reused value in cache identity.
+- Preserve a correct disabled and miss path; never reuse after validity inputs
+  change.
+- Isolate request-scoped state across concurrent requests and define ownership
+  for model-scoped state.
+- Make reset, invalidation, eviction, capacity, and teardown explicit on
+  completion, cancellation, failure, model reload, and shutdown.
+- Keep cache optimizations behind the owning runtime's hooks rather than adding
+  a second request lifecycle.
+
+Test hit, miss, invalidation, disabled behavior, concurrency, capacity, reset,
+and teardown, with quality comparison when reuse is approximate.
@@ -0,0 +1,24 @@
+# vLLM-Omni Configuration
+
+Primary design: [vLLM-Omni Configuration](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/module/vllm_omni_config/).
+
+Use for config construction, deploy YAML, registries, schema, defaults,
+aliases, CLI projection, stage config, and topology. Inspect the reviewed
+page's status: while its contract remains deferred or draft, current code and
+tests—not proposed precedence rules—are authoritative.
+
+## Contract checks
+
+- Resolve each supported structured, legacy, CLI, environment, default, alias,
+  and stage-override producer through its actual runtime consumer.
+- Reject unknown or owner-mismatched fields explicitly; keep process-local
+  runtime objects out of transportable configuration.
+- Verify default, explicit, and feature-off values reach every live factory and
+  consumer without silent reinterpretation.
+- Treat stage count, placement, connector, parallelism, device, and memory
+  fields as one topology contract and reject impossible combinations early.
+- Preserve supported construction-path parity until a documented migration
+  removes a path.
+
+Require normalization and schema tests, a live-consumer assertion, and docs for
+public keys, defaults, constraints, precedence, or migration behavior.
@@ -0,0 +1,33 @@
+# Diffusion Module Family
+
+Primary design: [Diffusion Modules](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/module/diffusion/).
+
+Use for diffusion-stage behavior, then load the exact submodule page from the
+reviewed head:
+
+| Submodule | Signals | Design |
+| --- | --- | --- |
+| Runtime | Admission, scheduling, execution, progress, outputs, cancellation, hooks, cleanup | [Diffusion Runtime](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/module/diffusion/diffusion_runtime/) |
+| Model Integration | Pipelines, registry, loader, adapters, shared layers, model-specific processing | [Diffusion Model Integration](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/module/diffusion/diffusion_model_integration/) |
+| Continuous Batching | Compatibility, request/step batches, per-request progress, capacity | [Continuous Batching](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/module/diffusion/continuous_batching/) |
+| Parallelism | Rank topology, process groups, sharding, collectives, distributed execution | [Parallelism](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/module/diffusion/parallelism/) |
+| Offloader | Residency, transfers, memory accounting, prefetch, teardown | [Offloader](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/module/diffusion/offloader/) |
+
+## Family-wide checks
+
+- Keep one scheduler-owned lifecycle from admission through completion,
+  cancellation, failure, and cleanup; optional features must use defined hooks.
+- Select pipelines through registry and loader contracts instead of scattered
+  model-name conditionals; keep genuine model differences in model directories.
+- Trace latent, conditioning, timestep, generator, shape, layout, dtype, device,
+  batch expansion, and output conversion through the actual consumer.
+- Make batch compatibility and per-request isolation explicit; never associate
+  output by unstable batch position.
+- Derive topology from validated configuration, define every sharding boundary,
+  keep collectives symmetric, and preserve a supported single-rank path.
+- Track one residency owner per offloaded component, wait for transfer readiness,
+  preserve model state, bound retained copies, and clean up deterministically.
+
+Load matching feature designs for cache, quantization, parallelism, continuous
+batching, or offload. Require feature-off correctness, representative inference,
+terminal-path tests, and quality evidence for optimizations.
@@ -0,0 +1,23 @@
+# Engine Orchestration
+
+Primary design: [Engine Orchestration](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/module/engine_orchestration/).
+
+Use for request-state creation, cross-stage routing, output ordering, companion
+tracking, RPC correlation, cancellation propagation, and terminal convergence.
+Orchestration does not own replica placement, stage-process startup, public
+rendering, connector transport, or payload and error schemas.
+
+## Contract checks
+
+- Keep one owner for cross-stage routing; entrypoints, models, and stage clients
+  must not independently select the next logical stage.
+- Preserve request identity and output order across concurrent stages and
+  correlate every control or RPC result to the originating request and wave.
+- Make terminal state monotonic: no new forwarding after success, abort,
+  request failure, or fatal engine failure.
+- Propagate cancellation and failure to every active stage and release
+  orchestrator-owned tasks, queues, companions, and request state.
+- Preserve shutdown ordering across the orchestrator and stage runtime.
+
+Test representative multi-stage routing, ordering, correlation, cancellation,
+failure, and repeated shutdown.
@@ -0,0 +1,25 @@
+# Entrypoints and Serving Boundaries
+
+Primary design: [Entrypoints and Serving Boundaries](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/module/entrypoints/).
+
+Use for offline APIs, CLI and server composition, OpenAI-compatible routes,
+request validation, response conversion, streaming, sessions, and engine
+handoff. Entrypoints adapt public contracts; they do not own cross-stage
+routing, stage lifecycle, configuration precedence, payload schemas, or error
+classification.
+
+## Contract checks
+
+- Normalize and validate public values once before engine submission.
+- Preserve request identity, modality, ordering, and exactly one terminal event
+  across streaming, disconnect, cancellation, and failure.
+- Translate internal outputs and errors explicitly for each advertised offline,
+  HTTP, SSE, WebSocket, or asynchronous-job path.
+- Keep model-specific behavior behind a common adapter or processor instead of
+  adding model-name policy to generic entrypoints.
+- Require API or CLI changes to explain necessity, alternatives, compatibility,
+  migration, and the supported route or transport matrix.
+
+Test the affected public protocol, normalization, engine handoff, rendering,
+and terminal outcomes. Route orchestration, I/O schema, and semantic error
+changes to their module contracts.
@@ -0,0 +1,24 @@
+# Error Classification, Propagation, and Rendering
+
+Primary design: [Error Classification, Propagation, and Rendering](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/module/error_contracts/).
+
+Use when a change classifies failures, crosses process boundaries, decides
+fatal versus request-scoped impact, sanitizes context, or renders offline,
+HTTP, SSE, WebSocket, or asynchronous-job errors. The I/O module owns the wire
+schema; this module owns the meaning of its error fields.
+
+## Contract checks
+
+- Keep request-scoped failure isolated unless the underlying engine is no
+  longer usable.
+- Preserve stable classification and request context across process and
+  protocol boundaries without exposing prompts, outputs, secrets, or internal
+  stack details.
+- Keep cancellation distinguishable from internal failure.
+- Terminate each public transport exactly once with consistent status, body,
+  event, finish reason, and metrics classification.
+- Maintain compatibility across mixed or older producers only when the
+  reviewed contract defines that behavior.
+
+Test table-driven classification, wire round trips, fatal/request-scoped
+propagation, cancellation, sanitization, and every affected public renderer.
@@ -0,0 +1,23 @@
+# Execution Platforms
+
+Primary design: [Execution Platforms](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/module/execution_platforms/).
+
+Use for hardware capability detection, worker selection, patches, kernels,
+attention backends, and vendor-specific configuration or connectors.
+
+## Contract checks
+
+- Guard hardware behavior through platform selection or capability detection
+  and fail clearly when unsupported.
+- Keep portable modules free of direct vendor imports when a common selector
+  exists; verify portable imports without the target accelerator.
+- Override only behavior that differs from common or upstream implementations,
+  and compare sibling worker hooks when inheritance does not provide parity.
+- Preserve dtype, layout, rank, stream, synchronization, workspace,
+  graph-capture, and resource-lifetime contracts.
+- Do not let simplified CPU doubles hide platform MRO, initialization, or
+  dispatch failures.
+
+Run portable checks first, then the smallest matching-device test in a fresh
+environment. Report unavailable hardware as a validation gap, never simulated
+device evidence.
@@ -0,0 +1,23 @@
+# Input, Output, and Modality Contracts
+
+Primary design: [Input, Output, and Modality Contracts](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/module/input_output_modality_contracts/).
+
+Use for requests, prompt and modality types, message schemas, serialization,
+output types, accumulation, completion, and compatibility shims crossing
+entrypoint, orchestration, stage, connector, or model boundaries.
+
+## Contract checks
+
+- Give every boundary value an explicit modality and validated typed schema.
+- Preserve request and stage identity across conversion, serialization,
+  streaming updates, batching, cancellation, and errors.
+- Distinguish partial, final, empty, and failed output states for every modality.
+- Verify producer and consumer agreement on shape, layout, dtype, device,
+  ownership, metadata, ordering, and optional or unknown fields.
+- Keep internal objects out of public protocols; translate them at the owning
+  entrypoint.
+- Preserve wire compatibility deliberately and give deprecated shims a bounded
+  migration path.
+
+Require construction, round-trip, accumulation, completion, rejection, and
+compatibility tests at each changed producer-consumer boundary.
@@ -0,0 +1,25 @@
+# Model Integration
+
+Primary design: [Model Integration](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/module/model_integration/).
+
+Use for model registration, configuration adapters, preprocessing, checkpoint
+loading, stage inputs, model-specific hooks, shared layers, runners, and model
+execution. Use the diffusion family reference for diffusion models.
+
+## Contract checks
+
+- Declare explicitly how the model class, loader, input processor, stage
+  configuration, and runtime capabilities are selected.
+- Preserve weight-name mapping, dtype/device placement, optional-dependency
+  errors, and checkpoint-specific loading semantics.
+- Trace each typed input and output through preprocessing, runner state, model
+  call, and its actual consumer.
+- Reuse upstream vLLM behavior when sufficient; document and test the precise
+  difference for every override or patch.
+- Keep model code out of admission, cross-stage routing, public policy, and
+  terminal lifecycle ownership.
+- Place reusable behavior in shared modules and test it with more than one
+  integration.
+
+Require focused registry, loader, processor, runner, and representative model
+execution tests. Apply the model-addition check for new or expanded support.
@@ -0,0 +1,21 @@
+# Observability
+
+Primary design: [Observability](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/module/observability/).
+
+Use for metrics, logs, correlation fields, and cross-stage request visibility.
+
+## Contract checks
+
+- Preserve stable request and stage identity across entrypoints, orchestration,
+  stages, workers, errors, and terminal cleanup.
+- Treat metric names, units, labels, aggregation, and lifecycle as public
+  contracts; document any semantic change.
+- Keep labels bounded: never use request IDs, prompts, outputs, paths, or other
+  unbounded user values.
+- Register and reset metrics exactly once across repeated startup and shutdown,
+  and keep disabled instrumentation inexpensive.
+- Align emitted values with their actual timing and lifecycle owner rather than
+  reconstructing them from ambiguous client strings.
+
+Test registration, label sets, units, resets, error classification, and
+representative multi-stage emission.
@@ -0,0 +1,24 @@
+# OmniConnector
+
+Primary design: [OmniConnector](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/module/omni_connector/).
+
+Use for connector contracts and model-agnostic data transfer across stages,
+processes, devices, or nodes. Connector implementations transport and
+synchronize data; they do not choose the next stage or implement model policy.
+
+## Contract checks
+
+- Prove producer and consumer agreement on request/stage identity, shape,
+  layout, dtype, placement, ownership, ordering, and completion.
+- Keep backend selection and metadata/key construction explicit; do not silently
+  reinterpret or drop payloads across SHM, store, network, or vendor paths.
+- Bound connect, put/get, retry, synchronization, and readiness waits with
+  backpressure, cancellation, and actionable errors.
+- Prevent route, endpoint, port, key, and buffer collisions across repeated
+  startup, concurrent requests, and replica replacement.
+- Release connections, handles, buffers, listener work, and stale sender state
+  on success, cancellation, failure, and shutdown.
+
+Load the matching communication feature design for a concrete backend. Require
+focused producer-consumer tests and the smallest runnable multi-process or
+multi-node path.
@@ -0,0 +1,21 @@
+# Profiling
+
+Primary design: [Profiling](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/module/profiling/).
+
+Use for opt-in instrumentation and trace collection across supported profiler
+backends.
+
+## Contract checks
+
+- Keep expensive collection disabled by default and bounded by explicit start,
+  stop, timeout, and cleanup behavior.
+- Preserve scheduling, generated outputs, resource ownership, and concurrency
+  semantics apart from documented synchronization overhead.
+- Include stage, rank, worker, operation, and request correlation where useful
+  without embedding user payloads or secrets.
+- Make repeated start/stop and partial failure safe, including multi-rank trace
+  naming and artifact ownership.
+- Measure disabled and enabled overhead before making performance claims.
+
+Test the disabled path, repeated lifecycle, failure cleanup, trace context, and
+each affected backend.
@@ -0,0 +1,22 @@
+# Quantization
+
+Primary design: [Quantization](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/module/quantization/).
+
+Use for method registration, checkpoint interpretation, layer replacement,
+reduced precision, platform capability, and runtime constraints.
+
+## Contract checks
+
+- Validate checkpoint metadata, selected method, supported layers, device
+  capability, required dependencies, and parallel layout before loading.
+- Preserve loader and weight-name mapping semantics, dtype, scale/granularity,
+  zero point, padding, and buffer handling through the real kernel consumer.
+- Fail clearly for unsupported combinations; do not silently choose a different
+  method or precision.
+- Keep online and pre-quantized checkpoint paths explicit and ensure patches or
+  fallbacks do not bypass model-owned loaders.
+- Require numerical comparison against a defined reference with explicit
+  tolerance before treating performance results as valid.
+
+Test registration, loading, representative layers, metadata, platform guards,
+parallel combinations, and numerical correctness separately from performance.
@@ -0,0 +1,23 @@
+# Stage Runtime and Replica Lifecycle
+
+Primary design: [Stage Runtime and Replica Lifecycle](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/module/stage_runtime/).
+
+Use for local or distributed placement, startup, readiness, stage pools,
+replica identity and membership, selection, affinity, liveness, draining, and
+stage-client or process lifecycle. Cross-stage request policy belongs to engine
+orchestration; model scheduling belongs to the owning runtime.
+
+## Contract checks
+
+- Acquire, replace, and retire replicas through one stage-runtime lifecycle.
+- Preserve stable replica identity and request affinity until termination or a
+  declared replica loss.
+- Make readiness explicit before admission; surface partial-startup and
+  membership failures without leaving half-initialized capacity.
+- Keep local and distributed semantics aligned for supported modes.
+- Make drain, shutdown, and cleanup idempotent across repeated signals and
+  process failure.
+
+Test startup, readiness, selection, affinity, membership changes, abort,
+process loss, draining, and repeated shutdown on affected local/distributed
+paths.
@@ -0,0 +1,34 @@
+# Resolve Design Contracts
+
+Use after freezing the review head and before selecting module or feature
+references. The maintained [design index](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/)
+defines two routing axes:
+
+- **Module designs** own code boundaries and invariants. Select one primary
+  module and a second only when the live producer-consumer path crosses it.
+- **Feature designs** describe behavior spanning modules. Load every feature
+  page whose implementation, default, support, or contract changes.
+
+## Resolution order
+
+1. Open `docs/design/index.md` in the reviewed head, not the reviewer's checkout
+   or published latest tree.
+2. Match changed behavior against each module page's `primary_code_paths`,
+   `primary_path_exceptions`, ownership boundary, and `depends_on` metadata.
+3. Read `status`, `architecture_state`, `last_verified_commit`, and any linked
+   in-flight work before treating prose as current behavior.
+4. Use current code and tests when a page is draft, deferred, stale, absent, or
+   conflicts with the frozen implementation. Report material documentation
+   drift separately.
+5. Treat candidate invariants and promotion gates as review questions until
+   repository policy, current code/tests, or owner-approved normative text
+   makes them enforceable. Do not turn a draft identifier into an automatic
+   blocker.
+6. Use published latest docs only for discovery when the reviewed head lacks a
+   page; never import a newer contract into an older branch silently.
+
+When behavior changes an active module or feature contract, require the same PR
+to update that page. Require a new page only when repository policy, an accepted
+RFC/issue, or explicit maintainer acceptance criteria do so; otherwise treat it
+as a non-blocking design suggestion. Do not demand a second page when an existing
+design already owns the exact contract.
@@ -0,0 +1,134 @@
+# General Review Checks
+
+Apply these checks to every PR after freezing the review surface. A pattern is
+not a finding until the diff introduces or exposes a reachable trigger, impact,
+and smallest safe fix. Do not report unrelated backlog.
+
+Official docs: [contributing guide](https://docs.vllm.ai/projects/vllm-omni/en/latest/contributing/)
+[feature compatibility](https://docs.vllm.ai/projects/vllm-omni/en/latest/user_guide/feature_compatibility/),
+and [design documents](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/).
+
+## Contract and scope
+
+- Match the title/body claims to every changed production path, test, config,
+  dependency, generated artifact, and user-visible document.
+- Trace each changed value through public ingress, validation/defaulting,
+  producer, transformations, process or device boundaries, final consumer, and
+  terminal cleanup.
+- Treat unchanged code as in scope when the diff makes its value newly
+  load-bearing for a changed consumer; validate that value's origin and contract
+  rather than reviewing only added lines.
+- Check every applicable sync/async, offline/online, streaming/non-streaming,
+  feature-off, topology, and compatibility path without demanding unsupported
+  modes merely for symmetry.
+
+## Module, API, and class design
+
+Apply these rules to new or materially changed production code; do not require
+unrelated cleanup from untouched files.
+
+1. Start each source module with a concise module docstring that defines its
+   responsibility and boundaries and identifies its primary classes and key
+   functions. Place it after any required license header and before imports.
+2. Give every added or materially changed class and function a clear docstring
+   covering its responsibility, parameters, return value, raised exceptions,
+   and non-obvious invariants. Use explicit typed parameters at owned API
+   boundaries; avoid opaque `**kwargs` and `Any`. Allow them only when an
+   upstream or compatibility interface requires them, and document and validate
+   the accepted keys and concrete types.
+3. Move stable helpers shared by multiple features into an appropriately scoped
+   utility module. Do not let feature modules accumulate large collections of
+   unrelated private utility functions; keep a private helper local only when
+   it is single-purpose and tightly coupled to that feature.
+4. Keep every method on a key class within that class's stated responsibility,
+   and document non-obvious design decisions and invariants. Before adding a
+   method, prefer reuse, consolidation, or extension of an existing abstraction;
+   split the class when new behavior creates a separate responsibility or makes
+   the class unnecessarily long.
+5. Require changes to user-facing behavior to update `docs/`. When an existing
+   active module or feature design page owns the changed contract, update that
+   page. Require a new design page only when the reviewed branch's repository
+   policy, an accepted RFC/issue, or explicit maintainer acceptance criteria do
+   so; otherwise suggest it as non-blocking. For a major architectural change,
+   apply the branch's contributing policy for a design RFC or issue; the current
+   policy uses more than 500 changed production-code lines, excluding kernel,
+   data, config, and test changes. Resolve page status and authority through
+   [design-contracts.md](design-contracts.md), and do not demand a duplicate page
+   when an existing design owns the exact contract. When required, cover scope,
+   data flow, public contracts, failure behavior, alternatives, validation,
+   limitations, and rollout.
+6. Require every performance PR to provide reproducible base-versus-head A/B
+   results under the same hardware, software, model, workload, precision,
+   topology, warmup, repetitions, and measurement scope. Include exact commands,
+   variability, and a correctness or quality guard; one-sided or non-comparable
+   numbers are incomplete evidence. For a new model absent from the frozen base,
+   compare head with its pinned canonical reference implementation under rule 12
+   instead. Follow
+   [perf-verification.md](../checks/perf-verification.md) for the full protocol.
+7. Require every addition, removal, default change, or semantic change to a
+   user-facing inference API or CLI to explain why it is necessary, which
+   alternatives were considered, and why the chosen interface is preferable.
+   Record compatibility, deprecation or migration behavior, and the rationale
+   in the PR. Update the corresponding module and feature design documents when
+   they define the changed interface or workflow, or when policy requires one.
+8. Require every bug-fix PR to add an automated regression test that reproduces
+   the defect on the frozen base and passes on the fixed head. Exercise the
+   production path or nearest stable boundary and assert the corrected behavior,
+   not only process survival. Record the exact base/head command and result, and
+   place hardware- or model-specific coverage in the correct CI lane. A manual
+   reproduction supplements but does not replace the regression test.
+9. Require every new or materially changed recipe to follow
+   [`recipes/TEMPLATE.md`](../../../../../recipes/TEMPLATE.md) and the naming and
+   layout conventions in
+   [`recipes/README.md`](../../../../../recipes/README.md).
+   Complete every applicable summary, use-case, reference, hardware,
+   environment, exact-command, verification, and notes section with tested,
+   reproducible details; explain any intentional deviation in the PR.
+10. Require every new model to verify its supported modalities, platforms,
+    execution modes, and features with production-path evidence. Update
+    `docs/models/supported_models.md` and every applicable feature support or
+    compatibility table; do not claim support that the PR does not validate.
+    Follow
+    [model-addition-checklist.md](../checks/model-addition-checklist.md) for the
+    model-specific documentation checks.
+11. Require every new model PR to add or update its in-repository model-family
+    recipe under `recipes/` and its `recipes/README.md` index entry. Apply rule
+    9, and keep the recipe's model identifiers, tasks, serving modes, hardware,
+    commands, supported features, and limitations aligned with validated PR
+    evidence. An external recipe link does not replace the in-repository recipe.
+12. Require every new model to compare accuracy or output quality and performance
+    against a pinned baseline or canonical reference implementation. Match the
+    checkpoint, inputs, precision, hardware, workload, and measurement method;
+    provide exact commands, results, and explanations for material differences.
+    Prefer a detailed timing split for each stage and explain when it is
+    unavailable. Follow
+    [model-addition-checklist.md](../checks/model-addition-checklist.md) and
+    [perf-verification.md](../checks/perf-verification.md) for the evidence
+    contract.
+
+## Blocking risk scan
+
+| Risk | Prove before reporting |
+| --- | --- |
+| Correctness | A live input reaches a wrong output, exception, silent drop, shape/dtype/device mismatch, or partially initialized state. |
+| Compatibility | A supported API, config key, default, serialization schema, model identifier, or caller breaks without validation, migration, or deprecation. |
+| Lifecycle | Allocation or startup lacks cleanup on partial failure, timeout, cancellation, shutdown, or repeated requests. |
+| Concurrency/distribution | Ordering, identity, rank, world size, completion, timeout, backpressure, or collective assumptions fail on a supported path. |
+| Cache/shared state | Identity omits a correctness-affecting input, or isolation, invalidation, disabled behavior, eviction, or cleanup is incomplete. |
+| Async behavior | Blocking I/O, sleep, a lock across `await`, or unnecessary serialization harms a reachable async path. |
+| Security/data handling | Secrets, user payloads, unsafe deserialization/eval/shell, unvalidated paths, or unbounded metric labels are reachable. |
+| Validation evidence | Tests bypass the production dispatcher, mock away the change, use unrealistic types/MRO, or cannot fail when the defect returns. |
+| User contract | User-visible model, feature, API, CLI, config, default, or compatibility behavior changes without accurate docs or examples. |
+
+Search bounded callers and sibling implementations before reporting dead code,
+duplication, or a missed consumer. Keep new abstractions only when they have a
+distinct live caller, invariant, or compatibility need.
+
+## Finding bar
+
+Anchor each finding to the exact `path:line` that causes the behavior; it may be
+unchanged when the diff makes it newly load-bearing, in which case name the
+changed call path that brings it into scope. Include the trigger, impact, and
+smallest fix direction. Treat pending CI, missing hardware, or unsupported
+measurements as validation gaps unless the repository contract makes that
+evidence a merge requirement.
@@ -0,0 +1,195 @@
+# Review Execution and Delivery
+
+Use this reference for every review. It defines snapshot collection, gates,
+validation records, delivery permissions, maintainer tone, and re-review.
+
+Official docs: [contributing guide](https://docs.vllm.ai/projects/vllm-omni/en/latest/contributing/)
+and [CI failure triage](https://docs.vllm.ai/projects/vllm-omni/en/latest/contributing/ci/failures/).
+
+## Contents
+
+- [Freeze the review surface](#freeze-the-review-surface)
+- [Report status before analysis](#report-status-before-analysis)
+- [Apply review gates](#apply-review-gates)
+- [Run bounded validation](#run-bounded-validation)
+- [Deliver maintainer-style findings](#deliver-maintainer-style-findings)
+- [Re-review safely](#re-review-safely)
+
+## Freeze the review surface
+
+For a GitHub PR, read the base/head SHA before and after fetching metadata and
+the diff:
+
+```bash
+gh api "repos/vllm-project/vllm-omni/pulls/<PR>" \
+  --jq '{base_sha: .base.sha, head_sha: .head.sha}'
+REVIEW_FIELDS="number,url,title,body,isDraft,baseRefName,headRefName,mergeable,mergeStateStatus,statusCheckRollup,files"
+gh pr view <PR> --repo vllm-project/vllm-omni \
+  --json "${REVIEW_FIELDS}"
+gh pr diff <PR> --repo vllm-project/vllm-omni
+gh api "repos/vllm-project/vllm-omni/pulls/<PR>" \
+  --jq '{base_sha: .base.sha, head_sha: .head.sha}'
+```
+
+Discard the snapshot if either SHA changed. Do not mix comments or validation
+from different heads.
+
+After the SHAs stabilize, fetch and materialize a trusted `head_sha` in an
+isolated detached worktree. Run every source read, `rg` search, import, and test
+from that snapshot, not the caller's checkout. A detached worktree provides
+snapshot isolation, not a security boundary.
+
+Treat a fork head as untrusted unless the user and environment policy explicitly
+establish trust. Never run its imports, tests, builds, hooks, package setup, or
+repo-configurable linters/plugins on the reviewer host. Execute them only in a
+disposable sandbox or VM with no credentials or inherited secrets, no host agent
+or service sockets, minimal read-only host mounts, disabled or explicitly
+allowlisted network, resource/time limits, and destruction after the run. If
+that boundary is unavailable, limit the review to the remote diff, SHA-addressed
+reads such as `git show <head_sha>:<path>`, and existing CI evidence; report all
+executable validation as a gap.
+
+Before the first source read, record a pristine snapshot fingerprint outside
+the reviewed worktree. Include `HEAD`, the NUL-delimited index entries and blob
+IDs, and a NUL-safe manifest of every worktree entry except Git metadata,
+including tracked, untracked, and ignored paths with file type, mode, symlink
+target, and content hash. Before each validation group and delivery, recompute
+and byte-compare the fingerprint as well as asserting `HEAD == head_sha`. If a
+tool changes or creates any entry, discard affected evidence and recreate the
+snapshot before another group; do not rely on `HEAD` alone or clean an unknown
+worktree in place. If an exact snapshot cannot be created, use SHA-addressed
+reads and report filesystem-dependent validation as a gap.
+
+For a local branch/worktree, determine the target ref from the user, current PR,
+or configured upstream; never infer it from the branch name. Resolve the target
+once and include all task-owned worktree state:
+
+```bash
+git status --porcelain=v2 -z
+git rev-parse HEAD
+git rev-parse <target-ref>
+git merge-base <target-base-sha> HEAD
+git diff --stat <comparison-commit>
+git diff --name-status <comparison-commit>
+git diff --binary <comparison-commit>
+git diff --cached --binary <comparison-commit>
+git diff --binary
+git ls-files --others --exclude-standard -z
+```
+
+Freeze the exact bytes of every in-scope untracked file from the NUL-delimited
+list, plus a NUL-safe path, file-type, mode, and content-hash manifest. Also
+fingerprint `HEAD`, the target and merge base, porcelain status, and both binary
+index and worktree patches. Recording only untracked names is insufficient.
+Keep these snapshots in the evidence packet and do not mutate the reviewed
+checkout during a read-only review.
+
+For local changes without PR metadata, mark the title/body as synthetic. Base
+them on the user request and commit messages without inventing validation claims.
+
+If no PR, branch, or usable worktree is identifiable, ask for a PR URL/number or
+explicit base and head rather than guessing.
+
+## Report status before analysis
+
+Within 60 seconds of starting, and before source searches or tests, send this
+host update:
+
+```text
+Pinned head: <SHA>
+Base/comparison: <ref and SHA>
+CI: <pass/fail/pending/not applicable>
+Mergeability: <state/not applicable>
+Preliminary findings: <brief finding or none yet>
+```
+
+Mark early findings as preliminary and continue. This is not a GitHub comment.
+
+## Apply review gates
+
+- Report draft/WIP state. Continue a local review when explicitly requested,
+  but do not publish review events without separate authorization.
+- Record DCO, pre-commit, required CI, and mergeability. Pending or unknown gates
+  do not block source review.
+- Treat a failed gate as its own evidence. Do not restate its formatting/lint
+  output as a new code-review finding.
+- Open CI logs only when the first failing step overlaps the frozen diff or
+  blocks the verdict. Start with the first error, not the last cascade.
+- Treat the PR body as navigation. Commands, benchmark tables, and claimed tests
+  become evidence only after their provenance and relevance are checked.
+
+## Run bounded validation
+
+Keep one evidence packet for files read, bounded searches, callers, tests, CI,
+hardware, routes, and findings. Reuse it rather than refetching facts.
+
+Before pytest, run a short import/version compatibility check. Record each
+validation result with:
+
+```text
+repo, head SHA, command, result, Python/platform, dependency or lock fingerprint
+```
+
+After preflight, run targeted tests and low-cost static checks alongside source
+inspection when possible. Imports, tests, builds, and linters may create caches
+or rewrite files, so isolate each potentially mutating group in a disposable
+snapshot or recreate the pinned snapshot before continuing. Map source symbols
+to tests with bounded `rg` searches rather than assuming the test directory
+mirrors production paths.
+
+Classify failures as code, test, infrastructure, or flaky before reporting.
+Skipped hardware tests are gaps, not passes. For available hardware, verify the
+smallest representative unit/E2E path and compare actual output with PR claims.
+
+## Deliver maintainer-style findings
+
+Default to roughly 1-5 short comments, but treat that as calibration rather than
+a quota: report every real blocker and report no finding when there is no issue.
+
+Write each finding as:
+
+```text
+[P1] Short imperative title — path/to/file.py:<line>
+<Trigger or call path>. <Current behavior and impact>. <Smallest fix direction>.
+```
+
+Use [maintainer-style-study.md](../delivery/maintainer-style-study.md) for
+comment tone.
+Keep rule IDs, grades, and audit matrices internal unless the user asks for the
+complete audit.
+
+Verify every inline `path:line` against the frozen diff before delivery. For a
+PR, re-read the remote head and byte-compare the detached snapshot fingerprint,
+or recreate it from `head_sha`, before asserting current lines. For a local
+review, recompute and byte-compare the frozen `HEAD`, target, merge base, status,
+index patch, worktree patch, and untracked-content manifest. Any mismatch makes
+the review stale; discard the affected evidence and restart. Prefer one
+root-cause comment to several symptom comments.
+
+When the root cause is an unchanged line made newly load-bearing by the diff,
+cite it in the consolidated review body instead of moving the comment to an
+unrelated changed line merely to make it inline.
+
+Local presentation is the default. Only explicit authorization permits GitHub
+posting. When authorized, publish one consolidated final review after
+validation; do not post preliminary or incremental comments. Submit a review
+event (`APPROVE`, `COMMENT`, or `REQUEST_CHANGES`) only when the user explicitly
+chooses that event.
+
+Reviewer requests and owner `@mention` comments are separate external writes;
+follow [review-requests.md](../delivery/review-requests.md) when the user asks
+for them.
+
+## Re-review safely
+
+Freeze the new head and compare it with the previously reviewed SHA. Then:
+
+1. inspect the delta and every file affected by conflict resolution or rebase;
+2. revalidate earlier findings against current lines and behavior;
+3. read unresolved, non-outdated threads and the author's evidence;
+4. rerun only checks invalidated by the delta;
+5. look for new regressions introduced by the fix;
+6. do not repeat resolved or outdated comments.
+
+If the target head changes during re-review, discard stale validation and start
+again from the new snapshot.
@@ -0,0 +1,77 @@
+# Module and Feature Review Routing
+
+Read [design-contracts.md](design-contracts.md) first. Route from the frozen
+head's live producer-consumer behavior, then confirm the result against its
+`docs/design/` metadata. Titles and paths are evidence, not the deciding rule.
+
+## Primary module contract
+
+Choose one primary module and a second only when the live call path crosses a
+documented dependency or primary-path exception.
+
+| Module | Live behavior signals | Read |
+| --- | --- | --- |
+| Entrypoints | Offline/CLI/API ingress, validation, rendering, streaming, sessions | [entrypoints.md](../modules/entrypoints.md) |
+| Configuration | Schema, defaults, deploy/stage construction, registry, topology | [configuration.md](../modules/configuration.md) |
+| I/O and Modality | Requests, messages, serialization, output types, accumulation, completion | [input-output-modality.md](../modules/input-output-modality.md) |
+| Error Contracts | Classification, fatality, propagation, sanitization, public rendering | [error-contracts.md](../modules/error-contracts.md) |
+| Engine Orchestration | Cross-stage routing, request state, ordering, RPC correlation, terminal convergence | [engine-orchestration.md](../modules/engine-orchestration.md) |
+| Stage Runtime | Placement, startup, readiness, replicas, affinity, membership, shutdown | [stage-runtime.md](../modules/stage-runtime.md) |
+| OmniConnector | Cross-stage/process/device/node transport and synchronization | [omni-connector.md](../modules/omni-connector.md) |
+| Model Integration | Registration, preprocessing, loading, runners, model-specific execution | [model-integration.md](../modules/model-integration.md) |
+| AR Runtime | Scheduling, request/cache state, adapters, workers, upstream vLLM semantics | [ar-runtime.md](../modules/ar-runtime.md) |
+| Diffusion Family | Diffusion runtime, models, batching, parallelism, or offload | [diffusion.md](../modules/diffusion.md) |
+| Execution Platforms | Hardware selection, capabilities, vendor workers, kernels, patches | [execution-platforms.md](../modules/execution-platforms.md) |
+| Cache Management | Cache identity, validity, reuse, eviction, reset, teardown | [cache-management.md](../modules/cache-management.md) |
+| Quantization | Method/metadata selection, layer mapping, reduced precision, constraints | [quantization.md](../modules/quantization.md) |
+| Observability | Metrics, logs, labels, units, correlation, lifecycle | [observability.md](../modules/observability.md) |
+| Profiling | Opt-in instrumentation, traces, start/stop lifecycle, overhead | [profiling.md](../modules/profiling.md) |
+| Benchmarking | Workloads, metric computation, benchmark CLI, result metadata | [benchmarking.md](../modules/benchmarking.md) |
+
+For tests-, docs-, CI-, or recipe-only changes, route to the production module
+whose contract they protect. When none exists, use the applicable evidence
+check without inventing a production owner.
+
+## Feature-design overlays
+
+Load every category whose documented behavior, enablement, default, support, or
+compatibility changes.
+
+| Feature-design section | Read |
+| --- | --- |
+| Runtime and stage execution | [runtime-stage-execution.md](../features/runtime-stage-execution.md) |
+| Communication and concrete connectors | [communication.md](../features/communication.md) |
+| Diffusion acceleration | [diffusion-acceleration.md](../features/diffusion-acceleration.md) |
+| Infrastructure and performance | [infrastructure-performance.md](../features/infrastructure-performance.md) |
+
+## Evidence and change overlays
+
+| Signal | Read | Optional repo-local skill |
+| --- | --- | --- |
+| New or expanded model, loader, processor, registry, or stage config | [model-addition-checklist.md](../checks/model-addition-checklist.md) | [`add-tts-model`](../../../add-tts-model/SKILL.md) or [`add-diffusion-model`](../../../add-diffusion-model/SKILL.md) |
+| Latency, throughput, memory, scaling, precision, or quality claim | [perf-verification.md](../checks/perf-verification.md) | [`diffusion-perf-opt`](../../../diffusion-perf-opt/SKILL.md) for diffusion |
+| Config/CLI/environment/default precedence; endpoint, host, port, address, connector metadata, or `*_extra_config` | [config-provenance.md](../checks/config-provenance.md) | None |
+| Tests changed, absent for risky behavior, or test-only | [test-quality-evaluation.md](../checks/test-quality-evaluation.md) | [`vllm-omni-test`](../../../vllm-omni-test/SKILL.md) |
+| CI, examples, docs, public behavior, or contributor evidence | [tests-docs-checklist.md](../checks/tests-docs-checklist.md) | None |
+| Suitable hardware/server and runnable affected path | [verification.md](../checks/verification.md) | None |
+| User asks who should review or requests owner notification | [review-requests.md](../delivery/review-requests.md) | None |
+
+Use [`quantization`](../../../quantization/SKILL.md) for a quantization module
+change and [`$vllm-omni-npu-model-runner-upgrade`](../../../vllm-omni-npu-upgrade/SKILL.md)
+for an NPU execution-platform change when those skills are available.
+
+For a bug fix, require a reachable reproduction and regression test. For a
+refactor, prove parity and remove obsolete paths. For a feature, verify the
+public/config contract, module integration, compatibility/default behavior,
+production dispatch, feature design, and docs.
+
+## Calibrate findings
+
+- **P0:** security exposure, data corruption, or broad project unusability.
+- **P1:** reachable runtime failure, wrong output, compatibility break, or
+  unsafe lifecycle in the changed behavior.
+- **P2:** real non-blocking defect with a concrete future failure mode.
+
+Treat draft candidate invariants, missing hardware, and unsupported claims as
+questions or validation gaps unless current code, tests, or policy makes them a
+merge requirement.
@@ -61,6 +61,13 @@ repos:
      types: [python]
      files: ^tests/

+    - id: check-tts-adapter-migration
+      name: Ratchet TTS per-model branches out of serving_speech.py
+      entry: python tools/pre_commit/check_tts_adapter.py
+      language: python
+      pass_filenames: false
+      files: ^vllm_omni/entrypoints/openai/(serving_speech\.py|tts_adapters/__init__\.py)$
+
    # Keep `suggestion` last
    - id: suggestion
      name: Suggestion
@@ -9,7 +9,7 @@ Easy, fast, and cheap omni-modality model serving for everyone
 </h3>

 <p align="center">
-| <a href="https://vllm-omni.readthedocs.io/en/latest/"><b>Documentation</b></a> | <a href="https://deepwiki.com/vllm-project/vllm-omni"><b>DeepWiki</b></a> | <a href="https://discuss.vllm.ai"><b>User Forum</b></a> | <a href="https://slack.vllm.ai"><b>Developer Slack</b></a> | <a href="docs/assets/WeChat.jpg"><b>WeChat</b></a> | <a href="https://arxiv.org/abs/2602.02204"><b>Paper</b></a> | <a href="https://docs.google.com/presentation/d/111-L8zF7A1j_YI_cR8JsblofdScdRr2f/edit?usp=sharing&ouid=110473603432222024453&rtpof=true&sd=true"><b>Slides</b></a> |
+| <a href="https://vllm-omni.readthedocs.io/en/latest/"><b>Documentation</b></a> | <a href="https://deepwiki.com/vllm-project/vllm-omni"><b>DeepWiki</b></a> | <a href="https://discuss.vllm.ai"><b>User Forum</b></a> | <a href="https://slack.vllm.ai"><b>Developer Slack</b></a> | <a href="docs/assets/WeChat.jpg"><b>WeChat</b></a> | <a href="https://arxiv.org/abs/2602.02204"><b>Paper</b></a> | <a href="https://docs.google.com/presentation/d/1aPj0OGl_-ZVoib-Qne5dGDAlrRFB-PdHl6E-EE99g8E/edit?usp=sharing"><b>Slides</b></a> |
 </p>


@@ -63,5 +63,5 @@ vLLM-Omni seamlessly supports most popular open-source models on HuggingFace, in

 For more information, checkout the following:

- [vllm-omni architecture design and recent roadmaps](https://docs.google.com/presentation/d/1XJWgv79lORl8rbaVvp2d5Sqs6ZEBgAgj/edit?slide=id.p1#slide=id.p1)
+- [vllm-omni architecture design and recent roadmaps](https://docs.google.com/presentation/d/1aPj0OGl_-ZVoib-Qne5dGDAlrRFB-PdHl6E-EE99g8E/edit?usp=sharing)
 - [vllm-omni announcement blogpost](https://blog.vllm.ai/2025/11/30/vllm-omni.html)
@@ -672,100 +672,147 @@ down and spawn a new one mid-module. A few rules that save real CI debugging tim

 ## Online Serving Integration

-To expose your model through the `/v1/audio/speech` OpenAI-compatible endpoint, add
-**all five** of the following integration points to
-`vllm_omni/entrypoints/openai/serving_speech.py` in a **single commit**. Adding them
-piecemeal causes partial-integration failures that are hard to debug.
+To expose your model through the `/v1/audio/speech` OpenAI-compatible endpoint, write
+one adapter under `vllm_omni/entrypoints/openai/tts_adapters/`. The adapter declares
+how your model is recognized and owns its request handling; `serving_speech.py` should
+not need an edit.

-### 1. Stage constant
+### 1. Write the adapter

-Near the top of the file, alongside the other `_*_TTS_MODEL_STAGES` constants:
+Create `vllm_omni/entrypoints/openai/tts_adapters/your_model.py`:

 ```python
-_YOUR_MODEL_TTS_MODEL_STAGES = {"your_model_stage_key"}
+# SPDX-License-Identifier: Apache-2.0
+"""YourModel serving adapter."""
+
+from typing import TYPE_CHECKING
+
+from vllm_omni.entrypoints.openai.tts_adapters import register_tts_adapter
+from vllm_omni.entrypoints.openai.tts_adapters.base import ARTTSAdapter, PreparedRequest
+
+if TYPE_CHECKING:
+    from vllm_omni.entrypoints.openai.protocol.audio import OpenAICreateSpeechRequest
+
+
+@register_tts_adapter
+class YourModelAdapter(ARTTSAdapter):
+    # The registry key, and the model-type string used in logs.
+    name = "your_model"
+    # The engine ``model_stage`` value(s) your deployment yaml sets. Detection,
+    # stage discovery and `/v1/audio/speech` routing all derive from this.
+    stage_keys = frozenset({"your_model_stage_key"})
+
+    def validate(self, request: "OpenAICreateSpeechRequest") -> str | None:
+        """Return an error string, or None if the request is valid."""
+        if not request.input or not request.input.strip():
+            return "Input text cannot be empty"
+        return None
+
+    async def build(
+        self,
+        request: "OpenAICreateSpeechRequest",
+        sampling_params_list: list,
+        has_inline_ref_audio: bool,
+    ) -> PreparedRequest:
+        """Build the engine prompt and per-model parameters."""
+        params: dict[str, list] = {"text": [request.input]}
+        if request.voice is not None:
+            params["voice"] = [request.voice]
+        return PreparedRequest(
+            prompt={"prompt": request.input},
+            tts_params=params,
+            model_type=self.name,
+        )
 ```

-### 2. Union into `_TTS_MODEL_STAGES`
+Then add it to the import block at the bottom of
+`vllm_omni/entrypoints/openai/tts_adapters/__init__.py` so it registers on import.
+That import line is the only shared file a new model has to touch.

-Add to the `_TTS_MODEL_STAGES` set union:
+> **Pure-diffusion TTS does not go through adapters yet.** When the server is built
+> by `OmniOpenAIServingSpeech.for_diffusion()`, `create_speech()` routes straight to
+> `_create_diffusion_speech()` and never calls `validate()` or `build()`.
+> `DiffusionTTSAdapter` is scaffolding for that migration and has no production
+> subclass today, so validation or prompt construction placed in one would silently
+> never run. If your model is served by the diffusion engine rather than
+> `engine_client`, follow the existing diffusion path and say so on #4855 — wiring
+> that path through the adapter layer is open work, not something to do inside a
+> new-model PR.
+
+### 2. If a stage key alone cannot identify your model
+
+Two escape hatches, in order of preference:

 ```python
-_TTS_MODEL_STAGES: set[str] = (
-    ...
-    | _YOUR_MODEL_TTS_MODEL_STAGES
-)
+# The architecture is authoritative (e.g. your model has no dedicated
+# model_stage value, or shares one with another model).
+model_archs = frozenset({"YourModelForConditionalGeneration"})
+
+# Set this too if your model owns NO stage key, so stage discovery has to find
+# your AR entry stage by architecture.
+arch_identifies_entry_stage = True
 ```

-### 3. Model type detection
-
-In `_detect_tts_model_type()`, add before the final `return None`:
+If your rule is not a set-membership test, override `matches()` — see
+`covo_audio.py`, which shares the generic `fused_thinker_talker` stage key with
+non-CoVo deployments and confirms with the architecture:

 ```python
-if model_stage in _YOUR_MODEL_TTS_MODEL_STAGES:
-    return "your_model"
+@classmethod
+def matches(cls, model_stage: str | None, model_arch: str | None) -> bool:
+    return super().matches(model_stage, model_arch) and bool(model_arch) and "CovoAudio" in model_arch
 ```

-### 4. Request validation dispatch
+If your model can match the same deployment as another adapter, give one of them an
+explicit `detect_priority` (lower runs first). `tests/entrypoints/openai_api/
+test_tts_detection.py` fails if two same-priority detectors can both match one input,
+so an unordered overlap cannot be merged.

-In `_validate_tts_request()`, add before the fallback `return`:
+Models that are speech-capable only in some deployment topologies override
+`stage_serves_speech()` — see `audex.py`, whose omni thinker is text-final unless the
+speech decoder is deployed alongside it.

-```python
-if self._tts_model_type == "your_model":
-    return self._validate_your_model_request(request)
-```
+### 3. Reuse the shared helpers

-### 5. Validation and parameter-builder methods
+Adapters reach shared serving helpers through `self.ctx.server`: reference-audio
+resolution (`_resolve_ref_audio`), uploaded-speaker handling
+(`_apply_uploaded_speaker`), format validation (`_validate_ref_audio_format`), and
+prompt-length limits (`_max_instructions_length`). Read a comparable adapter before
+writing your own — `fish_speech.py` for voice cloning, `higgs_audio_v3.py` for
+parameter-heavy models, `moss_tts.py` for a family sharing one base class.

-Add two new methods:
+### 4. Test it

-```python
-def _validate_your_model_request(
-    self, request: OpenAICreateSpeechRequest
-) -> str | None:
-    """Validate YourModel request. Returns an error string or None."""
-    if not request.input or not request.input.strip():
-        return "Input text cannot be empty"
-    return None
+`tests/entrypoints/openai_api/test_tts_adapter.py` covers registry invariants and
+`test_tts_detection.py` covers detection. Add your model to
+`EXPECTED_MODEL_TYPES` in the former, and a case to the latter if you added a
+`model_archs` or `matches()` rule.

-def _build_your_model_params(
-    self, request: OpenAICreateSpeechRequest
-) -> dict[str, Any]:
-    """Build additional_information dict for YourModel."""
-    params: dict[str, Any] = {"text": [request.input]}
-    if request.voice is not None:
-        params["voice"] = [request.voice]
-    # Add any other model-specific fields here
-    return params
-```
+### Do not add branches to `serving_speech.py`

-Then wire `_build_your_model_params` into the request-dispatch block in
-`_create_tts_request()` (search for the equivalent `_build_*_params` call for an
-existing model to find the right location). If the model supports voice cloning
-(`ref_audio` → `prompt_audio_path`, `ref_text` → `prompt_text`), add those mappings
-here too — follow any existing `_build_<model>_params` in `serving_speech.py` (e.g.
-`_build_moss_tts_params` for the voice-cloning variant) for the pattern.
+Older models predate the adapter framework and still have per-model branches in
+`serving_speech.py`; they are being migrated out (RFC #4327, #4855). Do not copy
+that pattern. `tools/pre_commit/check_tts_adapter.py` is a ratchet on the remaining
+`self._tts_model_type == ...` comparisons and fails the commit if the count grows.

-> **Two dispatch patterns coexist:** Fish Speech uses a `self._is_fish_speech` boolean
-> checked *before* `elif self._is_tts`. All newer models use the `_tts_model_type`
-> string pattern shown above. For new models, always use the string pattern — do not
-> add new `_is_*` boolean flags.
-
-> **Note on unused variables:** Only extract parameters in `_build_your_model_params`
-> that you actually pass to the model's generate / `inference_stream` call. Extracting
-> a variable without forwarding it will trigger a `ruff F841` pre-commit failure.
+If your model genuinely needs behaviour that no adapter hook can express, that is a
+missing hook — propose it on the RFC rather than adding a branch.

 ### Merge conflicts

-`serving_speech.py` is modified by every new model PR and is the most common source of
-rebase conflicts. When rebasing onto `main` and a conflict appears here, the resolution
-is always to **keep both** the upstream model's additions and your own — never discard
-either side. After resolving:
+Your adapter is a new file, so it cannot conflict. The one shared line is your entry
+in the import block at the bottom of `tts_adapters/__init__.py`, which conflicts only
+when another model lands at the same time. The resolution is always to **keep both**
+imports — never discard either side. After resolving:

 ```bash
-git add vllm_omni/entrypoints/openai/serving_speech.py
+git add vllm_omni/entrypoints/openai/tts_adapters/__init__.py
 git rebase --continue
 ```

+If you find yourself resolving a conflict inside `serving_speech.py`, something in
+your model is still going through the legacy path; see the note above.
+
 ## Single-Stage Models

 Some TTS models (e.g. MOSS-TTS-Nano) do not use a two-stage pipeline. Instead the
@@ -1,6 +1,6 @@
 # Architecture Overview

-This document outlines the architectural design for vLLM-Omni.
+This document outlines the architecture of vLLM-Omni.

 <p align="center">
  <picture>
@@ -9,194 +9,456 @@ This document outlines the architectural design for vLLM-Omni.
  </picture>
 </p>

-# Goals
+## Design goals

-The primary goal of the vLLM-Omni project is to build the fastest and easiest-to-use open-source Omni-Modality model inference & serving engine. vLLM-Omni extends the original vLLM, which was created to support large language models for text-based autoregressive (AR) generation tasks. vLLM-Omni is designed to support:
+The primary goal of vLLM-Omni is to provide a fast and easy-to-use inference
+and serving engine for omni-modality models. vLLM-Omni extends vLLM's
+text-oriented autoregressive (AR) runtime with stage-based execution for
+non-textual outputs and non-autoregressive model components.

-* **Non-textual Output:** Enables the integration, efficient processing and output of various data types, including but not limited to, images, audio, video, and action trajectories, alongside text.
-* **Non-Autoregressive Structure:** Support model structure beyond autoregressive, especially Diffusion Transformer (DiT), which is widely used in visual and audio generation.
-* **Integration with vLLM Core:** Maintain compatibility and leverage existing vLLM key modules and optimizations where applicable.
-* **Extensibility:** Design a modular and flexible architecture that can easily accommodate new modalities, model architectures, and output formats.
+The architecture is designed to:

+* support text, image, audio, video, and action inputs and outputs;
+* compose autoregressive, generation, and diffusion stages in one pipeline;
+* reuse vLLM scheduling, cache, distributed-execution, and serving
+  primitives where they fit; and
+* keep model topology, deployment placement, runtime lifecycle, transport,
+  and public API concerns in separate layers.

-# Representative omni-modality models
+## Model execution

-According to analysis for current popular open-source models, most of them have the combination of AR+DiT. Specifically, they can be further categorized into 3 types below:
+These four examples show how vLLM-Omni maps model components to stage-local
+execution policies. Each summary covers the model-owned path, batching,
+attention, parallelism, quantization, and primary serving targets. The profiles
+are representative; a model may support multiple `PipelineConfig` and
+`DeployConfig` combinations.

-**DiT as a main structure, with AR as text encoder (e.g.: Qwen-Image)**
-  A powerful image generation foundation model capable of complex text rendering and precise image editing.
+| Model | Model-owned path | vLLM-Omni stage view | Primary serving target |
+| --- | --- | --- | --- |
+| Qwen3-Omni | Thinker → Talker → Code2Wav | Three stages with asynchronous chunk hand-off | TTFT/TTFP, TPOT, E2EL, audio RTF, and concurrent throughput |
+| HunyuanImage-3.0 | Multimodal AR understanding/reasoning plus image-generation DiT | AR-only, DiT-only, or split AR → DiT deployment | DiT E2EL/denoising latency and image throughput; TTFT/TPOT for AR tasks |
+| MiniMax-H3 | Text encoder → task-specific FL2VA or Ref2VA DiT → video/audio VAE decoder | Three logical component/phase boundaries within one registered diffusion stage | Video/audio E2EL and media throughput; RTF when audio streaming is measured |
+| Cosmos3 | Unified MoT reasoner tower plus diffusion generator tower | One diffusion pipeline materializes both towers inside the stage | Image/video/action E2EL and throughput; RTF for synchronized audio |
+
+### Serving metric vocabulary
+
+The target metric follows the output modality, not just the model name.
+
+| Metric | Meaning in this document | Most useful for |
+| --- | --- | --- |
+| **TTFT** | Time to the first text token | AR reasoning or text output |
+| **TPOT** | Time per output token after the first token | AR decode stages |
+| **TTFP** | Time to the first streamed media packet, such as audio | Interactive speech and multimodal chat |
+| **E2EL** | End-to-end latency from request admission to the final output | Image, video, audio, and action requests |
+| **RTF** | Wall-clock processing time divided by generated audio duration; lower is faster than real time | Audio generation |
+| **Throughput** | Requests, tokens, or media seconds produced per wall-clock second | Offline and concurrent serving |
+
+### Qwen3-Omni: streaming AR pipeline
+
+Qwen3-Omni illustrates a multi-stage streaming AR pipeline: Thinker → Talker →
+Code2Wav. The [technical report](https://arxiv.org/abs/2509.17765) describes
+the model components, and the figure shows the stage-level streaming schedule
+used by vLLM-Omni.

 <p align="center">
-  <picture>
-    <source media="(prefers-color-scheme: dark)" src="https://raw.githubusercontent.com/vllm-project/vllm-omni/refs/heads/main/docs/source/architecture/dit-main-architecture.png">
-    <img alt="Qwen-Image" src="https://raw.githubusercontent.com/vllm-project/vllm-omni/refs/heads/main/docs/source/architecture/dit-main-architecture.png" width=30%>
-  </picture>
+  <a href="https://github.com/vllm-project/vllm-omni/blob/main/docs/source/architecture/qwen3-omni-async-chunk.png">
+    <img alt="Qwen3-Omni Thinker, Talker, and Code2Wav streaming stages" src="https://raw.githubusercontent.com/vllm-project/vllm-omni/refs/heads/main/docs/source/architecture/qwen3-omni-async-chunk.png" width="80%">
+  </a>
 </p>

-**AR as a main structure, with DiT as multi-modal generator (e.g. BAGEL)**
-  A unified multimodal comprehension and generation model, with cot text output and visual generation.
+#### Model architecture by stage
+
+| Stage | Model component | Stage output |
+| --- | --- | --- |
+| **Thinker** | AuT audio encoder, SigLIP2 vision encoder, and the 30B-total/3B-active Thinker MoE Transformer | Text tokens and conditioning for the Talker |
+| **Talker** | 3B-total/0.3B-active Talker MoE Transformer with the MTP code-prediction path | Text tokens and audio codec tokens |
+| **Code2Wav** | Approximately 200M-parameter ConvNet audio decoder | Waveform/audio chunks |
+
+#### Representative stage execution
+
+| Stage | Batching | Attention and execution | Parallelism | Quantization |
+| --- | --- | --- | --- | --- |
+| **Thinker** | Continuous batching with stage-local `max_num_seqs` and token budgets | vLLM KV-cached AR attention; CUDA Graph execution when the stage is not eager | Independent stage placement; use the AR runtime's configured tensor parallelism | Supported ModelOpt FP8/NVFP4 or AutoRound checkpoint paths target the Thinker; encoders remain BF16 in the documented paths |
+| **Talker** | Continuous batching over autoregressive decode requests | KV-cached token decode with async-chunk output to Code2Wav | Independent stage placement; typically a separate GPU or replica from Thinker | BF16 baseline; no generic Talker quantization path is assumed |
+| **Code2Wav** | Static/chunk batching for codec-to-waveform decode | Non-AR chunked audio decode; eager execution is commonly retained | Separate stage or colocated placement, depending on the deployment | BF16 baseline; keep codec/vocoder precision model-specific |
+
+For this pipeline, the main serving target is interactive responsiveness:
+TTFT for text, TTFP for the first audio packet, TPOT for autoregressive decode,
+E2EL for the complete answer, and RTF/throughput for sustained audio serving.
+Async chunking and streaming primarily reduce TTFP and improve stage overlap;
+batching and CUDA Graphs primarily improve E2EL and throughput. See the
+[Qwen3-Omni recipe](https://github.com/vllm-project/vllm-omni/blob/main/recipes/Qwen/Qwen3-Omni.md)
+and [async chunk design](feature/async_chunk.md) for the deployment details.
+
+### HunyuanImage-3.0: shared multimodal model, split deployment choices
+
+HunyuanImage-3.0 illustrates a shared multimodal model with flexible AR, DiT,
+and split AR → DiT deployment. The official [technical
+report](https://arxiv.org/abs/2509.23951) provides the model view; the tables
+below map it to vLLM-Omni stages.

 <p align="center">
-  <picture>
-    <source media="(prefers-color-scheme: dark)" src="https://raw.githubusercontent.com/vllm-project/vllm-omni/refs/heads/main/docs/source/architecture/ar-main-architecture.png">
-    <img alt="Bagel" src="https://raw.githubusercontent.com/vllm-project/vllm-omni/refs/heads/main/docs/source/architecture/ar-main-architecture.png" width=30%>
-  </picture>
+  <a href="https://github.com/Tencent-Hunyuan/HunyuanImage-3.0/blob/main/assets/framework.png">
+    <img alt="HunyuanImage-3.0 framework for understanding, language modeling, and image generation" src="https://raw.githubusercontent.com/Tencent-Hunyuan/HunyuanImage-3.0/main/assets/framework.png" width="95%">
+  </a>
 </p>

-**AR+DiT (e.g. Qwen-Omni)**
-  A natively end-to-end omni-modal LLM for multimodal inputs (text/image/audio/video...) and outputs (text/audio...).
+#### Model architecture by stage
+
+| Stage or component | Model component | Stage output |
+| --- | --- | --- |
+| **AR understanding/reasoning** | Understanding and generation encoders feeding the Hunyuan-A13B decoder-only Transformer and text detokenizer | Text responses or image-generation conditioning |
+| **Image generation** | Generation encoder, diffusion prediction path/Gen. Decoder, and the image VAE decode path | Image latents and final images |
+
+The vLLM-Omni deployment may expose the AR path only, the DiT path only, or
+both as an AR → DiT pipeline with a connector between them. That is a runtime
+decomposition of the model's capabilities; it should not be read as a claim
+that the published framework figure contains two unrelated backbones.
+
+#### Representative stage execution
+
+| Stage | Batching | Attention and execution | Parallelism | Quantization |
+| --- | --- | --- | --- | --- |
+| **AR** | Standard vLLM continuous batching when serving understanding/text tasks | KV-cached causal attention | Tensor/expert parallelism follows the selected AR deployment | Checkpoint-specific; do not infer DiT FP8 settings for the AR checkpoint |
+| **DiT + VAE** | Request batching or step-wise batching; Hunyuan step batching with more than one request requires `TORCH_SDPA` | `TORCH_SDPA` is used because the model mixes causal and full attention | Validated profiles include TP4, TP2 + sequence parallelism, TP2 + CFG parallelism, and expert parallelism for MoE weights | Online FP8 and ModelOpt mixed FP8/NVFP4 are documented for the DiT; VAE precision stays separate |
+
+Image generation is therefore measured mainly with per-image E2EL/denoising
+latency, peak memory, and images-per-second throughput. TTFT and TPOT remain
+meaningful for the optional AR understanding path, but they are not the right
+headline metrics for the DiT path. See the
+[HunyuanImage-3.0 recipe](https://github.com/vllm-project/vllm-omni/blob/main/recipes/Tencent/HunyuanImage-3.0-Instruct.md)
+for the validated batching, attention, and quantization combinations.
+
+### MiniMax-H3: conditioning, denoising, and decode
+
+MiniMax-H3 illustrates logical component and phase boundaries between text
+encoding, task-specific denoising, and media decoding:
+`Text Encoder → DiT → VAE Decoder`. The current registered topology co-locates
+these components in one diffusion stage. These boundaries are useful for
+profiling and offload; independent placement would require a future
+multi-stage `PipelineConfig`. The stage mapping follows the implementation and
+serving recipe; the [release overview](https://minimaxi.com/blog/minimax-h3)
+notes that a detailed technical report is forthcoming.
+
+#### Model architecture by component
+
+| Logical component | Model component | Component output |
+| --- | --- | --- |
+| **Text Encoder** | Tokenizer/processor and Qwen3-VL text encoder; vision/audio references are prepared alongside the text condition | Packed multimodal conditioning for denoising |
+| **DiT** | Task-specific `FL2VA` DiT for text/first-frame generation or `Ref2VA` DiT for mixed references | Video and audio latent tokens |
+| **VAE Decoder** | Video VAE and audio VAE decode the generated latents and assemble the synchronized output | Decoded video frames and stereo waveform, with FPS and audio sample rate metadata |
+
+Both DiT partitions remain behind one task-selection point; `task` chooses
+which partition runs. In the current implementation, the three logical
+components execute inside one diffusion stage. This is different from
+Qwen3-Omni's three independently scheduled stages, even though both models
+produce synchronized audio and video-like outputs.
+
+#### Representative component execution
+
+| Logical component | Batching | Attention and execution | Parallelism | Quantization |
+| --- | --- | --- | --- | --- |
+| **Text Encoder** | The current co-located pipeline follows the diffusion request scheduler; independent prompt/reference batching would require a separate multi-stage topology | Qwen3-VL attention and multimodal preprocessing | `--text-encoder-tp-size N` shards the encoder across the first `N` DiT ranks; independent placement is not part of the current topology | BF16/FP32 baseline; the H3 DiT FP8 path does not quantize the text encoder |
+| **DiT** | The current H3 implementation executes one generation request per diffusion batch; use concurrency for service-level throughput | cuDNN attention for the validated two-GPU consumer profile; TRTLLM attention or FlashAttention-4 on supported Blackwell profiles | TP2 plus text-encoder TP and Ulysses/VAE parallel groups on multi-GPU profiles; DLO/CPU offload trade memory for transfer time | Online FP8 applies to eligible DiT linears and is incompatible with layerwise offload |
+| **VAE Decoder** | Decode follows each generation request; tile/patch work can be distributed even when denoising is not batched | VAE decode kernels rather than DiT attention | VAE patch parallelism and native tiled decode within the diffusion stage | BF16/FP32 baseline; the documented H3 FP8 path leaves both VAEs unchanged |
+
+The primary target is video/audio E2EL and media throughput, with the logical
+component boundaries making prompt encoding, denoising, and VAE decode costs
+separately measurable. RTF is useful when the audio stream is evaluated
+independently. TTFT and TPOT do not describe the main H3 generation path
+because the user-visible output is diffusion media rather than a token stream.
+The
+[MiniMax-H3 recipe](https://github.com/vllm-project/vllm-omni/blob/main/recipes/MiniMaxAI/MiniMax-H3.md)
+contains the hardware-specific profiles and warmup requirements.
+
+### Cosmos3: unified MoT reasoner and generator
+
+Cosmos3 illustrates a unified Mixture-of-Transformers (MoT) reasoner and
+diffusion generator. The official [technical
+report](https://research.nvidia.com/labs/cosmos-lab/cosmos3/technical-report.pdf)
+and NVIDIA's [technical explanation](https://developer.nvidia.com/blog/develop-physical-ai-reasoning-world-and-action-models-with-nvidia-cosmos-3/)
+provide the two-tower model view used below.

 <p align="center">
-  <picture>
-    <source media="(prefers-color-scheme: dark)" src="https://raw.githubusercontent.com/vllm-project/vllm-omni/refs/heads/main/docs/source/architecture/ar-dit-main-architecture.png">
-    <img alt="Qwen-Omni" src="https://raw.githubusercontent.com/vllm-project/vllm-omni/refs/heads/main/docs/source/architecture/ar-dit-main-architecture.png" width=30%>
-  </picture>
+  <a href="https://developer.nvidia.com/blog/develop-physical-ai-reasoning-world-and-action-models-with-nvidia-cosmos-3/">
+    <img alt="Cosmos3 autoregressive reasoner and diffusion generator towers" src="https://developer-blogs.nvidia.com/wp-content/uploads/2026/05/image-11-e1780000686151.webp" width="95%">
+  </a>
 </p>

-# vLLM-Omni main architecture
+#### Model architecture by stage

-<p align="center">
-  <picture>
-    <source media="(prefers-color-scheme: dark)" src="https://raw.githubusercontent.com/vllm-project/vllm-omni/refs/heads/main/docs/source/architecture/vllm-omni-main-architecture.png">
-    <img alt="vLLM-Omni Main Architecture" src="https://raw.githubusercontent.com/vllm-project/vllm-omni/refs/heads/main/docs/source/architecture/vllm-omni-main-architecture.png" width=55%>
-  </picture>
-</p>
+| Stage or component | Model component | Stage output |
+| --- | --- | --- |
+| **Reasoner tower** | Autoregressive VLM path with causal attention over language and optional vision/action context; runs once per generation request | Discrete reasoning/context tokens |
+| **Generator tower** | Diffusion path with full attention over the generator tokens and the reasoner K/V context; runs for each denoising step | Image, video, audio, or action tokens |
+| **Shared pipeline** | `Cosmos3OmniDiffusersPipeline` plus modality encoders/decoders and VAE paths | Final image/video/audio or action response |

-## Key Components
+The reasoner and generator are model-owned components, not two independent
+vLLM-Omni stages by default. The current Cosmos3 pipeline materializes both
+inside one diffusion stage; model-level CPU offload can swap the reasoner and
+generator components between their phases.

-| Component         | Description                                                                                                                              |
-| ----------------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
-| **OmniRouter**    | provide an intelligent router for Omni-modality requests dispatch                                                                        |
-| **EntryPoints**   | define the APIs for offline/online serving (APIServer, Omni/AsyncOmni), while `AsyncOmniEngine` and `Orchestrator` coordinate multi-stage AR/DiT execution |
-| **AR**            | adapted for omni-modality models while inheriting efficient features from vLLM, such as cache management                                 |
-| **Diffusion**     | natively implemented and optimized using acceleration components                                                                         |
-| **OmniConnector** | supports fully disaggregation based on E/P/D/G (Encoding/Processing/Decoding/Generation) disaggregation across stages                    |
+#### Representative stage execution

-Disaggregated stages are managed through stage configuration. In Qwen3-Omni, Thinker/Talker/Code2wav are declared as separate configured stages, and runtime routing is handled by `Orchestrator` over `StageEngineCoreClient` / `StageDiffusionClient`.
+| Stage | Batching | Attention and execution | Parallelism | Quantization |
+| --- | --- | --- | --- | --- |
+| **Reasoner + generator pipeline** | Use request batching or step execution only when the selected pipeline advertises that capability; otherwise use `max_num_seqs=1` | Causal attention for reasoner tokens and full attention for diffusion tokens; the validated recipe uses a platform-selected aiter FlashAttention path with `TORCH_SDPA` fallback | Ulysses/context parallelism or tensor parallelism for the transformer; VAE tiling and layerwise/model-level offload address activation and weight memory | Online FP8 is supported for the diffusion pipeline; AutoRound/checkpoint-specific paths and quality validation remain separate from the BF16 baseline |
+
+For Cosmos3, generation E2EL and output throughput are the primary targets.
+Action-policy workloads add a control-loop latency target, while synchronized
+audio workloads add RTF. TTFT/TPOT are appropriate for a reasoner-only service,
+but not for the full diffusion generation path. See the
+[Cosmos3 recipe](https://github.com/vllm-project/vllm-omni/blob/main/recipes/cosmos3/Cosmos3-Nano.md)
+and [diffusion execution modes](../user_guide/diffusion/execution_modes.md)
+for the current runtime choices.
+
+## Why the stage-based architecture follows from these examples
+
+The four representative pipelines show that omni-modality serving is not one
+execution problem. Autoregressive decoding, diffusion denoising, multimodal
+encoding, and media decoding have different scheduling, attention, parallelism,
+memory, and quantization requirements. The architecture therefore needs a
+unit that can be optimized and resourced independently while remaining part of
+one request pipeline.
+
+| Observation | Design implication | vLLM-Omni response |
+| --- | --- | --- |
+| AR, diffusion, encoder, and decoder components have different execution loops | One scheduler and one execution policy cannot fit every component | Specialized AR, generation, and diffusion stage runtimes |
+| A model can be deployed as AR-only, DiT-only, or AR → DiT | Model structure must be separate from deployment topology | `PipelineConfig` describes logical stages and relationships; `DeployConfig` describes placement and resources |
+| Text encoders, DiTs, and VAE decoders have different batch and memory behavior | Batching, attention, parallelism, and quantization must be stage-local | Each stage carries its own runtime and engine configuration |
+| MiniMax-H3 can co-locate or logically split its text encoder and VAE decoder | A stage boundary should be configurable rather than forcing a process boundary | Logical stages can be co-located or assigned independent runtime placement |
+| Qwen3-Omni transfers hidden states and codec chunks across stages | Large intermediate data needs typed transport and synchronization | `OmniConnector` transports payloads and KV-cache data without owning model routing |
+| Requests span multiple stages and may be cancelled or produce ordered streaming output | Cross-stage request state needs a separate control plane | `Orchestrator` owns correlation, cancellation, routing state, and output ordering |
+| Qwen3-Omni optimizes first-packet latency while image/video models optimize E2EL and throughput | Performance objectives must be model- and stage-aware | TTFT/TPOT/TTFP, E2EL, RTF, and throughput are measured according to the output modality |
+
+This leads to a precise definition of a stage: it is a logical execution unit
+with its own model component, execution policy, resource ownership, and
+performance objectives. A stage is not necessarily a separate process. Cosmos3
+keeps its reasoner and generator inside one diffusion pipeline, while
+MiniMax-H3 exposes useful logical boundaries between text encoding, denoising,
+and VAE decoding. Both are valid mappings of the same abstraction.
+
+The resulting separation can be summarized as:
+
+```text
+Model structure       What is computed and how components are related
+        ↓
+PipelineConfig        Which logical stages exist and how they are connected
+        ↓
+Stage runtime policy  How each stage batches, attends, parallelizes, and quantizes
+        ↓
+DeployConfig           Where stages run, with which devices, replicas, and connectors
+```
+
+`AsyncOmniEngine` and `Orchestrator` coordinate the pipeline, `OmniConnector`
+transports stage outputs, and `StageRuntime` materializes the selected
+placement. This separation lets one logical model support multiple serving
+topologies without coupling model code to process layout or deployment
+resources.
+
+## System architecture
+
+The figure shows the main components of the stage-based runtime: entrypoints,
+the shared `AsyncOmniEngine`, request orchestration, stage lifecycle
+management, AR and diffusion execution modules, model/layer operations,
+connector transport, and multimodal outputs. `AsyncOmniEngine` is the
+composition root between the entrypoints and the orchestrator.
+
+```mermaid
+flowchart TB
+    subgraph omni["vLLM-Omni"]
+        subgraph entry["Entry points"]
+            api["API server"]
+            client["Omni / AsyncOmni"]
+        end
+        engine["AsyncOmniEngine<br/>composition root and background loop"]
+        orchestrator["Orchestrator<br/>request state and cross-stage routing"]
+        runtime["StageRuntime<br/>placement, replicas, readiness, and lifecycle"]
+        ar["AR module<br/>vLLM engine, scheduler, cache, executor, worker, and model runner"]
+        diffusion["Diffusion module<br/>scheduler, worker, model runner, and pipeline"]
+        ops["Model / layer / ops<br/>attention, parallelism, and quantization"]
+        connector["OmniConnector<br/>payload and KV transport plus synchronization"]
+        outputs["Multimodal outputs<br/>streaming chunks and final artifacts"]
+
+        api --> engine
+        client --> engine
+        engine --> orchestrator
+        engine --> runtime
+        orchestrator --> runtime
+        runtime --> ar
+        runtime --> diffusion
+        ar -. uses .-> ops
+        diffusion -. uses .-> ops
+        ar -. transports through .-> connector
+        diffusion -. transports through .-> connector
+        ar --> outputs
+        diffusion --> outputs
+    end
+```
+
+### Key components
+
+| Component | Responsibility |
+| --- | --- |
+| **Entrypoints** | Translate offline, CLI, OpenAI-compatible, and duplex requests into engine operations and render outputs back to public protocols. |
+| **Configuration resolution** | Combines pipeline topology, deployment settings, model metadata, and user overrides into a validated control-plane configuration. |
+| **AsyncOmniEngine** | Owns engine composition, the background event loop, stage initialization, request submission, and output collection. |
+| **Orchestrator** | Owns cross-stage request state, stage-to-stage routing, request correlation, cancellation, and output ordering. It does not own model selection or deployment placement. |
+| **StageRuntime** | Expands logical stages into local or distributed replicas, starts stage clients and processes, and manages readiness, affinity, failure, and shutdown. |
+| **AR runtime** | Extends vLLM's scheduler, KV-cache, worker, and model-runner path for omni-modality inputs and inter-stage outputs. |
+| **Diffusion runtime** | Schedules and executes denoising workloads through diffusion executors, workers, pipelines, acceleration backends, and output materialization. |
+| **OmniConnector** | Transports stage payloads and KV-cache data and provides synchronization. Connectors transport data; they do not choose the next logical stage. |
+| **Multimodal outputs** | `MultimodalPayload` separates tensor content from metadata, while `OmniRequestOutput` carries pipeline and diffusion results through the common output path. |
+
+## Configuration and runtime resolution
+
+The control plane has five conceptual layers. Authoring inputs are resolved
+once into a complete, transport-safe configuration before runtime processes are
+started. This keeps stage topology and model capabilities distinct from
+deployment placement and from process-local engine objects.
+
+The figure shows only the primary hand-off object for each layer; the detailed
+inputs, fields, and ownership rules are described below.
+
+```mermaid
+flowchart TB
+    layer1["Layer 1 · Authoring inputs<br/>PipelineConfig + DeployConfig"]
+    layer2["Layer 2 · Resolve once<br/>StageConfigFactory.create_from_model()<br/>VllmOmniConfig.from_pipeline_config()"]
+    layer3["Layer 3 · Transport-safe control plane<br/>VllmOmniConfig"]
+    layer4["Layer 4 · Runtime launch planning<br/>StageRuntime"]
+    layer5["Layer 5 · Engine materialization<br/>VllmConfig / OmniDiffusionConfig"]
+
+    layer1 --> layer2 --> layer3 --> layer4 --> layer5
+```
+
+The single resolution boundary is implemented by
+`StageConfigFactory.create_from_model()` and
+`VllmOmniConfig.from_pipeline_config()` in
+[`vllm_omni/config`](https://github.com/vllm-project/vllm-omni/tree/main/vllm_omni/config).
+The legacy `stage_args` YAML path remains only for models that have not yet
+migrated to `PipelineConfig` and `DeployConfig`.
+
+In the typed path, each stage configuration derives from
+`BaseVllmOmniStageConfig` and is specialized as
+`VllmOmniARStageConfig`, `VllmOmniGenerationStageConfig`, or
+`VllmOmniDiffusionStageConfig`. The request and engine-spec fields belong to the
+control-plane boundary; the current implementation stores their equivalent
+projections in the structured stage configuration and materializes
+backend-specific engine objects during stage initialization.
+
+The important ownership rules are:
+
+1. `PipelineConfig` is the source of truth for stage topology, execution type,
+   model capabilities, and stage relationships.
+2. `DeployConfig` describes placement, replicas, devices, connectors, and
+   deploy-time defaults; it does not redefine the model graph.
+3. CLI and Python overrides are applied at the resolution boundary, with
+   per-stage overrides taking precedence over global values where supported.
+4. `StageRuntime` owns launch planning and replica lifecycle. `ReplicaInitPlan`
+   is runtime-private state, not a user configuration object.
+5. `VllmConfig` and the enriched `OmniDiffusionConfig` are materialized in the
+   process that owns the corresponding engine.

 ## Main features

-vLLM-Omni aims to be fast, flexible, and easy to use with the following features:
+The feature surface is grouped to match the
+[feature design documents](index.md#feature-design-documents). This page
+summarizes each feature's architectural role; the linked design document is
+the source for configuration, compatibility, and implementation details.

-### Performance and Acceleration
+### Runtime and stage execution

-The framework achieves high performance through several optimization techniques:
+* **Disaggregated inference:** Logical stages can run in separate processes,
+  devices, or nodes while the orchestrator preserves their declared
+  relationships. `OmniConnector` implementations transfer stage data and
+  control-plane metadata. See [Disaggregated Inference](feature/disaggregated_inference.md).
+* **Asynchronous stage and output execution:** [Async Chunk](feature/async_chunk.md)
+  forwards partial stage outputs as they become available. [Async Diffusion
+  Output](feature/async_diffusion_output.md) overlaps device-to-host output
+  packing with the next diffusion request, while [Async Omni Output
+  Materialization](feature/omni_async_output_materialization.md) moves
+  CPU-side payload construction off the AR decode critical path.
+* **Automatic prefix caching:** [Automatic Prefix Caching in Omni Models](feature/prefix_caching.md)
+  reuses KV-cache-aligned stage outputs and multimodal tensors for requests
+  with common prefixes.

-* **Efficient AR Support:** Leverages efficient KV cache management inherited from vLLM.
-* **Pipelined Execution:** Uses pipelined stage execution overlapping to ensure high throughput.
-* **Full Disaggregation:** Relies on the OmniConnector and dynamic resource allocation across stages.
-* **Diffusion Acceleration:** Includes integrated support for diffusion acceleration. This is managed by the acceleration layer, which handles:
-    * **Cache:** Includes DBCache, TeaCache and third-party integration(e.g., [cache-dit](https://github.com/vipshop/cache-dit)).
-    * **Parallelism:** Supports TP, CP, USP, and CFG.
-    * **Attention:** Provides an interface for third-party integration (e.g., FA3, SAGE, MindIE-SD).
-    * **Quantization:** Supports various quantization implementations including FP8 and AWQ.
-    * **FusedOps:** Allows for custom and third-party integration.
+### Communication

-### Classifier-Free Guidance (CFG) Companion Flow
+* **OmniConnector transport:** The connector contract carries tensors, KV-cache
+  data, and transport metadata across stage boundaries. The available
+  implementations cover shared memory and multi-node Mooncake, Mori, and
+  Yuanrong transports; see [Disaggregated Inference](feature/disaggregated_inference.md)
+  for the connector choices and configuration model.

-vLLM-Omni natively models Classifier-Free Guidance (CFG) across disaggregated multi-stage setups via a "companion request" paradigm, eliminating redundant textual/multimodal context computation boundaries:
-1. **Prompt Expansion:** In the initial autoregressive (AR) stage, a customized `prompt_expand_func` hook intercepts incoming generation prompts and pairs them directly with negative companion prompts (e.g., a default negative prompt) on the fly, tagging the secondary prompt with a specific internal role (`cfg_text`).
-2. **Synchronized KV Cache Transfer:** The AR stage evaluates both the primary and companion sequence batches concurrently. The `OmniConnector` captures these specific structural dependencies and reliably passes the positive and negative outcome KV caches seamlessly across stage boundaries via shared memory or network protocols.
-3. **KV Cache Collection & Injection:** Upon reaching the downstream Diffusion (DiT) Engine, an assigned `cfg_kv_collect_func` automatically intercepts the mapped companion caches (`cfg_text_past_key_values`). These auxiliary dependencies are natively gathered and seamlessly bound to the primary generation sequence variables, enabling the DiT Engine to cleanly implement cross-attention CFG guidance over accurate conditioning and unconditioning structures in parallel.
+### Diffusion acceleration

-### Flexibility and Usability
+* **Request and step batching:** [Diffusion Continuous Batching](feature/diffusion_continuous_batching.md)
+  defines request-batch and step-batch execution, scheduler admission, and the
+  common streaming output path.
+* **Composable parallelism:** Diffusion stages can combine [CFG-Parallel](feature/cfg_parallel.md),
+  [Expert Parallel](feature/expert_parallel.md), [HSDP](feature/hsdp.md),
+  [Pipeline Parallel](feature/pipeline_parallel.md), [Sequence Parallel](feature/sequence_parallel.md),
+  [Tensor Parallel](feature/tensor_parallel.md), and [VAE Patch Parallelism](feature/vae_parallel.md)
+  according to the pipeline and hardware topology.
+* **Attention and cache acceleration:** [Skip-Softmax](feature/skip_softmax.md),
+  [Cache-DiT](feature/cache_dit.md), and [TeaCache](feature/teacache.md)
+  provide backend and denoising-step optimizations without changing the
+  stage contract.
+* **Quantization and memory efficiency:** [Quantization](feature/quantization.md)
+  resolves per-pipeline or per-component quantization configurations, while
+  [Distributed Layerwise Offload](feature/distributed_layerwise_offload.md)
+  streams diffusion blocks from host memory within the existing parallel
+  topology.

-vLLM-Omni is designed to be flexible and straightforward for users:
+## Interfaces

-* **Heterogeneous Pipeline Abstraction:** Manages complex model workflows effectively.
-* **Hugging Face Integration:** Offers seamless integration with popular Hugging Face models.
-* **Distributed Inference:** Supports tensor, pipeline, data, and expert parallelism.
-* **Streaming Outputs:** Supports streaming outputs.
-* **Unified API:** Provides a consistent and unified API interface compatible with vLLM.
-* **OpenAI-compatible API Server:** Includes a FastAPI-based server for online serving that is compatible with the OpenAI API.
-
-# Interface design
-
-If you use vLLM, then you know how to use vLLM-Omni from Day 0:
-
-<p align="center">
-  <picture>
-    <source media="(prefers-color-scheme: dark)" src="https://raw.githubusercontent.com/vllm-project/vllm-omni/refs/heads/main/docs/source/architecture/vllm-omni-user-interface.png">
-    <img alt="vLLM-Omni interface design" src="https://raw.githubusercontent.com/vllm-project/vllm-omni/refs/heads/main/docs/source/architecture/vllm-omni-user-interface.png" width=55%>
-  </picture>
-</p>
-
-Taking **Qwen3-Omni** as an example:
-
-## Offline Inference
-The **Omni** class provides a Python interface for offline batched inference. Users initialize the Omni class with a Hugging Face model name and use the generate method, passing inputs that include both text prompts and multi-modal data:
+The public interfaces map onto the same engine and stage boundaries:

+```mermaid
+flowchart LR
+    offline["Offline Python<br/>Omni.generate()"] --> engine["AsyncOmniEngine"]
+    online["OpenAI-compatible serving<br/>vllm serve ... --omni"] --> engine
+    duplex["Experimental duplex WebSocket<br/>/v1/duplex or realtime duplex"] --> engine
+    engine --> stages["Configured AR and diffusion stages"]
+    stages --> result["Streaming or final multimodal output"]
 ```
-# Create an omni runtime with HF model name.
+
+### Offline inference
+
+The **Omni** class provides a Python interface for offline batched inference:
+
+```python
 from vllm_omni.entrypoints.omni import Omni

 omni = Omni(model="Qwen/Qwen3-Omni-30B-A3B-Instruct")

-# Example prompts.
-om_inputs = {"prompt": prompt,
-             "multi_modal_data": {
-                 "video": video_frames,
-                 "audio": audio_signal,
-             }}
+om_inputs = {
+    "prompt": prompt,
+    "multi_modal_data": {
+        "video": video_frames,
+        "audio": audio_signal,
+    },
+}

-# Generate texts and audio from the multi-modality inputs.
 outputs = omni.generate(om_inputs, sampling_params_list)
 ```

-## Online Serving
-Similar to vLLM, vLLM-Omni also provides a FastAPI-based server for online serving. Users can launch the server using the vllm serve command with the `--omni` flag:
+### Online serving

-```
+The OpenAI-compatible server uses the same stage configuration and engine
+boundaries:
+
+```bash
 vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091
 ```

-Users can send requests to the server using curl:
+For example, a Qwen3-Omni chat request can contain text, image, audio, or
+video content and a `sampling_params_list` for its configured stages. See the
+[Qwen3-Omni serving example](../user_guide/examples/online_serving/qwen3_omni.md)
+and the [examples](https://github.com/vllm-project/vllm-omni/tree/main/examples)
+for complete requests.

-```
-# prepare user content
-user_content='[
-        {
-          "type": "video_url",
-          "video_url": {
-            "url": "'"$SAMPLE_VIDEO_URL"'"
-          }
-        },
-        {
-          "type": "text",
-          "text": "Why is this video funny?"
-        }
-      ]'
-    sampling_params_list='[
-      '"$thinker_sampling_params"',
-      '"$talker_sampling_params"',
-      '"$code2wav_sampling_params"'
-    ]'
-    mm_processor_kwargs="{}"
-
-# send the request
-curl -sS -X POST http://localhost:8091/v1/chat/completions \
-    -H "Content-Type: application/json" \
-    -d @- <<EOF
-{
-  "model": "Qwen/Qwen3-Omni-30B-A3B-Instruct",
-  "sampling_params_list": $sampling_params_list,
-  "mm_processor_kwargs": $mm_processor_kwargs,
-  "messages": [
-    {
-      "role": "system",
-      "content": [
-        {
-          "type": "text",
-          "text": "You are Qwen, a virtual human developed by the Qwen Team, Alibaba Group, capable of perceiving auditory and visual inputs, as well as generating text and speech."
-        }
-      ]
-    },
-    {
-      "role": "user",
-      "content": $user_content
-    }
-  ]
-}
-```
-
-For more usages, please refer to [examples](https://github.com/vllm-project/vllm-omni/tree/main/examples).
+Some pipelines expose additional OpenAI-compatible endpoints, such as joint
+video/audio generation. Endpoint support remains model-specific; consult the
+relevant model guide before assuming that every OpenAI route applies to every
+pipeline.
@@ -20,11 +20,12 @@ extracts it into a shared, typed contract:
  (matching the bespoke caches it replaces), and reports per-device byte usage
  via `stats()` for observability.

-Attention KV is *not* managed here: for DreamZero it is owned by the
-AR-Diffusion engine's paged KV pool (PR
-[#4534](https://github.com/vllm-project/vllm-omni/pull/4534)). The manager
-covers the model's non-KV session state; integrating the engine's per-session
-KV handle into the same byte accounting is RFC #4480 Phase 1.
+Autoregressive generation KV is *not* managed here: for DreamZero it is owned
+by the AR-Diffusion engine's paged KV pool (PR
+[#4534](https://github.com/vllm-project/vllm-omni/pull/4534)). Cosmos3's fixed,
+encode-once UND conditioning K/V is different and is covered by the dense
+Cosmos3 adapter described below. Integrating the engine's per-session paged KV
+handle into the same byte accounting is RFC #4480 Phase 1.

 ## Enabling

@@ -158,3 +159,62 @@ of `num_frame_per_block` frames instead: identical output, bounded memory, and
 the per-step quantisation no longer scales with session length. Asking for more
 frames than the ring retains raises rather than silently returning a shorter
 window. The bespoke path is unchanged.
+
+### Cosmos3
+
+When `enable_session_state_manager` is enabled, the Cosmos3 adapter stores the
+dense, encode-once UND text K/V for every transformer layer and active CFG
+branch, together with that branch's GEN M-RoPE cosine/sine tensors. These are
+GPU-resident tensors and can contribute to CUDA OOM. A useful description of
+the retained storage on one rank is:
+
+```text
+sum_over_branches(
+  sum_over_layers(storage_bytes(K) + storage_bytes(V))
+  + storage_bytes(freqs_gen_cos) + storage_bytes(freqs_gen_sin)
+)
+```
+
+This is not a peak-memory upper bound: construction of the next branch can
+temporarily overlap the already retained branch with UND activations, the new
+full K/V result, RoPE tensors, and generator activations. K/V slices also keep
+their underlying allocation alive, so physical storage can follow the padded
+text length rather than only the exposed trimmed length. The manager's byte
+accounting deliberately measures storage for this reason.
+
+Branch placement changes the per-rank cost:
+
+- no CFG retains one conditional branch;
+- sequential CFG can retain both conditional and unconditional branches on the
+  same GPU;
+- CFG-parallel retains one branch on each CFG rank;
+- UND K/V is replicated across sequence-parallel ranks rather than divided by
+  the SP degree.
+
+Consequently, risk increases with text batch/sequence length, media token count
+(which sizes `freqs_gen`), sequential CFG, dtype width, and concurrent live
+requests. Phase 0 records `nbytes:cuda:*` but does not enforce `byte_budget`, so
+the manager cannot prevent an allocation from exhausting the device. The
+count-based session limit is not an OOM guarantee.
+
+The adapter transfers ownership by tensor reference; it does not clone K/V or
+RoPE tensors. For a normal serial request this should not materially increase
+tensor storage relative to Cosmos3's existing bespoke cond/uncond cache, which
+retains the same branch values. It does add small CPU-side session/object
+metadata. This claim is based on the reference flow in the implementation;
+there is not yet a current-head GPU peak-memory A/B result for this PR.
+
+Request-finally cleanup drops the manager-owned session, so branches do not
+accumulate with completed request IDs. The transformer handoff field still
+references the last loaded branch until `reset_cache()` runs at the start of
+the next generation (or the pipeline is destroyed). Thus this is bounded
+post-request residency, not growth per completed request, but manager
+`stats()` after `drop_session()` no longer includes that last transformer-held
+alias. A profiler that only reads manager statistics can therefore under-report
+live process memory.
+
+For OOM validation, measure complete cache-off and true flag-on generations in
+otherwise identical fresh processes. Synchronize CUDA and report process-level
+peak memory plus PyTorch peak allocated and peak reserved memory. Include the
+period between requests when checking cleanup; do not infer safety from the
+manager byte total, a single denoising step, or an operator-only profile.
@@ -97,3 +97,10 @@ th {
 |`HiDreamImagePipeline` | HiDream-I1-Full | `HiDream-ai/HiDream-I1-Full` | ✅︎ | ✅︎ | | |

 ✅︎ indicates the model is supported on that backend. Empty cells mean not listed as supported on that backend.
+
+<sup>H3</sup> MiniMax H3 AMD GPU support is validated on gfx942 (MI300X) and
+gfx950 (MI350) in BF16, using the AITER `FLASH_ATTN` diffusion attention backend.
+Other AMD SKUs (e.g. MI325X) are not listed until their
+own validation evidence is added. See the
+[MiniMax H3 recipe](https://recipes.vllm.ai/MiniMaxAI/MiniMax-H3) for
+per-architecture serving commands.
@@ -58,6 +58,10 @@ FlashAttention-4 extra:
 uv pip install -e '.[fa4]'
 ```

+On AMD ROCm, install without the `[fa4]` extra (FA4 is CUDA-only) and use
+`--diffusion-attention-backend FLASH_ATTN`; see
+[AMD ROCm (gfx942 / gfx950)](#amd-rocm-gfx942--gfx950).
+
 `ffmpeg` and `ffprobe` must be available on `PATH`. They are used for
 reference-video preparation and MP4 output.

@@ -387,6 +391,101 @@ incompatible with H3 layerwise offload because the offload path produces a
 weight stride rejected by the Cutlass FP8 kernel. Use resident FP8 with tensor
 parallelism and VAE tiling instead.

+## AMD ROCm (gfx942 / gfx950)
+
+The sections above describe the NVIDIA CUDA path. This section covers the AMD
+ROCm path; use it instead of the CUDA commands when running on AMD Instinct
+GPUs.
+
+MiniMax H3 runs on AMD Instinct GPUs (gfx942 / gfx950) in BF16. Use
+`--diffusion-attention-backend FLASH_ATTN`, which resolves to AITER packed varlen
+attention on both architectures.
+
+Select the device with `HIP_VISIBLE_DEVICES` (not `CUDA_VISIBLE_DEVICES`), drop the
+CUDA-only `FLASHINFER_DISABLE_VERSION_CHECK`, and install without the `[fa4]` extra
+(FA4 is CUDA-only). The VAE uses AITER GroupNorm on ROCm.
+
+Install (ROCm wheel + source vLLM-Omni):
+
+```bash
+pip install "vllm==0.26.0+rocm723" \
+  --extra-index-url https://wheels.vllm.ai/rocm/0.26.0/rocm723
+VLLM_OMNI_TARGET_DEVICE=rocm pip install -e . --no-build-isolation
+```
+
+Prebuilt image: `vllm/vllm-omni-rocm:minimax-h3`. All tasks work out of the box:
+the image bundles TorchCodec (for image+audio Ref2VA) and `ffmpeg` (for
+video-reference Ref2VA).
+
+### ROCm single GPU
+
+Single GPU with model-level CPU offload keeps the Qwen3-VL encoder and DiT from
+being co-resident:
+
+```bash
+export MODEL="${MODEL_ROOT}/FL2VA"
+export PORT=8091
+
+HIP_VISIBLE_DEVICES=0 \
+VLLM_WORKER_MULTIPROC_METHOD=spawn \
+VLLM_OMNI_VIDEO_SYNC_TIMEOUT=1800 \
+vllm serve "${MODEL}" \
+  --omni --host 0.0.0.0 --port "${PORT}" --trust-remote-code \
+  --num-gpus 1 --enable-cpu-offload \
+  --diffusion-attention-backend FLASH_ATTN
+```
+
+### ROCm four GPUs
+
+The best-practice CUDA four-GPU configuration works on ROCm with the changes above.
+It mirrors the CUDA command including Ulysses sequence parallelism, VAE patch
+parallelism, and text-encoder tensor parallelism, with no CPU offload when the model
+shards across the GPUs:
+
+```bash
+export MODEL="${MODEL_ROOT}/FL2VA"
+export PORT=8091
+
+HIP_VISIBLE_DEVICES=0,1,2,3 \
+VLLM_WORKER_MULTIPROC_METHOD=spawn \
+VLLM_OMNI_VIDEO_SYNC_TIMEOUT=1800 \
+vllm serve "${MODEL}" \
+  --omni \
+  --host 0.0.0.0 \
+  --port "${PORT}" \
+  --trust-remote-code \
+  --num-gpus 4 \
+  --usp 4 \
+  --ring 1 \
+  --text-encoder-tp-size 4 \
+  --vae-patch-parallel-size 4 \
+  --vae-parallel-mode tile \
+  --vae-use-tiling \
+  --diffusion-attention-backend FLASH_ATTN
+```
+
+As on CUDA, H3 is CFG-distilled, so keep `--cfg-parallel-size` at 1, and the H3 VAE
+supports only its native `tile` mode. `--text-encoder-tp-size` is validated on
+gfx942; on gfx950 it was not exercised.
+
+### Validated ROCm evidence
+
+vLLM-Omni with MiniMax H3 support, BF16. gfx942 rows measured with the
+`vllm/vllm-omni-rocm:minimax-h3` image; gfx950 rows measured with the
+`0.26.0+rocm723` wheel (HIP 7.2).
+
+| Workload | Configuration | Observed result |
+|----------|---------------|-----------------|
+| T2VA, 1344x768, 209 frames, 50 steps | 4x gfx942 (MI300X), FLASH_ATTN, USP4, text-enc TP4, VAE PP4 tile | encode 0.09 s, denoise 244.04 s, decode 4.15 s, 267.42 s client E2E; H.264 24 FPS + 32 kHz stereo AAC |
+| FL2VA, 1344x768, 209 frames, 50 steps | 4x gfx942 (MI300X), FLASH_ATTN, USP4, text-enc TP4, VAE PP4 tile | encode 13.98 s, denoise 257.58 s, decode 4.11 s, 287.07 s client E2E; H.264 24 FPS + 32 kHz stereo AAC |
+| T2VA, 832x480, ~4 s, 40 steps | 1x gfx950 (MI350), FLASH_ATTN, CPU offload | valid MP4 (H.264 + synced audio); ~0.73 s/denoise-step (~1.37 it/s), ~55 s client E2E incl. warmup |
+
+gfx942 figures are the mean of three requests after one excluded warmup
+(external evidence: vllm-project/recipes#732). gfx950 figures are
+functional-correctness validations, not tuned throughput; the first request
+includes lazy regional compilation. MI325X (gfx942) and other MI355X SKUs are not
+listed until their own evidence is added.
+
 ## HTTP API examples

 The following requests use the synchronous endpoint so the returned body can
@@ -658,12 +757,22 @@ reduction.
 - H3 currently executes one generation request per diffusion batch.
 - The first regional-compile request is a warmup and should not be included in
  steady-state performance measurements.
+- The serving path accepts fewer references than the model supports. H3 documents up
+  to 9 images, 3 video clips, and 3 audio clips (12 files) per Omni Reference
+  request; the current vLLM-Omni path takes exactly one image plus one audio
+  reference, or one or more videos with no separate `audio_reference` (it uses the
+  source soundtracks).
+- The 768 px short-edge mode is available for T2VA and FL2VA; 1344x768 is the
+  documented 16:9 request shape.
+- `--cfg-parallel-size > 1` is rejected by design (CFG-distilled, no negative branch).
+- VAE patch parallelism requires size 1 or the full DiT group size and supports the
+  H3 native `tile` mode only.
+- A U2 x Ring2 hybrid currently fails with an attention-mask length mismatch; use
+  pure Ulysses.
 - Online FP8 is not compatible with layerwise offload.
- Image+audio Ref2VA accepts exactly one image and one audio reference.
- Video Ref2VA accepts one or more video files, but not an additional standalone
-  audio reference.
- VAE patch parallelism requires size 1 or the full DiT group size and supports
-  the H3 native `tile` mode only.
+- Pure Ulysses still replicates the full DiT on every rank, so smaller-memory GPUs
+  cannot use `--usp N --tp 1` as a resident capacity path. Use DiT tensor parallelism
+  or model-level CPU offload; text-encoder TP alone is not sufficient.

 ## Additional resources

@@ -1805,6 +1805,7 @@ class TestQwen2_5OmniPipeline:
        assert s.owns_tokenizer is True
        assert s.engine_output_type == "latent"
        assert s.requires_multimodal_data is True
+        assert s.hf_config_name == "thinker_config"

    def test_talker(self):
        p = resolve_pipeline_config("qwen2_5_omni")
@@ -1813,6 +1814,7 @@ class TestQwen2_5OmniPipeline:
        s = p.get_stage(1)
        assert isinstance(s, StagePipelineConfig)
        assert s.input_sources == (0,)
+        assert s.hf_config_name == "talker_config"
        assert s.sampling_constraints["stop_token_ids"] == [8294]
        # thinker2talker was removed: qwen2_5_omni has no async_chunk support,
        # so sync_process_input_func always wins and custom_process_input_func
@@ -1829,6 +1831,7 @@ class TestQwen2_5OmniPipeline:
        assert s.execution_type == StageExecutionType.LLM_GENERATION
        assert s.final_output_type == "audio"
        assert s.engine_output_type == "audio"
+        assert s.hf_config_name == "thinker_config"


 class TestQwen3TTSPipeline:
@@ -0,0 +1,192 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
+"""Equivalence and session-isolation tests for Cosmos3StateAdapter (RFC #4480).
+
+These drive the adapter directly with tiny CPU tensors -- no model, no GPU --
+and assert it stores/returns the per-branch UND K/V exactly and isolates
+durable state between sessions. They do not cover the transformer's mutable
+load-to-forward handoff.
+"""
+
+from __future__ import annotations
+
+import pytest
+import torch
+
+from vllm_omni.experimental.world_models.adapters.state_cosmos3_adapter import (
+    Cosmos3StateAdapter,
+)
+from vllm_omni.experimental.world_models.session_state import SessionStateManager
+
+pytestmark = [pytest.mark.core_model, pytest.mark.cpu]
+
+LAYERS = 3
+SEQ, DIM = 3, 4
+
+
+def _fake_cached_kv(seed: int) -> list[tuple[torch.Tensor, torch.Tensor]]:
+    """A list of per-layer (K, V); contents seeded so branches/sessions differ."""
+    g = torch.Generator().manual_seed(seed)
+    return [(torch.randn(1, SEQ, DIM, generator=g), torch.randn(1, SEQ, DIM, generator=g)) for _ in range(LAYERS)]
+
+
+def _adapter(session_id: str, manager: SessionStateManager | None = None) -> Cosmos3StateAdapter:
+    # `manager is not None`, not `manager or ...`: an empty SessionStateManager is
+    # falsy (it defines __len__), so `or` would silently swap in a fresh manager.
+    return Cosmos3StateAdapter(session_id, manager if manager is not None else SessionStateManager())
+
+
+def _assert_kv_equal(got: list | None, expected: list) -> None:
+    assert got is not None and len(got) == len(expected) == LAYERS
+    for (k, v), (ke, ve) in zip(got, expected):
+        torch.testing.assert_close(k, ke)
+        torch.testing.assert_close(v, ve)
+
+
+class _FakeTransformer:
+    cached_kv = None
+    cached_freqs_gen = None
+
+
+# ----------------------------- equivalence -----------------------------
+
+
+def test_branch_starts_uninitialized() -> None:
+    a = _adapter("s0")
+    for is_neg in (False, True):
+        assert a.is_branch_initialized(is_neg) is False
+        assert a.get_branch_kv(is_neg) is None
+
+
+def test_set_then_get_roundtrip() -> None:
+    a = _adapter("s0")
+    kv = _fake_cached_kv(1)
+    a.set_branch_kv(False, kv)
+    assert a.is_branch_initialized(False) is True
+    _assert_kv_equal(a.get_branch_kv(False), kv)
+
+
+def test_cond_uncond_branches_isolated() -> None:
+    a = _adapter("s0")
+    kv_pos, kv_neg = _fake_cached_kv(1), _fake_cached_kv(2)
+    a.set_branch_kv(False, kv_pos)
+    a.set_branch_kv(True, kv_neg)
+    _assert_kv_equal(a.get_branch_kv(False), kv_pos)
+    _assert_kv_equal(a.get_branch_kv(True), kv_neg)
+
+
+def test_encode_once_capture_is_noop_when_initialized() -> None:
+    a = _adapter("s0")
+    kv = _fake_cached_kv(1)
+    a.set_branch_kv(False, kv)
+    tr = _FakeTransformer()
+    tr.cached_kv = _fake_cached_kv(999)  # different content
+    a.capture_from_transformer(tr, False)
+    _assert_kv_equal(a.get_branch_kv(False), kv)  # still the first one
+
+
+def test_capture_writes_when_uninitialized() -> None:
+    a = _adapter("s0")
+    tr = _FakeTransformer()
+    tr.cached_kv = _fake_cached_kv(5)
+    tr.cached_freqs_gen = (torch.ones(2), torch.zeros(2))
+    a.capture_from_transformer(tr, False)
+    _assert_kv_equal(a.get_branch_kv(False), _fake_cached_kv(5))
+
+
+def test_capture_without_freqs_does_not_partially_initialize() -> None:
+    a = _adapter("s0")
+    tr = _FakeTransformer()
+    tr.cached_kv = _fake_cached_kv(5)
+    tr.cached_freqs_gen = None
+    with pytest.raises(RuntimeError, match="did not produce GEN RoPE frequencies"):
+        a.capture_from_transformer(tr, False)
+    assert a.is_branch_initialized(False) is False
+
+
+def test_load_into_transformer_without_freqs_sets_kv() -> None:
+    a = _adapter("s0")
+    kv = _fake_cached_kv(1)
+    a.set_branch_kv(False, kv)
+    tr = _FakeTransformer()
+    a.load_into_transformer(tr, False)
+    _assert_kv_equal(tr.cached_kv, kv)
+    assert tr.cached_freqs_gen is None
+
+
+def test_set_then_get_preserves_per_layer_values() -> None:
+    # The stored value must reflect the tensors passed in (per-layer).
+    a = _adapter("s0")
+    kv = _fake_cached_kv(3)
+    a.set_branch_kv(False, kv)
+    _assert_kv_equal(a.get_branch_kv(False), kv)
+
+
+def test_load_into_transformer_restores_kv_and_freqs_gen() -> None:
+    a = _adapter("s0")
+    tr = _FakeTransformer()
+    tr.cached_kv = _fake_cached_kv(3)
+    tr.cached_freqs_gen = (torch.ones(2), torch.zeros(2))
+    a.capture_from_transformer(tr, False)
+    tr.cached_kv = None
+    tr.cached_freqs_gen = None
+    a.load_into_transformer(tr, False)
+    _assert_kv_equal(tr.cached_kv, _fake_cached_kv(3))
+    assert tr.cached_freqs_gen is not None
+    torch.testing.assert_close(tr.cached_freqs_gen[0], torch.ones(2))
+
+
+def test_partial_branch_raises() -> None:
+    a = _adapter("s0")
+    a.set_branch_kv(False, _fake_cached_kv(1))
+    a._session.get("und_kv/1").reset()  # type: ignore[union-attr]
+    with pytest.raises(RuntimeError, match="partially initialized"):
+        a.get_branch_kv(False)
+
+
+# ----------------------------- reset -----------------------------
+
+
+def test_reset_clears_both_branches() -> None:
+    a = _adapter("s0")
+    a.set_branch_kv(False, _fake_cached_kv(1))
+    a.set_branch_kv(True, _fake_cached_kv(2))
+    a.reset()
+    for is_neg in (False, True):
+        assert a.is_branch_initialized(is_neg) is False
+        assert a.get_branch_kv(is_neg) is None
+
+
+# ----------------------- concurrency isolation (core) -----------------------
+
+
+def test_two_sessions_do_not_clobber() -> None:
+    manager = SessionStateManager()
+    a, b = _adapter("A", manager), _adapter("B", manager)
+    kv_a, kv_b = _fake_cached_kv(10), _fake_cached_kv(20)
+    a.set_branch_kv(False, kv_a)
+    assert b.is_branch_initialized(False) is False  # B not seeing A's state
+    b.set_branch_kv(False, kv_b)
+    _assert_kv_equal(a.get_branch_kv(False), kv_a)
+    _assert_kv_equal(b.get_branch_kv(False), kv_b)
+
+
+def test_fresh_adapter_same_session_sees_prior_state() -> None:
+    manager = SessionStateManager()
+    kv = _fake_cached_kv(7)
+    _adapter("s", manager).set_branch_kv(False, kv)
+    again = Cosmos3StateAdapter("s", manager)
+    _assert_kv_equal(again.get_branch_kv(False), kv)
+
+
+def test_adapter_survives_session_lru_eviction() -> None:
+    # An in-use adapter keeps its state even after the manager evicts its
+    # session id from the lookup table (LRU bounded by max_sessions).
+    manager = SessionStateManager(max_sessions=2)
+    a = _adapter("active", manager)
+    kv = _fake_cached_kv(11)
+    a.set_branch_kv(False, kv)
+    for i in range(3):
+        _adapter(f"other{i}", manager)
+    assert "active" not in manager
+    _assert_kv_equal(a.get_branch_kv(False), kv)
@@ -244,7 +244,7 @@ def test_mindie_full_dim_rope_keeps_h3_cos_sin_layout(monkeypatch: pytest.Monkey
    captured: dict[str, object] = {}

    def rotary_position_embedding(x, cos, sin, **kwargs):
-        captured.update(x=x, cos=cos, sin=sin, kwargs=kwargs)
+        captured.update(cos=cos, sin=sin, kwargs=kwargs)
        return x

    monkeypatch.setitem(
@@ -256,10 +256,8 @@ def test_mindie_full_dim_rope_keeps_h3_cos_sin_layout(monkeypatch: pytest.Monkey
    cos = torch.randn(11, 96, dtype=torch.bfloat16)
    sin = torch.randn(11, 96, dtype=torch.bfloat16)

-    actual = apply_rotary_emb_mindiesd(x, cos, sin, interleaved=False, half_head_dim=False)
+    apply_rotary_emb_mindiesd(x, cos, sin, interleaved=False, half_head_dim=False)

-    assert actual is x
-    assert captured["x"] is x
    assert captured["cos"] is cos
    assert captured["sin"] is sin
    assert captured["kwargs"] == {
@@ -267,3 +265,31 @@ def test_mindie_full_dim_rope_keeps_h3_cos_sin_layout(monkeypatch: pytest.Monkey
        "head_first": False,
        "fused": True,
    }
+
+
+def test_mindie_rope_restores_3d_layout(monkeypatch: pytest.MonkeyPatch) -> None:
+    """mindiesd rotary_position_embedding only accepts 4D ``x``."""
+    captured: dict[str, torch.Tensor] = {}
+
+    def rotary_position_embedding(x, cos, sin, **kwargs):
+        # Mirror mindiesd's own input guard.
+        assert x.dim() == 4, f"mindiesd requires 4D x, got {x.dim()}"
+        captured.update(x=x)
+        return x
+
+    monkeypatch.setitem(
+        sys.modules,
+        "mindiesd",
+        types.SimpleNamespace(rotary_position_embedding=rotary_position_embedding),
+    )
+
+    S, H, D = 11, 3, 96
+    x = torch.randn(S, H, D, dtype=torch.bfloat16)
+    cos = torch.randn(S, D, dtype=torch.bfloat16)
+    sin = torch.randn(S, D, dtype=torch.bfloat16)
+
+    actual = apply_rotary_emb_mindiesd(x, cos, sin, interleaved=False, half_head_dim=False)
+
+    assert captured["x"].shape == (1, S, H, D)
+    assert actual.shape == (S, H, D)
+    torch.testing.assert_close(actual, x, atol=0, rtol=0)
@@ -16,6 +16,7 @@ from PIL import Image
 from torch import nn

 from vllm_omni.diffusion.worker.request_batch import DiffusionRequestBatch
+from vllm_omni.experimental.world_models.session_state import SessionStateManager

 pytestmark = [pytest.mark.core_model, pytest.mark.cpu, pytest.mark.diffusion]

@@ -425,6 +426,29 @@ def test_distilled_generation_rejects_robolab_policy(make_cosmos3_pipeline) -> N
        pipeline._forward_robolab_policy(make_sampling_params(), None, 0.0)


+def test_forward_threads_request_id_to_robolab(make_cosmos3_pipeline) -> None:
+    pipeline = make_cosmos3_pipeline()
+    robolab_inputs = object()
+    captured: dict[str, Any] = {}
+    expected = object()
+    pipeline._build_robolab_policy_inputs = lambda sp, prompt, request_id: robolab_inputs
+
+    def fake_forward_robolab(sp, inputs, pipeline_start, session_id=None):
+        del sp, inputs, pipeline_start
+        captured["session_id"] = session_id
+        return expected
+
+    pipeline._forward_robolab_policy = fake_forward_robolab
+    request = SimpleNamespace(
+        prompts=["policy"],
+        sampling_params=make_sampling_params(),
+        request_id="robolab-request-7",
+    )
+
+    assert pipeline.forward(request) is expected
+    assert captured["session_id"] == "robolab-request-7"
+
+
@pytest.mark.parametrize(
    ("prompt", "sampling_params", "message"),
    [
@@ -595,6 +619,7 @@ def _make_od_config(
    return SimpleNamespace(
        enable_cpu_offload=False,
        enable_diffusion_pipeline_profiler=False,
+        enable_session_state_manager=False,
        model="/nonexistent/model/path",
        dtype=torch.float32,
        flow_shift=None,
@@ -1673,6 +1698,32 @@ def test_diffuse_covers_cfg_i2v_and_multimodal_steps(make_cosmos3_pipeline) -> N
    torch.testing.assert_close(action_result, torch.full((), 44.0).expand_as(action_result))


+def test_diffuse_drops_session_when_progress_iteration_fails(make_cosmos3_pipeline) -> None:
+    pipeline = make_cosmos3_pipeline()
+    pipeline._use_session_state = True
+    pipeline._memory_manager = SessionStateManager()
+
+    class FailingProgress:
+        def __iter__(self):
+            raise RuntimeError("progress failed before first step")
+
+    pipeline.progress_bar = lambda timesteps: FailingProgress()
+    with pytest.raises(RuntimeError, match="progress failed"):
+        pipeline.diffuse(
+            latents=torch.zeros(1, 2, 1, 1, 1),
+            timesteps=torch.tensor([7]),
+            cond_ids=_ids(2),
+            cond_mask=_mask(),
+            uncond_ids=_ids(1),
+            uncond_mask=_mask(),
+            guidance_scale=1.0,
+            shared_kwargs={"video_shape": (1, 1, 1), "fps": 24.0},
+            session_id="request-that-fails",
+        )
+
+    assert "request-that-fails" not in pipeline._memory_manager
+
+
 def test_diffuse_transfer_applies_control_cfg(make_cosmos3_pipeline, sequential_cfg_parallel) -> None:
    pipeline = make_cosmos3_pipeline()
    latents = torch.zeros(1, 2, 1, 1, 1)
@@ -1702,6 +1753,28 @@ def test_diffuse_transfer_applies_control_cfg(make_cosmos3_pipeline, sequential_
    torch.testing.assert_close(result, torch.full_like(latents, 254.0))


+def test_diffuse_transfer_rejects_session_state_manager(make_cosmos3_pipeline) -> None:
+    pipeline = make_cosmos3_pipeline()
+    pipeline._use_session_state = True
+
+    with pytest.raises(NotImplementedError, match="richer cache keys"):
+        pipeline.diffuse_transfer(
+            latents=torch.zeros(1, 2, 1, 1, 1),
+            timesteps=torch.tensor([7]),
+            cond_ids=_ids(2),
+            cond_mask=_mask(),
+            uncond_ids=_ids(1),
+            uncond_mask=_mask(),
+            guidance_scale=1.0,
+            control_guidance=1.0,
+            control_guidance_interval=None,
+            control_latents=[],
+            shared_kwargs={},
+            velocity_mask=torch.ones(1, 1, 1, 1, 1),
+            condition_latents=torch.zeros(1, 2, 1, 1, 1),
+        )
+
+
 def test_diffuse_transfer_skips_idle_cfg_branches(make_cosmos3_pipeline, sequential_cfg_parallel) -> None:
    latents = torch.zeros(1, 2, 1, 1, 1)
    velocity_mask = torch.ones(1, 1, 1, 1, 1)
@@ -9,6 +9,7 @@ from unittest.mock import Mock
 import numpy as np
 import pytest
 import torch
+import torch.nn as nn

 pytestmark = [pytest.mark.core_model, pytest.mark.cpu, pytest.mark.diffusion]

@@ -1465,3 +1466,149 @@ def test_g4_reference_video_metadata_validation(field, value, message, tmp_path)
    metadata[field] = value
    with pytest.raises(ValueError, match=message):
        _validate_reference_video_metadata(metadata, index=0, source=str(tmp_path / "reference.mp4"))
+
+
+_ENCODER_HIDDEN = 4
+_ENCODER_HEAD_DIM = 2
+_ENCODER_NUM_HEADS = 2
+_ENCODER_NUM_KV_HEADS = 1
+_ENCODER_INTERMEDIATE = 4
+
+# One distinct value per checkpoint tensor, none of them a parameter initializer
+# (fused weights start as `torch.empty`, norms as ones). A uniform fill would
+# prove only that every row was written, not that each source landed in its own
+# row range, which is the invariant the fused loaders have to get right.
+_SOURCE_FILL = {
+    "self_attn.q_proj.weight": 3.0,
+    "self_attn.k_proj.weight": 4.0,
+    "self_attn.v_proj.weight": 5.0,
+    "mlp.gate_proj.weight": 6.0,
+    "mlp.up_proj.weight": 7.0,
+    "input_layernorm.weight": 8.0,
+}
+
+_QKV_TARGET = "text_model.layers.0.self_attn.qkv_proj.weight"
+_GATE_UP_TARGET = "text_model.layers.0.mlp.gate_up_proj.weight"
+_NORM_TARGET = "text_model.layers.0.input_layernorm.weight"
+
+
+def _one_layer_text_encoder():
+    """Encoder stub holding both fused weights plus one plain parameter."""
+    from vllm_omni.diffusion.models.minimax_h3.encoder import (
+        MiniMaxH3Qwen3VLEncoder,
+        MiniMaxH3Qwen3VLMergedColumnParallelLinear,
+        MiniMaxH3Qwen3VLQKVParallelLinear,
+        MiniMaxH3Qwen3VLRMSNorm,
+    )
+    from vllm_omni.diffusion.models.minimax_h3.pipeline_minimax_h3 import _SingleRankEncoderGroup
+
+    group = _SingleRankEncoderGroup(0)
+    layer = nn.Module()
+    layer.self_attn = nn.Module()
+    layer.self_attn.qkv_proj = MiniMaxH3Qwen3VLQKVParallelLinear(
+        group,
+        hidden_size=_ENCODER_HIDDEN,
+        num_heads=_ENCODER_NUM_HEADS,
+        num_kv_heads=_ENCODER_NUM_KV_HEADS,
+        head_dim=_ENCODER_HEAD_DIM,
+        dtype=torch.float32,
+    )
+    layer.mlp = nn.Module()
+    layer.mlp.gate_up_proj = MiniMaxH3Qwen3VLMergedColumnParallelLinear(
+        group,
+        input_size=_ENCODER_HIDDEN,
+        intermediate_size=_ENCODER_INTERMEDIATE,
+        dtype=torch.float32,
+    )
+    layer.input_layernorm = MiniMaxH3Qwen3VLRMSNorm(_ENCODER_HIDDEN)
+
+    encoder = object.__new__(MiniMaxH3Qwen3VLEncoder)
+    nn.Module.__init__(encoder)
+    encoder.text_model = nn.Module()
+    encoder.text_model.layers = nn.ModuleList([layer])
+    return encoder
+
+
+def _write_text_encoder_checkpoint(path, *, omit=()):
+    """Write a one-layer Qwen3-VL-named checkpoint, one fill value per source."""
+    import json
+
+    import safetensors.torch
+
+    prefix = "model.language_model.layers.0"
+    q_rows = _ENCODER_NUM_HEADS * _ENCODER_HEAD_DIM
+    kv_rows = _ENCODER_NUM_KV_HEADS * _ENCODER_HEAD_DIM
+    shapes = {
+        "self_attn.q_proj.weight": (q_rows, _ENCODER_HIDDEN),
+        "self_attn.k_proj.weight": (kv_rows, _ENCODER_HIDDEN),
+        "self_attn.v_proj.weight": (kv_rows, _ENCODER_HIDDEN),
+        "mlp.gate_proj.weight": (_ENCODER_INTERMEDIATE, _ENCODER_HIDDEN),
+        "mlp.up_proj.weight": (_ENCODER_INTERMEDIATE, _ENCODER_HIDDEN),
+        "input_layernorm.weight": (_ENCODER_HIDDEN,),
+    }
+    tensors = {f"{prefix}.{source}": torch.full(shape, _SOURCE_FILL[source]) for source, shape in shapes.items()}
+    for source in omit:
+        del tensors[f"{prefix}.{source}"]
+    safetensors.torch.save_file(tensors, str(path / "model.safetensors"))
+    index = {"weight_map": dict.fromkeys(tensors, "model.safetensors")}
+    (path / "model.safetensors.index.json").write_text(json.dumps(index))
+
+
+@pytest.mark.parametrize(
+    ("omitted_sources", "expected_detail", "unreported_target"),
+    [
+        (("self_attn.q_proj.weight",), f"{_QKV_TARGET}: ['q']", _GATE_UP_TARGET),
+        (("self_attn.k_proj.weight",), f"{_QKV_TARGET}: ['k']", _GATE_UP_TARGET),
+        (("self_attn.v_proj.weight",), f"{_QKV_TARGET}: ['v']", _GATE_UP_TARGET),
+        (("mlp.gate_proj.weight",), f"{_GATE_UP_TARGET}: ['gate']", _QKV_TARGET),
+        (("mlp.up_proj.weight",), f"{_GATE_UP_TARGET}: ['up']", _QKV_TARGET),
+        (
+            ("self_attn.q_proj.weight", "self_attn.k_proj.weight", "self_attn.v_proj.weight"),
+            f"{_QKV_TARGET}: ['q', 'k', 'v']",
+            _GATE_UP_TARGET,
+        ),
+    ],
+)
+def test_encoder_load_reports_missing_fused_source_shards(
+    tmp_path, omitted_sources, expected_detail, unreported_target
+):
+    encoder = _one_layer_text_encoder()
+    _write_text_encoder_checkpoint(tmp_path, omit=omitted_sources)
+
+    with pytest.raises(RuntimeError) as excinfo:
+        encoder._load_weights(str(tmp_path))
+
+    message = str(excinfo.value)
+    assert expected_detail in message
+    assert unreported_target not in message
+
+
+def test_encoder_load_rejects_unloaded_plain_param(tmp_path):
+    encoder = _one_layer_text_encoder()
+    _write_text_encoder_checkpoint(tmp_path, omit=("input_layernorm.weight",))
+
+    with pytest.raises(RuntimeError, match=r"weights not loaded.*input_layernorm"):
+        encoder._load_weights(str(tmp_path))
+
+
+def test_encoder_load_places_every_source_in_its_own_rows(tmp_path):
+    encoder = _one_layer_text_encoder()
+    _write_text_encoder_checkpoint(tmp_path)
+
+    encoder._load_weights(str(tmp_path))
+
+    params = dict(encoder.named_parameters())
+    q_rows = _ENCODER_NUM_HEADS * _ENCODER_HEAD_DIM
+    kv_rows = _ENCODER_NUM_KV_HEADS * _ENCODER_HEAD_DIM
+    qkv = params[_QKV_TARGET]
+    gate_up = params[_GATE_UP_TARGET]
+    slices = {
+        "self_attn.q_proj.weight": qkv[:q_rows],
+        "self_attn.k_proj.weight": qkv[q_rows : q_rows + kv_rows],
+        "self_attn.v_proj.weight": qkv[q_rows + kv_rows :],
+        "mlp.gate_proj.weight": gate_up[:_ENCODER_INTERMEDIATE],
+        "mlp.up_proj.weight": gate_up[_ENCODER_INTERMEDIATE:],
+        "input_layernorm.weight": params[_NORM_TARGET],
+    }
+    for source, rows in slices.items():
+        assert torch.equal(rows, torch.full_like(rows, _SOURCE_FILL[source])), source
@@ -12,7 +12,7 @@ from unittest.mock import Mock
 import pytest
 from omegaconf import OmegaConf
 from pydantic import ValidationError
-from transformers import PretrainedConfig
+from transformers import PretrainedConfig, Qwen3OmniMoeConfig
 from vllm.engine.arg_utils import EngineArgs

 from vllm_omni.config.model import OmniModelConfig
@@ -244,6 +244,28 @@ def test_stage_specific_text_config_override():
    assert omni_config.get_head_size() == talker_head_dim


+def test_non_override_ar_stage_inputs_embeds_size_matches_hidden_size():
+    """A stage without an embedding-size override keeps the default input width."""
+    vllm_config = EngineArgs().create_model_config()
+    vllm_config.hf_config = Qwen3OmniMoeConfig(enable_audio_output=True)
+
+    omni_config = OmniModelConfig.from_vllm_model_config(
+        vllm_config,
+        model_stage="talker",
+        hf_config_name="talker_config",
+    )
+
+    assert (
+        getattr(
+            omni_config.hf_config.talker_config,
+            "embedding_size",
+            None,
+        )
+        is None
+    )
+    assert omni_config.get_inputs_embeds_size() == omni_config.hf_text_config.hidden_size
+
+
 def test_stage_configs_path_field():
    """OmniEngineArgs with stage_configs_path should construct without error."""
    args = OmniEngineArgs(stage_configs_path="/some/path.yaml")
@@ -20,6 +20,10 @@ from vllm_omni.entrypoints.openai.serving_speech import (
    OmniOpenAIServingSpeech,
 )

+# Serving-layer only: the engine is stubbed, so these run on the CPU lane
+# alongside the other tests/entrypoints/openai_api suites.
+pytestmark = [pytest.mark.core_model, pytest.mark.cpu]
+

 def _serving(model_type: str = "audex") -> OmniOpenAIServingSpeech:
    serving = OmniOpenAIServingSpeech.__new__(OmniOpenAIServingSpeech)
@@ -3445,16 +3445,17 @@ class TestCosyVoice3Serving:
    def test_cosyvoice3_model_type_detection(self, cosyvoice3_server):
        assert cosyvoice3_server._tts_model_type == "cosyvoice3"
        assert cosyvoice3_server._is_tts is True
-        assert cosyvoice3_server._is_cosyvoice3 is True

    def test_cosyvoice3_stage_registered(self):
-        from vllm_omni.entrypoints.openai.serving_speech import (
-            _COSYVOICE3_TTS_MODEL_STAGES,
-            _TTS_MODEL_STAGES,
+        from vllm_omni.entrypoints.openai.tts_adapters import (
+            all_tts_stage_keys,
+            detect_tts_model_type,
+            resolve_adapter,
        )

-        assert "cosyvoice3_talker" in _COSYVOICE3_TTS_MODEL_STAGES
-        assert "cosyvoice3_talker" in _TTS_MODEL_STAGES
+        assert "cosyvoice3_talker" in resolve_adapter("cosyvoice3").stage_keys
+        assert "cosyvoice3_talker" in all_tts_stage_keys()
+        assert detect_tts_model_type("cosyvoice3_talker", None) == "cosyvoice3"

    def test_validate_cosyvoice3_empty_input(self, cosyvoice3_server):
        request = OpenAICreateSpeechRequest(input="", ref_audio="data:audio/wav;base64,abc", ref_text="hello")
@@ -0,0 +1,345 @@
+# SPDX-License-Identifier: Apache-2.0
+"""TTS model detection: registry-driven stage -> model-type resolution.
+
+``serving_speech.py`` used to carry a hand-written 20-branch ladder mapping
+``model_stage``/``model_arch`` to a model-type string, duplicating the
+``stage_keys`` every adapter already declared. Detection is now derived from the
+adapters. ``test_detection_matches_legacy_ladder`` pins the new implementation
+against a verbatim copy of the ladder it replaced, over every input the ladder
+could distinguish, so the migration is checkable rather than reviewable-by-eye.
+"""
+
+import itertools
+
+import pytest
+
+from vllm_omni.entrypoints.openai.tts_adapters import (
+    LEGACY_TTS_DETECTORS,
+    TTS_ADAPTER_REGISTRY,
+    all_tts_stage_keys,
+    detect_tts_model_type,
+    iter_tts_detectors,
+    resolve_adapter,
+    tts_entry_stage_archs,
+)
+
+pytestmark = [pytest.mark.core_model, pytest.mark.cpu]
+
+
+# --------------------------------------------------------------------------
+# The legacy ladder, copied verbatim from serving_speech.py at 67c54777b
+# (the commit this refactor is based on), with the module constants inlined.
+# Do not "clean up" this function: its value is being a frozen oracle.
+# --------------------------------------------------------------------------
+_LEGACY_MING_TTS_MODEL_ARCHS = {"MingTTSForConditionalGeneration"}
+_LEGACY_VOXTRAL_TTS_MODEL_STAGES = {"audio_generation"}
+_LEGACY_QWEN3_TTS_MODEL_STAGES = {"qwen3_tts"}
+_LEGACY_FISH_TTS_MODEL_STAGES = {"fish_speech_slow_ar"}
+_LEGACY_COSYVOICE3_TTS_MODEL_STAGES = {"cosyvoice3_talker"}
+_LEGACY_OMNIVOICE_TTS_MODEL_STAGES = {"omnivoice_generator"}
+_LEGACY_COVO_AUDIO_MODEL_STAGES = {"fused_thinker_talker"}
+_LEGACY_VOXCPM2_TTS_MODEL_STAGES = {"latent_generator"}
+_LEGACY_MING_TTS_MODEL_STAGES = {"ming_tts"}
+_LEGACY_MOSS_TTS_MODEL_STAGES = {"moss_tts_nano"}
+_LEGACY_MOSS_TTS_FULL_MODEL_STAGES = {"moss_tts", "moss_tts_codec"}
+_LEGACY_MOSS_TTS_LOCAL_MODEL_STAGES = {"moss_tts_local", "moss_tts_local_codec"}
+_LEGACY_HIGGS_AUDIO_V2_TTS_MODEL_STAGES = {"higgs_audio_v2"}
+_LEGACY_HIGGS_V3_TTS_MODEL_STAGES = {"higgs_audio_v3"}
+_LEGACY_GLM_TTS_MODEL_STAGES = {"glm_tts"}
+_LEGACY_STEP_AUDIO2_TTS_MODEL_STAGES = {"step_audio2_thinker"}
+_LEGACY_INDEXTTS2_TTS_MODEL_STAGES = {"indextts2_talker"}
+_LEGACY_AUDEX_TTS_MODEL_STAGES = {"audex_thinker", "audex_omni"}
+_LEGACY_AUDEX_TTA_MODEL_STAGES = {"audex_tta_thinker"}
+
+_LEGACY_TTS_MODEL_STAGES = (
+    _LEGACY_VOXTRAL_TTS_MODEL_STAGES
+    | _LEGACY_QWEN3_TTS_MODEL_STAGES
+    | _LEGACY_FISH_TTS_MODEL_STAGES
+    | _LEGACY_COSYVOICE3_TTS_MODEL_STAGES
+    | _LEGACY_OMNIVOICE_TTS_MODEL_STAGES
+    | _LEGACY_HIGGS_AUDIO_V2_TTS_MODEL_STAGES
+    | _LEGACY_HIGGS_V3_TTS_MODEL_STAGES
+    | _LEGACY_COVO_AUDIO_MODEL_STAGES
+    | _LEGACY_VOXCPM2_TTS_MODEL_STAGES
+    | _LEGACY_MING_TTS_MODEL_STAGES
+    | _LEGACY_MOSS_TTS_MODEL_STAGES
+    | _LEGACY_MOSS_TTS_FULL_MODEL_STAGES
+    | _LEGACY_MOSS_TTS_LOCAL_MODEL_STAGES
+    | _LEGACY_GLM_TTS_MODEL_STAGES
+    | _LEGACY_STEP_AUDIO2_TTS_MODEL_STAGES
+    | _LEGACY_INDEXTTS2_TTS_MODEL_STAGES
+    | _LEGACY_AUDEX_TTS_MODEL_STAGES
+    | _LEGACY_AUDEX_TTA_MODEL_STAGES
+)
+
+
+def _legacy_detect(model_stage, model_arch):
+    if model_arch == "VoxCPM2TalkerForConditionalGeneration":
+        return "voxcpm2"
+    if model_stage in _LEGACY_QWEN3_TTS_MODEL_STAGES:
+        return "qwen3_tts"
+    if model_stage in _LEGACY_VOXTRAL_TTS_MODEL_STAGES:
+        return "voxtral_tts"
+    if model_stage in _LEGACY_FISH_TTS_MODEL_STAGES:
+        return "fish_tts"
+    if model_stage in _LEGACY_COSYVOICE3_TTS_MODEL_STAGES:
+        return "cosyvoice3"
+    if model_stage in _LEGACY_OMNIVOICE_TTS_MODEL_STAGES:
+        return "omnivoice"
+    if model_stage in _LEGACY_COVO_AUDIO_MODEL_STAGES:
+        if model_arch and "CovoAudio" in model_arch:
+            return "covo_audio"
+    if model_stage in _LEGACY_VOXCPM2_TTS_MODEL_STAGES:
+        return "voxcpm2"
+    if model_stage in _LEGACY_MING_TTS_MODEL_STAGES:
+        return "ming_flash_omni_tts"
+    if model_arch in _LEGACY_MING_TTS_MODEL_ARCHS:
+        return "ming_tts"
+    if model_stage in _LEGACY_MOSS_TTS_MODEL_STAGES:
+        return "moss_tts_nano"
+    if model_stage in _LEGACY_MOSS_TTS_FULL_MODEL_STAGES:
+        return "moss_tts"
+    if model_stage in _LEGACY_MOSS_TTS_LOCAL_MODEL_STAGES:
+        return "moss_tts"
+    if model_stage in _LEGACY_HIGGS_AUDIO_V2_TTS_MODEL_STAGES:
+        return "higgs_audio_v2"
+    if model_stage in _LEGACY_HIGGS_V3_TTS_MODEL_STAGES:
+        return "higgs_audio_v3"
+    if model_stage in _LEGACY_GLM_TTS_MODEL_STAGES:
+        return "glm_tts"
+    if model_stage in _LEGACY_STEP_AUDIO2_TTS_MODEL_STAGES:
+        return "step_audio2"
+    if model_stage in _LEGACY_INDEXTTS2_TTS_MODEL_STAGES:
+        return "indextts2"
+    if model_stage in _LEGACY_AUDEX_TTS_MODEL_STAGES:
+        return "audex"
+    if model_stage in _LEGACY_AUDEX_TTA_MODEL_STAGES:
+        return "audex_tta"
+    return None
+
+
+# Every ``model_stage`` value any in-tree pipeline emits, plus any stage key an
+# adapter claims, so the parametrization below covers the real deployment space
+# rather than a hand-picked subset. Refresh the pipeline half with:
+#   grep -rhoE 'model_stage="[^"]+"' --include='pipeline*.py' vllm_omni/model_executor/ \
+#     | sed 's/model_stage="//;s/"$//' | sort -u
+# ``test_pipeline_stage_list_is_complete`` enforces the adapter half.
+_PIPELINE_STAGES = [
+    "AR",
+    "ar",
+    "asr",
+    "audex_code2wav",
+    "audex_omni",
+    "audex_thinker",
+    "audex_tta_thinker",
+    "audex_xcodec",
+    "audio_generation",
+    "audio_tokenizer",
+    "audio_vae",
+    "aura",
+    "code2wav",
+    "cosyvoice3_code2wav",
+    "cosyvoice3_talker",
+    "dac_decoder",
+    "diffusion",
+    "dit",
+    "fish_speech_slow_ar",
+    "fused_thinker_talker",
+    "glm_tts",
+    "glm_tts_dit",
+    "higgs_audio_v2",
+    "higgs_audio_v3",
+    "indextts2_s2mel_decoder",
+    "indextts2_talker",
+    "latent_generator",
+    "llm",
+    "ming_tts",
+    "moss_tts",
+    "moss_tts_codec",
+    "moss_tts_local",
+    "moss_tts_local_codec",
+    "moss_tts_nano",
+    # Claimed by the OmniVoice adapter but emitted by no pipeline: OmniVoice is
+    # served through the diffusion path (``OmniOpenAIServingSpeech.for_diffusion``
+    # sets ``_tts_model_type`` directly and leaves ``_tts_stage`` None), so
+    # detection is never consulted for it. The key is vestigial in the adapter
+    # exactly as it was in the deleted ladder; kept here so detection is still
+    # exercised on it.
+    "omnivoice_generator",
+    "qwen3_tts",
+    "step_audio2_thinker",
+    "talker",
+    "thinker",
+    "token2audio",
+    "token2image",
+    "token2text",
+    "token2wav",
+    "tts",
+]
+
+_STAGES = [*_PIPELINE_STAGES, None, "vae", "not_a_real_stage"]
+_ARCHS = [
+    None,
+    "VoxCPM2TalkerForConditionalGeneration",
+    "MingTTSForConditionalGeneration",
+    "CovoAudioForConditionalGeneration",
+    "MyCovoAudioThing",
+    "Qwen3TTSForConditionalGeneration",
+]
+
+#: Architectures that no adapter claims. Paired with any stage key these are
+#: inert, so the full cross product is meaningful for them.
+_UNCLAIMED_ARCHS = [None, "Qwen3TTSForConditionalGeneration"]
+
+#: Architectures a detector matches on. Pairing one of these with *another*
+#: model's stage key describes a deployment no pipeline can produce (the
+#: architecture comes from the checkpoint, the stage key from the pipeline
+#: definition), and the two implementations deliberately disagree there —
+#: see ``test_arch_matching_is_a_fallback_not_an_override``.
+_CLAIMED_ARCH_CASES = [
+    # Ming dense: its AR stage is the generic ``llm``.
+    ("llm", "MingTTSForConditionalGeneration"),
+    ("audio_vae", "MingTTSForConditionalGeneration"),
+    (None, "MingTTSForConditionalGeneration"),
+    # Ming flash-omni owns the ``ming_tts`` stage key and must win it.
+    ("ming_tts", "MingTTSForConditionalGeneration"),
+    # VoxCPM2: talker architecture plus its own stage key.
+    ("latent_generator", "VoxCPM2TalkerForConditionalGeneration"),
+    (None, "VoxCPM2TalkerForConditionalGeneration"),
+    # CoVo shares ``fused_thinker_talker`` with non-CoVo fused deployments.
+    ("fused_thinker_talker", "CovoAudioForConditionalGeneration"),
+    ("fused_thinker_talker", "MyCovoAudioThing"),
+    ("fused_thinker_talker", None),
+    ("fused_thinker_talker", "Qwen3TTSForConditionalGeneration"),
+]
+
+_REACHABLE = list(itertools.product(_STAGES, _UNCLAIMED_ARCHS)) + _CLAIMED_ARCH_CASES
+
+
+@pytest.mark.parametrize(("model_stage", "model_arch"), _REACHABLE)
+def test_detection_matches_legacy_ladder(model_stage, model_arch):
+    """Registry detection is identical to the ladder it replaced.
+
+    Covers every ``model_stage`` any in-tree pipeline emits (``_PIPELINE_STAGES``,
+    plus negatives) against every architecture no adapter claims, and each
+    claimed architecture against the stage keys it really ships with. Pairing a
+    claimed architecture with an unrelated model's stage key is excluded because
+    no pipeline can produce it, and is covered separately by
+    ``test_arch_matching_is_a_fallback_not_an_override``.
+    """
+    assert detect_tts_model_type(model_stage, model_arch) == _legacy_detect(model_stage, model_arch)
+
+
+def test_arch_matching_is_a_fallback_not_an_override():
+    """The one deliberate divergence from the ladder, pinned.
+
+    The ladder tested Ming's architecture at a fixed position partway down, so a
+    Ming-architecture model deployed under *another* model's stage key resolved
+    to ``ming_tts`` if that key was checked below Ming, and to the other model if
+    above. Detection now treats an architecture-only match as a fallback that
+    runs after every adapter owning a real stage key, so the stage key always
+    wins. That is both self-consistent and unreachable in practice: a
+    checkpoint's architecture and a pipeline's stage key cannot be mixed across
+    models.
+    """
+    assert detect_tts_model_type("audex_omni", "MingTTSForConditionalGeneration") == "audex"
+    assert _legacy_detect("audex_omni", "MingTTSForConditionalGeneration") == "ming_tts"
+    # Where the two real Ming deployments live, they agree.
+    assert detect_tts_model_type("llm", "MingTTSForConditionalGeneration") == "ming_tts"
+    assert detect_tts_model_type("ming_tts", "MingTTSForConditionalGeneration") == "ming_flash_omni_tts"
+
+
+def test_stage_keys_cover_legacy_stage_set():
+    """No stage key was dropped when the module constants were deleted."""
+    assert all_tts_stage_keys() == frozenset(_LEGACY_TTS_MODEL_STAGES)
+
+
+def test_pipeline_stage_list_is_complete():
+    """``_PIPELINE_STAGES`` must still list every stage key adapters claim.
+
+    Guards the parametrization above from silently narrowing: a new adapter
+    whose stage key is missing here would be detected but never exercised.
+    Regenerate the list from the pipelines (see the comment beside it) when a
+    new model lands.
+    """
+    missing = all_tts_stage_keys() - set(_PIPELINE_STAGES)
+    assert not missing, f"stage keys claimed by adapters but absent from _PIPELINE_STAGES: {sorted(missing)}"
+
+
+def test_entry_stage_archs_is_ming_only():
+    """Only Ming dense identifies its entry stage by architecture.
+
+    Widening this set would change which stage ``_find_tts_stage`` selects in
+    mixed deployments — notably VoxCPM2, which declares ``model_archs`` but is
+    found by its ``latent_generator`` stage key.
+    """
+    assert tts_entry_stage_archs() == frozenset({"MingTTSForConditionalGeneration"})
+
+
+def test_every_detected_type_is_adapter_backed_or_declared_legacy():
+    """Detection may only name a model that has an adapter or an explicit
+    :data:`LEGACY_TTS_DETECTORS` entry — never an undeclared string."""
+    legacy_names = {d.name for d in LEGACY_TTS_DETECTORS}
+    for stage, arch in itertools.product(_STAGES, _ARCHS):  # full cross product on purpose
+        detected = detect_tts_model_type(stage, arch)
+        if detected is None:
+            continue
+        assert detected in TTS_ADAPTER_REGISTRY or detected in legacy_names, (
+            f"detection produced undeclared model type {detected!r} for stage={stage!r} arch={arch!r}"
+        )
+
+
+def test_legacy_detectors_have_no_adapter():
+    """A legacy entry that gained an adapter must be deleted from the list."""
+    for detector in LEGACY_TTS_DETECTORS:
+        assert resolve_adapter(detector.name) is None, (
+            f"{detector.name!r} now has an adapter; remove it from LEGACY_TTS_DETECTORS"
+        )
+
+
+def test_no_ambiguous_detectors_at_equal_priority():
+    """Two detectors that can match the same deployment must be ordered.
+
+    This is what makes ``detect_tts_model_type`` independent of adapter import
+    order: any real overlap has to be resolved by an explicit
+    ``detect_priority``, not by whichever module happened to register first.
+    """
+    detectors = iter_tts_detectors()
+    for left, right in itertools.combinations(detectors, 2):
+        if left.detect_priority != right.detect_priority:
+            continue
+        for stage, arch in itertools.product(_STAGES, _ARCHS):
+            assert not (left.matches(stage, arch) and right.matches(stage, arch)), (
+                f"{left.name!r} and {right.name!r} both match stage={stage!r} arch={arch!r} "
+                f"at priority {left.detect_priority}; give one an explicit detect_priority"
+            )
+
+
+def test_detection_order_is_total_and_stable():
+    """Resolution order must not depend on registry insertion order."""
+    keys = [(d.detect_priority, d.name) for d in iter_tts_detectors()]
+    assert keys == sorted(keys)
+    assert len(set(keys)) == len(keys)
+
+
+def test_adapters_declare_at_least_one_match_rule():
+    """An adapter with neither stage keys nor architectures is unreachable."""
+    for name, cls in TTS_ADAPTER_REGISTRY.items():
+        assert cls.stage_keys or cls.model_archs, f"adapter {name!r} declares no stage_keys and no model_archs"
+
+
+def test_audex_omni_requires_speech_decoder():
+    """The Audex omni thinker only serves speech alongside its decoder."""
+    audex = resolve_adapter("audex")
+    assert audex.stage_serves_speech("audex_omni", frozenset({"audex_omni", "audex_code2wav"})) is True
+    assert audex.stage_serves_speech("audex_omni", frozenset({"audex_omni"})) is False
+    # The dedicated TTS thinker is unconditional.
+    assert audex.stage_serves_speech("audex_thinker", frozenset({"audex_thinker"})) is True
+
+
+def test_default_stage_serves_speech_is_unconditional():
+    for name, cls in TTS_ADAPTER_REGISTRY.items():
+        if name == "audex":
+            continue
+        for key in cls.stage_keys:
+            assert cls.stage_serves_speech(key, frozenset({key})) is True
@@ -105,6 +105,9 @@ class _FakeHiFT(nn.Module):
        generated_source = speech[:, None]
        return speech, generated_source

+    def inference(self, mel, source):
+        return self.forward(mel, source)
+

 class _FakeToken2Wav:
    def __init__(self):
@@ -0,0 +1,144 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
+
+from types import SimpleNamespace
+from unittest.mock import Mock
+
+import pytest
+import torch
+import torch.nn as nn
+
+from vllm_omni.model_executor.models.cosyvoice3.code2wav_core.hifigan import (
+    HiFTGenerator,
+)
+from vllm_omni.model_executor.models.minicpmo_4_5.cuda_graph_wrapper import (
+    HiFTGraphWrapper,
+)
+
+pytestmark = [pytest.mark.core_model]
+
+
+class _F0Predictor(nn.Module):
+    def __init__(self) -> None:
+        super().__init__()
+        self.proj = nn.Conv1d(80, 1, 1)
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        return self.proj(x).squeeze(1).abs()
+
+
+class _DeterministicSineGen(nn.Module):
+    """Remove source RNG so eager and replay compare only execution paths."""
+
+    def __init__(self, num_harmonics: int) -> None:
+        super().__init__()
+        self.num_harmonics = num_harmonics
+
+    def forward(self, f0: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor]:
+        shape = (*f0.shape[:-1], self.num_harmonics)
+        sine = f0.new_zeros(shape)
+        uv = f0.new_ones((*f0.shape[:-1], 1))
+        return sine, uv, sine
+
+
+def _small_hift() -> HiFTGenerator:
+    hift = HiFTGenerator(
+        base_channels=32,
+        sampling_rate=24000,
+        upsample_rates=[8, 5, 3],
+        upsample_kernel_sizes=[16, 11, 7],
+        source_resblock_kernel_sizes=[7, 7, 11],
+        source_resblock_dilation_sizes=[[1, 3, 5]] * 3,
+        f0_predictor=_F0Predictor(),
+    )
+    hift.m_source.l_sin_gen = _DeterministicSineGen(hift.nb_harmonics + 1)
+    return hift.eval().cuda()
+
+
+@pytest.mark.skipif(not torch.cuda.is_available(), reason="CUDA is required")
+def test_hift_graph_replay_matches_eager_for_uncached_and_cached_shapes() -> None:
+    torch.manual_seed(0)
+    hift = _small_hift()
+    token2wav = SimpleNamespace(
+        hift=hift,
+        flow=SimpleNamespace(
+            encoder=SimpleNamespace(pre_lookahead_layer=SimpleNamespace(pre_lookahead_len=3)),
+            token_mel_ratio=2,
+        ),
+        mel_cache_len=2,
+        source_cache_len=960,
+    )
+    wrapper = HiFTGraphWrapper(
+        token2wav,
+        connector_config={"codec_chunk_frames": 2, "codec_left_context_frames": 3},
+        capture_batch_sizes=[1],
+    )
+    wrapper.capture()
+
+    cases = (
+        (torch.randn(1, 80, 4, device="cuda"), torch.zeros(1, 1, 0, device="cuda")),
+        (torch.randn(1, 80, 6, device="cuda"), torch.randn(1, 1, 960, device="cuda")),
+    )
+    with torch.inference_mode():
+        for speech_feat, cache_source in cases:
+            expected_speech, expected_source = hift.inference(speech_feat, cache_source)
+            actual_speech, actual_source = wrapper.replay(speech_feat, cache_source)
+            torch.testing.assert_close(actual_speech, expected_speech, rtol=1e-4, atol=1e-5)
+            torch.testing.assert_close(actual_source, expected_source, rtol=1e-4, atol=1e-5)
+
+
+class _FakeGraph:
+    def replay(self) -> None:
+        return None
+
+
+def _fake_wrapper(monkeypatch: pytest.MonkeyPatch) -> HiFTGraphWrapper:
+    monkeypatch.setattr(torch.cuda, "is_current_stream_capturing", lambda: False)
+    wrapper = object.__new__(HiFTGraphWrapper)
+    wrapper.capture_batch_sizes = [1]
+    wrapper.graph = {}
+    wrapper.static_speech_inputs = {}
+    wrapper.static_cache_source_inputs = {}
+    wrapper.static_magnitude_outputs = {}
+    wrapper.static_phase_outputs = {}
+    wrapper.static_cache_source_outputs = {}
+    wrapper.lazy_graph_count = 0
+    wrapper.max_lazy_graphs = 1
+    wrapper.decode_fn = Mock(return_value=(torch.tensor([[99.0]]), torch.tensor([[[98.0]]])))
+    wrapper.finalize_fn = lambda magnitude, phase: magnitude + phase
+
+    def capture(batch_size: int, num_frames: int, cache_len: int) -> None:
+        key = (batch_size, num_frames, cache_len)
+        wrapper.graph[key] = _FakeGraph()
+        wrapper.static_speech_inputs[key] = torch.zeros(batch_size, 80, num_frames)
+        wrapper.static_cache_source_inputs[key] = torch.zeros(batch_size, 1, cache_len)
+        wrapper.static_magnitude_outputs[key] = torch.ones(batch_size, 1, num_frames)
+        wrapper.static_phase_outputs[key] = torch.ones(batch_size, 1, num_frames)
+        wrapper.static_cache_source_outputs[key] = torch.ones(batch_size, 1, num_frames)
+
+    wrapper._capture = Mock(side_effect=capture)
+    return wrapper
+
+
+def test_unseen_shape_is_lazily_captured(monkeypatch: pytest.MonkeyPatch) -> None:
+    wrapper = _fake_wrapper(monkeypatch)
+    speech, source = wrapper.replay(torch.randn(1, 80, 7), torch.zeros(1, 1, 0))
+
+    wrapper._capture.assert_called_once_with(1, 7, 0)
+    assert wrapper.lazy_graph_count == 1
+    assert speech.shape == (1, 1, 7)
+    assert source.shape == (1, 1, 7)
+    wrapper.decode_fn.assert_not_called()
+
+
+def test_lazy_capture_limit_falls_back_to_eager(monkeypatch: pytest.MonkeyPatch) -> None:
+    wrapper = _fake_wrapper(monkeypatch)
+    wrapper.lazy_graph_count = wrapper.max_lazy_graphs
+    speech_feat = torch.randn(1, 80, 9)
+    cache_source = torch.zeros(1, 1, 0)
+
+    result = wrapper.replay(speech_feat, cache_source)
+
+    wrapper._capture.assert_not_called()
+    wrapper.decode_fn.assert_called_once_with(speech_feat, cache_source)
+    assert result is wrapper.decode_fn.return_value
@@ -157,6 +157,7 @@ class TestDeployTopology:
        assert connector["name"] == "SharedMemoryConnector"
        assert connector["extra"]["codec_chunk_frames"] == 25
        assert connector["extra"]["codec_left_context_frames"] == 3
+        assert connector["extra"]["enable_hift_graph"] is True
        assert connector["extra"]["connector_get_max_wait_first_chunk"] == 3000
        assert connector["extra"]["connector_get_max_wait"] == 300
        expected_processor = "tts2code2wav_async_chunk" if deploy.async_chunk else "tts2code2wav_full_payload"
@@ -0,0 +1,54 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
+
+import pytest
+import torch
+
+from vllm_omni.model_executor.models.cosyvoice3.utils import mel_spectrogram
+from vllm_omni.model_executor.models.step_audio2.step_audio2_token2wav import _build_hift
+
+pytestmark = [pytest.mark.core_model, pytest.mark.cpu]
+
+flashcosyvoice_hifigan = pytest.importorskip("flashcosyvoice.modules.hifigan")
+flashcosyvoice_audio = pytest.importorskip("flashcosyvoice.utils.audio")
+
+
+def test_vendored_hift_matches_flashcosyvoice() -> None:
+    reference = flashcosyvoice_hifigan.HiFTGenerator().eval()
+    vendored = _build_hift().eval()
+
+    # Strict loading verifies that the vendored architecture remains compatible
+    # with the Step-Audio2 checkpoint produced for flashcosyvoice.
+    vendored.load_state_dict(reference.state_dict(), strict=True)
+
+    speech_feat = torch.randn(1, 80, 4)
+    cache_source = torch.zeros(1, 1, 0)
+    with torch.inference_mode():
+        torch.manual_seed(0)
+        expected_speech, expected_source = reference(speech_feat, cache_source)
+        torch.manual_seed(0)
+        actual_speech, actual_source = vendored.inference(speech_feat, cache_source)
+
+    torch.testing.assert_close(actual_source, expected_source)
+    torch.testing.assert_close(actual_speech, expected_speech)
+
+
+def test_vendored_mel_spectrogram_matches_flashcosyvoice() -> None:
+    reference = flashcosyvoice_audio.mel_spectrogram
+    vendored = mel_spectrogram
+
+    audio = torch.randn(1, 48000)
+    kwargs = {
+        "n_fft": 1920,
+        "num_mels": 80,
+        "sampling_rate": 24000,
+        "hop_size": 480,
+        "win_size": 1920,
+        "fmin": 0,
+        "fmax": 8000,
+    }
+    with torch.inference_mode():
+        expected_spec = reference(audio, **kwargs)
+        actual_spec = vendored(audio, **kwargs)
+
+    torch.testing.assert_close(actual_spec, expected_spec)
@@ -0,0 +1,79 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Tests for the TTS adapter migration ratchet.
+
+The gate only earns its place if it actually catches the ways a per-model branch
+gets reintroduced. These pin what it detects — and, just as importantly, the one
+shape it is known to miss, so nobody mistakes it for an adversarial sandbox.
+"""
+
+from __future__ import annotations
+
+import ast
+import sys
+from pathlib import Path
+
+import pytest
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[2] / "tools" / "pre_commit"))
+
+from check_tts_adapter import (  # noqa: E402
+    MAX_LEGACY_DETECTORS,
+    MAX_MODEL_TYPE_BRANCHES,
+    _legacy_detectors,
+    _model_type_branches,
+    main,
+)
+
+pytestmark = [pytest.mark.core_model, pytest.mark.cpu]
+
+
+def _count(src: str) -> int:
+    return len(_model_type_branches(ast.parse(src)))
+
+
+@pytest.mark.parametrize(
+    ("label", "src", "expected"),
+    [
+        ("direct comparison", 'if self._tts_model_type == "foo": pass', 1),
+        ("negated comparison", 'if self._tts_model_type != "foo": pass', 1),
+        ("membership test", 'if self._tts_model_type in ("a", "b"): pass', 1),
+        ("local alias", 'kind = self._tts_model_type\nif kind == "foo": pass', 1),
+        ("getattr read", 'if getattr(self, "_tts_model_type") == "foo": pass', 1),
+        ("getattr with default", 'if getattr(self, "_tts_model_type", None) == "foo": pass', 1),
+        ("match statement counts each case", 'match self._tts_model_type:\n case "a": pass\n case _: pass', 2),
+        ("alias then match", 'k = getattr(self, "_tts_model_type")\nmatch k:\n case "a": pass', 1),
+        ("unrelated attribute is ignored", 'if self._other == "foo": pass', 0),
+        ("unrelated local is ignored", 'kind = self._backend\nif kind == "foo": pass', 0),
+    ],
+)
+def test_branch_detection(label, src, expected):
+    assert _count(src) == expected, label
+
+
+def test_dict_dispatch_is_a_known_gap():
+    """Documented limitation, asserted so it is a decision and not a surprise.
+
+    Table dispatch keyed on the model type reads as zero branches. The ratchet
+    guards against drift in the shape people actually write; catching every
+    indirection is a review question, not a lint one.
+    """
+    assert _count('TABLE = {"foo": 1}\nvalue = TABLE[self._tts_model_type]') == 0
+
+
+def test_legacy_detector_counting():
+    src = 'X = (LegacyDetector(name="a", stage_keys=frozenset({"s"})), LegacyDetector(name="b"))'
+    found = _legacy_detectors(ast.parse(src))
+    assert [name for _, name in found] == ["a", "b"]
+
+
+def test_gate_passes_on_current_tree():
+    """The recorded budgets must match reality, or the ratchet is decorative."""
+    assert main([]) == 0
+
+
+def test_budgets_are_the_real_counts():
+    root = Path(__file__).resolve().parents[2]
+    serving = root / "vllm_omni" / "entrypoints" / "openai" / "serving_speech.py"
+    adapters = root / "vllm_omni" / "entrypoints" / "openai" / "tts_adapters" / "__init__.py"
+    assert len(_model_type_branches(ast.parse(serving.read_text()))) == MAX_MODEL_TYPE_BRANCHES
+    assert len(_legacy_detectors(ast.parse(adapters.read_text()))) == MAX_LEGACY_DETECTORS
@@ -0,0 +1,181 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Ratchet on the TTS adapter migration (RFC #4327, #4855).
+
+Per-model behaviour is being moved out of ``serving_speech.py`` and into
+``entrypoints/openai/tts_adapters/``. The migration has stalled before, because
+nothing stopped a new model from adding one more ``self._tts_model_type == ...``
+branch faster than the migration removed them.
+
+This gate makes the two debt counters monotonically non-increasing:
+
+1. ``self._tts_model_type`` comparisons in ``serving_speech.py``
+2. entries in ``tts_adapters.LEGACY_TTS_DETECTORS`` (models with no adapter)
+
+Lowering a budget after removing branches is expected and encouraged — the check
+insists on it, so the recorded numbers cannot silently drift upward. Raising one
+requires editing this file, which makes it a reviewable decision instead of an
+invisible one.
+
+Source-only (``ast``), so it runs in pre-commit without importing vllm.
+"""
+
+from __future__ import annotations
+
+import argparse
+import ast
+import sys
+from pathlib import Path
+
+REPO_ROOT = Path(__file__).resolve().parents[2]
+SERVING_SPEECH = Path("vllm_omni/entrypoints/openai/serving_speech.py")
+ADAPTERS_INIT = Path("vllm_omni/entrypoints/openai/tts_adapters/__init__.py")
+
+# Budgets. These may only go DOWN. See the module docstring.
+MAX_MODEL_TYPE_BRANCHES = 29
+MAX_LEGACY_DETECTORS = 1
+
+_GUIDANCE = """
+Move the per-model behaviour into its adapter under
+``vllm_omni/entrypoints/openai/tts_adapters/`` instead of branching in the
+shared serving module. ``docs/contributing/model/adding_tts_model.md`` walks
+through it. If a branch genuinely cannot be expressed as an adapter hook, raise
+the budget in this file and say why in the PR description.
+""".strip()
+
+
+_ATTR = "_tts_model_type"
+
+
+def _reads_model_type(node: ast.AST) -> bool:
+    """Whether ``node`` evaluates to the model-type discriminator."""
+    if isinstance(node, ast.Attribute) and node.attr == _ATTR:
+        return True
+    # getattr(self, "_tts_model_type") / getattr(self, "_tts_model_type", None)
+    return (
+        isinstance(node, ast.Call)
+        and isinstance(node.func, ast.Name)
+        and node.func.id == "getattr"
+        and len(node.args) >= 2
+        and isinstance(node.args[1], ast.Constant)
+        and node.args[1].value == _ATTR
+    )
+
+
+def _local_aliases(tree: ast.AST) -> set[str]:
+    """Local names bound directly to the model type, e.g. ``kind = self._tts_model_type``.
+
+    Shallow on purpose: one hop is what an author reaches for when a branch gets
+    long, which is the drift this ratchet exists to catch.
+    """
+    aliases: set[str] = set()
+    for node in ast.walk(tree):
+        value = None
+        if isinstance(node, ast.Assign):
+            value = node.value
+        elif isinstance(node, ast.AnnAssign | ast.AugAssign):
+            value = node.value
+        if value is None or not _reads_model_type(value):
+            continue
+        targets = node.targets if isinstance(node, ast.Assign) else [node.target]
+        for target in targets:
+            if isinstance(target, ast.Name):
+                aliases.add(target.id)
+    return aliases
+
+
+def _model_type_branches(tree: ast.AST) -> list[tuple[int, str]]:
+    """Every branch keyed on the model-type discriminator.
+
+    Counts direct comparisons, ``getattr`` reads, ``match`` subjects (one per
+    ``case``), and comparisons against a local alias of the attribute. This is a
+    ratchet against drift, not an adversarial sandbox: dispatch tables and other
+    indirections still slip past, and are a review question.
+    """
+    aliases = _local_aliases(tree)
+
+    def _is_subject(node: ast.AST) -> bool:
+        return _reads_model_type(node) or (isinstance(node, ast.Name) and node.id in aliases)
+
+    found: list[tuple[int, str]] = []
+    for node in ast.walk(tree):
+        if isinstance(node, ast.Compare):
+            if any(_is_subject(op) for op in [node.left, *node.comparators]):
+                found.append((node.lineno, ast.unparse(node)))
+        elif isinstance(node, ast.Match) and _is_subject(node.subject):
+            subject = ast.unparse(node.subject)
+            for case in node.cases:
+                found.append((case.pattern.lineno, f"match {subject}: case {ast.unparse(case.pattern)}"))
+    found.sort()
+    return found
+
+
+def _legacy_detectors(tree: ast.AST) -> list[tuple[int, str]]:
+    """Every ``LegacyDetector(...)`` construction."""
+    found: list[tuple[int, str]] = []
+    for node in ast.walk(tree):
+        if isinstance(node, ast.Call) and isinstance(node.func, ast.Name) and node.func.id == "LegacyDetector":
+            name = "?"
+            for kw in node.keywords:
+                if kw.arg == "name" and isinstance(kw.value, ast.Constant):
+                    name = str(kw.value.value)
+            found.append((node.lineno, name))
+    found.sort()
+    return found
+
+
+def _check(label: str, path: Path, actual: int, budget: int, sample: list[str], budget_name: str) -> list[str]:
+    if actual > budget:
+        listing = "\n".join(f"      {line}" for line in sample)
+        return [
+            f"{path}: {actual} {label} exceeds the budget of {budget}.\n{listing}\n\n{_GUIDANCE}\n",
+        ]
+    if actual < budget:
+        return [
+            f"{path}: {actual} {label}, down from a budget of {budget}. "
+            f"Lower {budget_name} to {actual} in {Path(__file__).relative_to(REPO_ROOT)} "
+            f"so the ratchet holds the new ground.",
+        ]
+    return []
+
+
+def main(argv: list[str] | None = None) -> int:
+    parser = argparse.ArgumentParser(description=__doc__)
+    # pre-commit passes the staged filenames; we always audit the same two files,
+    # so they are accepted and ignored.
+    parser.add_argument("filenames", nargs="*")
+    parser.parse_args(argv)
+
+    serving_path = REPO_ROOT / SERVING_SPEECH
+    adapters_path = REPO_ROOT / ADAPTERS_INIT
+    for path in (serving_path, adapters_path):
+        if not path.is_file():
+            print(f"check_tts_adapter: expected file not found: {path}", file=sys.stderr)
+            return 1
+
+    branches = _model_type_branches(ast.parse(serving_path.read_text()))
+    detectors = _legacy_detectors(ast.parse(adapters_path.read_text()))
+
+    errors = _check(
+        "`_tts_model_type` comparisons",
+        SERVING_SPEECH,
+        len(branches),
+        MAX_MODEL_TYPE_BRANCHES,
+        [f"line {lineno}: {src}" for lineno, src in branches],
+        "MAX_MODEL_TYPE_BRANCHES",
+    )
+    errors += _check(
+        "unmigrated model types in LEGACY_TTS_DETECTORS",
+        ADAPTERS_INIT,
+        len(detectors),
+        MAX_LEGACY_DETECTORS,
+        [f"line {lineno}: {name}" for lineno, name in detectors],
+        "MAX_LEGACY_DETECTORS",
+    )
+
+    for error in errors:
+        print(f"check_tts_adapter: {error}", file=sys.stderr)
+    return 1 if errors else 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())
@@ -178,6 +178,14 @@ class OmniModelConfig(ModelConfig):
                return override
        return super().embedding_size

+    def get_inputs_embeds_size(self) -> int:
+        if self.hf_config_name is not None:
+            stage_config = getattr(self.hf_config, self.hf_config_name, None)
+            override = getattr(stage_config, "embedding_size", None)
+            if override is not None:
+                return override
+        return super().get_inputs_embeds_size()
+
    def get_model_arch_config(self):
        # For multi-stage omni models, use a stage-aware convertor so that
        # only the correct stage's quantization config is surfaced.
@@ -14,6 +14,11 @@ connectors:
      connector_get_max_wait: 300
      codec_chunk_frames: 25
      codec_left_context_frames: 3
+      # HiFT capture buckets are derived from codec_chunk_frames and
+      # codec_left_context_frames. (#5869)
+      # A future initial_codec_chunk_frames shape will use lazy capture unless
+      # HiFTGraphWrapper.derive_capture_bucket_size explicitly includes it.
+      enable_hift_graph: true

 stages:
  - stage_id: 0
@@ -17,6 +17,11 @@ connectors:
      connector_get_max_wait: 300
      codec_chunk_frames: 25
      codec_left_context_frames: 3
+      # HiFT capture buckets are derived from codec_chunk_frames and
+      # codec_left_context_frames. (#5869)
+      # A future initial_codec_chunk_frames shape will use lazy capture unless
+      # HiFTGraphWrapper.derive_capture_bucket_size explicitly includes it.
+      enable_hift_graph: true

 stages:
  - stage_id: 0
@@ -17,6 +17,11 @@ connectors:
      connector_get_max_wait: 300
      codec_chunk_frames: 25
      codec_left_context_frames: 3
+      # HiFT capture buckets are derived from codec_chunk_frames and
+      # codec_left_context_frames. (#5869)
+      # A future initial_codec_chunk_frames shape will use lazy capture unless
+      # HiFTGraphWrapper.derive_capture_bucket_size explicitly includes it.
+      enable_hift_graph: true

 stages:
  - stage_id: 0
@@ -28,6 +28,11 @@ connectors:
      connector_get_max_wait: 300
      codec_chunk_frames: 25
      codec_left_context_frames: 3
+      # HiFT capture buckets are derived from codec_chunk_frames and
+      # codec_left_context_frames. (#5869)
+      # A future initial_codec_chunk_frames shape will use lazy capture unless
+      # HiFTGraphWrapper.derive_capture_bucket_size explicitly includes it.
+      enable_hift_graph: true

 stages:
  - stage_id: 0
@@ -46,6 +46,8 @@ def apply_rotary_emb_mindiesd(
 ) -> torch.Tensor:
    from mindiesd import rotary_position_embedding

+    x, squeezed = _ensure_batch_dim(x)
+
    if cos.dim() == 3:
        # (B, S, D/2) -> (S, D/2)
        cos = cos[0]
@@ -57,13 +59,14 @@ def apply_rotary_emb_mindiesd(
            seqlen = cos.shape[0]
            sin = sin.unsqueeze(0).unsqueeze(2).unsqueeze(-1).expand(-1, -1, -1, -1, 2).reshape(1, seqlen, 1, -1)
            cos = cos.unsqueeze(0).unsqueeze(2).unsqueeze(-1).expand(-1, -1, -1, -1, 2).reshape(1, seqlen, 1, -1)
-        return rotary_position_embedding(x, cos, sin, rotated_mode="rotated_interleaved", head_first=False, fused=True)
+        out = rotary_position_embedding(x, cos, sin, rotated_mode="rotated_interleaved", head_first=False, fused=True)
    else:
        if half_head_dim:
            seqlen = cos.shape[0]
            sin = sin.unsqueeze(0).unsqueeze(2).repeat(1, 1, 1, 2)
            cos = cos.unsqueeze(0).unsqueeze(2).repeat(1, 1, 1, 2)
-        return rotary_position_embedding(x, cos, sin, rotated_mode="rotated_half", head_first=False, fused=True)
+        out = rotary_position_embedding(x, cos, sin, rotated_mode="rotated_half", head_first=False, fused=True)
+    return _restore_batch_dim(out, squeezed)


 def _ensure_batch_dim(x: torch.Tensor) -> tuple[torch.Tensor, bool]:
@@ -51,6 +51,7 @@ from vllm_omni.diffusion.data import DiffusionOutput, OmniDiffusionConfig
 from vllm_omni.diffusion.distributed.autoencoders.autoencoder_kl_wan import DistributedAutoencoderKLWan
 from vllm_omni.diffusion.distributed.cfg_parallel import CFGParallelMixin
 from vllm_omni.diffusion.distributed.parallel_state import (
+    get_classifier_free_guidance_rank,
    get_classifier_free_guidance_world_size,
 )
 from vllm_omni.diffusion.distributed.utils import get_local_device
@@ -71,6 +72,13 @@ from vllm_omni.diffusion.profiler.diffusion_pipeline_profiler import DiffusionPi
 from vllm_omni.diffusion.request import OmniDiffusionRequest
 from vllm_omni.diffusion.worker.request_batch import DiffusionRequestBatch
 from vllm_omni.entrypoints.openai.video_api_utils import positive_float
+from vllm_omni.experimental.world_models.adapters.state_cosmos3_adapter import (
+    Cosmos3StateAdapter,
+)
+from vllm_omni.experimental.world_models.session_state import (
+    SessionStateManager,
+    resolve_session_state_config,
+)
 from vllm_omni.inputs.data import OmniDiffusionSamplingParams

 from .action import (
@@ -910,6 +918,24 @@ class Cosmos3OmniDiffusersPipeline(
        )
        self.is_edge_model = transformer_cls is Cosmos3EdgeVFMTransformer

+        # Opt-in: route the per-CFG-branch UND text K/V through the shared
+        # SessionStateManager (RFC #4480) instead of the transformer-instance
+        # cache. This establishes request-keyed state isolation; it does not by
+        # itself make the mutable pipeline safe for concurrent execution.
+        # Default off -> the bespoke transformer-instance path below is unchanged.
+        self._use_session_state, mm_max_sessions = resolve_session_state_config(
+            enable=od_config.enable_session_state_manager,
+        )
+        self._memory_manager: SessionStateManager | None = (
+            SessionStateManager(max_sessions=mm_max_sessions) if self._use_session_state else None
+        )
+        # Forward declaration for the future BDE paged path (Stage C/D). Nothing
+        # writes this field in Phase 0; a non-None value is rejected by the guards
+        # below. None means the dense session adapter or the bespoke path.
+        self._bde_kv_state = None
+        if getattr(self, "_use_session_state", False):
+            logger.info("Cosmos3: session state manager enabled (max_sessions=%d)", mm_max_sessions)
+
        # --- Scheduler ---
        # Distilled model differs from regular one only by scheduler,
        # distilled one uses FlowMatchEulerDiscreteScheduler, while
@@ -1463,6 +1489,7 @@ class Cosmos3OmniDiffusersPipeline(
        sp: OmniDiffusionSamplingParams,
        inputs: RoboLabPolicyInputs,
        pipeline_start: float,
+        session_id: str | None = None,
    ) -> DiffusionOutput:
        if self.is_distilled_model:
            raise self._distilled_unsupported_error("RoboLab/action policy requests are unsupported.")
@@ -1576,6 +1603,7 @@ class Cosmos3OmniDiffusersPipeline(
            guidance_interval=None,
            raw_action_dim=raw_action_dim,
            scheduler=scheduler,
+            session_id=session_id,
            generator=generator,
        )

@@ -2415,6 +2443,49 @@ class Cosmos3OmniDiffusersPipeline(
        action_velocity_mask = 1.0 - condition_mask
        return action_latents, action_velocity_mask, clean_action, raw_action_dim

+    # -- Session-memory bypass for UND text K/V (RFC #4480) -----------------
+    # Mirrors DreamZero's `_kv_*` pattern: route through the BDE pool when set
+    # (Stage C/D), else the session adapter (Stage B), else the bespoke
+    # transformer-instance cache. All gated so the default path is unchanged.
+
+    def _new_cosmos3_state(self, session_id: str | None) -> Cosmos3StateAdapter | None:
+        if getattr(self, "_memory_manager", None) is None:
+            return None
+        return Cosmos3StateAdapter(session_id, self._memory_manager)
+
+    def _kv_load_und(self, state: Cosmos3StateAdapter | None, is_negative: bool) -> bool:
+        """Load this branch's UND K/V onto the transformer before forward.
+
+        Returns True if the session/BDE path handled the load (caller skips the
+        bespoke ``cond_cache`` assignment), False for the bespoke path.
+        """
+        if getattr(self, "_bde_kv_state", None) is not None:
+            raise NotImplementedError("BDE pool path for Cosmos3 UND K/V is Stage C/D")
+        if state is not None:
+            state.load_into_transformer(self.transformer, is_negative)
+            return True
+        return False
+
+    def _kv_capture_und(self, state: Cosmos3StateAdapter | None, is_negative: bool) -> None:
+        """After forward, store the freshly computed UND K/V for this branch (encode-once)."""
+        if getattr(self, "_bde_kv_state", None) is not None:
+            raise NotImplementedError("BDE pool path for Cosmos3 UND K/V is Stage C/D")
+        if state is not None:
+            state.capture_from_transformer(self.transformer, is_negative)
+
+    def _kv_reset_und(self, state: Cosmos3StateAdapter | None) -> None:
+        """Reset UND K/V at the start of a generation (replaces ``transformer.reset_cache()``).
+
+        Always clears the transformer-instance cache so no stale UND K/V leaks into
+        paths that read it directly (cfg-parallel / no-CFG / bespoke fallback); also
+        clears the session objects when session state is active.
+        """
+        if getattr(self, "_bde_kv_state", None) is not None:
+            raise NotImplementedError("BDE pool path for Cosmos3 UND K/V is Stage C/D")
+        self.transformer.reset_cache()
+        if state is not None:
+            state.reset()
+
    # -- Denoising loop (shared by T2V and I2V) -----------------------------

    def diffuse(
@@ -2438,6 +2509,7 @@ class Cosmos3OmniDiffusersPipeline(
        guidance_interval: tuple[float, float] | None = None,
        raw_action_dim: int | None = None,
        scheduler: Any | None = None,
+        session_id: str | None = None,
        generator: torch.Generator | None = None,
    ) -> torch.Tensor | tuple[torch.Tensor, ...]:
        """Denoising loop with 3-mode CFG support (parallel, sequential, none).
@@ -2470,7 +2542,9 @@ class Cosmos3OmniDiffusersPipeline(
        do_cfg = guidance_scale > 1.0
        cfg_parallel = self._cfg_parallel_active() and do_cfg
        step_scheduler = scheduler if scheduler is not None else self.scheduler
-        self.transformer.reset_cache()
+        # Session-keyed UND K/V (RFC #4480); None => bespoke transformer-instance cache.
+        kv_state = self._new_cosmos3_state(session_id)
+        self._kv_reset_und(kv_state)

        def _cfg_active_at(t: torch.Tensor) -> bool:
            if guidance_interval is None:
@@ -2606,18 +2680,57 @@ class Cosmos3OmniDiffusersPipeline(
            if sound_latents is not None:
                sound_latents = step_out[idx]

-        if cfg_parallel:
-            for t in self.progress_bar(timesteps):
-                timestep = t.unsqueeze(0)
-                # Out-of-interval steps run with effective scale 1.0 so the
-                # combined output equals the cond branch (uncond is dropped).
-                # All ranks still execute both branches; no CFG-Parallel
-                # divergence.
-                step_scale = guidance_scale if _cfg_active_at(t) else 1.0
-                noise_pred = self.predict_noise_maybe_with_cfg(
-                    do_true_cfg=True,
-                    true_cfg_scale=step_scale,
-                    positive_kwargs=dict(
+        try:
+            if cfg_parallel:
+                # Each CFG-parallel rank runs exactly one branch (rank 0 -> cond,
+                # else uncond), so session keying loads/stores only this rank's branch.
+                cfg_rank_is_negative = get_classifier_free_guidance_rank() != 0
+                for t in self.progress_bar(timesteps):
+                    timestep = t.unsqueeze(0)
+                    # Outside the interval, scale=1 makes the combined output equal
+                    # the cond branch. Every rank remains on the same iteration and
+                    # collective schedule.
+                    step_scale = guidance_scale if _cfg_active_at(t) else 1.0
+                    self._kv_load_und(kv_state, is_negative=cfg_rank_is_negative)
+                    noise_pred = self.predict_noise_maybe_with_cfg(
+                        do_true_cfg=True,
+                        true_cfg_scale=step_scale,
+                        positive_kwargs=dict(
+                            hidden_states=latents,
+                            timestep=timestep,
+                            text_ids=cond_ids,
+                            text_mask=cond_mask,
+                            action_latents=action_latents,
+                            sound_latents=sound_latents,
+                            **shared_kwargs,
+                        ),
+                        negative_kwargs=dict(
+                            hidden_states=latents,
+                            timestep=timestep,
+                            text_ids=uncond_ids,
+                            text_mask=uncond_mask,
+                            action_latents=action_latents,
+                            sound_latents=sound_latents,
+                            **shared_kwargs,
+                        ),
+                        cfg_normalize=False,
+                    )
+                    if kv_state is not None:
+                        self._kv_capture_und(kv_state, is_negative=cfg_rank_is_negative)
+                    _assign_step_out(_step(noise_pred, t, latents, action_latents, sound_latents))
+
+            elif do_cfg:
+                cond_cache: tuple = (None, None)
+                uncond_cache: tuple = (None, None)
+                keep_uncond_for_cache = self._cache_requires_paired_cfg()
+
+                for t in self.progress_bar(timesteps):
+                    timestep = t.unsqueeze(0)
+                    cfg_active = _cfg_active_at(t)
+
+                    if not self._kv_load_und(kv_state, is_negative=False):
+                        self.transformer.cached_kv, self.transformer.cached_freqs_gen = cond_cache
+                    noise_cond = self.transformer(
                        hidden_states=latents,
                        timestep=timestep,
                        text_ids=cond_ids,
@@ -2625,79 +2738,67 @@ class Cosmos3OmniDiffusersPipeline(
                        action_latents=action_latents,
                        sound_latents=sound_latents,
                        **shared_kwargs,
-                    ),
-                    negative_kwargs=dict(
+                    )
+                    if kv_state is not None:
+                        self._kv_capture_und(kv_state, is_negative=False)
+                    elif cond_cache[0] is None:
+                        cond_cache = (self.transformer.cached_kv, self.transformer.cached_freqs_gen)
+
+                    if cfg_active or keep_uncond_for_cache:
+                        if not self._kv_load_und(kv_state, is_negative=True):
+                            self.transformer.cached_kv, self.transformer.cached_freqs_gen = uncond_cache
+                        noise_uncond = self.transformer(
+                            hidden_states=latents,
+                            timestep=timestep,
+                            text_ids=uncond_ids,
+                            text_mask=uncond_mask,
+                            action_latents=action_latents,
+                            sound_latents=sound_latents,
+                            **shared_kwargs,
+                        )
+                        if kv_state is not None:
+                            self._kv_capture_und(kv_state, is_negative=True)
+                        elif uncond_cache[0] is None:
+                            uncond_cache = (self.transformer.cached_kv, self.transformer.cached_freqs_gen)
+                        # Outside the interval, scale=1.0 makes the combined result
+                        # equal to noise_cond; the uncond pass is computed only to
+                        # preserve cache-dit's cond/uncond parity.
+                        step_scale = guidance_scale if cfg_active else 1.0
+                        noise_pred = self.combine_cfg_noise(
+                            noise_cond,
+                            noise_uncond,
+                            step_scale,
+                            cfg_normalize=False,
+                        )
+                    else:
+                        noise_pred = noise_cond
+
+                    _assign_step_out(_step(noise_pred, t, latents, action_latents, sound_latents))
+
+            else:
+                # No CFG: a single cond branch per step. Bespoke (state None) keeps
+                # using the transformer-instance cache exactly as before.
+                for t in self.progress_bar(timesteps):
+                    timestep = t.unsqueeze(0)
+                    self._kv_load_und(kv_state, is_negative=False)
+                    noise_pred = self.transformer(
                        hidden_states=latents,
                        timestep=timestep,
-                        text_ids=uncond_ids,
-                        text_mask=uncond_mask,
-                        action_latents=action_latents,
-                        sound_latents=sound_latents,
-                        **shared_kwargs,
-                    ),
-                    cfg_normalize=False,
-                )
-                _assign_step_out(_step(noise_pred, t, latents, action_latents, sound_latents))
-
-        elif do_cfg:
-            cond_cache: tuple = (None, None)
-            uncond_cache: tuple = (None, None)
-
-            keep_uncond_for_cache = self._cache_requires_paired_cfg()
-
-            for t in self.progress_bar(timesteps):
-                timestep = t.unsqueeze(0)
-                cfg_active = _cfg_active_at(t)
-
-                self.transformer.cached_kv, self.transformer.cached_freqs_gen = cond_cache
-                noise_cond = self.transformer(
-                    hidden_states=latents,
-                    timestep=timestep,
-                    text_ids=cond_ids,
-                    text_mask=cond_mask,
-                    action_latents=action_latents,
-                    sound_latents=sound_latents,
-                    **shared_kwargs,
-                )
-                if cond_cache[0] is None:
-                    cond_cache = (self.transformer.cached_kv, self.transformer.cached_freqs_gen)
-
-                if cfg_active or keep_uncond_for_cache:
-                    self.transformer.cached_kv, self.transformer.cached_freqs_gen = uncond_cache
-                    noise_uncond = self.transformer(
-                        hidden_states=latents,
-                        timestep=timestep,
-                        text_ids=uncond_ids,
-                        text_mask=uncond_mask,
+                        text_ids=cond_ids,
+                        text_mask=cond_mask,
                        action_latents=action_latents,
                        sound_latents=sound_latents,
                        **shared_kwargs,
                    )
-                    if uncond_cache[0] is None:
-                        uncond_cache = (self.transformer.cached_kv, self.transformer.cached_freqs_gen)
-                    # Outside the interval, scale=1.0 makes the combined result
-                    # equal to noise_cond; the uncond pass is computed only to
-                    # preserve cache-dit's cond/uncond parity.
-                    step_scale = guidance_scale if cfg_active else 1.0
-                    noise_pred = self.combine_cfg_noise(noise_cond, noise_uncond, step_scale, cfg_normalize=False)
-                else:
-                    noise_pred = noise_cond
-
-                _assign_step_out(_step(noise_pred, t, latents, action_latents, sound_latents))
-
-        else:
-            for t in self.progress_bar(timesteps):
-                timestep = t.unsqueeze(0)
-                noise_pred = self.transformer(
-                    hidden_states=latents,
-                    timestep=timestep,
-                    text_ids=cond_ids,
-                    text_mask=cond_mask,
-                    action_latents=action_latents,
-                    sound_latents=sound_latents,
-                    **shared_kwargs,
-                )
-                _assign_step_out(_step(noise_pred, t, latents, action_latents, sound_latents))
+                    if kv_state is not None:
+                        self._kv_capture_und(kv_state, is_negative=False)
+                    _assign_step_out(_step(noise_pred, t, latents, action_latents, sound_latents))
+        finally:
+            # Cosmos3 currently receives a unique request_id rather than a
+            # reusable rollout session id. Retaining its state would only pin
+            # K/V buffers on device after this generation finishes.
+            if kv_state is not None:
+                self._memory_manager.drop_session(session_id)

        outputs = [latents]
        if action_latents is not None:
@@ -2794,6 +2895,13 @@ class Cosmos3OmniDiffusersPipeline(
        guidance_interval: tuple[float, float] | None = None,
        generator: torch.Generator | None = None,
    ) -> torch.Tensor:
+        if getattr(self, "_use_session_state", False):
+            raise NotImplementedError(
+                "Cosmos3 transfer does not yet support SessionStateManager: "
+                "its control/text CFG branches require richer cache keys. "
+                "Disable the session state manager for transfer requests."
+            )
+
        def _active_at(t: torch.Tensor, interval: tuple[float, float] | None) -> bool:
            if interval is None:
                return True
@@ -3178,7 +3286,12 @@ class Cosmos3OmniDiffusersPipeline(
        sp = req.sampling_params
        robolab_inputs = self._build_robolab_policy_inputs(sp, prompt_data, getattr(req, "request_id", None))
        if robolab_inputs is not None:
-            return self._forward_robolab_policy(sp, robolab_inputs, pipeline_start)
+            return self._forward_robolab_policy(
+                sp,
+                robolab_inputs,
+                pipeline_start,
+                session_id=getattr(req, "request_id", None),
+            )

        if isinstance(prompt_data, str):
            prompt = prompt_data
@@ -3543,6 +3656,7 @@ class Cosmos3OmniDiffusersPipeline(
                guidance_interval=guidance_interval,
                raw_action_dim=raw_action_dim,
                scheduler=scheduler,
+                session_id=getattr(req, "request_id", None),
                generator=generator,
            )

@@ -1674,8 +1674,11 @@ class Cosmos3VFMTransformer(nn.Module):
            sound_tokens = self.pack_sound(sound_latents)
            s_sound = sound_tokens.shape[1]

-        # Run UND pathway once and cache K/V (replicated across all ranks)
-        if self.cached_kv is None:
+        # Run UND pathway once and cache K/V (replicated across all ranks).
+        # freqs_gen (M-RoPE cos/sin) is derived purely from shape/fps and is
+        # cached alongside UND K/V by both the bespoke and session-state paths.
+        need_kv = self.cached_kv is None
+        if need_kv or self.cached_freqs_gen is None:
            freqs_und, freqs_gen = self._compute_rope_freqs(
                text_mask,
                t,
@@ -1691,13 +1694,14 @@ class Cosmos3VFMTransformer(nn.Module):
                num_vision_items=len(control_latent_list) + 1,
                share_vision_temporal_positions=transfer_share_vision_temporal_positions,
            )
-            with self._offload_context("reasoner"):
-                cached_kv_full = self.language_model(text_ids, freqs_und)
            self.cached_freqs_gen = freqs_gen

-            # Trim to real text length (remove padding).  K/V stay replicated;
-            # the framework Attention layer head-slices them via joint_key/value.
-            self.cached_kv = [(k[:, :max_real_len], v[:, :max_real_len]) for k, v in cached_kv_full]
+            if need_kv:
+                with self._offload_context("reasoner"):
+                    cached_kv_full = self.language_model(text_ids, freqs_und)
+                # Trim to real text length (remove padding).  K/V stay replicated;
+                # the framework Attention layer head-slices them via joint_key/value.
+                self.cached_kv = [(k[:, :max_real_len], v[:, :max_real_len]) for k, v in cached_kv_full]

        with self._offload_context("generator"):
            # Patchify latents and project to hidden space after UND cache
@@ -206,6 +206,38 @@ class MiniMaxH3Qwen3VLQKVParallelLinear(nn.Module):
            raise ValueError(f"unexpected QKV shard id {loaded_shard_id!r}")


+# A fused weight is filled from several checkpoint tensors, so a partial load leaves
+# uninitialized `torch.empty` rows that no later check would catch. Keyed by the class
+# owning the multi-shard `weight_loader`; each entry is the source name reported to the
+# user paired with the shard id that loader expects, in projection order.
+_FUSED_SOURCE_SHARDS: dict[type[nn.Module], tuple[tuple[str, str | int], ...]] = {
+    MiniMaxH3Qwen3VLQKVParallelLinear: (("q", "q"), ("k", "k"), ("v", "v")),
+    MiniMaxH3Qwen3VLMergedColumnParallelLinear: (("gate", 0), ("up", 1)),
+}
+
+
+def _fused_source_shards(module: nn.Module) -> tuple[tuple[str, str | int], ...] | None:
+    for cls, sources in _FUSED_SOURCE_SHARDS.items():
+        if isinstance(module, cls):
+            return sources
+    return None
+
+
+def _verify_fused_shards_complete(
+    expected: dict[str, tuple[tuple[str, str | int], ...]],
+    loaded: dict[str, set[str | int]],
+) -> None:
+    missing = {
+        name: [source for source, shard_id in sources if shard_id not in loaded[name]]
+        for name, sources in expected.items()
+    }
+    missing = {name: sources for name, sources in missing.items() if sources}
+    if not missing:
+        return
+    details = "; ".join(f"{name}: {sources}" for name, sources in sorted(missing.items()))
+    raise RuntimeError(f"MiniMax H3 text encoder fused weights are missing source shards: {details}")
+
+
 class MiniMaxH3Qwen3VLRowParallelLinear(nn.Module):
    def __init__(
        self,
@@ -952,7 +984,7 @@ class MiniMaxH3Qwen3VLEncoder(nn.Module):
    def tp_size(self) -> int:
        return self._tp_size

-    def _map_weight_name(self, name: str) -> tuple[str, str | None] | None:
+    def _map_weight_name(self, name: str) -> tuple[str, str | int | None] | None:
        if name == "lm_head.weight" or name == "model.language_model.norm.weight":
            return None
        if name.startswith("model.visual."):
@@ -1004,6 +1036,15 @@ class MiniMaxH3Qwen3VLEncoder(nn.Module):
        files = sorted(set(weight_map.values()))
        params = dict(self.named_parameters())
        loaded: set[str] = set()
+        # Seeded from the modules that own a multi-shard loader, so a fused weight
+        # that receives no source shard at all is caught too, not only a partially
+        # filled one.
+        expected_fused_shards: dict[str, tuple[tuple[str, str | int], ...]] = {
+            f"{module_name}.weight": sources
+            for module_name, module in self.named_modules()
+            if (sources := _fused_source_shards(module)) is not None
+        }
+        loaded_fused_shards: dict[str, set[str | int]] = {name: set() for name in expected_fused_shards}
        for shard in files:
            tensors = safetensors.torch.load_file(os.path.join(model_path, shard), device="cpu")
            for name, tensor in tensors.items():
@@ -1018,13 +1059,13 @@ class MiniMaxH3Qwen3VLEncoder(nn.Module):
                weight_loader = getattr(param, "weight_loader", _default_weight_loader)
                weight_loader(param, tensor, shard_id)
                loaded.add(param_name)
+                if shard_id is not None and param_name in loaded_fused_shards:
+                    loaded_fused_shards[param_name].add(shard_id)
+        _verify_fused_shards_complete(expected_fused_shards, loaded_fused_shards)
        missing = sorted(set(params) - loaded)
        if missing:
-            logger.warning(
-                "MiniMax H3 text encoder weights not loaded (retained-layer subset): %d params, e.g. %s",
-                len(missing),
-                missing[:5],
-            )
+            # Listed in full: this aborts startup, so the message is the only diagnostic.
+            raise RuntimeError(f"MiniMax H3 text encoder weights not loaded: {len(missing)} params: {missing}")

    def load_to_device(self) -> None:
        if not self.is_loaded:
@@ -724,7 +724,10 @@ class MiniMaxH3Pipeline(
            loaded_with_prefix.update(prefix + name for name in loaded)
        # The text encoder and both VAEs load eagerly in ``__init__`` rather
        # than through ``weights_sources``. Record them for the runner's strict
-        # missing-parameter check.
+        # missing-parameter check. For the text encoder that report is exact
+        # because its loader raises on any unloaded parameter or partially filled
+        # fused parameter; weaken that and gaps here become invisible. The two
+        # VAEs carry no equivalent guarantee.
        for component_name in ("text_encoder", "video_vae", "audio_vae"):
            component = getattr(self, component_name)
            loaded_with_prefix.update(f"{component_name}.{name}" for name, _ in component.named_parameters())
@@ -53,7 +53,10 @@ from vllm_omni.entrypoints.openai.speech_usage import (
 )
 from vllm_omni.entrypoints.openai.tts_adapters import (
    SpeechServingContext,
+    all_tts_stage_keys,
+    detect_tts_model_type,
    resolve_adapter,
+    tts_entry_stage_archs,
 )
 from vllm_omni.entrypoints.utils import coerce_param_message_types
 from vllm_omni.model_executor.models.fish_speech.prompt_utils import (
@@ -78,52 +81,17 @@ from vllm_omni.utils.speaker_cache import (
 logger = init_logger(__name__)

 # TTS Configuration
-_MING_TTS_MODEL_ARCHS = {"MingTTSForConditionalGeneration"}
-_VOXTRAL_TTS_MODEL_STAGES = {"audio_generation"}
-_QWEN3_TTS_MODEL_STAGES = {"qwen3_tts"}
-_FISH_TTS_MODEL_STAGES = {"fish_speech_slow_ar"}
-_COSYVOICE3_TTS_MODEL_STAGES = {"cosyvoice3_talker"}
+#
+# The stage-key -> model-type mapping is NOT declared here: it is derived from
+# the ``stage_keys`` / ``model_archs`` each adapter declares, via
+# ``tts_adapters.detect_tts_model_type``. Adding a TTS model must not require an
+# edit to this module.
+#
 # CosyVoice3 talker expects its reference transcript wrapped in the model
 # instruction template; without the delimiter the talker re-speaks the
 # reference (issue #4644). Matches the offline example/test and upstream demo.
 _COSYVOICE3_PROMPT_DELIMITER = "<|endofprompt|>"
 _COSYVOICE3_PROMPT_PREFIX = f"You are a helpful assistant.{_COSYVOICE3_PROMPT_DELIMITER}"
-_OMNIVOICE_TTS_MODEL_STAGES = {"omnivoice_generator"}
-_COVO_AUDIO_MODEL_STAGES = {"fused_thinker_talker"}
-_VOXCPM2_TTS_MODEL_STAGES = {"latent_generator"}
-_MING_TTS_MODEL_STAGES = {"ming_tts"}
-_MOSS_TTS_MODEL_STAGES = {"moss_tts_nano"}
-_MOSS_TTS_FULL_MODEL_STAGES = {"moss_tts", "moss_tts_codec"}
-_MOSS_TTS_LOCAL_MODEL_STAGES = {"moss_tts_local", "moss_tts_local_codec"}
-_HIGGS_AUDIO_V2_TTS_MODEL_STAGES = {"higgs_audio_v2"}
-_HIGGS_V3_TTS_MODEL_STAGES = {"higgs_audio_v3"}
-_GLM_TTS_MODEL_STAGES = {"glm_tts"}
-_STEP_AUDIO2_TTS_MODEL_STAGES = {"step_audio2_thinker"}
-_INDEXTTS2_TTS_MODEL_STAGES = {"indextts2_talker"}
-# audex_omni covers the audex_s2s S2S deployment, whose
-# TTS pass uses the same /v1/audio/speech surface.
-_AUDEX_TTS_MODEL_STAGES = {"audex_thinker", "audex_omni"}
-_AUDEX_TTA_MODEL_STAGES = {"audex_tta_thinker"}
-_TTS_MODEL_STAGES: set[str] = (
-    _VOXTRAL_TTS_MODEL_STAGES
-    | _QWEN3_TTS_MODEL_STAGES
-    | _FISH_TTS_MODEL_STAGES
-    | _COSYVOICE3_TTS_MODEL_STAGES
-    | _OMNIVOICE_TTS_MODEL_STAGES
-    | _HIGGS_AUDIO_V2_TTS_MODEL_STAGES
-    | _HIGGS_V3_TTS_MODEL_STAGES
-    | _COVO_AUDIO_MODEL_STAGES
-    | _VOXCPM2_TTS_MODEL_STAGES
-    | _MING_TTS_MODEL_STAGES
-    | _MOSS_TTS_MODEL_STAGES
-    | _MOSS_TTS_FULL_MODEL_STAGES
-    | _MOSS_TTS_LOCAL_MODEL_STAGES
-    | _GLM_TTS_MODEL_STAGES
-    | _STEP_AUDIO2_TTS_MODEL_STAGES
-    | _INDEXTTS2_TTS_MODEL_STAGES
-    | _AUDEX_TTS_MODEL_STAGES
-    | _AUDEX_TTA_MODEL_STAGES
-)
 _SAMPLING_MAX_TOKENS_TTS_MODEL_TYPES = {
    "fish_tts",
    "qwen3_tts",
@@ -446,7 +414,6 @@ class OmniOpenAIServingSpeech(OpenAIServing, AudioMixin):
        instance._diffusion_stage_configs = stage_configs
        instance._tts_model_type = "omnivoice"
        instance._is_tts = False
-        instance._is_fish_speech = False
        # Diffusion-only instances don't have a TTS stage; set None so any
        # ``_is_tts_model()`` / ``_tts_stage`` access doesn't raise AttributeError.
        instance._tts_stage = None
@@ -462,10 +429,6 @@ class OmniOpenAIServingSpeech(OpenAIServing, AudioMixin):
        # Find and cache the TTS stage (if any) during initialization
        self._tts_stage = self._find_tts_stage()
        self._is_tts = self._tts_stage is not None
-        self._is_fish_speech = (
-            self._tts_stage is not None
-            and getattr(getattr(self._tts_stage, "engine_args", None), "model_stage", None) == "fish_speech_slow_ar"
-        )
        self._fish_speech_tokenizer = None
        self._covo_audio_tokenizer = None
        # Cached per process: the CosyVoice3 Qwen tokenizer + resolved model
@@ -474,11 +437,6 @@ class OmniOpenAIServingSpeech(OpenAIServing, AudioMixin):
        # TTFP critical path) in _apply_cosyvoice3_dynamic_tokens.
        self._cosyvoice3_tokenizer = None

-        self._is_cosyvoice3 = (
-            self._tts_stage is not None
-            and getattr(getattr(self._tts_stage, "engine_args", None), "model_stage", None)
-            in _COSYVOICE3_TTS_MODEL_STAGES
-        )
        # Determine TTS model type or None
        self._tts_model_type = self._detect_tts_model_type()
        self.precomputed_speakers = self._load_precomputed_speakers()
@@ -687,74 +645,43 @@ class OmniOpenAIServingSpeech(OpenAIServing, AudioMixin):

    def _find_tts_stage(self):
        """Find and return the TTS stage config, or None if not found."""
-        all_stages = {getattr(stage.engine_args, "model_stage", None) for stage in self.engine_client.stage_configs}
+        tts_stage_keys = all_tts_stage_keys()
+        entry_stage_archs = tts_entry_stage_archs()
+        all_stages = frozenset(
+            getattr(stage.engine_args, "model_stage", None) for stage in self.engine_client.stage_configs
+        )
        for stage in self.engine_client.stage_configs:
            engine_args = stage.engine_args
            model_stage = engine_args.model_stage
            model_arch = getattr(engine_args, "model_arch", None)
            worker_type = getattr(engine_args, "worker_type", None)
-            if model_stage in _TTS_MODEL_STAGES:
-                # The audio-capable Audex thinker is only speech-capable when
-                # deployed WITH the speech decoder (audex_s2s);
-                # the thinker-only deployment is text-final and must not
-                # accept /v1/audio/speech requests.
-                if model_stage == "audex_omni" and "audex_code2wav" not in all_stages:
+            if model_stage in tts_stage_keys:
+                # Owning the stage key is not always enough: a model may be
+                # speech-capable only in some deployment topologies. Ask the
+                # adapter. Stages that resolve to no adapter keep the legacy
+                # behaviour of being accepted here (and detected as ``None``).
+                adapter_cls = resolve_adapter(detect_tts_model_type(model_stage, model_arch))
+                if adapter_cls is not None and not adapter_cls.stage_serves_speech(model_stage, all_stages):
                    continue
                return stage
-            # Ming dense identifies its AR entry stage by architecture because
-            # it does not use a dedicated TTS model_stage value.
-            if model_arch in _MING_TTS_MODEL_ARCHS and worker_type == "ar":
+            # Models with no dedicated TTS model_stage value identify their AR
+            # entry stage by architecture (Ming dense).
+            if model_arch in entry_stage_archs and worker_type == "ar":
                return stage
        return None

    def _detect_tts_model_type(self) -> str | None:
-        """Detect TTS model type from the stage's model_stage attribute."""
+        """Detect TTS model type from the resolved stage's deployment metadata.
+
+        The mapping lives on the adapters (``stage_keys`` / ``model_archs``);
+        this only supplies the stage under inspection.
+        """
        if self._tts_stage is None:
            return None
-        model_stage = getattr(self._tts_stage.engine_args, "model_stage", None)
-        model_arch = getattr(self._tts_stage.engine_args, "model_arch", None)
-        if model_arch == "VoxCPM2TalkerForConditionalGeneration":
-            return "voxcpm2"
-        if model_stage in _QWEN3_TTS_MODEL_STAGES:
-            return "qwen3_tts"
-        if model_stage in _VOXTRAL_TTS_MODEL_STAGES:
-            return "voxtral_tts"
-        if model_stage in _FISH_TTS_MODEL_STAGES:
-            return "fish_tts"
-        if model_stage in _COSYVOICE3_TTS_MODEL_STAGES:
-            return "cosyvoice3"
-        if model_stage in _OMNIVOICE_TTS_MODEL_STAGES:
-            return "omnivoice"
-        if model_stage in _COVO_AUDIO_MODEL_STAGES:
-            if model_arch and "CovoAudio" in model_arch:
-                return "covo_audio"
-        if model_stage in _VOXCPM2_TTS_MODEL_STAGES:
-            return "voxcpm2"
-        if model_stage in _MING_TTS_MODEL_STAGES:
-            return "ming_flash_omni_tts"
-        if model_arch in _MING_TTS_MODEL_ARCHS:
-            return "ming_tts"
-        if model_stage in _MOSS_TTS_MODEL_STAGES:
-            return "moss_tts_nano"
-        if model_stage in _MOSS_TTS_FULL_MODEL_STAGES:
-            return "moss_tts"
-        if model_stage in _MOSS_TTS_LOCAL_MODEL_STAGES:
-            return "moss_tts"
-        if model_stage in _HIGGS_AUDIO_V2_TTS_MODEL_STAGES:
-            return "higgs_audio_v2"
-        if model_stage in _HIGGS_V3_TTS_MODEL_STAGES:
-            return "higgs_audio_v3"
-        if model_stage in _GLM_TTS_MODEL_STAGES:
-            return "glm_tts"
-        if model_stage in _STEP_AUDIO2_TTS_MODEL_STAGES:
-            return "step_audio2"
-        if model_stage in _INDEXTTS2_TTS_MODEL_STAGES:
-            return "indextts2"
-        if model_stage in _AUDEX_TTS_MODEL_STAGES:
-            return "audex"
-        if model_stage in _AUDEX_TTA_MODEL_STAGES:
-            return "audex_tta"
-        return None
+        return detect_tts_model_type(
+            getattr(self._tts_stage.engine_args, "model_stage", None),
+            getattr(self._tts_stage.engine_args, "model_arch", None),
+        )

    def _get_custom_voice_dir(self) -> str | None:
        try:
@@ -1,12 +1,16 @@
 # SPDX-License-Identifier: Apache-2.0
-"""Registry of TTS serving adapters.
+"""Registry of TTS serving adapters, and the model detection built on it.

-Adapters register themselves by their ``name`` (the model-type discriminator
-returned by ``OmniOpenAIServingSpeech._detect_tts_model_type``) via
-``@register_tts_adapter``. Resolution is by that name: detection already encodes
-the (order-sensitive, partly ``model_arch``-based) stage logic, so keying the
-registry on the resolved type is both simpler and correct by construction.
-``stage_keys`` is retained on each adapter as documentation metadata.
+Adapters register themselves by their ``name`` (the model-type discriminator)
+via ``@register_tts_adapter``, and declare the deployment metadata that
+identifies them: ``stage_keys``, plus ``model_archs`` for the few models a stage
+key alone cannot distinguish.
+
+That metadata is the single source of truth. :func:`detect_tts_model_type` and
+:func:`all_tts_stage_keys` derive the whole stage -> model-type mapping from it,
+so a new TTS model is one adapter file and needs no edit to ``serving_speech.py``.
+Models still awaiting an adapter are listed explicitly in
+:data:`LEGACY_TTS_DETECTORS`.
 """

 from vllm.logger import init_logger
@@ -14,6 +18,7 @@ from vllm.logger import init_logger
 from vllm_omni.entrypoints.openai.tts_adapters.base import (
    ARTTSAdapter,
    DiffusionTTSAdapter,
+    LegacyDetector,
    OutputPolicy,
    PreparedRequest,
    SpeechServingContext,
@@ -24,6 +29,19 @@ logger = init_logger(__name__)

 TTS_ADAPTER_REGISTRY: dict[str, type[TTSModelAdapter]] = {}

+#: Model types the serving layer detects that have no adapter yet. See
+#: :class:`LegacyDetector`. This list must only ever shrink.
+LEGACY_TTS_DETECTORS: tuple[LegacyDetector, ...] = (
+    # Ming *flash-omni* claims the ``ming_tts`` stage key, while Ming *dense*
+    # (which does have an adapter) is identified by architecture. The stage key
+    # must therefore be tested first, hence the explicit priority.
+    LegacyDetector(
+        name="ming_flash_omni_tts",
+        stage_keys=frozenset({"ming_tts"}),
+        detect_priority=50,
+    ),
+)
+

 def register_tts_adapter(cls: type[TTSModelAdapter]) -> type[TTSModelAdapter]:
    """Class decorator: index ``cls`` under its ``name`` (model-type)."""
@@ -48,6 +66,56 @@ def resolve_adapter(model_type: str | None) -> type[TTSModelAdapter] | None:
    return TTS_ADAPTER_REGISTRY.get(model_type)


+def iter_tts_detectors() -> list[type[TTSModelAdapter] | LegacyDetector]:
+    """Every detector, in resolution order.
+
+    Sorted by ``detect_priority`` then ``name``, so the order is total and does
+    not depend on adapter import order.
+    """
+    detectors: list[type[TTSModelAdapter] | LegacyDetector] = [
+        *TTS_ADAPTER_REGISTRY.values(),
+        *LEGACY_TTS_DETECTORS,
+    ]
+    detectors.sort(key=lambda d: (d.detect_priority, d.name))
+    return detectors
+
+
+def detect_tts_model_type(model_stage: str | None, model_arch: str | None) -> str | None:
+    """Resolve a deployed stage to its TTS model-type, or ``None``.
+
+    The first detector that matches wins; ties are impossible for distinct model
+    types because overlapping detectors must declare an explicit
+    ``detect_priority`` (enforced by ``test_tts_detection.py``).
+    """
+    for detector in iter_tts_detectors():
+        if detector.matches(model_stage, model_arch):
+            return detector.name
+    return None
+
+
+def all_tts_stage_keys() -> frozenset[str]:
+    """Every ``model_stage`` value served by ``/v1/audio/speech``."""
+    keys: set[str] = set()
+    for detector in iter_tts_detectors():
+        keys |= detector.stage_keys
+    return frozenset(keys)
+
+
+def tts_entry_stage_archs() -> frozenset[str]:
+    """``model_arch`` values that identify an AR entry stage on their own.
+
+    Only for models that own no ``model_stage`` key; stage discovery falls back
+    to these. Deliberately *not* every declared ``model_arch``: a model that also
+    owns a stage key is found by that key, and widening this would change which
+    stage is selected in mixed deployments.
+    """
+    archs: set[str] = set()
+    for detector in iter_tts_detectors():
+        if detector.arch_identifies_entry_stage:
+            archs |= detector.model_archs
+    return frozenset(archs)
+
+
 # Import adapter modules for their registration side effects. Keep at the bottom
 # so the registry helpers above are defined first.
 from vllm_omni.entrypoints.openai.tts_adapters import (  # noqa: E402,F401
@@ -72,12 +140,18 @@ from vllm_omni.entrypoints.openai.tts_adapters import (  # noqa: E402,F401
 __all__ = [
    "ARTTSAdapter",
    "DiffusionTTSAdapter",
+    "LegacyDetector",
    "OutputPolicy",
    "PreparedRequest",
    "SpeechServingContext",
    "TTSModelAdapter",
+    "LEGACY_TTS_DETECTORS",
    "TTS_ADAPTER_REGISTRY",
    "all_tts_model_types",
+    "all_tts_stage_keys",
+    "detect_tts_model_type",
+    "iter_tts_detectors",
    "register_tts_adapter",
    "resolve_adapter",
+    "tts_entry_stage_archs",
 ]
@@ -29,6 +29,15 @@ class AudexAdapter(ARTTSAdapter):
    stage_keys = frozenset({"audex_thinker", "audex_omni"})
    name = "audex"

+    @classmethod
+    def stage_serves_speech(cls, model_stage: str | None, all_stage_keys: frozenset[str]) -> bool:
+        """``audex_omni`` covers the audex_s2s deployment, whose TTS pass uses
+        the same ``/v1/audio/speech`` surface. Without the speech decoder the
+        same thinker is text-final and must not accept speech requests."""
+        if model_stage == "audex_omni":
+            return "audex_code2wav" in all_stage_keys
+        return True
+
    def validate(self, request: "OpenAICreateSpeechRequest") -> str | None:
        if not request.input or not request.input.strip():
            return "Audex TTS requires non-empty input text"
@@ -99,8 +99,20 @@ class TTSModelAdapter(ABC):

    #: Stable discriminator string (the model-type from detection); registry key.
    name: ClassVar[str]
-    #: Engine ``model_stage`` key(s) this model uses, for documentation only.
+    #: Engine ``model_stage`` key(s) this model uses. Load-bearing: the serving
+    #: layer discovers the TTS stage and resolves the model type from these.
    stage_keys: ClassVar[frozenset[str]] = frozenset()
+    #: Engine ``model_arch`` value(s) that identify this model type. Checked
+    #: before ``stage_keys`` in :meth:`matches`. Only needed by models whose
+    #: ``model_stage`` alone is ambiguous or absent.
+    model_archs: ClassVar[frozenset[str]] = frozenset()
+    #: Set when the model has no dedicated ``model_stage`` value and its AR
+    #: entry stage must be discovered by ``model_archs`` instead (Ming dense).
+    arch_identifies_entry_stage: ClassVar[bool] = False
+    #: Detection order; lower runs first. Only set this to break a genuine
+    #: overlap with another adapter — see ``tests/.../test_tts_detection.py``,
+    #: which fails if two same-priority detectors can match the same input.
+    detect_priority: ClassVar[int] = 100
    #: Serving backend: ``"ar"`` (engine_client) or ``"diffusion"``.
    backend: ClassVar[str] = "ar"

@@ -111,6 +123,31 @@ class TTSModelAdapter(ABC):
    def __init__(self, ctx: SpeechServingContext) -> None:
        self.ctx = ctx

+    @classmethod
+    def matches(cls, model_stage: str | None, model_arch: str | None) -> bool:
+        """Whether a deployed stage with this ``model_stage``/``model_arch``
+        is served by this adapter.
+
+        Architecture wins over stage key, so a model that declares both is
+        recognized even when deployed under a stage key it shares with another
+        model. Override only for match rules that are not a set membership
+        test (see ``covo_audio``).
+        """
+        if model_arch is not None and model_arch in cls.model_archs:
+            return True
+        return model_stage is not None and model_stage in cls.stage_keys
+
+    @classmethod
+    def stage_serves_speech(cls, model_stage: str | None, all_stage_keys: frozenset[str]) -> bool:
+        """Whether a matched stage really accepts ``/v1/audio/speech`` in *this*
+        deployment.
+
+        Lets a model that is speech-capable only in some topologies say so
+        (Audex: its omni thinker is text-final unless the speech decoder is
+        deployed alongside). Default: always.
+        """
+        return True
+
    def normalize(self, request: "OpenAICreateSpeechRequest") -> None:
        """In-place request normalization/mutation (e.g. infer task type,
        lowercase voice). Default: no-op."""
@@ -189,10 +226,37 @@ class DiffusionTTSAdapter(TTSModelAdapter):
        return frozenset(params) if params is not None else frozenset()


+@dataclass(frozen=True)
+class LegacyDetector:
+    """A model type the serving layer detects but has no adapter for yet.
+
+    Detection has to keep naming these models so ``serving_speech.py`` can route
+    them down its legacy path, but they own none of the adapter contract. Each
+    entry is a migration debt: delete it when the model gets an adapter. The
+    pre-commit gate (``tools/pre_commit/check_tts_adapter.py``) fails if the list
+    grows.
+
+    Exposes the subset of the :class:`TTSModelAdapter` class surface that
+    detection reads, so both kinds live in one sorted sequence.
+    """
+
+    name: str
+    stage_keys: frozenset[str] = frozenset()
+    model_archs: frozenset[str] = frozenset()
+    arch_identifies_entry_stage: bool = False
+    detect_priority: int = 100
+
+    def matches(self, model_stage: str | None, model_arch: str | None) -> bool:
+        if model_arch is not None and model_arch in self.model_archs:
+            return True
+        return model_stage is not None and model_stage in self.stage_keys
+
+
 # Re-exported here to avoid import cycles at call sites.
 __all__ = [
    "ARTTSAdapter",
    "DiffusionTTSAdapter",
+    "LegacyDetector",
    "OutputPolicy",
    "PreparedRequest",
    "SpeechServingContext",
@@ -15,6 +15,13 @@ class CovoAudioAdapter(ARTTSAdapter):
    stage_keys = frozenset({"fused_thinker_talker"})
    name = "covo_audio"

+    @classmethod
+    def matches(cls, model_stage: str | None, model_arch: str | None) -> bool:
+        """``fused_thinker_talker`` is a generic stage key that non-CoVo fused
+        deployments also use, so the architecture has to confirm it. A fused
+        stage from another model falls through to "not a TTS model"."""
+        return super().matches(model_stage, model_arch) and bool(model_arch) and "CovoAudio" in model_arch
+
    def validate(self, request: "OpenAICreateSpeechRequest") -> str | None:
        if not request.input or not request.input.strip():
            return "Input text cannot be empty"
@@ -17,8 +17,16 @@ logger = init_logger(__name__)

@register_tts_adapter
 class MingTTSAdapter(ARTTSAdapter):
-    # Detected by model_arch (MingTTSForConditionalGeneration), not stage key.
+    # Ming dense has no dedicated model_stage value, so both stage discovery and
+    # model-type detection go through the architecture. ``ming_flash_omni_tts``
+    # is the model that owns the ``ming_tts`` *stage key*, and is matched first.
    name = "ming_tts"
+    model_archs = frozenset({"MingTTSForConditionalGeneration"})
+    arch_identifies_entry_stage = True
+    # Ming dense deploys its AR stage as the generic ``model_stage="llm"``, so
+    # this is an architecture *fallback*: it must run after every adapter that
+    # owns a real stage key, or it would claim stages those adapters own.
+    detect_priority = 200

    def validate(self, request: "OpenAICreateSpeechRequest") -> str | None:
        """Validate Ming TTS request parameters. Returns error message or None."""
@@ -16,7 +16,12 @@ class VoxCPM2Adapter(ARTTSAdapter):
    stage is present (and/or via ``model_arch``)."""

    stage_keys = frozenset({"latent_generator"})
+    model_archs = frozenset({"VoxCPM2TalkerForConditionalGeneration"})
    name = "voxcpm2"
+    # The talker architecture is authoritative and is tested ahead of every
+    # stage key, so a VoxCPM2 talker deployed under a stage key another model
+    # claims still resolves to VoxCPM2.
+    detect_priority = 10

    def validate(self, request: "OpenAICreateSpeechRequest") -> str | None:
        """Validate VoxCPM2 request parameters. Returns error message or None."""
@@ -0,0 +1,167 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
+"""Adapter routing Cosmos3's per-CFG-branch UND text K/V through session state.
+
+Cosmos3's cross-step cache is the UND (reasoner) text K/V, currently kept as
+instance fields on ``Cosmos3VFMTransformer`` (``cached_kv``).
+``Cosmos3StateAdapter`` moves its durable ownership into the shared
+``SessionStateManager`` (RFC #4480), behind the
+``enable_session_state_manager`` opt-in flag. The load-to-transformer handoff
+still uses the transformer's fields, so this is session-state groundwork rather
+than synchronization for overlapping forwards.
+
+Each (layer, CFG branch) pair is a model-specific ``Cosmos3UNDKV`` state object.
+The shared package deliberately does not define a dense attention-KV object:
+DreamZero's attention KV belongs to the paged AR-Diffusion engine, whereas this
+is Cosmos3's fixed, encode-once conditioning KV. Keeping the concrete object in
+the model adapter preserves that ownership boundary while retaining the shared
+lifecycle and byte accounting.
+
+``freqs_gen`` (M-RoPE cos/sin for the GEN pathway) is also session-scoped. It
+is constant for one request, so retaining it in ``SessionState.attrs`` avoids
+recomputing both UND and GEN RoPE frequencies on every denoise step.
+"""
+
+from __future__ import annotations
+
+from typing import Protocol
+
+import torch
+
+from vllm_omni.experimental.world_models.session_state.accounting import (
+    SeenStorages,
+    device_bytes,
+)
+from vllm_omni.experimental.world_models.session_state.base import StateObject
+from vllm_omni.experimental.world_models.session_state.manager import (
+    SessionStateManager,
+)
+
+KVPair = tuple[torch.Tensor, torch.Tensor]
+
+
+class _Cosmos3KVTransformer(Protocol):
+    cached_kv: list[KVPair] | None
+    cached_freqs_gen: tuple[torch.Tensor, torch.Tensor] | None
+
+
+class Cosmos3UNDKV(StateObject[KVPair, KVPair]):
+    """One layer of fixed, encode-once Cosmos3 UND conditioning K/V."""
+
+    def __init__(self) -> None:
+        self._kv: KVPair | None = None
+
+    def allocate(self) -> None:
+        self._kv = None
+
+    def commit(self, payload: KVPair | None = None) -> None:
+        value = self._staged if payload is None else payload
+        if value is None:
+            raise RuntimeError("Cosmos3UNDKV.commit requires K/V payload")
+        self._kv = value
+        self.discard()
+
+    def view(self, *, include_staged: bool = True) -> KVPair:
+        if include_staged and self._staged is not None:
+            return self._staged
+        if self._kv is None:
+            raise RuntimeError("Cosmos3UNDKV is not resident")
+        return self._kv
+
+    def reset(self) -> None:
+        self._kv = None
+        self.discard()
+
+    def nbytes_by_device(self, seen: SeenStorages) -> dict[str, int]:
+        return {} if self._kv is None else device_bytes(self._kv, seen)
+
+    @property
+    def resident(self) -> bool:
+        return self._kv is not None
+
+
+def _layer_key(layer_index: int, is_negative: bool) -> str:
+    """Session key for one (layer, CFG branch) UND K/V object (cf. DreamZero ``_xattn_key``)."""
+    return f"und_kv_neg/{layer_index}" if is_negative else f"und_kv/{layer_index}"
+
+
+def _freqs_key(is_negative: bool) -> str:
+    return "und_freqs_gen_neg" if is_negative else "und_freqs_gen"
+
+
+class Cosmos3StateAdapter:
+    """Session-keyed view over Cosmos3's per-branch UND text K/V.
+
+    A fresh adapter is built per forward; the durable state lives in the manager
+    (mirroring ``DreamZeroStateAdapter``).
+    """
+
+    def __init__(self, session_id: str | None, manager: SessionStateManager) -> None:
+        self._session_id = session_id
+        # Pin the session: the manager may evict it from its lookup table under
+        # LRU pressure, but an adapter holding a reference keeps its state alive.
+        self._session = manager.get_or_create_session(session_id)
+
+    @property
+    def _num_layers(self) -> int | None:
+        value = self._session.attrs.get("_num_layers")
+        return value if isinstance(value, int) else None
+
+    def is_branch_initialized(self, is_negative: bool) -> bool:
+        """Whether this CFG branch's UND K/V has been stored (encode-once)."""
+        if self._num_layers is None:
+            return False
+        obj = self._session.get(_layer_key(0, is_negative))
+        return obj is not None and obj.resident
+
+    def get_branch_kv(self, is_negative: bool) -> list[KVPair] | None:
+        """Return the branch's ``cached_kv`` as ``list[(K, V)]`` by layer, or ``None``."""
+        if not self.is_branch_initialized(is_negative):
+            return None
+        num_layers = self._num_layers
+        assert num_layers is not None
+        out: list[KVPair] = []
+        for i in range(num_layers):
+            obj = self._session.get(_layer_key(i, is_negative))
+            if not isinstance(obj, Cosmos3UNDKV) or not obj.resident:
+                raise RuntimeError("Cosmos3 UND K/V is partially initialized.")
+            out.append(obj.view())
+        return out
+
+    def set_branch_kv(self, is_negative: bool, cached_kv: list[KVPair]) -> None:
+        """Store this branch's freshly computed ``cached_kv`` once."""
+        self._session.attrs["_num_layers"] = len(cached_kv)
+        for i, (k, v) in enumerate(cached_kv):
+            obj = self._session.get(_layer_key(i, is_negative))
+            if not isinstance(obj, Cosmos3UNDKV):
+                obj = Cosmos3UNDKV()
+                obj.allocate()
+                self._session.put(_layer_key(i, is_negative), obj)
+            obj.commit((k, v))
+
+    def get_branch_freqs_gen(self, is_negative: bool) -> tuple[torch.Tensor, torch.Tensor] | None:
+        value = self._session.attrs.get(_freqs_key(is_negative))
+        if isinstance(value, tuple) and len(value) == 2 and all(isinstance(freq, torch.Tensor) for freq in value):
+            return value
+        return None
+
+    def load_into_transformer(self, transformer: _Cosmos3KVTransformer, is_negative: bool) -> None:
+        """Assign this branch's UND K/V and GEN RoPE frequencies onto the transformer."""
+        transformer.cached_kv = self.get_branch_kv(is_negative)
+        transformer.cached_freqs_gen = self.get_branch_freqs_gen(is_negative)
+
+    def capture_from_transformer(self, transformer: _Cosmos3KVTransformer, is_negative: bool) -> None:
+        """Capture the freshly computed ``cached_kv`` into the session if not already stored."""
+        if not self.is_branch_initialized(is_negative):
+            cached_kv = transformer.cached_kv
+            if cached_kv is None:
+                raise RuntimeError("Cosmos3 transformer did not produce UND K/V")
+            freqs_gen = transformer.cached_freqs_gen
+            if freqs_gen is None:
+                raise RuntimeError("Cosmos3 transformer did not produce GEN RoPE frequencies")
+            self.set_branch_kv(is_negative, cached_kv)
+            self._session.attrs[_freqs_key(is_negative)] = freqs_gen
+
+    def reset(self) -> None:
+        """Clear all UND K/V objects + metadata for this session (replaces ``reset_cache()``)."""
+        self._session.reset()
@@ -200,6 +200,11 @@ class SineGen2(torch.nn.Module):
        self.flag_for_pulse = flag_for_pulse
        self.upsample_scale = upsample_scale
        self.causal = causal
+        self.register_buffer(
+            "harmonic_ids",
+            torch.arange(1, self.harmonic_num + 2, dtype=torch.float32).view(1, 1, -1),
+            persistent=False,
+        )
        if causal is True:
            self.rand_ini = torch.rand(1, 9)
            self.rand_ini[:, 0] = 0
@@ -277,7 +282,7 @@ class SineGen2(torch.nn.Module):
        output uv: tensor(batchsize=1, length, 1)
        """
        # fundamental component
-        fn = torch.multiply(f0, torch.FloatTensor([[range(1, self.harmonic_num + 2)]]).to(f0.device))
+        fn = torch.multiply(f0, self.harmonic_ids)

        # generate sine waveforms
        sine_waves = self._f02sine(fn) * self.sine_amp
@@ -463,7 +468,11 @@ class HiFTGenerator(nn.Module):
        self.ups.apply(init_weights)
        self.conv_post.apply(init_weights)
        self.reflection_pad = nn.ReflectionPad1d((1, 0))
-        self.stft_window = torch.from_numpy(get_window("hann", istft_params["n_fft"], fftbins=True).astype(np.float32))
+        self.register_buffer(
+            "stft_window",
+            torch.from_numpy(get_window("hann", istft_params["n_fft"], fftbins=True).astype(np.float32)),
+            persistent=False,
+        )
        self.f0_predictor = f0_predictor

    def remove_weight_norm(self):
@@ -485,7 +494,7 @@ class HiFTGenerator(nn.Module):
            self.istft_params["n_fft"],
            self.istft_params["hop_len"],
            self.istft_params["n_fft"],
-            window=self.stft_window.to(x.device),
+            window=self.stft_window,
            return_complex=True,
        )
        spec = torch.view_as_real(spec)  # [B, F, TT, 2]
@@ -500,11 +509,15 @@ class HiFTGenerator(nn.Module):
            self.istft_params["n_fft"],
            self.istft_params["hop_len"],
            self.istft_params["n_fft"],
-            window=self.stft_window.to(magnitude.device),
+            window=self.stft_window,
        )
        return inverse_transform

-    def decode(self, x: torch.Tensor, s: torch.Tensor = torch.zeros(1, 1, 0)) -> torch.Tensor:
+    def _decode_pre_istft(
+        self,
+        x: torch.Tensor,
+        s: torch.Tensor,
+    ) -> tuple[torch.Tensor, torch.Tensor]:
        s_stft_real, s_stft_imag = self._stft(s.squeeze(1))
        s_stft = torch.cat([s_stft_real, s_stft_imag], dim=1)

@@ -533,11 +546,17 @@ class HiFTGenerator(nn.Module):
        x = self.conv_post(x)
        magnitude = torch.exp(x[:, : self.istft_params["n_fft"] // 2 + 1, :])
        phase = torch.sin(x[:, self.istft_params["n_fft"] // 2 + 1 :, :])  # actually, sin is redundancy
+        return magnitude, phase

+    def _finalize_decode(self, magnitude, phase):
        x = self._istft(magnitude, phase)
        x = torch.clamp(x, -self.audio_limit, self.audio_limit)
        return x

+    def decode(self, x: torch.Tensor, s: torch.Tensor = torch.zeros(1, 1, 0)) -> torch.Tensor:
+        magnitude, phase = self._decode_pre_istft(x, s)
+        return self._finalize_decode(magnitude, phase)
+
    def forward(
        self,
        batch: dict,
@@ -555,7 +574,11 @@ class HiFTGenerator(nn.Module):
        return generated_speech, f0

    @torch.inference_mode()
-    def inference(self, speech_feat: torch.Tensor, cache_source: torch.Tensor = torch.zeros(1, 1, 0)) -> torch.Tensor:
+    def _inference_pre_istft(
+        self,
+        speech_feat: torch.Tensor,
+        cache_source: torch.Tensor,
+    ) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor]:
        # mel->f0
        f0 = self.f0_predictor(speech_feat)
        # f0->source
@@ -565,7 +588,13 @@ class HiFTGenerator(nn.Module):
        # use cache_source to avoid glitch
        if cache_source.shape[2] != 0:
            s[:, :, : cache_source.shape[2]] = cache_source
-        generated_speech = self.decode(x=speech_feat, s=s)
+        magnitude, phase = self._decode_pre_istft(speech_feat, s)
+        return magnitude, phase, s
+
+    @torch.inference_mode()
+    def inference(self, speech_feat: torch.Tensor, cache_source: torch.Tensor = torch.zeros(1, 1, 0)) -> torch.Tensor:
+        magnitude, phase, s = self._inference_pre_istft(speech_feat, cache_source)
+        generated_speech = self._finalize_decode(magnitude, phase)
        return generated_speech, s


@@ -4,6 +4,7 @@

 from __future__ import annotations

+from collections.abc import Mapping
 from contextlib import nullcontext
 from dataclasses import dataclass
 from typing import Any
@@ -11,6 +12,11 @@ from typing import Any
 import torch
 import torch.nn as nn
 import torch.nn.functional as F
+from vllm.logger import init_logger
+
+from .cuda_graph_wrapper import HiFTGraphWrapper
+
+logger = init_logger(__name__)

 _SILENCE_TOKEN = 4218

@@ -60,7 +66,14 @@ class BatchedToken2Wav(nn.Module):
    asset loader and prompt feature extractor.
    """

-    def __init__(self, token2wav: Any, trt_stepper: Any | None = None):
+    def __init__(
+        self,
+        token2wav: Any,
+        trt_stepper: Any | None = None,
+        *,
+        connector_config: Mapping[str, int] | None = None,
+        hift_graph_config: Mapping[str, Any] | None = None,
+    ):
        super().__init__()
        self._token2wav = token2wav
        # Optional TrtDiTStepper (step_audio2_dit_trt): replaces only the
@@ -82,7 +95,7 @@ class BatchedToken2Wav(nn.Module):
                _autocast_disabled(device),
            ):
                # 50 mel frames match the default first streamed vocoder chunk.
-                speech, source = self.hift(
+                speech, source = self.hift.inference(
                    torch.zeros((1, mel_channels, 50), device=device, dtype=dtype),
                    torch.zeros((1, 1, 0), device=device, dtype=dtype),
                )
@@ -98,8 +111,41 @@ class BatchedToken2Wav(nn.Module):
            token2wav.speech_window.detach().clone(),
            persistent=False,
        )
+        self.hift_graph_wrapper: HiFTGraphWrapper | None = None
+        graph_config = dict(hift_graph_config or {})
+        if bool(graph_config.get("enabled", False)):
+            if hift_parameter is None:
+                raise ValueError("MiniCPM-o HiFT Graph requires a parameterized HiFT module")
+            if hift_parameter.device.type != "cuda":
+                logger.info("HiFT CUDA Graph is disabled on device type %s", hift_parameter.device.type)
+            else:
+                if connector_config is None:
+                    raise ValueError("MiniCPM-o HiFT CUDA Graph requires connector chunk configuration")
+                if self.mel_cache_len <= 0 or self.source_cache_len % self.mel_cache_len != 0:
+                    raise ValueError(
+                        "MiniCPM-o HiFT CUDA Graph requires source_cache_len to be divisible by mel_cache_len"
+                    )
+                capture_batch_sizes = graph_config.get("capture_batch_sizes", [1])
+                logger.info("Enabling HiFT CUDA Graph with batch sizes %s", capture_batch_sizes)
+                self.hift_graph_wrapper = HiFTGraphWrapper(
+                    token2wav=token2wav,
+                    connector_config=dict(connector_config),
+                    capture_batch_sizes=capture_batch_sizes,
+                )
+                with torch.inference_mode(), _autocast_disabled(hift_parameter.device):
+                    self.hift_graph_wrapper.capture()
+                logger.info("HiFT CUDA Graph captured successfully")
        self._prompt_features: dict[tuple[str, str], PromptFeatures] = {}

+    def _hift_inference(
+        self,
+        mel: torch.Tensor,
+        source_cache: torch.Tensor,
+    ) -> tuple[torch.Tensor, torch.Tensor]:
+        if self.hift_graph_wrapper is None:
+            return self.hift.inference(mel, source_cache)
+        return self.hift_graph_wrapper.replay(mel, source_cache)
+
    def prepare_prompt(self, prompt_cache_id: str, prompt_wav: str) -> PromptFeatures:
        cache_key = (prompt_cache_id, prompt_wav)
        cached = self._prompt_features.get(cache_key)
@@ -463,7 +509,7 @@ class BatchedToken2Wav(nn.Module):
        old_source = torch.cat([state.hift_cache["source"] for state in states], dim=0)
        old_speech = torch.cat([state.hift_cache["speech"] for state in states], dim=0)
        mel = torch.cat((old_mel, chunk_mel), dim=2)
-        speech, source = self.hift(mel, old_source)
+        speech, source = self._hift_inference(mel, old_source)
        if old_speech.shape[-1] > 0:
            window = self.speech_window.to(device=speech.device, dtype=speech.dtype)
            speech = self._fade_in_out(speech, old_speech, window)
@@ -0,0 +1,135 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
+
+import torch
+from torch.cuda import CUDAGraph
+from vllm.logger import init_logger
+from vllm.platforms import current_platform
+
+logger = init_logger(__name__)
+
+
+class HiFTGraphWrapper:
+    def __init__(self, token2wav, connector_config, capture_batch_sizes):
+        self.decode_fn = token2wav.hift.inference
+        self.graph_fn = token2wav.hift._inference_pre_istft
+        self.finalize_fn = token2wav.hift._finalize_decode
+        self.codec_chunk_frames = connector_config["codec_chunk_frames"]
+        self.codec_left_context_frames = connector_config["codec_left_context_frames"]
+        lookahead_layer = getattr(token2wav.flow.encoder, "pre_lookahead_layer", None)
+        pre_lookahead_len = getattr(lookahead_layer, "pre_lookahead_len", None)
+        self.pre_lookahead_len = int(pre_lookahead_len) if pre_lookahead_len is not None else 3
+        self.mel_cache_len = int(token2wav.mel_cache_len)
+        self.source_cache_len = int(token2wav.source_cache_len)
+        self.mel_frames = int(token2wav.hift.conv_pre.in_channels)
+        self.flow_upsample_rate = int(getattr(token2wav.flow, "token_mel_ratio", 2))
+        self.capture_bucket_size, self.capture_source_cache_len = self.derive_capture_bucket_size()
+        self.capture_batch_sizes = capture_batch_sizes
+        self.graph: dict[tuple[int, int, int], torch.cuda.CUDAGraph] = {}
+        self.static_speech_inputs: dict[tuple[int, int, int], torch.Tensor] = {}
+        self.static_magnitude_outputs: dict[tuple[int, int, int], torch.Tensor] = {}
+        self.static_phase_outputs: dict[tuple[int, int, int], torch.Tensor] = {}
+        self.static_cache_source_inputs: dict[tuple[int, int, int], torch.Tensor] = {}
+        self.static_cache_source_outputs: dict[tuple[int, int, int], torch.Tensor] = {}
+        parameter = next(token2wav.hift.parameters())
+        self.device = parameter.device
+        self.dtype = parameter.dtype
+        self.max_lazy_graphs = 8
+        self.lazy_graph_count = 0
+
+    def derive_capture_bucket_size(self):
+        chunk_mel_frames = (
+            self.codec_chunk_frames + self.codec_left_context_frames - self.pre_lookahead_len
+        ) * self.flow_upsample_rate
+
+        return [chunk_mel_frames, chunk_mel_frames + self.mel_cache_len], [
+            0,
+            self.source_cache_len,
+        ]
+
+    def capture(self):
+        for batch_size in self.capture_batch_sizes:
+            for mel_frames, source_cache_len in zip(
+                self.capture_bucket_size,
+                self.capture_source_cache_len,
+                strict=True,
+            ):
+                self._capture(batch_size, mel_frames, source_cache_len)
+
+    def _capture(
+        self,
+        batch_size: int,
+        mel_frames: int,
+        source_cache_len: int,
+    ):
+        if torch.cuda.is_current_stream_capturing():
+            raise RuntimeError("Cannot capture HiFT graph during an active stream capture")
+
+        key = (batch_size, mel_frames, source_cache_len)
+
+        if key in self.graph:
+            return
+
+        static_mel = torch.zeros(batch_size, self.mel_frames, mel_frames, device=self.device, dtype=self.dtype)
+        static_source_cache = torch.zeros(batch_size, 1, source_cache_len, device=self.device, dtype=self.dtype)
+        current_stream = torch.cuda.current_stream(self.device)
+        warmup_stream = torch.cuda.Stream(device=self.device)
+        warmup_stream.wait_stream(current_stream)
+        with torch.cuda.stream(warmup_stream), torch.no_grad():
+            for _ in range(3):
+                warmup_outputs = self.graph_fn(static_mel, static_source_cache)
+        current_stream.wait_stream(warmup_stream)
+        del warmup_outputs
+
+        graph = CUDAGraph()
+        with torch.cuda.graph(graph, pool=current_platform.get_global_graph_pool()):
+            static_magnitude_output, static_phase_output, static_cache_source_output = self.graph_fn(
+                static_mel,
+                static_source_cache,
+            )
+
+        self.graph[key] = graph
+        self.static_speech_inputs[key] = static_mel
+        self.static_cache_source_inputs[key] = static_source_cache
+
+        self.static_magnitude_outputs[key] = static_magnitude_output
+        self.static_phase_outputs[key] = static_phase_output
+        self.static_cache_source_outputs[key] = static_cache_source_output
+        logger.info("Captured HiFT CUDA Graph for shape %s", key)
+
+    def replay(self, speech_feat, cache_source):
+        if torch.cuda.is_current_stream_capturing():
+            logger.info("Falling back to eager HiFT inference during an active stream capture")
+            return self.decode_fn(speech_feat, cache_source)
+
+        batch_size = speech_feat.shape[0]
+        num_frames = speech_feat.shape[2]
+        cache_source_len = cache_source.shape[2]
+        target_b = next((b for b in sorted(self.capture_batch_sizes) if b >= batch_size), None)
+
+        if target_b is None:
+            logger.info("Falling back to eager HiFT inference for unsupported batch size %d", batch_size)
+            return self.decode_fn(speech_feat, cache_source)
+
+        key = (target_b, num_frames, cache_source_len)
+
+        if key not in self.graph:
+            if self.lazy_graph_count >= self.max_lazy_graphs:
+                logger.info("Falling back to eager HiFT inference after reaching the lazy Graph limit")
+                return self.decode_fn(speech_feat, cache_source)
+            logger.info("Lazily capturing HiFT CUDA Graph for shape %s", key)
+            self._capture(*key)
+            self.lazy_graph_count += 1
+
+        static_speech_inputs = self.static_speech_inputs[key].zero_()
+        static_speech_inputs[:batch_size].copy_(speech_feat)
+        static_cache_sources = self.static_cache_source_inputs[key].zero_()
+        static_cache_sources[:batch_size].copy_(cache_source)
+
+        self.graph[key].replay()
+        static_magnitude_output = self.static_magnitude_outputs[key]
+        static_phase_output = self.static_phase_outputs[key]
+        static_cache_source_output = self.static_cache_source_outputs[key]
+        cache_source = static_cache_source_output[:batch_size].clone()
+        speech = self.finalize_fn(static_magnitude_output[:batch_size], static_phase_output[:batch_size]).clone()
+        return speech, cache_source
@@ -150,6 +150,18 @@ class MiniCPMO45Code2Wav(nn.Module):
            prefix="minicpmo45-runtime-prompts-",
        )
        extra = self._extra_config()
+        self._connector_config = {
+            "codec_chunk_frames": int(extra.get("codec_chunk_frames", 25)),
+            "codec_left_context_frames": int(extra.get("codec_left_context_frames", 3)),
+        }
+        if self._connector_config["codec_chunk_frames"] <= 0 or self._connector_config["codec_left_context_frames"] < 0:
+            raise ValueError(f"Invalid MiniCPM-o connector chunk configuration: {self._connector_config}")
+        raw_capture_batch_sizes = extra.get("hift_graph_capture_batch_sizes")
+        capture_batch_sizes = [1] if raw_capture_batch_sizes is None else raw_capture_batch_sizes
+        self._hift_graph_config = {
+            "enabled": bool(extra.get("enable_hift_graph", False)),
+            "capture_batch_sizes": capture_batch_sizes,
+        }
        self._min_batch_size = int(extra.get("code2wav_min_batch_size", 1))
        if self._min_batch_size < 1:
            raise ValueError("MiniCPM-o Code2Wav code2wav_min_batch_size must be >= 1")
@@ -785,4 +797,9 @@ class MiniCPMO45Code2Wav(nn.Module):
            logger.info("MiniCPM-o Code2Wav: DiT estimator running on TensorRT (%s)", dtype_name)
            token2wav.enable_trt_spk_embedding()
            logger.info("MiniCPM-o Code2Wav: campplus speaker embedding running on TensorRT")
-        self.backend = BatchedToken2Wav(token2wav, trt_stepper=trt_stepper)
+        self.backend = BatchedToken2Wav(
+            token2wav,
+            trt_stepper=trt_stepper,
+            connector_config=self._connector_config,
+            hift_graph_config=self._hift_graph_config,
+        )
@@ -29,6 +29,7 @@ QWEN2_5_OMNI_PIPELINE = PipelineConfig(
            final_output_type="text",
            owns_tokenizer=True,
            requires_multimodal_data=True,
+            hf_config_name="thinker_config",
            engine_output_type="latent",
            sampling_constraints={"detokenize": True},
            custom_process_next_stage_input_func=f"{_PROC}.thinker2talker_full_payload",
@@ -38,6 +39,7 @@ QWEN2_5_OMNI_PIPELINE = PipelineConfig(
            model_stage="talker",
            execution_type=StageExecutionType.LLM_AR,
            input_sources=(0,),
+            hf_config_name="talker_config",
            engine_output_type="latent",
            sync_process_input_func=f"{_PROC}.thinker2talker_token_only",
            custom_process_next_stage_input_func=f"{_PROC}.talker2code2wav_full_payload",
@@ -53,6 +55,7 @@ QWEN2_5_OMNI_PIPELINE = PipelineConfig(
            input_sources=(1,),
            final_output=True,
            final_output_type="audio",
+            hf_config_name="thinker_config",
            engine_output_type="audio",
            sync_process_input_func=f"{_PROC}.talker2code2wav_token_only",
            sampling_constraints={"detokenize": True},
@@ -75,6 +78,7 @@ QWEN2_5_OMNI_THINKER_ONLY_PIPELINE = PipelineConfig(
            final_output_type="text",
            owns_tokenizer=True,
            requires_multimodal_data=True,
+            hf_config_name="thinker_config",
            engine_output_type="latent",
            sampling_constraints={"detokenize": True},
            custom_process_next_stage_input_func=f"{_PROC}.thinker2talker_full_payload",
@@ -13,9 +13,12 @@ import torch
 import torch.nn as nn
 import torchaudio
 import torchaudio.compliance.kaldi as kaldi
-from flashcosyvoice.modules.hifigan import HiFTGenerator
-from flashcosyvoice.utils.audio import mel_spectrogram
 from hyperpyyaml import load_hyperpyyaml
+
+try:
+    from torch.nn.utils.parametrizations import weight_norm
+except ImportError:
+    from torch.nn.utils import weight_norm
 from vllm.config import VllmConfig
 from vllm.logger import init_logger
 from vllm.model_executor.models.interfaces import SupportsPP
@@ -25,6 +28,8 @@ from vllm.v1.outputs import SamplerOutput
 from vllm.v1.sample.metadata import SamplingMetadata
 from vllm.v1.sample.sampler import Sampler

+from vllm_omni.model_executor.models.cosyvoice3.code2wav_core.hifigan import HiFTGenerator
+from vllm_omni.model_executor.models.cosyvoice3.utils import mel_spectrogram
 from vllm_omni.model_executor.models.output_templates import OmniOutput
 from vllm_omni.model_executor.models.step_audio2.step_audio2_constants import (
    DEFAULT_STREAM_CONFIG,
@@ -66,6 +71,30 @@ class _StreamState:
    finished: bool = False


+class _ConvRNNF0Predictor(nn.Module):
+    def __init__(self):
+        super().__init__()
+        layers: list[nn.Module] = []
+        for in_channels in [80, 512, 512, 512, 512]:
+            layers.extend([weight_norm(nn.Conv1d(in_channels, 512, 3, padding=1)), nn.ELU()])
+        self.condnet = nn.Sequential(*layers)
+        self.classifier = nn.Linear(512, 1)
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        return self.classifier(self.condnet(x).transpose(1, 2)).squeeze(-1).abs()
+
+
+def _build_hift() -> HiFTGenerator:
+    return HiFTGenerator(
+        sampling_rate=24000,
+        upsample_rates=[8, 5, 3],
+        upsample_kernel_sizes=[16, 11, 7],
+        source_resblock_kernel_sizes=[7, 7, 11],
+        source_resblock_dilation_sizes=[[1, 3, 5]] * 3,
+        f0_predictor=_ConvRNNF0Predictor(),
+    )
+
+
 class StepAudio2Token2WavCore(nn.Module):
    """Core Token2Wav model - handles token to waveform conversion"""

@@ -142,8 +171,8 @@ class StepAudio2Token2WavCore(nn.Module):
            torch.load(f"{self.model_path}/flow.pt", map_location="cpu", weights_only=True), strict=True
        )
        self._flow.to(self.device).eval()
+        self._hift = _build_hift()

-        self._hift = HiFTGenerator()
        hift_state_dict = {
            k.replace("generator.", ""): v
            for k, v in torch.load(f"{self.model_path}/hift.pt", map_location="cpu", weights_only=True).items()
@@ -232,7 +261,21 @@ class StepAudio2Token2WavCore(nn.Module):
        if sample_rate != 24000:
            audio_24k = librosa.resample(y=audio_24k, orig_sr=sample_rate, target_sr=24000)
        audio_24k_t = torch.from_numpy(audio_24k).unsqueeze(0)  # [1, T]
-        prompt_mel = mel_spectrogram(audio_24k_t).transpose(1, 2).squeeze(0)  # [T, num_mels]
+        prompt_mel = (
+            mel_spectrogram(
+                audio_24k_t,
+                n_fft=1920,
+                num_mels=80,
+                sampling_rate=24000,
+                hop_size=480,
+                win_size=1920,
+                fmin=0,
+                fmax=8000,
+                center=False,
+            )
+            .transpose(1, 2)
+            .squeeze(0)
+        )  # [T, num_mels]
        prompt_mels = prompt_mel.unsqueeze(0).to(self.device)
        prompt_mels_lens = torch.tensor([prompt_mels.shape[1]], dtype=torch.int32, device=self.device)
        prompt_mels = torch.nn.functional.pad(
@@ -278,7 +321,7 @@ class StepAudio2Token2WavCore(nn.Module):
                self.n_timesteps,
            )

-        wav, _ = self.hift(speech_feat=mel)
+        wav, _ = self.hift.inference(speech_feat=mel)

        if return_bytes:
            output = io.BytesIO()
@@ -366,7 +409,7 @@ class StepAudio2Token2WavCore(nn.Module):
        hift_cache_speech = state.hift_cache_dict["speech"]

        mel = torch.cat([hift_cache_mel, chunk_mel], dim=2)
-        speech, source = self.hift(mel, hift_cache_source)
+        speech, source = self.hift.inference(mel, hift_cache_source)

        if hift_cache_speech.shape[-1] > 0:
            speech = fade_in_out(speech, hift_cache_speech, self.speech_window)
@@ -303,7 +303,16 @@ class GPUARModelRunner(OmniGPUModelRunner, OmniConnectorModelRunnerMixin):
        self.input_ids = self._make_buffer(self.max_num_tokens, dtype=torch.int32)
        # each model stage has their own hidden size
        self.hidden_size = self.model_config.hf_text_config.hidden_size
-        self.inputs_embeds = self._make_buffer(self.max_num_tokens, self.hidden_size, dtype=self.dtype, numpy=False)
+        # The width of embeddings received by this stage. It can differ from
+        # the stage's internal hidden size (e.g., Qwen2.5-Omni Talker receives
+        # 3584-wide Thinker states and projects them to a 896-wide hidden state).
+        self.inputs_embeds_size = self.model_config.get_inputs_embeds_size()
+        self.inputs_embeds = self._make_buffer(
+            self.max_num_tokens,
+            self.inputs_embeds_size,
+            dtype=self.dtype,
+            numpy=False,
+        )
        # Initialize KV cache manager (preserve vllm_config fallback behavior)
        self.kv_transfer_manager = OmniKVTransferManager.from_vllm_config(self.vllm_config, self.model_config)
        self._async_chunk = getattr(self.model_config, "async_chunk", False)
Author	SHA1	Message	Date
Hongsheng Liu	48bd050974	docs(review): enforce configuration provenance Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>	2026-08-09 22:39:57 +08:00
boatman	fc8946fcf5	[Perf][MiniCPM-O-4.5]Add CUDA Graph For HiFTGenerator (#5869 ) Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com>	2026-08-09 22:29:49 +08:00
amd-xiaoyu12	f01e4bdddc	[Recipe] Document MiniMax H3 ROCm (gfx950) BF16 serving (#5723 ) Signed-off-by: Xiao Yu <xiao.yu@amd.com> Signed-off-by: amd-xiaoyu12 <xiao.yu@amd.com>	2026-08-09 22:18:24 +08:00
Weiming LiaoandHongsheng Liu	592637015f	[BugFix][NPU] MiniMax-H3 RoPE crash: add the missing batch dim before the mindiesd kernel (#5896 ) Signed-off-by: Weiming Liao <liaowm5@gmail.com> Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>	2026-08-09 14:08:43 +00:00
Yueqian Lin	ebe93f5d1e	[Refactor][TTS] Derive TTS model detection from adapter metadata (#5682 ) Signed-off-by: Yueqian Lin <linyueqian@outlook.com>	2026-08-09 21:30:05 +08:00
linzhenpl07andClaude Opus 4.8	8e2e9b6b53	[Feat] Cosmos3 session-memory port for UND text K/V (RFC #4480 Phase 0) (#4657 ) Signed-off-by: linzhenpl07 <linzhenpl07@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>	2026-08-09 20:53:17 +08:00
Hongsheng Liu	9159cedbd7	docs: update vLLM-Omni architecture overview (#5914 ) Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>	2026-08-09 09:16:18 +08:00
Caspian ChenandSYLAR	593b404539	[Quantization][Qwen2.5-Omni] Support thinker-only ModelOpt NVFP4 W4A4 checkpoints (#5073 ) Signed-off-by: Caspian443 <scrisis843@gmail.com> Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>	2026-08-09 02:45:57 +08:00
Hongsheng Liu	f663d120c7	[Misc] Add vLLM-Omni PR review skill (#5871 ) Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>	2026-08-08 22:18:54 +08:00
Shenglei FuandClaude Opus 5	81b48e83ec	[Model] Fail MiniMax H3 encoder load when a weight or fused shard is missing (#5824 ) Signed-off-by: Shenglei Fu <sfu@confluent.io> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-08 13:14:00 +08:00