[Rebase] Rebase to vLLM 0.30.0 (#7820)

Signed-off-by: tzhouam <tzhouam@connect.ust.hk> Signed-off-by: Gao Han <hgaoaf@connect.ust.hk> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
2026-09-22 15:52:01 +00:00
1 parent ca14e635f5
commit 59db6a4285
@@ -10,7 +10,7 @@ steps:
          DOCKER_BUILDKIT: "1"
          # Buildkite will automatically replace this with the actual commit hash
          VLLM_IMAGE_TAG: "${BUILDKITE_COMMIT}"
-          VLLM_VERSION: "v0.29.0"
+          VLLM_VERSION: "v0.30.0"
        priority: 100
        timeout_in_minutes: 60
        soft_fail: false
@@ -16,9 +16,9 @@ What changed:
  `/v1/images/edits` endpoint to produce benchmark outputs in the expected
  directory structure.
 - The evaluator keeps the same VIEScore-style decomposition:
-  - `semantics_score`
-  - `quality_score`
-  - `overall_score = sqrt(semantics_score * quality_score)`
+    - `semantics_score`
+    - `quality_score`
+    - `overall_score = sqrt(semantics_score * quality_score)`
 - Judge calls are routed to a local OpenAI-compatible model served by
  `vllm-omni`, not a remote provider.

@@ -12,14 +12,14 @@ Upstream mapping:
 What changed:

 - Generation calls the local OpenAI-compatible `vllm-omni` endpoints:
-  - `/v1/images/generations` for text-only frame generation
-  - `/v1/images/edits` for image-conditioned GUI transition generation
+    - `/v1/images/generations` for text-only frame generation
+    - `/v1/images/edits` for image-conditioned GUI transition generation
 - Evaluation still keeps the GEBench scoring dimensions:
-  - `goal`
-  - `logic`
-  - `cons`
-  - `ui`
-  - `qual`
+    - `goal`
+    - `logic`
+    - `cons`
+    - `ui`
+    - `qual`
 - Judge calls are also routed to a local OpenAI-compatible model served by
  `vllm-omni` instead of a remote service.

@@ -31,6 +31,7 @@ docker run -it \
 ```

 Parameter explanation:
+
 - `--cap-add=SYS_PTRACE`: Allow reading system topology information
 - `--cap-add=IPC_LOCK`: Allow memory locking (required for RDMA memory registration)
 - `--security-opt seccomp=unconfined`: Disable seccomp restrictions
@@ -60,6 +61,7 @@ When running single-node tests (producer and consumer on the same machine), ensu
 InfiniBand devices use LID (Local Identifier) for routing. Different devices have different LIDs and cannot communicate directly. If no device is specified, Mooncake may assign different devices to connectors, causing handshake failures.

 Common error:
+
 ```
 [Handshake] Failed to modify QP to RTR, check mtu, gid, peer lid, peer qp num: Invalid argument [22]
 ```
@@ -249,7 +251,7 @@ python cross_node_mooncake_transfer_engine.py \
 ### Transfer Modes

 | Mode | Description | Example |
-|------|-------------|---------|
+| ------ | ------------- | --------- |
 | `copy` | Normal path — tensor copied to RDMA pool (default) | `--mode copy` |
 | `zerocopy` | Zero-copy path — data created directly in RDMA pool | `--mode zerocopy` |
 | `gpu` | GPU transfer — RDMA pool on GPU, uses GPUDirect | `--mode gpu --gpu-id 0` |
@@ -281,7 +283,7 @@ python cross_node_mooncake_transfer_engine.py \
 ### Cross-Node Test Options

 | Option | Description | Default |
-|--------|-------------|---------|
+| -------- | ------------- | --------- |
 | `--role` | `producer` or `consumer` | Required |
 | `--local-host` | Local RDMA IP address | Required |
 | `--remote-host` | Remote RDMA IP address | Required |
@@ -303,6 +305,7 @@ python cross_node_mooncake_transfer_engine.py \
 **Cause**: QP handshake failed, usually due to device configuration mismatch.

 **Solution**:
+
 ```bash
 # Force using the same device
 export RDMA_DEVICE_NAME='mlx5_0'
@@ -313,6 +316,7 @@ export RDMA_DEVICE_NAME='mlx5_0'
 **Cause**: Mooncake not installed or import failed.

 **Solution**:
+
 ```bash
 # Check Mooncake installation
 python -c "from mooncake.engine import TransferEngine; print('OK')"
@@ -329,6 +333,7 @@ uv pip install mooncake-transfer-engine
 **Cause**: Container lacks IB device access permissions.

 **Solution**:
+
 ```bash
 docker run --device=/dev/infiniband --cap-add=IPC_LOCK ...
 ```
@@ -338,6 +343,7 @@ docker run --device=/dev/infiniband --cap-add=IPC_LOCK ...
 **Cause**: RDMA connection establishment failed or network latency.

 **Solution**:
+
 ```bash
 # Check network status
 ibstat
@@ -349,6 +355,7 @@ ibstatus
 **Cause**: CUDA environment not configured or GPU unavailable.

 **Solution**:
+
 ```bash
 # Check CUDA
 python -c "import torch; print(torch.cuda.is_available())"
@@ -362,7 +369,7 @@ docker run --gpus all ...
 ## Environment Variables Reference

 | Variable | Description | Example |
-|----------|-------------|---------|
+| ---------- | ------------- | --------- |
 | `RDMA_DEVICE_NAME` | Specify RDMA device name | `mlx5_0` |
 | `RDMA_TEST_HOST` | Specify test host IP | `10.0.0.1` |
 | `MC_TE_METRIC` | Enable Mooncake metrics | `1` |
@@ -373,7 +380,7 @@ docker run --gpus all ...
 ## Test Files Overview

 | File | Description | Auto-discovered by pytest |
-|------|-------------|--------------------------|
+| ------ | ------------- | -------------------------- |
 | `test_mooncake_transfer_engine_rdma.py` | Integration tests for MooncakeTransferEngineConnector (basic, E2E, lifecycle, GPU) | Yes |
 | `test_mooncake_transfer_engine_buffer.py` | Memory pool and buffer management unit tests | Yes |
 | `cross_node_mooncake_transfer_engine.py` | Cross-node (multi-machine) testing script — run manually | No (filename does not start with `test_`) |
@@ -381,7 +388,7 @@ docker run --gpus all ...
 ### test_mooncake_transfer_engine_rdma.py — Test Classes

 | Test Class | Memory Pool | Marker | Description |
-|------------|-------------|--------|-------------|
+| ------------ | ------------- | -------- | ------------- |
 | `TestBasicConnector` | CPU | — | Initialization, put tensor/bytes/object, cleanup, pool exhaustion |
 | `TestEndToEnd` | CPU | — | E2E RDMA transfer: tensor, bytes, object, zero-copy, large payload (100MB), mixed types, concurrency |
 | `TestLifecycle` | CPU | — | Close, context manager, double-close safety |
@@ -391,7 +398,7 @@ docker run --gpus all ...
 ### test_mooncake_transfer_engine_buffer.py — Test Classes

 | Test Class | Marker | Description |
-|------------|--------|-------------|
+| ------------ | -------- | ------------- |
 | `TestBufferAllocator` | — | Basic alloc/free, alignment, exhaustion/recovery, thread safety |
 | `TestAllocatorInvariants` | `slow` | Double-free safety, overlap corruption detection, adjacent-block merging, fragmentation/defrag |
 | `TestManagedBuffer` | — | Tensor views, context manager |
@@ -5,7 +5,7 @@ Benchmark GLM-Image T2I (text-to-image) and I2I (image-to-image) performance acr
 ## Benchmarks

 | Benchmark | Script | Description |
-|-----------|--------|-------------|
+| ----------- | -------- | ------------- |
 | HuggingFace Baseline | `huggingface/inference.py` | Single-GPU transformers + diffusers pipeline |
 | vLLM-Omni Offline | `vllm-omni/inference.py` | Offline inference with continuous batching |
 | vLLM-Omni Online | `benchmark_glm_image.py` | Online serving via `/v1/chat/completions` |
@@ -27,7 +27,7 @@ CUDA_VISIBLE_DEVICES=0 python benchmarks/glm_image/huggingface/inference.py \
 ### Options

 | Flag | Default | Description |
-|------|---------|-------------|
+| ------ | --------- | ------------- |
 | `--model-path` | `zai-org/GLM-Image` | Model path |
 | `--mode` | `t2i` | `t2i` or `i2i` |
 | `--dataset-path` | `prompt/prompt.json` | Path to prompt.json |
@@ -54,7 +54,7 @@ CUDA_VISIBLE_DEVICES=0,1 python benchmarks/glm_image/vllm-omni/inference.py \
 ### Options

 | Flag | Default | Description |
-|------|---------|-------------|
+| ------ | --------- | ------------- |
 | `--model-path` | `zai-org/GLM-Image` | Model path |
 | `--deploy-config` | - | Deploy config YAML |
 | `--mode` | `t2i` | `t2i` or `i2i` |
@@ -100,7 +100,7 @@ python benchmarks/glm_image/benchmark_glm_image.py \
 ### Options

 | Flag | Default | Description |
-|------|---------|-------------|
+| ------ | --------- | ------------- |
 | `--mode` | `t2i` | `t2i` or `i2i` |
 | `--dataset` | `prompt` | `prompt`, `random`, or `custom` |
 | `--dataset-path` | - | JSON file path (required for `custom`) |
@@ -134,7 +134,7 @@ Custom datasets use the same JSON format and can be provided via `--dataset-path
 All three benchmarks report per-stage pipeline timings (in milliseconds):

 | Key | Description |
-|-----|-------------|
+| ----- | ------------- |
 | `preprocess_ms` | Input preprocessing (tokenization, multimodal encoding) |
 | `stage_0_gen_ms` | AR (autoregressive) model generation time |
 | `ar2diffusion_ms` | AR output to diffusion input conversion |
@@ -148,7 +148,7 @@ The stages are ordered by execution: `preprocess → stage_0 (AR) → ar2diffusi
 Tested on 2x GPU with 10 prompts, 1024x1024, 50 denoising steps:

 | Backend | Mode | Latency Mean (s) | Throughput (img/s) |
-|---------|------|-------------------|--------------------|
+| --------- | ------ | ------------------- | -------------------- |
 | HuggingFace | T2I | 72.6 | 0.014 |
 | HuggingFace | I2I | 70.9 | 0.014 |
 | vLLM-Omni Offline | T2I | 35.0 | 0.044 |
@@ -1,10 +1,10 @@
-# Target vLLM version: v0.29.0 (tag @ 98dff2a81d74).
+# Target vLLM version: v0.30.0 (tag @ ced6857afa0e).
 # Use the matching release image and its coherent CUDA/PyTorch/FlashInfer
 # runtime. Do not reinstall a commit wheel or repair dependencies on top of
 # an older base image at a release boundary. For a future unreleased target,
 # restore an explicit wheel pin and validate its complete runtime stack.
 ARG VLLM_BASE_IMAGE=vllm/vllm-openai
-ARG VLLM_BASE_TAG=v0.29.0
+ARG VLLM_BASE_TAG=v0.30.0

 # We use registry-based layer caching for the dependencies,
 # where the tag is determined based on the hash of the contents
@@ -1,4 +1,4 @@
-ARG BASE_IMAGE=vllm/vllm-openai:v0.29.0
+ARG BASE_IMAGE=vllm/vllm-openai:v0.30.0
 FROM ${BASE_IMAGE}

 ARG COMMON_WORKDIR=/app
@@ -1,7 +1,7 @@
 # Keep the ROCm runtime aligned with the vLLM release targeted by
 # docker/Dockerfile.ci. A stale base image can build successfully while failing
 # at runtime when vLLM-Omni imports APIs added by the newer vLLM release.
-ARG BASE_IMAGE=vllm/vllm-openai-rocm:v0.29.0
+ARG BASE_IMAGE=vllm/vllm-openai-rocm:v0.30.0

 # Keep the dependency-heavy portion independent from the source tree. The
 # registry cache is keyed on these inputs, while the final source copy can be
@@ -16,7 +16,7 @@ FROM ${BASE_IMAGE} AS deps
 # intentional custom source pin must update that regression in the same change,
 # and should return to the stable release as soon as one is available.
 ARG USE_NIGHTLY_BUILD=0
-ARG VLLM_VERSION_OR_COMMIT_HASH=v0.29.0
+ARG VLLM_VERSION_OR_COMMIT_HASH=v0.30.0
 ARG ARG_PYTORCH_ROCM_ARCH
 ENV PYTORCH_ROCM_ARCH=${ARG_PYTORCH_ROCM_ARCH:-${PYTORCH_ROCM_ARCH}}

@@ -1,4 +1,4 @@
-# Target vLLM version: v0.29.0. Keep VLLM_VERSION aligned with the target vLLM
+# Target vLLM version: v0.30.0. Keep VLLM_VERSION aligned with the target vLLM
 # release on every rebase.
 #
 # This file layers vLLM-Omni on top of the published upstream XPU image, which
@@ -18,7 +18,7 @@
 #
 # .buildkite/intel/scripts/run-xpu-test.sh automates that pull-then-build fallback.
 ARG VLLM_BASE_IMAGE=vllm/vllm-openai-xpu
-ARG VLLM_VERSION=v0.29.0
+ARG VLLM_VERSION=v0.30.0
 ARG VLLM_BASE=${VLLM_BASE_IMAGE}:${VLLM_VERSION}

 FROM ${VLLM_BASE} AS vllm-omni
@@ -1,7 +1,7 @@
 # Installation

 !!! important
-    vLLM-Omni is released against the matching upstream vLLM major/minor version. The 0.29 development line requires vLLM 0.29.x; the source-install instructions pin vLLM 0.29.0. Published vLLM-Omni 0.28.0 wheels and images still require vLLM 0.28.x.
+    vLLM-Omni is released against the matching upstream vLLM major/minor version. The 0.30 development line requires vLLM 0.30.x; the source-install instructions pin vLLM 0.30.0. Published vLLM-Omni 0.28.0 wheels and images still require vLLM 0.28.x.

 vLLM-Omni supports the following hardware platforms:

@@ -34,7 +34,7 @@ vLLM-Omni is a Python library that supports the following GPU variants. The libr

 ### Pre-built wheels

-The source-install instructions target vLLM 0.29.0. Until vLLM-Omni 0.29.0 artifacts are published, the CUDA/ROCm pre-built examples below remain on the matching 0.28.0 release pair.
+The source-install instructions target vLLM 0.30.0. Until vLLM-Omni 0.30.0 artifacts are published, the CUDA/ROCm pre-built examples below remain on the matching 0.28.0 release pair.

 Note: Pre-built wheels are currently available for vLLM-Omni 0.11.0rc1, 0.12.0rc1, 0.14.0rc1, 0.14.0, 0.16.0, 0.18.0, 0.20.0, 0.21.0, 0.22.0, 0.23.0, 0.24.0, 0.25.0, 0.26.0, and 0.28.0. If you need a newer unreleased revision, please [build from source](https://docs.vllm.ai/projects/vllm-omni/en/latest/getting_started/installation/gpu/#build-wheel-from-source).

@@ -5,7 +5,7 @@
 # --8<-- [end:requirements]
 # --8<-- [start:set-up-using-python]

-vLLM-Omni depends on the matching major/minor release of vLLM. The 0.29 development line uses vLLM 0.29.x. Published 0.28.0 wheels and images use vLLM 0.28.x.
+vLLM-Omni depends on the matching major/minor release of vLLM. The 0.30 development line uses vLLM 0.30.x. Published 0.28.0 wheels and images use vLLM 0.28.x.

 !!! note
    PyTorch installed via `conda` will statically link `NCCL` library, which can cause issues when vLLM tries to use `NCCL`. See <gh-issue:8420> for more details.
@@ -18,7 +18,7 @@ Therefore, it is recommended to install vLLM and vLLM-Omni with a **fresh new**

 #### Installation of vLLM

-These pre-built wheel instructions install the published vLLM-Omni 0.28.0 release. For the 0.29 development line, use the source-install instructions below.
+These pre-built wheel instructions install the published vLLM-Omni 0.28.0 release. For the 0.30 development line, use the source-install instructions below.

 vLLM-Omni is built based on vLLM. Please install it with command below.
 ```bash
@@ -41,13 +41,13 @@ uv pip install 'vllm-omni[demo]==0.28.0'
 # --8<-- [start:build-wheel-from-source]

 #### Installation of vLLM
-If you do not need to modify source code of vLLM, you can directly install the stable 0.29.0 release version of the library
+If you do not need to modify source code of vLLM, you can directly install the stable 0.30.0 release version of the library

 ```bash
-uv pip install vllm==0.29.0 --torch-backend=auto
+uv pip install vllm==0.30.0 --torch-backend=auto
 ```

-The 0.29.0 release of vLLM ships CUDA 13.0-compatible binaries by default. If you need a different CUDA variant or want to reuse an existing PyTorch installation, build vLLM from source instead.
+The 0.30.0 release of vLLM ships CUDA 13.0-compatible binaries by default. If you need a different CUDA variant or want to reuse an existing PyTorch installation, build vLLM from source instead.

 #### Installation of vLLM-Omni
 Since vllm-omni is rapidly evolving, it's recommended to install it from source
@@ -68,12 +68,12 @@ If you want to check, modify or debug with source code of vLLM, install the libr
 ```bash
 git clone https://github.com/vllm-project/vllm.git
 cd vllm
-git checkout v0.29.0
+git checkout v0.30.0
 ```
 Set up environment variables to get pre-built wheels. If there are internet problems, just download the whl file manually. And set `VLLM_PRECOMPILED_WHEEL_LOCATION` as your local absolute path of whl file.
 ```bash
-#For CUDA 13.0 (the default for v0.29.0; the wheel filename has no `+cu130` suffix)
-export VLLM_PRECOMPILED_WHEEL_LOCATION=https://github.com/vllm-project/vllm/releases/download/v0.29.0/vllm-0.29.0-cp38-abi3-manylinux_2_28_x86_64.whl
+#For CUDA 13.0 (the default for v0.30.0; the wheel filename has no `+cu130` suffix)
+export VLLM_PRECOMPILED_WHEEL_LOCATION=https://github.com/vllm-project/vllm/releases/download/v0.30.0/vllm-0.30.0-cp38-abi3-manylinux_2_28_x86_64.whl
 ```
 Install vllm with command below (If you have no existing PyTorch).
 ```bash
@@ -133,7 +133,7 @@ If you want to specify the base vLLM version:
 ```bash
 DOCKER_BUILDKIT=1 docker build \
  -f docker/Dockerfile.cuda \
-  --build-arg BASE_IMAGE=vllm/vllm-openai:v0.29.0 \
+  --build-arg BASE_IMAGE=vllm/vllm-openai:v0.30.0 \
  -t vllm-omni-cuda .
 ```

@@ -10,8 +10,8 @@

 For ROCm, vLLM-Omni currently recommends the setup steps through Docker Images.

-vLLM-Omni depends on the matching major/minor release of vLLM. The 0.29
-development line uses vLLM 0.29.x. Published 0.28.0 wheels and images use vLLM
+vLLM-Omni depends on the matching major/minor release of vLLM. The 0.30
+development line uses vLLM 0.30.x. Published 0.28.0 wheels and images use vLLM
 0.28.x.

 The Dockerfile's `BASE_IMAGE` pin applies only to Docker builds. The
@@ -23,7 +23,7 @@ installing vLLM-Omni, as shown below.

 #### Installation of vLLM

-These pre-built wheel instructions install the published vLLM-Omni 0.28.0 release. For the 0.29 development line, use the source-install instructions below.
+These pre-built wheel instructions install the published vLLM-Omni 0.28.0 release. For the 0.30 development line, use the source-install instructions below.

 vLLM-Omni is built based on vLLM. Please install it with command below.

@@ -50,13 +50,13 @@ uv pip install onnxruntime-rocm

 #### Installation of vLLM

-If you do not need to modify source code of vLLM, you can directly install the stable 0.29.0 release version of the library
+If you do not need to modify source code of vLLM, you can directly install the stable 0.30.0 release version of the library

 ```bash
-uv pip install vllm==0.29.0+rocm723 --extra-index-url https://wheels.vllm.ai/rocm/0.29.0/rocm723
+uv pip install vllm==0.30.0+rocm723 --extra-index-url https://wheels.vllm.ai/rocm/0.30.0/rocm723
 ```

-The pre-built 0.29.0 vLLM wheel targets ROCm 7.2.3. If you need a different ROCm stack or want to reuse an existing PyTorch installation, build vLLM from source instead.
+The pre-built 0.30.0 vLLM wheel targets ROCm 7.2.3. If you need a different ROCm stack or want to reuse an existing PyTorch installation, build vLLM from source instead.

 #### Installation of vLLM-Omni

@@ -76,7 +76,7 @@ If you want to check, modify or debug with source code of vLLM, install the libr
 ```bash
 git clone https://github.com/vllm-project/vllm.git
 cd vllm
-git checkout v0.29.0
+git checkout v0.30.0
 python3 -m pip install -r requirements/rocm.txt
 python3 setup.py develop
 ```
@@ -88,7 +88,7 @@ python3 setup.py develop
 #### Build docker image

 The source build defaults to the published upstream base image
-`vllm/vllm-openai-rocm:v0.29.0`, aligned with the vLLM release used by CI.
+`vllm/vllm-openai-rocm:v0.30.0`, aligned with the vLLM release used by CI.
 Older or custom bases must provide the vLLM APIs checked by the Dockerfile's
 image-build canary. This upstream base is distinct from the prebuilt
 `vllm/vllm-omni-rocm` images discussed below; their published-tag availability
@@ -103,7 +103,7 @@ To select the upstream ROCm base explicitly, pass `BASE_IMAGE`:
 ```bash
 DOCKER_BUILDKIT=1 docker build \
  -f docker/Dockerfile.rocm \
-  --build-arg BASE_IMAGE=vllm/vllm-openai-rocm:v0.29.0 \
+  --build-arg BASE_IMAGE=vllm/vllm-openai-rocm:v0.30.0 \
  -t vllm-omni-rocm .
 ```

@@ -24,7 +24,7 @@ DOCKER_BUILDKIT=1 docker build -f docker/Dockerfile.xpu -t vllm-omni-xpu --shm-s
 ```

 This layers vLLM-Omni on top of the published `vllm/vllm-openai-xpu:<VLLM_VERSION>`
-base image, which Docker pulls automatically. To target a different vLLM release,
+base image (v0.30.0 by default), which Docker pulls automatically. To target a different vLLM release,
 pass `--build-arg VLLM_VERSION=<tag>`. If that tag has not been published yet, build
 the base from upstream's own Dockerfile first and point the build at it:

@@ -19,10 +19,10 @@ uv venv --python 3.12 --seed
 source .venv/bin/activate

 # On CUDA
-uv pip install vllm==0.29.0 --torch-backend=auto
+uv pip install vllm==0.30.0 --torch-backend=auto

 # On ROCm
-uv pip install vllm==0.29.0+rocm723 --extra-index-url https://wheels.vllm.ai/rocm/0.29.0/rocm723
+uv pip install vllm==0.30.0+rocm723 --extra-index-url https://wheels.vllm.ai/rocm/0.30.0/rocm723

 git clone https://github.com/vllm-project/vllm-omni.git
 cd vllm-omni
@@ -34,7 +34,7 @@ For additional installation methods — please see the [installation guide](inst
 !!! note
    It is important to install the same major & minor version of vLLM and vLLM Omni, otherwise things may not work as expected. If the versions are misaligned, you will see a warning when you import vLLM Omni.

-    If you are seeing strange behavior with the `vllm` command not handling the `--omni` flag correctly, you most likely have a version mismatch with vLLM < `0.29.0` and vLLM Omni `0.29.0`, as vLLM Omni no longer hijacks the vLLM entrypoint. Updating vLLM should resolve this issue.
+    If the `vllm` command does not handle `--omni` correctly, verify that the active environment contains the matching vLLM release and vLLM-Omni installation. vLLM owns the CLI entrypoint and delegates `--omni` requests to vLLM-Omni.

 ## Offline Inference

@@ -1,16 +1,10 @@
 # Common dependencies for all platforms
-# transformers: required unified multimodal model classes are unavailable
-# before 5.10.1; 5.10.0 is yanked, and 5.15 has a known construction
-# regression. transformers 5.9 also changed
-# _adjust_missing_and_unexpected_keys from
-# list+list to set|set, breaking models that declare
-# _keys_to_ignore_on_load_unexpected as a list (e.g. OpenMOSS-Team/MOSS-TTS-Nano)
-# with "TypeError: unsupported operand type(s) for |: 'list' and 'set'".
-# That breakage is now handled in code instead of by pinning:
-# MossTTSNanoForGeneration wraps weight loading in
-# transformers_keys_to_ignore_compat (vllm_omni/model_executor/models/utils.py).
-# Floor is 5.13.0 so transformers.integrations.hub_kernels LayerRepository
-# entries always pass version=/revision= required by kernels>=0.16 (#6971).
+# kernels==0.16.1 requires the LayerRepository version=/revision= integration
+# introduced in transformers 5.13. Keep the tested upper bound: 5.15 has
+# a reported model-construction regression; later versions need validation.
+# This range also satisfies vLLM 0.30.0's transformers>=5.10.4 requirement.
+# MOSS-TTS-Nano's list/set weight-loading compatibility is handled by
+# transformers_keys_to_ignore_compat in model_executor/models/utils.py.
 transformers >= 5.13.0, < 5.15
 av>=14.0.0
 omegaconf>=2.3.0
@@ -1,5 +1,3 @@
 -r common.txt
 onnxruntime>=1.23.2
 fa3-fwd==0.0.3
-# nvidia-cublas-cu12 is pinned to 12.9.1.4 via force-reinstall in Dockerfile.ci
-# because torch pins its own cublas version, making it unsatisfiable at resolve time.
@@ -20,6 +20,7 @@ from vllm_omni.benchmarks.data_modules.seed_tts_dataset import (
    SeedTTSSampleRequest,
    SeedTTSTextSampleRequest,
 )
+from vllm_omni.benchmarks.patch import patch
 from vllm_omni.benchmarks.patch.patch import (
    MixRequestFuncOutput,
    _add_combined_video_form_references,
@@ -2011,3 +2012,36 @@ def test_image_metrics_persist_stage_durations_from_metrics() -> None:

 if __name__ == "__main__":
    pytest.main([__file__, "-v", "-s"])
+
+
+def test_get_samples_forwards_upstream_multimodal_backends_kwarg(mocker: MockerFixture) -> None:
+    """The patched ``datasets.get_samples`` must stay call-compatible upstream.
+
+    Upstream ``vllm.benchmarks.datasets.get_samples`` takes a keyword-only
+    ``multimodal_backends`` (``vllm/benchmarks/throughput.py`` passes it) and
+    ``patch.py`` rebinds that symbol module-wide, so a non-omni request must
+    forward the keyword to the original implementation instead of raising
+    ``TypeError`` or silently dropping it.
+    """
+    calls: list[tuple[Namespace, object, dict]] = []
+
+    def fake_get_samples_old(args, tokenizer, **kwargs):
+        calls.append((args, tokenizer, kwargs))
+        return ["delegated"]
+
+    mocker.patch.object(patch, "get_samples_old", fake_get_samples_old)
+
+    args = Namespace(
+        dataset_name="random",
+        backend="vllm-chat",
+        dataset_path=None,
+        hf_name=None,
+    )
+    sentinel = object()
+    mm_backends = ("openai-chat", "openai-audio")
+
+    assert patch.get_samples(args, sentinel, multimodal_backends=mm_backends) == ["delegated"]
+    assert calls == [(args, sentinel, {"multimodal_backends": mm_backends})]
+    # No upstream kwargs: unchanged legacy delegate call.
+    assert patch.get_samples(args, sentinel) == ["delegated"]
+    assert calls[-1] == (args, sentinel, {})
@@ -66,18 +66,25 @@ def _make_generation_scheduler(waiting_request, *, use_v2_model_runner=False):
    scheduler.log_stats = False
    scheduler.scheduler_config = SimpleNamespace(enable_chunked_prefill=True)
    scheduler.num_lookahead_tokens = 0
+    scheduler.num_spec_tokens = 0
+    scheduler.dynamic_sd_lookup = None
+    scheduler.reset_preempted_req_ids = set()
    scheduler.kv_cache_manager = SimpleNamespace(
        new_step_starts=lambda: None,
        allocate_slots=lambda *args, **kwargs: SimpleNamespace(get_block_ids=lambda: ([1],)),
        get_num_common_prefix_blocks=lambda request_id: [0],
-        take_new_block_ids=lambda: None,
+        take_new_block_ids=lambda: [],
+        take_boundary_state_offloads=lambda: {},
    )
    scheduler.kv_cache_config = SimpleNamespace(kv_cache_groups=[object()])
    scheduler.use_v2_model_runner = use_v2_model_runner
    scheduler._retains_state_across_chunks = False
    scheduler.needs_kv_cache_zeroing = False
    scheduler.finished_req_ids = set()
-    scheduler.encoder_cache_manager = SimpleNamespace(get_freed_mm_hashes=lambda: [])
+    scheduler.encoder_cache_manager = SimpleNamespace(
+        get_freed_mm_hashes=lambda: [],
+        get_manager_metadata=lambda: None,
+    )
    scheduler.connector = None
    scheduler.ec_connector = None
    scheduler.prev_step_scheduled_req_ids = set()
@@ -137,7 +137,7 @@ def _make_scheduler_stub(requests: list[_Request]) -> SimpleNamespace:
        running=list(requests),
        waiting=_RequestQueue(),
        skipped_waiting=_RequestQueue(),
-        structured_output_manager=SimpleNamespace(should_advance=lambda _request: False),
+        structured_output_manager=SimpleNamespace(accept_tokens=lambda _request, _token_ids: True),
        transfer_triggered_requests=set(),
        active_kv_transfers=set(),
        pending_stop_after_extraction=set(),
@@ -1,5 +1,5 @@
 # SPDX-License-Identifier: Apache-2.0
-# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project

 """Investigation tests for the num_stale_output_tokens drain (59b9e719).

@@ -89,7 +89,7 @@ def _make_drain_sched(session: Request) -> MagicMock:
    sched = MagicMock()
    sched.requests = {session.request_id: session}
    sched.perf_metrics = None
-    sched.structured_output_manager.should_advance.return_value = False
+    sched.structured_output_manager.accept_tokens.return_value = True
    sched._update_request_with_output.return_value = ([42], False)
    sched._process_kv_transfer_trigger.return_value = False
    sched.chunk_transfer_adapter = MagicMock()
@@ -150,7 +150,7 @@ def _run_resumable_segment_stop(
    sched = MagicMock()
    sched.requests = {session.request_id: session}
    sched.perf_metrics = None
-    sched.structured_output_manager.should_advance.return_value = False
+    sched.structured_output_manager.accept_tokens.return_value = True

    def stop_request(request: Request, _token_ids: list[int]):
        request.status = RequestStatus.FINISHED_STOPPED
@@ -298,7 +298,7 @@ def test_running_decode_step_without_inter_stage_payload_does_not_raise() -> Non
    sched = MagicMock()
    sched.requests = {session.request_id: session}
    sched.perf_metrics = None
-    sched.structured_output_manager.should_advance.return_value = False
+    sched.structured_output_manager.accept_tokens.return_value = True
    sched._update_request_with_output.return_value = ([42], False)
    sched._process_kv_transfer_trigger.return_value = False
    sched.chunk_transfer_adapter = MagicMock()
@@ -381,7 +381,7 @@ def test_stale_async_frame_is_dropped_before_output_processing() -> None:
    sched = MagicMock()
    sched.requests = {session.request_id: session}
    sched.perf_metrics = None
-    sched.structured_output_manager.should_advance.return_value = False
+    sched.structured_output_manager.accept_tokens.return_value = True

    def discard_stale_output(request: Request, token_ids: list[int]) -> tuple[list[int], bool]:
        request.async_tokens_to_discard = 0
@@ -1,3 +1,6 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
+
 """Unit tests for generation streaming session replacement.

 These tests pin the behavior of `_update_request_as_session` against
@@ -20,7 +23,7 @@ import pytest
 import vllm_omni  # noqa: F401 - import for side effects (patch vLLM)
 from vllm.sampling_params import SamplingParams
 from vllm.v1.core.sched.output import SchedulerOutput
-from vllm.v1.engine import EngineCoreEventType
+from vllm.v1.engine import EngineCoreEventType, FinishReason
 from vllm.v1.metrics.stats import PrefillStats
 from vllm.v1.outputs import ModelRunnerOutput
 from vllm.v1.request import Request, RequestStatus, StreamingUpdate
@@ -130,7 +133,7 @@ def test_resumable_generation_stop_marks_segment_boundary() -> None:
    sched.running = [session]
    sched.waiting = MagicMock()
    sched.skipped_waiting = MagicMock()
-    sched.structured_output_manager.should_advance.return_value = False
+    sched.structured_output_manager.accept_tokens.return_value = True
    # Async scheduling can observe the segment boundary in the next schedule()
    # before this model output is applied and defer the same request for finish.
    # update_from_output() must not re-arm the resumable request twice.
@@ -171,6 +174,77 @@ def test_resumable_generation_stop_marks_segment_boundary() -> None:
    assert sched._pending_finish_reqs == []


+@pytest.mark.parametrize("prompt_complete", [False, True])
+def test_rejected_grammar_finishes_and_frees_generation_request(prompt_complete: bool) -> None:
+    session = _make_request(request_id="req-generation-segment")
+    session.status = RequestStatus.RUNNING
+    session.resumable = True
+    session.num_computed_tokens = len(session.prompt_token_ids) if prompt_complete else 1
+    # schedule() has counted this step's token as in flight; update_from_output()
+    # must settle it before handling the segment boundary.
+    session.num_in_flight_tokens = 1
+
+    sched = MagicMock()
+    sched.requests = {session.request_id: session}
+    sched.perf_metrics = None
+    sched.chunk_transfer_adapter = SimpleNamespace(
+        is_done_receiving_chunks=lambda _request_id: True,
+        segment_finished_requests={session.request_id},
+    )
+
+    sched._async_chunk_transport_enabled.return_value = False
+    sched._native_data_plane = False
+    sched._handle_stopped_request.return_value = True
+    sched._free_request.return_value = (None, None)
+    sched.chunk_transfer_adapter.cleanup = MagicMock()
+    sched.running = [session]
+    sched._remove_stopped_requests_from_queues.side_effect = lambda running, preempted: (
+        OmniSchedulerMixin._remove_stopped_requests_from_queues(sched, running, preempted)
+    )
+    sched.waiting = MagicMock()
+    sched.skipped_waiting = MagicMock()
+    sched.structured_output_manager.accept_tokens.return_value = False
+    sched._pending_finish_reqs = []
+    sched.recompute_kv_load_failures = False
+    sched.connector = None
+    sched.kv_cache_manager.take_events.return_value = None
+    sched.kv_cache_manager.estimate_cached_tokens.return_value = 0
+    sched.finished_req_ids_dict = {}
+    sched.make_stats.return_value = None
+
+    scheduler_output = MagicMock(spec=SchedulerOutput)
+    scheduler_output.num_scheduled_tokens = {session.request_id: 1}
+    scheduler_output.scheduled_spec_decode_tokens = {}
+    scheduler_output.num_invalid_spec_tokens = 0
+
+    model_runner_output = MagicMock(spec=ModelRunnerOutput)
+    model_runner_output.sampled_token_ids = [[7]]
+    model_runner_output.logprobs = None
+    model_runner_output.prompt_logprobs_dict = {}
+    model_runner_output.pooler_output = None
+    model_runner_output.num_nans_in_logits = None
+    model_runner_output.kv_connector_output = None
+    model_runner_output.cudagraph_stats = None
+    model_runner_output.req_id_to_index = {session.request_id: 0}
+    model_runner_output.routed_experts = None
+
+    outputs = OmniGenerationScheduler.update_from_output(
+        sched,
+        scheduler_output,
+        model_runner_output,
+    )
+
+    output = outputs[session.client_index].outputs[0]
+    assert output.finish_reason == session.get_finished_reason() == FinishReason.ERROR
+    assert not output.is_segment_finished
+    assert session.status == RequestStatus.FINISHED_ERROR
+    assert session.resumable is False
+    sched._handle_stopped_request.assert_called_once_with(session)
+    sched._free_request.assert_called_once_with(session)
+    sched.chunk_transfer_adapter.cleanup.assert_called_once()
+    assert session not in sched.running
+
+
 def test_async_chunk_resumable_stop_rearms_connector_polling() -> None:
    sched = _AsyncChunkStopSchedulerStub()
    session = _make_request(request_id="req-async-chunk-next-segment")
@@ -123,7 +123,7 @@ def _make_finish_sched(request: Request) -> MagicMock:
    sched.requests = {request.request_id: request}
    sched.perf_metrics = None
    sched.defer_block_free = False
-    sched.structured_output_manager.should_advance.return_value = False
+    sched.structured_output_manager.accept_tokens.return_value = True
    sched._process_kv_transfer_trigger.return_value = False
    sched._maybe_decode_pooling_output.return_value = None
    sched._handle_stopped_request.return_value = True
@@ -56,7 +56,7 @@ def _make_sched(request: Request) -> MagicMock:
    sched.requests = {request.request_id: request}
    sched.perf_metrics = None
    sched.defer_block_free = False
-    sched.structured_output_manager.should_advance.return_value = False
+    sched.structured_output_manager.accept_tokens.return_value = True
    sched._process_kv_transfer_trigger.return_value = False
    sched._maybe_decode_pooling_output.return_value = None
    sched._handle_stopped_request.return_value = True
@@ -11,7 +11,7 @@ Currently only the **GPU** backend is implemented. The backend is selected with
 All functionality is provided by a single script and should be run from the repository root:

 | Subcommand | Description |
-|--------|------|
+| -------- | ------ |
 | `scripts/resource_monitor.sh start [--backend gpu\|cpu\|npu] [gpu_ids] [interval]` | Collect data in the background (currently only `gpu`: `nvidia-smi` writes CSV) |
 | `scripts/resource_monitor.sh finalize [--backend gpu\|cpu\|npu] [run_id]` | Bundle the current run, generate `report.html`, and print `GPU_MONITOR_BUNDLE_DIR=` / `RESOURCE_MONITOR_BUNDLE_DIR=` |
 | `scripts/resource_monitor.sh run [--backend gpu\|cpu\|npu] -- <command>` | Complete everything in one step: `start` -> run your command -> `finalize` |
@@ -23,7 +23,7 @@ All functionality is provided by a single script and should be run from the repo
 ## Environment Variables (monitoring script only)

 | Environment Variable | Description | Default |
-|----------|------|--------|
+| ---------- | ------ | -------- |
 | `RESOURCE_MONITOR_DATA_ROOT` | Root directory for monitoring data | `tests/dfx/stability/gpu_monitor_data` |
 | `RESOURCE_MONITOR_INTERVAL` | Sampling interval (seconds) | 5 |
 | `RESOURCE_MONITOR_LOG_INTERVAL` | Log print interval (seconds) | 15 |
@@ -83,8 +83,8 @@ echo "Report directory: $GPU_MONITOR_BUNDLE_DIR"

 - **Scripts**: `tests/dfx/stability/scripts/resource_monitor.sh` (entry point) and `scripts/generate_report.py` (called by `finalize` to generate HTML).
 - **Data directory**: `tests/dfx/stability/gpu_monitor_data/` by default (can be overridden with `RESOURCE_MONITOR_DATA_ROOT`).  
-  - Each run generates `run_<run_id>/gpu_metrics.csv`.  
-  - After `finalize`, you get `gpu_monitor_bundle_<run_id>/` containing `gpu_metrics.csv`, `report.html`, and `README.txt`.
+    - Each run generates `run_<run_id>/gpu_metrics.csv`.  
+    - After `finalize`, you get `gpu_monitor_bundle_<run_id>/` containing `gpu_metrics.csv`, `report.html`, and `README.txt`.
 - **View the report**: open `report.html` in the bundle directory with a browser to inspect memory usage curves and statistics.

 The script only generates `report.html` and CSV files. If you need to keep the report, archive or download it from the working directory yourself.
@@ -279,19 +279,19 @@ def _install_native_modules(monkeypatch: pytest.MonkeyPatch, events: list[tuple]
        group = SimpleNamespace(get_metadata_builder=lambda _index: SimpleNamespace(reorder_batch_threshold=None))
        return [[group], [group]], "cg-support", [4, 8]

-    # vLLM 0.29 dropped attn_groups and cache_dtype from init_kv_cache: the
-    # layout now rides on the resolved CacheConfig.kv_cache_layout instead.
+    # f2aad6aa70 (#56888): first arg is forward_context; return a cache dict.
    def init_cache(
-        runner_kv_caches,
        forward_context,
        kv_cache_config,
        device,
        kernel_block_sizes,
        vllm_config,
+        kv_cache_allocation_context=None,
+        *,
+        block_tables=None,
    ):
        events.append(("cache", kv_cache_config, kernel_block_sizes))
-        runner_kv_caches.extend(["cache-0", "cache-1"])
-        return {"layer-0": "tensor-0", "layer-1": "tensor-1"}
+        return {"layer-0": "cache-0", "layer-1": "cache-1"}

    monkeypatch.setattr(model_runner_backend_module, "init_attn_backend", init_backend)
    monkeypatch.setattr(model_runner_backend_module, "init_kv_cache", init_cache)
@@ -431,8 +431,7 @@ def test_initialize_failure_does_not_publish_partial_cache_state(
        ],
    )

-    def fail_init_cache(runner_kv_caches, *_args, **_kwargs):
-        runner_kv_caches.append("partial-cache")
+    def fail_init_cache(forward_context, *_args, **_kwargs):
        backend._kv_cache_layer_adapters["layer-0"].kv_cache = "partial-binding"
        raise RuntimeError("cache binding failed")

@@ -24,7 +24,16 @@ def producer(monkeypatch):
    worker.shutdown = Mock()  # No hardware constructor, sockets or listener thread.
    worker.is_kv_consumer = False
    worker.tp_rank, worker.tp_size = 0, 1
-    worker.transfer_topo = SimpleNamespace(handshake_target_ranks=lambda size: list(range(size)))
+    worker.transfer_topo = SimpleNamespace(
+        handshake_target_ranks=lambda size: list(range(size)),
+        total_num_kv_heads=1,
+    )
+    # vLLM 0.30 send_kv_to_decode also reads the PP size (ce08bb5b34,
+    # #56033), the MLA/Mamba flags and the head-resharding validator.
+    worker.pp_size = 1
+    worker.use_mla = False
+    worker.kv_cache_config = SimpleNamespace(has_mamba_layers=False)
+    worker._validate_head_resharding_layout = Mock(return_value=None)
    worker.reqs_need_send, worker.finished_sending_reqs = {}, set()
    for name in (
        "kv_caches_base_addr",
@@ -42,7 +51,9 @@ def producer(monkeypatch):
    worker._send_blocks = Mock(return_value=0)
    worker.xfer_stats = Mock()
    worker._build_transfer_params = AsyncMock(return_value=([1], [2], [16], [], None))
-    monkeypatch.setattr(mc, "_align_transfer_regions", lambda *_: ([], [], None))
+    # vLLM ce08bb5b34 (#56033) calls _align_transfer_regions with the
+    # allow_partial_layers keyword; the stub must accept it.
+    monkeypatch.setattr(mc, "_align_transfer_regions", lambda *_, **__: ([], [], None))
    monkeypatch.setattr(mc, "_validate_asymmetric_region_lengths", lambda **_: None)
    monkeypatch.setattr(mc.envs, "VLLM_MOONCAKE_ABORT_REQUEST_TIMEOUT", 0.05)
    connector = object.__new__(mc.MooncakeConnector)
@@ -1,6 +1,7 @@
 # SPDX-License-Identifier: Apache-2.0
 # SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project

+from collections.abc import Iterator
 from types import SimpleNamespace
 from unittest.mock import Mock

@@ -526,23 +527,30 @@ def test_drain_after_terminal_request_state_was_popped(monkeypatch):


@pytest.mark.parametrize("prefetch", [False, True])
-def test_real_active_connector_starts_required_load_before_polling(prefetch):
-    from vllm.v1.worker.gpu.kv_connector import ActiveKVConnector
+def test_real_active_connector_starts_required_load_before_polling(prefetch, monkeypatch):
+    from vllm.distributed.kv_transfer.kv_connector.v1.base import KVConnectorBase_V1
+    from vllm.v1.worker.gpu import kv_connector as native_kv_connector

    from vllm_omni.diffusion.diffusion_kv.kv_connector import wait_for_kv_load

-    active = object.__new__(ActiveKVConnector)
-    active._disabled = False
-    active._pending_load_start = False
-    active.kv_connector = Mock()
-    active.kv_connector.get_block_ids_with_load_errors.return_value = set()
-    active._start_load_kv = Mock()
+    connector = Mock(spec=KVConnectorBase_V1)
+    connector.get_block_ids_with_load_errors.return_value = set()
+    # Exercise upstream's compatibility adapter instead of returning an
+    # unconstrained Mock from the v0.30 get_transfer_results API.
+    connector.get_transfer_results.side_effect = lambda ids: KVConnectorBase_V1.get_transfer_results(connector, ids)
+    monkeypatch.setattr(native_kv_connector, "get_kv_transfer_group", lambda: connector)
+    context = object()
+    monkeypatch.setattr(native_kv_connector, "is_forward_context_available", lambda: True)
+    monkeypatch.setattr(native_kv_connector, "get_forward_context", lambda: context)
+    active = native_kv_connector.ActiveKVConnector(Mock(), {})
+
+    completions: Iterator[tuple[set[str], set[str]]] = iter([(set(), {"A0"}), (set(), set())])

    def finished(_):
-        active._start_load_kv.assert_called_once()
-        return set(), {"A0"}
+        connector.start_load_kv.assert_called_once_with(context)
+        return next(completions)

-    active.kv_connector.get_finished.side_effect = finished
+    connector.get_finished.side_effect = finished
    output = _output(submit=["A0"], required=["A0"], metadata=object())
    if prefetch:
        result = KVReceiveProgress().prepare(active, output, 1)
@@ -550,3 +558,6 @@ def test_real_active_connector_starts_required_load_before_polling(prefetch):
        output.kv_required_request_ids = None
        result = wait_for_kv_load(active, output, 1)
    assert result.finished_recving == {"A0"}
+    assert result.finished_sending == set()
+    connector.get_transfer_results.assert_called_once_with(set())
+    connector.start_load_kv.assert_called_once_with(context)
@@ -106,11 +106,9 @@ def _native_adapter(num_rows: int = 1):
            vllm_config,
            device,
        )
-        runner_kv_caches: list[torch.Tensor | list[torch.Tensor]] = []
-        # vLLM 0.29 dropped attn_groups and cache_dtype here: the layout now
-        # comes from the resolved CacheConfig.kv_cache_layout inside the call.
+        # f2aad6aa70 (#56888): init_kv_cache no longer mutates a runner list;
+        # first arg is forward_context and the allocated caches are returned.
        init_kv_cache(
-            runner_kv_caches,
            vllm_config.compilation_config.static_forward_context,
            kv_cache_config,
            device,
@@ -178,7 +178,13 @@ class TestGlmImageAdaLayerNormZeroQuantization:

    def test_accepts_quant_config_parameter(self, mocker: MockerFixture):
        """Verify the class accepts quant_config parameter."""
+        # vLLM 0.30 resolves online quantization in resolve_quant_method():
+        # a non-None online_quantization_config whose resolve_quant_method_cls()
+        # returns a target raises on a checkpoint-quantized layer. A bare
+        # MagicMock satisfies both, so declare no online quantization -- these
+        # tests only assert the layer accepts and stores a quant_config.
        mock_quant_config = mocker.MagicMock()
+        mock_quant_config.online_quantization_config = None
        layer = GlmImageAdaLayerNormZero(
            embedding_dim=512,
            dim=2560,
@@ -220,7 +226,13 @@ class TestGlmImageAdaLayerNormContinuousQuantization:

    def test_accepts_quant_config_parameter(self, mocker: MockerFixture):
        """Verify the class accepts quant_config parameter."""
+        # vLLM 0.30 resolves online quantization in resolve_quant_method():
+        # a non-None online_quantization_config whose resolve_quant_method_cls()
+        # returns a target raises on a checkpoint-quantized layer. A bare
+        # MagicMock satisfies both, so declare no online quantization -- these
+        # tests only assert the layer accepts and stores a quant_config.
        mock_quant_config = mocker.MagicMock()
+        mock_quant_config.online_quantization_config = None
        layer = GlmImageAdaLayerNormContinuous(
            embedding_dim=2560,
            conditioning_embedding_dim=512,
@@ -260,7 +272,13 @@ class TestGlmImageAttentionQuantization:

    def test_accepts_quant_config_parameter(self, mocker: MockerFixture):
        """Verify GlmImageAttention accepts quant_config parameter."""
+        # vLLM 0.30 resolves online quantization in resolve_quant_method():
+        # a non-None online_quantization_config whose resolve_quant_method_cls()
+        # returns a target raises on a checkpoint-quantized layer. A bare
+        # MagicMock satisfies both, so declare no online quantization -- these
+        # tests only assert the layer accepts and stores a quant_config.
        mock_quant_config = mocker.MagicMock()
+        mock_quant_config.online_quantization_config = None
        attn = GlmImageAttention(
            dim=2560,
            num_heads=64,
@@ -286,7 +304,13 @@ class TestColumnParallelModulesQuantization:

    def test_column_parallel_gelu_accepts_quant_config(self, mocker: MockerFixture):
        """Verify ColumnParallelGELU accepts quant_config."""
+        # vLLM 0.30 resolves online quantization in resolve_quant_method():
+        # a non-None online_quantization_config whose resolve_quant_method_cls()
+        # returns a target raises on a checkpoint-quantized layer. A bare
+        # MagicMock satisfies both, so declare no online quantization -- these
+        # tests only assert the layer accepts and stores a quant_config.
        mock_quant_config = mocker.MagicMock()
+        mock_quant_config.online_quantization_config = None
        layer = ColumnParallelGELU(
            dim_in=2560,
            dim_out=10240,
@@ -297,7 +321,13 @@ class TestColumnParallelModulesQuantization:

    def test_column_parallel_silu_accepts_quant_config(self, mocker: MockerFixture):
        """Verify ColumnParallelSiLU accepts quant_config."""
+        # vLLM 0.30 resolves online quantization in resolve_quant_method():
+        # a non-None online_quantization_config whose resolve_quant_method_cls()
+        # returns a target raises on a checkpoint-quantized layer. A bare
+        # MagicMock satisfies both, so declare no online quantization -- these
+        # tests only assert the layer accepts and stores a quant_config.
        mock_quant_config = mocker.MagicMock()
+        mock_quant_config.online_quantization_config = None
        layer = ColumnParallelSiLU(
            dim_in=2560,
            dim_out=10240,
@@ -312,7 +342,13 @@ class TestGlmImageFeedForwardQuantization:

    def test_accepts_quant_config_parameter(self, mocker: MockerFixture):
        """Verify GlmImageFeedForward accepts quant_config parameter."""
+        # vLLM 0.30 resolves online quantization in resolve_quant_method():
+        # a non-None online_quantization_config whose resolve_quant_method_cls()
+        # returns a target raises on a checkpoint-quantized layer. A bare
+        # MagicMock satisfies both, so declare no online quantization -- these
+        # tests only assert the layer accepts and stores a quant_config.
        mock_quant_config = mocker.MagicMock()
+        mock_quant_config.online_quantization_config = None
        ff = GlmImageFeedForward(
            dim=2560,
            dim_out=2560,
@@ -358,7 +394,13 @@ class TestGlmImageTransformerBlockQuantization:

    def test_accepts_quant_config_parameter(self, mocker: MockerFixture):
        """Verify GlmImageTransformerBlock accepts quant_config parameter."""
+        # vLLM 0.30 resolves online quantization in resolve_quant_method():
+        # a non-None online_quantization_config whose resolve_quant_method_cls()
+        # returns a target raises on a checkpoint-quantized layer. A bare
+        # MagicMock satisfies both, so declare no online quantization -- these
+        # tests only assert the layer accepts and stores a quant_config.
        mock_quant_config = mocker.MagicMock()
+        mock_quant_config.online_quantization_config = None
        parallel_config = DiffusionParallelConfig(
            tensor_parallel_size=1,
            sequence_parallel_size=1,
@@ -534,7 +576,13 @@ class TestGlmImageTransformer2DModelQuantization:
        """Verify the model accepts quant_config parameter."""
        from vllm_omni.diffusion.data import OmniDiffusionConfig

+        # vLLM 0.30 resolves online quantization in resolve_quant_method():
+        # a non-None online_quantization_config whose resolve_quant_method_cls()
+        # returns a target raises on a checkpoint-quantized layer. A bare
+        # MagicMock satisfies both, so declare no online quantization -- these
+        # tests only assert the layer accepts and stores a quant_config.
        mock_quant_config = mocker.MagicMock()
+        mock_quant_config.online_quantization_config = None
        parallel_config = DiffusionParallelConfig(
            tensor_parallel_size=1,
            sequence_parallel_size=1,
@@ -607,7 +655,13 @@ class TestGlmImageTransformer2DModelQuantization:
        """Verify norm_out (output layer) does NOT use quantization to preserve precision."""
        from vllm_omni.diffusion.data import OmniDiffusionConfig

+        # vLLM 0.30 resolves online quantization in resolve_quant_method():
+        # a non-None online_quantization_config whose resolve_quant_method_cls()
+        # returns a target raises on a checkpoint-quantized layer. A bare
+        # MagicMock satisfies both, so declare no online quantization -- these
+        # tests only assert the layer accepts and stores a quant_config.
        mock_quant_config = mocker.MagicMock()
+        mock_quant_config.online_quantization_config = None
        parallel_config = DiffusionParallelConfig(
            tensor_parallel_size=1,
            sequence_parallel_size=1,
@@ -1380,6 +1380,8 @@ def test_text_encoder_linear_delegates_quantization_to_vllm_factory():
    method = UnquantizedLinearMethod()
    quant_config = Mock()
    quant_config.get_quant_method.return_value = method
+    # vLLM 0.30: no online quantization for this checkpoint-quantized layer
+    quant_config.online_quantization_config = None
    prefix = "text_encoder.text_model.layers.0.mlp.gate_up_proj"

    with (
@@ -1396,7 +1398,9 @@ def test_text_encoder_linear_delegates_quantization_to_vllm_factory():
        )

    assert linear.quant_method is method
-    quant_config.get_quant_method.assert_called_once_with(linear, prefix=prefix)
+    # vLLM 0.30 resolves the method through resolve_quant_method(), which calls
+    # get_quant_method(layer, prefix) POSITIONALLY; 0.29 passed prefix= by keyword.
+    quant_config.get_quant_method.assert_called_once_with(linear, prefix)


 def test_distributed_layerwise_offload_stages_vae_component():
@@ -2391,7 +2391,7 @@ def test_generation_scheduler_calls_cleanup_on_finished(monkeypatch, mocker: Moc
    scheduler.log_stats = False
    scheduler.recompute_kv_load_failures = False
    scheduler.structured_output_manager = mocker.MagicMock()
-    scheduler.structured_output_manager.should_advance.return_value = False
+    scheduler.structured_output_manager.accept_tokens.return_value = True
    scheduler.finished_req_ids_dict = {}
    scheduler.kv_cache_manager.take_events.return_value = None
    scheduler.kv_cache_manager.estimate_cached_tokens.return_value = 0
@@ -2471,7 +2471,7 @@ def test_ar_scheduler_defers_cleanup_and_queues_save_on_finished(mocker: MockerF
    scheduler.log_stats = False
    scheduler.recompute_kv_load_failures = False
    scheduler.structured_output_manager = mocker.MagicMock()
-    scheduler.structured_output_manager.should_advance.return_value = False
+    scheduler.structured_output_manager.accept_tokens.return_value = True
    scheduler.finished_req_ids_dict = {}
    scheduler.kv_cache_manager = mocker.MagicMock()
    scheduler.kv_cache_manager.take_events.return_value = None
@@ -2979,7 +2979,7 @@ def _build_deferred_finish_scheduler(mocker, *, running, pending_finish_reqs):
    scheduler.log_stats = False
    scheduler.recompute_kv_load_failures = False
    scheduler.structured_output_manager = mocker.MagicMock()
-    scheduler.structured_output_manager.should_advance.return_value = False
+    scheduler.structured_output_manager.accept_tokens.return_value = True
    scheduler.finished_req_ids_dict = {}
    scheduler.kv_cache_manager.take_events.return_value = None
    scheduler.kv_cache_manager.estimate_cached_tokens.return_value = 0
@@ -1,6 +1,7 @@
 # SPDX-License-Identifier: Apache-2.0
-# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project

+import pytest
 import torch

 from vllm_omni.model_executor.models.indextts2 import codec as codec_module
@@ -84,10 +85,33 @@ def test_model_asset_resolver_supports_native_checkpoints_layout(tmp_path):
    assert preprocess_utils.resolve_model_file(str(tmp_path), "codec.pth") == str(checkpoint)


-def test_campplus_falls_back_to_hub_for_local_model_dir(tmp_path, monkeypatch):
-    import huggingface_hub
+def test_campplus_local_dir_does_not_silently_download(tmp_path, monkeypatch):
+    """A local bundle missing CAMPPlus must fail closed, not hit the Hub.
+
+    origin/main ``load_campplus`` treats ``os.path.isdir(model_path)`` as a
+    complete local snapshot and raises FileNotFoundError instead of
+    downloading ``funasr/campplus``. The old test name
+    ``test_campplus_falls_back_to_hub_for_local_model_dir`` encoded the
+    pre-guard behavior.
+    """
+    from vllm_omni.model_executor.models.indextts2.utils.campplus import dtdnn
+
+    class FakeCAMPPlus(torch.nn.Module):
+        def load_state_dict(self, state, strict=True):
+            self.loaded_state = (state, strict)
+
+    monkeypatch.setattr(dtdnn, "CAMPPlus", lambda **_kwargs: FakeCAMPPlus())
+    preprocess_utils._campplus_cache.clear()
+
+    with pytest.raises(FileNotFoundError, match="CAMPPlus checkpoint is missing from local bundle"):
+        preprocess_utils.load_campplus(str(tmp_path), torch.device("cpu"))
+
+
+def test_campplus_falls_back_to_hub_for_repo_id(tmp_path, monkeypatch):
+    from types import SimpleNamespace

    from vllm_omni.model_executor.models.indextts2.utils.campplus import dtdnn
+    from vllm_omni.transformers_utils import repo_utils

    checkpoint = tmp_path / "downloaded-campplus.bin"
    checkpoint.touch()
@@ -99,14 +123,16 @@ def test_campplus_falls_back_to_hub_for_local_model_dir(tmp_path, monkeypatch):

    monkeypatch.setattr(dtdnn, "CAMPPlus", lambda **_kwargs: FakeCAMPPlus())
    monkeypatch.setattr(
-        huggingface_hub,
-        "hf_hub_download",
-        lambda repo_id, filename: downloads.append((repo_id, filename)) or str(checkpoint),
+        repo_utils,
+        "hf_api",
+        lambda: SimpleNamespace(
+            hf_hub_download=lambda repo_id, filename: downloads.append((repo_id, filename)) or str(checkpoint)
+        ),
    )
    monkeypatch.setattr(preprocess_utils.torch, "load", lambda *_args, **_kwargs: {"weight": torch.ones(1)})
    preprocess_utils._campplus_cache.clear()

-    model = preprocess_utils.load_campplus(str(tmp_path), torch.device("cpu"))
+    model = preprocess_utils.load_campplus("IndexTeam/IndexTTS-2", torch.device("cpu"))

    assert isinstance(model, FakeCAMPPlus)
    assert downloads == [("funasr/campplus", "campplus_cn_common.bin")]
@@ -7,27 +7,37 @@ ModelOpt 0.44's FP32->FP8 E4M3 cast of per-block weight scales can emit
 literal NaN bytes (E4M3 encoding 0x7F / 0xFF) when the pre-cast scale
 rounds above the FP8 max of 448 after the global-scale division. A
 single NaN byte in any per-block weight_scale collapses the FlashInfer
-FP4 GEMM output to NaN and the served model emits `!!!!`. The patch
-overrides ``ModelOptNvFp4LinearMethod.process_weights_after_loading``
+FP4 GEMM output to NaN and the served model emits `!!!!`. On the legacy
+pin the patch overrides ``ModelOptNvFp4LinearMethod.process_weights_after_loading``
 to scan + clamp those bytes at load time.

-These tests cover the behavior of the override on a synthetic layer:
+vLLM #49381 replaced that class with the generic ``ModelOptLinearMethod``
+(and the ``LinearMethodCls`` attributes) and #52501 made a NaN
+weight_scale a hard error there ("... was never loaded (still NaN)"), so on
+that pin ``vllm_omni.patch`` self-extinguishes: it installs nothing and
+upstream's load-time RuntimeError stays observable. These tests cover both
+pins:

- the NaN bytes get clamped to the FP8 E4M3 max byte (0x7E)
- a finite weight_scale is left untouched (no spurious writes)
- the install asserts itself so silent fallback is impossible
- the env-var escape hatch path is reachable
+- the NaN bytes get clamped to the FP8 E4M3 max byte (0x7E) [helper]
+- a finite weight_scale is left untouched (no spurious writes) [helper]
+- the install asserts itself so silent fallback is impossible [either pin]
+- the env-var escape hatch path is reachable [either pin]
+- on the redesigned pin nothing is installed on ``ModelOptLinearMethod``
+  and upstream's unloaded-scale RuntimeError is still raised [redesigned]

 No GPU is required — the tests construct an in-memory FP8 ``weight_scale``
-tensor with planted NaN bytes and call the patched PWAL directly.
+tensor with planted NaN bytes and call either the patched helper or the
+upstream PWAL directly.
 """

+import importlib
+
 import pytest
 import torch

 # Importing vllm_omni runs vllm_omni.patch, which installs the override
 # we are testing. The import must come before any direct reference to
-# ``ModelOptNvFp4LinearMethod`` below.
+# the clamp target below.
 import vllm_omni  # noqa: F401

 pytestmark = [pytest.mark.core_model, pytest.mark.cpu]
@@ -36,6 +46,48 @@ pytestmark = [pytest.mark.core_model, pytest.mark.cpu]
 FP8_NAN_BYTES = (0x7F, 0xFF)
 FP8_MAX_BYTE = 0x7E  # encoding of finfo(float8_e4m3fn).max

+_MODELOPT_MODULE = "vllm.model_executor.layers.quantization.modelopt"
+_LEGACY_NVFP4_LINEAR_METHOD = "ModelOptNvFp4LinearMethod"
+_GENERIC_LINEAR_METHOD = "ModelOptLinearMethod"
+
+
+def _resolve_target(name: str):
+    """Return ``vllm.modelopt.<name>`` or None when upstream never had it.
+
+    Imports the module eagerly (a missing module is a hard upstream breakage
+    and must fail loudly); only the *attribute* lookup is tolerant, so
+    "upstream redesigned the class away" is distinguishable from "the module
+    moved" without swallowing real errors.
+    """
+    modelopt = importlib.import_module(_MODELOPT_MODULE)
+    return getattr(modelopt, name, None)
+
+
+def _resolve_legacy_target():
+    return _resolve_target(_LEGACY_NVFP4_LINEAR_METHOD)
+
+
+def _resolve_generic_target():
+    return _resolve_target(_GENERIC_LINEAR_METHOD)
+
+
+def _pwfn_source(cls) -> str:
+    """Source file of ``cls.process_weights_after_loading``."""
+    fn = cls.__dict__.get("process_weights_after_loading") or cls.process_weights_after_loading
+    return getattr(fn, "__code__", None) and fn.__code__.co_filename or ""
+
+
+def _last_stdout_line(stdout: str) -> str:
+    """Last non-empty stdout line of a subprocess.
+
+    Importing ``vllm_omni`` runs ``vllm_omni.patch``, whose INFO logs land on
+    stdout, so a sentinel printed by the child is the *last* line, never the
+    first (the previous ``stdout.strip().startswith(...)`` form silently broke
+    as soon as the patch module logged anything).
+    """
+    lines = [line.strip() for line in stdout.strip().splitlines() if line.strip()]
+    return lines[-1] if lines else ""
+

 def _fp8_tensor_with_nan_bytes(num_blocks: int, nan_indices: list[int]) -> torch.Tensor:
    """Return an FP8 E4M3 tensor where every position is the FP8 max byte
@@ -53,9 +105,8 @@ def _fp8_tensor_with_nan_bytes(num_blocks: int, nan_indices: list[int]) -> torch
 def clamp_fn():
    """Return the top-level NaN-clamp helper exposed by vllm_omni.patch.

-    Skipped when vllm doesn't expose ``ModelOptNvFp4LinearMethod`` (older
-    pin / unsupported build) and vllm_omni.patch consequently didn't
-    install the helper either.
+    Skipped when vllm_omni.patch did not expose the helper (e.g. an
+    incompatible vllm_omni install).
    """
    try:
        from vllm_omni.patch import _clamp_nvfp4_weight_scale_nans
@@ -130,36 +181,109 @@ def test_clamps_non_contiguous_weight_scale(clamp_fn):
 def test_install_is_self_asserting():
    # If the install assert is removed, a future regression could let the
    # override silently drop. There are three legitimate end states:
-    #   (a) upstream vLLM already has its own NaN clamp and vllm_omni.patch
-    #       intentionally skipped installing the override (self-extinguish),
-    #   (b) upstream is unpatched and our wrapper is installed,
-    #   (c) the env-var escape hatch is set OR the modelopt import failed,
+    #   (a) upstream vLLM already handles NaN weight_scale itself — either its
+    #       own in-place clamp, or (on the redesigned generic
+    #       ModelOptLinearMethod) a hard rejection — and vllm_omni.patch
+    #       intentionally skipped installing the override,
+    #   (b) upstream is unpatched (legacy per-format class) and our wrapper is
+    #       installed,
+    #   (c) the env-var escape hatch is set OR every modelopt target is missing,
    #       so nothing is installed by design — nothing to assert.
-    try:
-        from vllm.model_executor.layers.quantization.modelopt import (
-            ModelOptNvFp4LinearMethod,
-        )
-    except ImportError:
-        pytest.skip("vllm.modelopt.ModelOptNvFp4LinearMethod not available")
+    legacy = _resolve_legacy_target()
+    if legacy is None and _resolve_generic_target() is None:
+        pytest.skip("no ModelOpt linear method exposed by this vLLM build")

    from vllm_omni.patch import _already_patched_upstream, _clamp_installed

-    current = ModelOptNvFp4LinearMethod.process_weights_after_loading
-    name = getattr(current, "__qualname__", "") or getattr(current, "__name__", "")
    if _already_patched_upstream:
        # Self-extinguish path: presence of our marker here would mean the
-        # heuristic in vllm_omni.patch failed to detect upstream's clamp.
-        assert "_patched_nvfp4_pwal" not in name, (
-            f"vllm_omni override installed despite upstream PWAL already containing a "
-            f"NaN clamp ({name!r}); self-extinguish heuristic in vllm_omni.patch needs review"
-        )
+        # detection failed and we wrapped a class we must not touch.
+        assert not _clamp_installed, "clamp installed although upstream handles NaN weight_scale"
+        if legacy is not None:
+            name = getattr(legacy.process_weights_after_loading, "__qualname__", "") or getattr(
+                legacy.process_weights_after_loading, "__name__", ""
+            )
+            assert "_patched_nvfp4_pwal" not in name, (
+                f"vllm_omni override installed although upstream handles NaN weight_scale "
+                f"itself ({name!r}); the self-extinguish detection in vllm_omni.patch needs review"
+            )
    elif _clamp_installed:
+        assert legacy is not None, "clamp reported installed but the legacy ModelOptNvFp4LinearMethod does not exist"
+        current = legacy.process_weights_after_loading
+        name = getattr(current, "__qualname__", "") or getattr(current, "__name__", "")
        assert "patched_nvfp4_pwal" in name or "_patched_nvfp4_pwal" in name, (
            f"NVFP4 NaN-clamp PWAL override does not appear installed (saw {name!r}); "
            "another plugin may have re-patched ModelOptNvFp4LinearMethod"
        )
    else:
-        pytest.skip("NaN-clamp not installed by design (env-var escape hatch or import failure)")
+        pytest.skip("NaN-clamp not installed by design (env-var escape hatch or missing target)")
+
+
+def test_no_override_on_redesigned_upstream():
+    """On the redesigned ModelOpt linear path (#49381) nothing may be installed.
+
+    #52501 made any NaN weight_scale a hard load-time error there, so a clamp
+    would mask it and silently serve a partially loaded model. Assert both that
+    we installed nothing and that we did not touch the class at all.
+    """
+    generic = _resolve_generic_target()
+    if generic is None:
+        pytest.skip("ModelOptLinearMethod not available (legacy pin)")
+    if _resolve_legacy_target() is not None:
+        pytest.skip("legacy pin: the per-format override path applies instead")
+
+    from vllm_omni.patch import _already_patched_upstream, _clamp_installed
+
+    assert not _clamp_installed, (
+        "the NaN clamp must not be installed on the redesigned generic ModelOptLinearMethod: "
+        "it would mask upstream's unloaded-weight_scale RuntimeError"
+    )
+    assert _already_patched_upstream, (
+        "vllm_omni.patch must record that upstream itself rejects NaN weight_scale on this pin"
+    )
+    assert not hasattr(generic.process_weights_after_loading, "_vllm_omni_wrapped_pwal"), (
+        "our wrapper is present on ModelOptLinearMethod.process_weights_after_loading"
+    )
+    # Stronger identity check than the marker: a monkeypatched wrapper would be
+    # defined in vllm_omni/patch.py, so the class attribute must still come from
+    # vLLM itself.
+    assert "vllm_omni" not in _pwfn_source(generic), (
+        f"ModelOptLinearMethod.process_weights_after_loading was replaced by {_pwfn_source(generic)}"
+    )
+
+
+def _nvfp4_w4a4_method():
+    """Build the generic NVFP4 W4A4 linear method exactly as upstream does."""
+    from types import SimpleNamespace
+
+    from vllm.model_executor.layers.quantization.modelopt import (
+        ModelOptLinearMethod,
+        resolve,
+    )
+
+    spec, ctx, format_scheme = resolve("NVFP4", SimpleNamespace(group_size=16), "test")
+    return ModelOptLinearMethod(spec, ctx, format_scheme)
+
+
+def test_upstream_nan_sentinel_check_still_reachable():
+    """Upstream's unloaded-weight_scale guard must survive our import.
+
+    Drives the *public* entry point (``ModelOptLinearMethod``), not just the
+    scheme, so a future re-install of the override on that class would be
+    caught: a wrapper that clamped first would swallow the RuntimeError.
+    """
+    if _resolve_generic_target() is None:
+        pytest.skip("ModelOptLinearMethod not available (legacy pin)")
+
+    method = _nvfp4_w4a4_method()
+    layer = torch.nn.Module()
+    layer.weight_scale = torch.nn.Parameter(
+        _fp8_tensor_with_nan_bytes(num_blocks=8, nan_indices=list(range(8))),
+        requires_grad=False,
+    )
+
+    with pytest.raises(RuntimeError, match="never loaded"):
+        method.process_weights_after_loading(layer)


 def test_reload_does_not_nest_wrapper():
@@ -180,15 +304,13 @@ def test_reload_does_not_nest_wrapper():
        """
        import importlib
        import vllm_omni.patch as p
-        try:
-            from vllm.model_executor.layers.quantization.modelopt import (
-                ModelOptNvFp4LinearMethod,
-            )
-        except ImportError:
-            print("SKIP: modelopt not available")
+        modelopt = importlib.import_module("vllm.model_executor.layers.quantization.modelopt")
+        ModelOptNvFp4LinearMethod = getattr(modelopt, "ModelOptNvFp4LinearMethod", None)
+        if ModelOptNvFp4LinearMethod is None:
+            print("SKIP: legacy modelopt target not available")
            raise SystemExit(0)
        if not getattr(p, "_clamp_installed", False):
-            print("SKIP: clamp not installed (upstream patched or escape hatch)")
+            print("SKIP: clamp not installed (upstream handles NaN, or escape hatch)")
            raise SystemExit(0)
        importlib.reload(p)
        fn = ModelOptNvFp4LinearMethod.process_weights_after_loading
@@ -199,8 +321,12 @@ def test_reload_does_not_nest_wrapper():
        """
    )
    result = subprocess.run([sys.executable, "-c", code], capture_output=True, text=True, timeout=600)
-    if result.stdout.strip().startswith("SKIP:"):
-        pytest.skip(result.stdout.strip())
+    # The subprocess also emits vLLM's own INFO log lines on stdout (importing
+    # vllm_omni runs vllm_omni.patch), so the sentinel is the LAST stdout line,
+    # not the first.
+    last_line = _last_stdout_line(result.stdout)
+    if last_line.startswith("SKIP:"):
+        pytest.skip(last_line)
    assert "FLAT" in result.stdout, (
        f"reload nested the NVFP4 clamp wrapper — clamp would run twice per load.\n"
        f"stdout: {result.stdout}\nstderr: {result.stderr}"
@@ -244,18 +370,21 @@ def test_env_var_escape_hatch_disables_install(env_value, acceptable):

    code = textwrap.dedent(
        """
+        import importlib
        import vllm_omni  # noqa: F401
        import vllm_omni.patch as p
-        try:
-            from vllm.model_executor.layers.quantization.modelopt import (
-                ModelOptNvFp4LinearMethod,  # noqa: F401
-            )
-        except ImportError:
-            print("SKIP: modelopt not available")
+        modelopt = importlib.import_module("vllm.model_executor.layers.quantization.modelopt")
+        has_target = any(
+            getattr(modelopt, name, None) is not None
+            for name in ("ModelOptNvFp4LinearMethod", "ModelOptLinearMethod")
+        )
+        if not has_target:
+            print("SKIP: no modelopt linear method target available")
            raise SystemExit(0)
        if getattr(p, "_already_patched_upstream", False):
-            # Upstream vLLM already clamps; vllm_omni.patch deliberately
-            # skipped installing our wrapper (self-extinguish).
+            # Upstream handles NaN weight_scale itself (its own clamp, or the
+            # redesigned generic PWAL's hard rejection); vllm_omni.patch
+            # deliberately skipped installing our wrapper (self-extinguish).
            print("UPSTREAM_PATCHED")
        elif getattr(p, "_clamp_installed", False):
            print("INSTALLED")
@@ -271,11 +400,11 @@ def test_env_var_escape_hatch_disables_install(env_value, acceptable):
    # contention. The subprocess does a single import + flag print, so 600s
    # only bounds pathological hangs, not normal runtime.
    result = subprocess.run([sys.executable, "-c", code], capture_output=True, text=True, timeout=600, env=env)
-    if result.stdout.strip().startswith("SKIP:"):
-        pytest.skip("modelopt not available in subprocess")
+    state = _last_stdout_line(result.stdout)
+    if state.startswith("SKIP:"):
+        pytest.skip(state)
    # Exact token match on the final stdout line — not a substring test, since
    # "INSTALLED" is a substring of "NOT_INSTALLED".
-    state = result.stdout.strip().splitlines()[-1] if result.stdout.strip() else ""
    assert state in acceptable, (
        f"VLLM_OMNI_SKIP_NVFP4_NAN_CLAMP={env_value!r} expected one of {set(acceptable)!r} "
        f"but got {state!r}:\nstdout: {result.stdout}\nstderr: {result.stderr}"
@@ -1,5 +1,5 @@
 # SPDX-License-Identifier: Apache-2.0
-# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project

 import pytest
 from vllm.config import CompilationConfig, CompilationMode, DeviceConfig, VllmConfig
@@ -45,3 +45,26 @@ def test_cuda_default_ir_op_priority_with_oink(monkeypatch: pytest.MonkeyPatch)
    )
    assert priority.rms_norm == ["oink", "vllm_c", "native"]
    assert priority.fused_add_rms_norm == ["oink", "vllm_c", "native"]
+
+
+@pytest.mark.parametrize(
+    "mode",
+    [CompilationMode.NONE, CompilationMode.VLLM_COMPILE],
+)
+def test_cuda_default_ir_op_priority_names_only_registered_providers(mode: CompilationMode) -> None:
+    """Every field must name providers that are actually registered.
+
+    ``IrOpPriorityConfig.with_default`` fills any field it is not given with the
+    generic ``default`` list, so an override that does not name a field silently
+    inherits every provider in that list. Upstream added ``gelu_and_mul_sparse``
+    (implemented by ``triton``/``native`` only), which therefore must not inherit
+    CUDA's ``["vllm_c", "native"]`` default: ``IrOp._filter_priority_impls``
+    asserts on unregistered providers and is reached from ``WorkerBase.__init__``
+    and the omni diffusion forward context.
+    """
+    priority = CudaOmniPlatform.get_default_ir_op_priority(_vllm_config(mode=mode))
+    assert priority.gelu_and_mul_sparse == ["triton", "native"]
+    # Scoped context manager: runs the real upstream provider validation for
+    # every field and restores the previous priorities on exit.
+    with priority.set_priority():
+        pass
@@ -44,7 +44,8 @@ def _async_output(req_ids=("req-0",), **overrides) -> OmniAsyncOutput:
    return OmniAsyncOutput(**(kwargs | overrides))


-def test_async_output_blocking_event_and_routing_masks(monkeypatch) -> None:
+@pytest.mark.parametrize("compact_width", [1, 2])
+def test_async_output_blocking_event_and_routing_masks(monkeypatch, compact_width) -> None:
    event_kwargs = []
    monkeypatch.setattr(torch.cuda, "set_stream", lambda _stream: None)

@@ -53,7 +54,12 @@ def test_async_output_blocking_event_and_routing_masks(monkeypatch) -> None:
        return _FakeEvent()

    monkeypatch.setattr(torch.cuda, "Event", make_event)
-    masks = SamplingMaskTensors(torch.tensor([[5], [0]], dtype=torch.uint8), torch.tensor([2, 0]), 4)
+    masks = SamplingMaskTensors(
+        token_ids=torch.tensor([[0, 2], [0, 0]], dtype=torch.int32)[:, :compact_width],
+        packed_mask=torch.tensor([[5], [0]], dtype=torch.uint8),
+        counts=torch.tensor([2, 0]),
+        vocab_size=4,
+    )
    sampler_output = SamplerOutput(torch.tensor([[2], [0]]), None, None, torch.tensor([1, 0]), None, masks)
    routed = RoutedExpertsTensors(torch.tensor([[[2, 3]], [[4, 5]]]), torch.tensor([7, 9]))

@@ -65,7 +71,9 @@ def test_async_output_blocking_event_and_routing_masks(monkeypatch) -> None:
        routed_experts=routed,
    ).get_output()
    assert event_kwargs == [{"blocking": True}]  # blocking event by default
-    assert output.sampled_token_ids == [[2], []] and output.sampling_masks.cu_num_generated_tokens == [0, 1, 1]
+    assert output.sampled_token_ids == [[2], []]
+    assert output.sampling_masks.to_nested_list() == [[0, 2], []]
+    np.testing.assert_array_equal(output.sampling_masks.offsets, [0, 2, 2])
    np.testing.assert_array_equal(output.routed_experts.routing_data, [[[2, 3]], [[4, 5]]])
    np.testing.assert_array_equal(output.routed_experts.slot_mapping, [7, 9])
    np.testing.assert_array_equal(output.sampling_masks.token_ids, [0, 2])
@@ -3,6 +3,7 @@

 """MRV2 admission, capture, dispatch and request lifecycle contracts."""

+from contextlib import nullcontext
 from types import SimpleNamespace
 from unittest.mock import MagicMock, patch

@@ -164,3 +165,67 @@ def test_capture_contract_uses_model_declaration(stage, declared):
    runner._configure_cudagraph_output_contract()
    assert runner._model_returns_tuple is declared
    assert runner._exclude_full_graph is declared
+
+
+@pytest.mark.parametrize("runner_kind", ["gpu", "ar", "generation"])
+def test_dummy_forward_uses_upstream_execution_state(runner_kind, monkeypatch):
+    from vllm.v1.worker.gpu.input_batch import InputBatch
+    from vllm.v1.worker.gpu.model_runner import ExecuteModelState
+
+    from vllm_omni.worker_v2.omni_ar_model_runner import OmniARModelRunner
+    from vllm_omni.worker_v2.omni_generation_model_runner import OmniGenerationModelRunner
+
+    runner_cls = {"gpu": OmniGPUModelRunner, "ar": OmniARModelRunner, "generation": OmniGenerationModelRunner}[
+        runner_kind
+    ]
+    runner = object.__new__(runner_cls)
+    hidden = torch.ones(1, 2)
+    runner.model = MagicMock(
+        return_value=OmniOutput(text_hidden_states=hidden, multimodal_outputs={})
+        if runner_kind == "generation"
+        else hidden
+    )
+    runner._dummy_hidden = hidden
+    runner.model_config = SimpleNamespace()
+    runner.vllm_config = SimpleNamespace()
+    runner.req_states = SimpleNamespace()
+    runner.model_state = MagicMock()
+    runner.model_state.prepare_inputs.return_value = {}
+    runner._omni_data_plane = object()
+    runner.supports_mm_inputs = False
+    runner.lora_config = None
+    runner.is_encoder_decoder = False
+    runner.eplb = MagicMock()
+    runner.kv_connector = MagicMock()
+    runner.input_buffers = object()
+    runner.kv_cache_config = object()
+    runner.attn_groups = []
+    input_batch = SimpleNamespace(
+        input_ids=torch.tensor([1]),
+        positions=torch.tensor([0]),
+        num_tokens=1,
+        num_tokens_after_padding=1,
+        is_padding=None,
+    )
+    runner.prepare_dummy_attn = MagicMock(return_value=((), torch.empty(0)))
+    runner.gather_batch_req_state = MagicMock(return_value=(None, 1))
+    batch_desc = SimpleNamespace(
+        cg_mode=CUDAGraphMode.NONE, num_reqs=1, num_tokens=1, num_active_loras=0, max_query_len=1
+    )
+    runner._dispatch_batch_descriptor = MagicMock(return_value=(batch_desc, None))
+    monkeypatch.setattr(InputBatch, "make_dummy", lambda *args, **kwargs: input_batch)
+    monkeypatch.setattr("vllm_omni.worker_v2.omni_model_runner.build_slot_mappings_by_layer", lambda *args: {})
+    for module in ("omni_model_runner", "omni_generation_model_runner"):
+        monkeypatch.setattr(f"vllm_omni.worker_v2.{module}.set_forward_context", lambda *args, **kwargs: nullcontext())
+    scheduled = SchedulerOutput.make_empty()
+    scheduled.num_scheduled_tokens = {"_dummy_req_0": 1}
+    scheduled.total_num_scheduled_tokens = 1
+
+    # Upstream _dummy_run always supplies valid_dummy_state_slots, and the
+    # result must use the real upstream constructor rather than a mocked state.
+    assert runner.execute_model(scheduled, dummy_run=True, valid_dummy_state_slots=True) is None
+    assert isinstance(runner.execute_model_state, ExecuteModelState)
+    assert runner.execute_model_state.input_batch is input_batch
+    assert runner.execute_model_state.cudagraph_stats is None
+    if runner_kind != "generation":
+        runner.prepare_dummy_attn.assert_called_once_with(input_batch, True)
@@ -353,6 +353,38 @@ def test_rope_shim_propagates_type_error():
        _make_safe_get_rope(broken_get_rope)(SimpleNamespace(uses_mrope=False), object())


+def test_rope_shim_constructs_sequential_mrope_state():
+    from vllm.platforms.cpu import CpuPlatform
+    from vllm.v1.worker.gpu.mm.rope import RopeState, get_rope_state
+
+    model = torch.nn.Module()
+    config = SimpleNamespace(uses_mrope=True, mrope_num_dims=3)
+    # Exercise the upstream constructor, including its argument contract, while
+    # keeping the backing buffers on CPU for this unit test.
+    with (
+        patch("vllm.platforms.current_platform", CpuPlatform()),
+        patch("vllm.v1.worker.gpu.buffer_utils.is_uva_available", return_value=False),
+    ):
+        rope = _make_safe_get_rope(get_rope_state)(
+            config,
+            model,
+            max_num_reqs=2,
+            max_num_tokens=8,
+            max_model_len=16,
+            device=torch.device("cpu"),
+        )
+
+    assert isinstance(rope, RopeState)
+    assert rope.get_positions(4).shape == (3, 4)
+    positions, delta = model.get_mrope_input_positions([11, 22, 33, 44], [])
+    torch.testing.assert_close(positions, torch.arange(4).expand(3, -1))
+    assert delta == 0
+    rope.init_prefill_positions(1, model, [11, 22, 33, 44], [])
+    assert rope.prefill_delta.np[1] == 0
+    assert rope.prefill_positions._staged_write_indices == [3, 4, 5]
+    assert rope.prefill_positions._staged_write_contents == [0, 1, 2, 3] * 3
+
+
@pytest.mark.parametrize("output_key", [None, ("only_one",)])
 def test_mtp_requires_model_declared_output_key(output_key):
    state = _make_state()
@@ -503,7 +503,14 @@ def _videomme_repo_from_args(args, *, explicit: bool = False) -> str | None:
    return None


-def get_samples(args, tokenizer):
+def get_samples(args, tokenizer, **kwargs):
+    """Omni override of ``vllm.benchmarks.datasets.get_samples``.
+
+    ``**kwargs`` mirrors upstream's keyword-only arguments (today
+    ``multimodal_backends``, passed by ``vllm/benchmarks/throughput.py``) so that
+    any upstream caller reaching this patched replacement keeps working; they are
+    forwarded to the original implementation on every delegate path.
+    """
    # Daily-Omni: explicit dataset name, or hf + matching path/hf-name
    is_daily_omni = args.dataset_name == "daily-omni" or (
        args.dataset_name == "hf" and _daily_omni_repo_from_args(args) is not None
@@ -531,7 +538,7 @@ def get_samples(args, tokenizer):

    if not is_omni_backend and not is_omni_dataset:
        # Not an omni-related request, delegate to original implementation
-        return get_samples_old(args, tokenizer)
+        return get_samples_old(args, tokenizer, **kwargs)

    if is_omniinteract:
        dataset_path = getattr(args, "dataset_path", None)
@@ -830,7 +837,7 @@ def get_samples(args, tokenizer):
        )
        return input_requests
    else:
-        return get_samples_old(args, tokenizer)
+        return get_samples_old(args, tokenizer, **kwargs)


 datasets.get_samples = get_samples
@@ -217,16 +217,20 @@ class OmniARScheduler(OmniSchedulerMixin, VLLMScheduler):
        finished_status: RequestStatus,
    ) -> list[Request]:
        """Finish requests and discard any incomplete KV-wait timing."""
-        cleanup_ids: Iterable[str]
+        # Annotated explicitly so every branch keeps the same static type
+        # (typing-only; the runtime values are unchanged).
+        cleanup_ids: tuple[str, ...]
+        finish_request_ids: str | Iterable[str] | None
        if isinstance(request_ids, str):
            cleanup_ids = (request_ids,)
-            finish_request_ids: str | Iterable[str] | None = request_ids
+            finish_request_ids = request_ids
        elif request_ids is None:
            cleanup_ids = ()
            finish_request_ids = None
        else:
-            finish_request_ids = list(request_ids) if isinstance(request_ids, Iterator) else request_ids
-            cleanup_ids = tuple(finish_request_ids)
+            ids = list(request_ids) if isinstance(request_ids, Iterator) else request_ids
+            finish_request_ids = ids
+            cleanup_ids = tuple(ids)

        finished = super().finish_requests(finish_request_ids, finished_status)
        self._clear_kv_wait_starts(cleanup_ids)
@@ -422,14 +426,23 @@ class OmniARScheduler(OmniSchedulerMixin, VLLMScheduler):
        outputs: dict[int, list[EngineCoreOutput]] = defaultdict(list)
        spec_decoding_stats: SpecDecodingStats | None = None

-        failed_kv_load_req_ids = None
+        failed_kv_load_req_ids: set[str] = set()
+        if kv_connector_output and getattr(kv_connector_output, "failed_recving", None):
+            # Upstream d43bb2f37f (HiSparse #53781): a receive failure is
+            # reported per request identity, so hybrid / multi-pool layouts
+            # whose numeric block ids overlap fail closed instead of recovering
+            # the wrong request. The helper returns an empty set when the
+            # failure is being recomputed, matching _handle_failed_kv_load_outputs.
+            failed_kv_load_req_ids.update(self._handle_failed_recving(kv_connector_output.failed_recving))
        if kv_connector_output and kv_connector_output.invalid_block_ids:
            # These blocks contain externally computed tokens that failed to
            # load. Identify affected requests and adjust their computed token
            # count to trigger recomputation of the invalid blocks.
-            failed_kv_load_req_ids = self._handle_invalid_blocks(
-                kv_connector_output.invalid_block_ids,
-                num_scheduled_tokens,
+            failed_kv_load_req_ids.update(
+                self._handle_invalid_blocks(
+                    kv_connector_output.invalid_block_ids,
+                    num_scheduled_tokens,
+                )
            )

        # Pre-process KV extraction acks so that the per-request loop below
@@ -600,6 +613,18 @@ class OmniARScheduler(OmniSchedulerMixin, VLLMScheduler):
                if request.status != RequestStatus.FINISHED_ERROR:
                    request.status = RequestStatus.FINISHED_STOPPED
                stopped = True
+            elif (
+                # Upstream parity: an encoder-only instance publishes
+                # embeddings instead of sampling, so it stops as soon as the
+                # whole prompt is consumed. Encoder inputs are never scheduled
+                # past a multi-modal item the encoder cache could not admit, so
+                # a consumed prompt also means every item in it was encoded.
+                # getattr: upstream Scheduler.__init__ sets is_mm_encoder_only,
+                # but SimpleNamespace/__new__ unit-test stubs may not carry it.
+                getattr(self, "is_mm_encoder_only", False) and request.num_computed_tokens >= request.num_prompt_tokens
+            ):
+                request.status = RequestStatus.FINISHED_STOPPED
+                stopped = True

            # If criteria returns True, it means we must STOP the request.
            # If criteria returns False, it might have triggered a background
@@ -607,19 +632,8 @@ class OmniARScheduler(OmniSchedulerMixin, VLLMScheduler):
            if not stopped and self._process_kv_transfer_trigger(request, new_token_ids):
                stopped = True

-            if new_token_ids and self.structured_output_manager.should_advance(request):
-                struct_output_request = request.structured_output_request
-                assert struct_output_request is not None
-                assert struct_output_request.grammar is not None
-                if not struct_output_request.grammar.accept_tokens(req_id, new_token_ids):
-                    logger.error(
-                        "Unexpected: grammar rejected tokens %s for request %s. Terminating request.",
-                        new_token_ids,
-                        req_id,
-                    )
-                    request.status = RequestStatus.FINISHED_ERROR
-                    request.resumable = False
-                    stopped = True
+            if OmniSchedulerMixin._reject_invalid_grammar_tokens(self, request, new_token_ids):
+                stopped = True

            # Finalize prefill stats BEFORE stop handling (upstream v0.28
            # order): _free_request below releases the KV blocks, after which
@@ -772,6 +786,8 @@ class OmniARScheduler(OmniSchedulerMixin, VLLMScheduler):
            stopped_preempted_reqs,
        )

+        OmniSchedulerMixin._finish_error_requests(self, outputs)
+
        failed_requests = self._handle_failed_kv_load_outputs(
            failed_kv_load_req_ids,
            outputs,
@@ -931,8 +947,9 @@ class OmniARScheduler(OmniSchedulerMixin, VLLMScheduler):
            ),
            None,
        )
+        chunk_transfer_adapter = self.chunk_transfer_adapter
        update_streaming_prompt = getattr(
-            self.chunk_transfer_adapter,
+            chunk_transfer_adapter,
            "update_streaming_prompt_for_condition",
            None,
        )
@@ -948,8 +965,10 @@ class OmniARScheduler(OmniSchedulerMixin, VLLMScheduler):
                # This streaming update has already been dequeued. Report the
                # permanent contract failure so the next scheduling pass
                # finishes only this request instead of crashing EngineCore.
-                if self.chunk_transfer_adapter is not None:
-                    self.chunk_transfer_adapter.record_receive_failure(req_id, str(exc))
+                # callable(update_streaming_prompt) above implies a live
+                # adapter; the assert narrows it for the type checker.
+                assert chunk_transfer_adapter is not None
+                chunk_transfer_adapter.record_receive_failure(req_id, str(exc))
                return
            if replaced is not None:
                if replaced:
@@ -11,7 +11,7 @@ from vllm.compilation.cuda_graph import CUDAGraphStat
 from vllm.logger import init_logger
 from vllm.v1.core.kv_cache_manager import KVCacheBlocks
 from vllm.v1.core.sched.interface import PauseState
-from vllm.v1.core.sched.output import SchedulerOutput
+from vllm.v1.core.sched.output import KVConnectorBlockState, SchedulerOutput
 from vllm.v1.core.sched.request_queue import create_request_queue
 from vllm.v1.core.sched.scheduler import Scheduler as VLLMScheduler
 from vllm.v1.core.sched.utils import remove_all
@@ -57,6 +57,75 @@ class OmniGenerationScheduler(OmniSchedulerMixin, VLLMScheduler):
        self._retains_state_across_chunks = bool(getattr(model_config, "retains_state_across_chunks", False))
        self._pending_finish_reqs: list[Request] = []

+    def _build_generation_scheduler_output(
+        self,
+        new_reqs_data: list[OmniNewRequestData],
+        cached_reqs_data: OmniCachedRequestData,
+        num_scheduled_tokens: dict[str, int],
+        scheduled_spec_decode_tokens: dict[str, list[int]],
+        scheduled_encoder_inputs: dict[str, list[int]],
+        num_common_prefix_blocks: list[int],
+    ) -> SchedulerOutput:
+        """Build the worker output and consume scheduler-only connector state."""
+        # Dynamic speculative decoding: optimal K chosen by scheduler
+        # (upstream Scheduler.schedule parity; 0 when spec decode is unused).
+        num_spec_tokens_to_schedule = self.num_spec_tokens
+        if self.dynamic_sd_lookup is not None and len(num_scheduled_tokens) > 0:
+            num_spec_tokens_to_schedule = self.dynamic_sd_lookup[len(num_scheduled_tokens)]
+
+        # Mamba "align" boundary states must be handed off with exact block ids;
+        # they cannot be reconstructed from a connector's append-only block
+        # table. Drained every step so stale offers cannot accumulate.
+        boundary_state_offloads = self.kv_cache_manager.take_boundary_state_offloads()
+
+        # Connector-only block state: a producer-side connector resolves the
+        # current block table lazily for every request scheduled this step (not
+        # only the ones that allocated blocks). Built here and cleared again
+        # before the output is dispatched -- workers must never see it.
+        kv_connector_block_state = None
+        if self.connector is not None:
+            block_state_req_ids = set(num_scheduled_tokens)
+            block_state_req_ids.update(req_id for req_id in boundary_state_offloads if req_id in self.requests)
+            kv_connector_block_state = KVConnectorBlockState(
+                req_ids=block_state_req_ids,
+                resolve_block_ids=self.kv_cache_manager.get_block_ids,
+                boundary_state_offloads=boundary_state_offloads,
+            )
+
+        scheduler_output = SchedulerOutput(
+            scheduled_new_reqs=new_reqs_data,
+            scheduled_cached_reqs=cached_reqs_data,
+            num_scheduled_tokens=num_scheduled_tokens,
+            total_num_scheduled_tokens=sum(num_scheduled_tokens.values()),
+            scheduled_spec_decode_tokens=scheduled_spec_decode_tokens,
+            scheduled_encoder_inputs=scheduled_encoder_inputs,
+            num_common_prefix_blocks=num_common_prefix_blocks,
+            finished_req_ids=self.finished_req_ids,
+            free_encoder_mm_hashes=self.encoder_cache_manager.get_freed_mm_hashes(),
+            # Preemptions accumulated since the previous step; cleared by
+            # _update_after_schedule below, exactly like upstream.
+            preempted_req_ids=self.reset_preempted_req_ids,
+            # Upstream drains the manager-side list every step (unbounded growth
+            # otherwise) and applies _skip_zero_block_ids before zeroing.
+            new_block_ids_to_zero=self._get_new_block_ids_to_zero(),
+            ec_manager_metadata=self.encoder_cache_manager.get_manager_metadata(),
+            kv_connector_block_state=kv_connector_block_state,
+            num_spec_tokens_to_schedule=num_spec_tokens_to_schedule,
+        )
+
+        # KVTransfer: package metadata
+        if self.connector is not None:
+            meta = self._build_kv_connector_meta(self.connector, scheduler_output)
+            scheduler_output.kv_connector_metadata = meta
+        # EC Connector: package metadata
+        if self.ec_connector is not None:
+            ec_meta = self.ec_connector.build_connector_meta(scheduler_output)
+            scheduler_output.ec_connector_metadata = ec_meta
+        # Connector-only block state must not be dispatched to workers.
+        scheduler_output.kv_connector_block_state = None
+
+        return scheduler_output
+
    def _is_done_receiving_chunks(self, request_id: str) -> bool:
        if self.chunk_transfer_adapter is not None:
            return self.chunk_transfer_adapter.is_done_receiving_chunks(request_id)
@@ -390,33 +459,15 @@ class OmniGenerationScheduler(OmniSchedulerMixin, VLLMScheduler):
        self.prev_step_scheduled_req_ids.clear()
        self.prev_step_scheduled_req_ids.update(num_scheduled_tokens.keys())

-        new_block_ids_to_zero = (
-            (self.kv_cache_manager.take_new_block_ids() or None) if self.needs_kv_cache_zeroing else None
+        scheduler_output = self._build_generation_scheduler_output(
+            new_reqs_data,
+            cached_reqs_data,
+            num_scheduled_tokens,
+            scheduled_spec_decode_tokens,
+            scheduled_encoder_inputs,
+            num_common_prefix_blocks,
        )

-        scheduler_output = SchedulerOutput(
-            scheduled_new_reqs=new_reqs_data,
-            scheduled_cached_reqs=cached_reqs_data,
-            num_scheduled_tokens=num_scheduled_tokens,
-            total_num_scheduled_tokens=total_num_scheduled_tokens,
-            scheduled_spec_decode_tokens=scheduled_spec_decode_tokens,
-            scheduled_encoder_inputs=scheduled_encoder_inputs,
-            num_common_prefix_blocks=num_common_prefix_blocks,
-            finished_req_ids=self.finished_req_ids,
-            free_encoder_mm_hashes=self.encoder_cache_manager.get_freed_mm_hashes(),
-            preempted_req_ids=set(),
-            new_block_ids_to_zero=new_block_ids_to_zero,
-        )
-
-        # KVTransfer: package metadata
-        if self.connector is not None:
-            meta = self._build_kv_connector_meta(self.connector, scheduler_output)
-            scheduler_output.kv_connector_metadata = meta
-        # EC Connector: package metadata
-        if self.ec_connector is not None:
-            ec_meta = self.ec_connector.build_connector_meta(scheduler_output)
-            scheduler_output.ec_connector_metadata = ec_meta
-
        # Advance the fence only for non-empty steps (those that actually
        # write KV and have their output processed later in
        # update_from_output). Must precede _update_after_schedule, which
@@ -490,11 +541,20 @@ class OmniGenerationScheduler(OmniSchedulerMixin, VLLMScheduler):
        outputs: dict[int, list[EngineCoreOutput]] = defaultdict(list)
        spec_decoding_stats: SpecDecodingStats | None = None

-        failed_kv_load_req_ids = None
+        failed_kv_load_req_ids: set[str] = set()
+        if kv_connector_output and getattr(kv_connector_output, "failed_recving", None):
+            # Upstream d43bb2f37f (HiSparse #53781): a receive failure is
+            # reported per request identity, so hybrid / multi-pool layouts
+            # whose numeric block ids overlap fail closed instead of recovering
+            # the wrong request. The helper returns an empty set when the
+            # failure is being recomputed, matching _handle_failed_kv_load_outputs.
+            failed_kv_load_req_ids.update(self._handle_failed_recving(kv_connector_output.failed_recving))
        if kv_connector_output and getattr(kv_connector_output, "invalid_block_ids", None):
-            failed_kv_load_req_ids = self._handle_invalid_blocks(
-                kv_connector_output.invalid_block_ids,
-                num_scheduled_tokens,
+            failed_kv_load_req_ids.update(
+                self._handle_invalid_blocks(
+                    kv_connector_output.invalid_block_ids,
+                    num_scheduled_tokens,
+                )
            )

        # NOTE(woosuk): As len(num_scheduled_tokens) can be up to 1K or more,
@@ -617,6 +677,10 @@ class OmniGenerationScheduler(OmniSchedulerMixin, VLLMScheduler):
                # (does not affect protocol)
                stopped = True

+            # Validate grammar before terminal cleanup and finish-reason capture.
+            if OmniSchedulerMixin._reject_invalid_grammar_tokens(self, request, new_token_ids):
+                stopped = True
+
            # Finalize prefill stats BEFORE stop handling (upstream v0.28
            # order): _free_request below releases the KV blocks, after which
            # estimate_cached_tokens(request) reports 0. kv_transfer_params is
@@ -657,14 +721,6 @@ class OmniGenerationScheduler(OmniSchedulerMixin, VLLMScheduler):
            if request.sampling_params is not None and request.sampling_params.num_logprobs is not None and logprobs:
                new_logprobs = logprobs.slice_request(req_index, len(new_token_ids))

-            if new_token_ids and self.structured_output_manager.should_advance(request):
-                # NOTE: structured_output_request should not be None if
-                # use_structured_output, we have check above, so safe to ignore
-                # type warning
-                request.structured_output_request.grammar.accept_tokens(  # type: ignore[union-attr]  # noqa: E501
-                    req_id, new_token_ids
-                )
-
            # spec_token_ids comes from the model runner output
            if num_nans_in_logits is not None and req_id in num_nans_in_logits:
                request.num_nans_in_logits = num_nans_in_logits[req_id]
@@ -731,6 +787,8 @@ class OmniGenerationScheduler(OmniSchedulerMixin, VLLMScheduler):
            stopped_preempted_reqs,
        )

+        OmniSchedulerMixin._finish_error_requests(self, outputs)
+
        failed_requests = self._handle_failed_kv_load_outputs(
            failed_kv_load_req_ids,
            outputs,
@@ -730,6 +730,38 @@ class OmniSchedulerMixin:
            output.finished_requests = finished_set
        finished_req_ids.clear()

+    def _reject_invalid_grammar_tokens(self, request: Request, new_token_ids: list[int]) -> bool:
+        """Mark rejected tokens terminal before callers capture the finish reason."""
+        if not new_token_ids or self.structured_output_manager.accept_tokens(request, new_token_ids):
+            return False
+        logger.error(
+            "Unexpected: grammar rejected tokens %s for request %s. Terminating request.",
+            new_token_ids,
+            request.request_id,
+        )
+        request.status = RequestStatus.FINISHED_ERROR
+        request.resumable = False
+        return True
+
+    def _finish_error_requests(self, outputs: dict[int, list[EngineCoreOutput]]) -> None:
+        """Drain grammar-compilation and unavailable-encoder errors into terminal outputs."""
+        grammar_error_reqs = getattr(self, "grammar_compile_error_reqs", None)
+        error_req_ids = set(grammar_error_reqs or ())
+        if grammar_error_reqs:
+            grammar_error_reqs.clear()
+        ec_connector = getattr(self, "ec_connector", None)
+        if ec_connector is not None:
+            error_req_ids.update(ec_connector.take_unavailable_requests())
+        if error_req_ids:
+            for request in self.finish_requests(error_req_ids, RequestStatus.FINISHED_ERROR):
+                OmniSchedulerMixin._append_request_output(
+                    self,
+                    outputs,
+                    request,
+                    new_token_ids=[],
+                    finish_reason=request.get_finished_reason(),
+                )
+
    def _remove_stopped_requests_from_queues(
        self,
        stopped_running_reqs: set[Request],
@@ -260,7 +260,6 @@ class DiffusionKVModelRunnerBackend:
        cp_rank = get_dcp_group().rank_in_group if cp_size > 1 else 0
        cp_interleave = parallel_config.cp_kv_cache_interleave_size

-        kv_caches: list[torch.Tensor | list[torch.Tensor]] = []
        previous_adapter_caches = {
            layer_name: adapter.kv_cache for layer_name, adapter in self._kv_cache_layer_adapters.items()
        }
@@ -292,19 +291,20 @@ class DiffusionKVModelRunnerBackend:
                    layers=self._kv_cache_layer_adapters,
                    resolve_row=self._resolve_paged_attention_row,
                )
-                # vLLM 0.29 reads the resolved physical layout when allocating,
-                # and dropped attn_groups/cache_dtype from the signature.
+                # f2aad6aa70 (#56888): first arg is forward_context; caches
+                # are returned as a dict instead of appended to a runner list.
                adopt_kv_cache_layout(self.vllm_config, kv_cache_config)
                for layer_adapter in self._kv_cache_layer_adapters.values():
                    assert_backend_layout_supported(self.vllm_config, getattr(layer_adapter, "attn_backend", None))
-                init_kv_cache(
-                    kv_caches,
+                kv_caches_dict = init_kv_cache(
                    self.vllm_config.compilation_config.static_forward_context,
                    kv_cache_config,
                    self.device,
                    kernel_block_sizes,
                    self.vllm_config,
+                    block_tables=block_tables,
                )
+                kv_caches = list(kv_caches_dict.values())
        except Exception:
            for layer_name, previous_cache in previous_adapter_caches.items():
                self._kv_cache_layer_adapters[layer_name].kv_cache = previous_cache
@@ -1,5 +1,5 @@
 # SPDX-License-Identifier: Apache-2.0
-# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project

 """Format-specific live-weight validation and reference A16 materialization."""

@@ -7,14 +7,31 @@ from __future__ import annotations

 import torch
 import torch.nn.functional as F
-from vllm.model_executor.layers.quantization.modelopt import (
-    ModelOptFp8LinearMethod,
-    ModelOptNvFp4LinearMethod,
-)
+from vllm.config.quantization import QuantSpec
+from vllm.model_executor.layers.quantization.modelopt import ModelOptLinearMethod
 from vllm.model_executor.layers.quantization.utils import nvfp4_emulation_utils
+from vllm.model_executor.layers.quantization.utils.quant_utils import (
+    kFp8StaticTensorSym,
+    kNvfp4Dynamic,
+    kNvfp4Static,
+)

 _NVFP4_BLOCK_SIZE = 16

+# vLLM #49381 replaced the per-format ModelOpt linear methods
+# (ModelOptFp8LinearMethod / ModelOptNvFp4LinearMethod, reached through the
+# removed ``LinearMethodCls`` attributes) with one generic
+# ``ModelOptLinearMethod`` built from a ``(weight, activation)`` QuantSpec pair
+# produced by ``resolve()``. Identify the formats the same way upstream does:
+# by that exact spec, so the set of accepted methods stays as narrow as it was
+# with the class checks (PcPt / PbWo / W4A16 are still not accepted here).
+_FP8_W8A8_SPEC = QuantSpec(weight=kFp8StaticTensorSym, activation=kFp8StaticTensorSym)
+_NVFP4_W4A4_SPEC = QuantSpec(weight=kNvfp4Static, activation=kNvfp4Dynamic)
+
+
+def _has_spec(method: object | None, spec: QuantSpec) -> bool:
+    return isinstance(method, ModelOptLinearMethod) and getattr(method, "spec", None) == spec
+

 class Cosmos3PrecisionStrategy:
    """Quantization-format boundary behind the shared precision schedule."""
@@ -70,7 +87,7 @@ class Fp8W8A8W8A16Strategy(Cosmos3PrecisionStrategy):
    """Use native ModelOpt W8A8 or reference dense W8A16."""

    def accepts(self, method: object | None) -> bool:
-        return isinstance(method, ModelOptFp8LinearMethod)
+        return _has_spec(method, _FP8_W8A8_SPEC)

    def validate_before_processing(
        self,
@@ -131,7 +148,7 @@ class Nvfp4W4A4W4A16Strategy(Cosmos3PrecisionStrategy):
    """Use native ModelOpt W4A4 or reference dense W4A16."""

    def accepts(self, method: object | None) -> bool:
-        return isinstance(method, ModelOptNvFp4LinearMethod)
+        return _has_spec(method, _NVFP4_W4A4_SPEC)

    def validate_before_processing(
        self,
@@ -186,11 +186,16 @@ def _setup_diffusion_worker_proc_title_and_log_prefix(
 def _force_cutlass_fp8_linear_kernel(quant_config: object | None) -> Iterator[None]:
    import vllm.model_executor.layers.quantization.modelopt as vllm_modelopt

-    linear_method_cls = getattr(quant_config, "LinearMethodCls", None)
-    if linear_method_cls in {
-        vllm_modelopt.ModelOptFp8LinearMethod,
-        vllm_modelopt.ModelOptFp8PcPtLinearMethod,
-    }:
+    # vLLM #49381 replaced the per-format ModelOpt linear methods with the
+    # generic ``ModelOptLinearMethod`` and removed the ``LinearMethodCls``
+    # attributes this used to match on. The same two formats are identified by
+    # the ModelOpt quant-algo string carried on the config
+    # (``ModelOptQuantConfigBase.quant_method``): "FP8" used to select
+    # ``ModelOptFp8LinearMethod`` and "FP8_PER_CHANNEL_PER_TOKEN" used to select
+    # ``ModelOptFp8PcPtLinearMethod``. "FP8_PB_WO" / "NVFP4" / "W4A16_NVFP4"
+    # were never matched here and still are not.
+    quant_algo = getattr(quant_config, "quant_method", None)
+    if quant_algo in ("FP8", "FP8_PER_CHANNEL_PER_TOKEN"):
        from vllm.platforms import current_platform

        if current_platform.is_cuda() and current_platform.has_device_capability(89):
@@ -51,6 +51,42 @@ class AsyncOmniEngine(OmniEngineBase):
            return value
        return [value]

+    _DEFAULT_MM_HASHER_ALGORITHM = "blake3"
+
+    def _resolve_mm_hasher_algorithm(self) -> str:
+        """Return the MM hash algorithm configured for stage 0.
+
+        Upstream vLLM 0.29 removed ``MultiModalHasher``'s ``_get_mm_hasher_algorithm``
+        helper and the ``VLLM_MM_HASHER_ALGORITHM`` env var; the algorithm now lives
+        only on ``MultiModalConfig.mm_hasher_algorithm`` (``--mm-hasher-algorithm``,
+        default ``"blake3"``). Resolve it from stage-0's config so the pre-computed
+        replica-scoped uuid uses the same algorithm as the stage-0 renderer built by
+        ``build_stage0_input_processor``. Falls back to the upstream default when the
+        engine has no stage config (unit tests built via ``object.__new__``) or the
+        model has no multimodal config.
+        """
+
+        stage_vllm_configs = getattr(self, "stage_vllm_configs", None) or []
+        if not stage_vllm_configs:
+            return self._DEFAULT_MM_HASHER_ALGORITHM
+
+        model_config = getattr(stage_vllm_configs[0], "model_config", None)
+        if model_config is None:
+            return self._DEFAULT_MM_HASHER_ALGORITHM
+
+        mm_config = None
+        get_multimodal_config = getattr(model_config, "get_multimodal_config", None)
+        if callable(get_multimodal_config):
+            try:
+                mm_config = get_multimodal_config()
+            except ValueError:
+                # Model is not multimodal.
+                mm_config = None
+        if mm_config is None:
+            mm_config = getattr(model_config, "multimodal_config", None)
+
+        return getattr(mm_config, "mm_hasher_algorithm", None) or self._DEFAULT_MM_HASHER_ALGORITHM
+
    def _ensure_stage_replica_mm_uuids(
        self,
        prompt: Any,
@@ -75,9 +111,10 @@ class AsyncOmniEngine(OmniEngineBase):
        if not isinstance(mm_data, dict) or not mm_data:
            return

-        from vllm.config.multimodal import _get_mm_hasher_algorithm
        from vllm.multimodal.hasher import MultiModalHasher

+        mm_hasher_algorithm = self._resolve_mm_hasher_algorithm()
+
        existing_uuids = prompt.get("multi_modal_uuids")
        if not isinstance(existing_uuids, dict):
            existing_uuids = {}
@@ -102,7 +139,7 @@ class AsyncOmniEngine(OmniEngineBase):
                    base_uuid = None
                else:
                    base_uuid = MultiModalHasher.hash_kwargs(
-                        _get_mm_hasher_algorithm(),
+                        mm_hasher_algorithm,
                        model_id=model_id,
                        **{modality: item},
                    )
@@ -115,7 +115,7 @@ class StageEngineCoreProc(EngineCoreProc):
        scheduler_request, current_wave = super().preprocess_add_request(request)
        scheduler_request.additional_information = request.additional_information
        scheduler_request.external_req_id = getattr(request, "external_req_id", request.request_id)
-        scheduler_request.payload_sender_info = request.payload_sender_info
+        scheduler_request.payload_sender_info = getattr(request, "payload_sender_info", None)
        return scheduler_request, current_wave

    @staticmethod
@@ -119,6 +119,7 @@ class AsyncOmni(AsyncOmniBase, EngineClient):
        trace_headers: Mapping[str, str] | None = None,
        priority: int = 0,
        data_parallel_rank: int | None = None,
+        session_id: str | None = None,
        reasoning_ended: bool | None = None,
        reasoning_parser_kwargs: dict[str, Any] | None = None,
        arrival_time: float | None = None,
@@ -129,6 +130,9 @@ class AsyncOmni(AsyncOmniBase, EngineClient):
        through all stages in the pipeline and yields outputs as they become
        available.

+        ``session_id`` is accepted for EngineClient protocol compatibility
+        and is not duplex-session plumbing.
+
        **Diffusion batching:**
        Diffusion stages accept only a single prompt per request.  Passing a
        ``list`` of prompts to a diffusion stage will raise ``ValueError``.
@@ -1,3 +1,7 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
+
+import importlib
 import logging
 import os
 import sys
@@ -94,20 +98,35 @@ assert _installed is _patched_cp, (
 # inference bug. Newly calibrated, clean checkpoints pay no runtime cost
 # (the clamp is a no-op when no NaN bytes are present).
 #
-# SCOPE: ModelOptNvFp4LinearMethod (W4A4 NVFP4 Linear) only. NvFp4FusedMoE /
-# NvFp4W4A16 / CompressedTensors / Quark NVFP4 paths are not covered.
+# SCOPE: the W4A4 NVFP4 linear PWAL only. NvFp4FusedMoE / NvFp4W4A16 /
+# CompressedTensors / Quark NVFP4 paths are not covered.
 #
-# SELF-EXTINGUISH: `_already_patched_upstream` heuristically detects when
-# vLLM's own PWAL contains an in-place `masked_fill_` against `weight_scale`
-# / `isnan` — the structure the upstream fix is expected to take when it is
-# filed (planned as a follow-up PR after this one merges). Once vllm-omni's
-# vllm pin moves to a release with that upstream fix, the override is
-# skipped at import and this block can be deleted. NOTE: the heuristic only
-# matches "vLLM PWAL clamps NaN in-place with `masked_fill_`"; if the
-# upstream fix lands as `nan_to_num_` or as a clamp before the FP32→FP8
-# cast, the check won't fire and this override stays active. The override
-# is idempotent so the overlap is a warning log, not a correctness issue —
-# but the heuristic should be revisited when the upstream PR is filed.
+# SELF-EXTINGUISH (structural): vLLM #49381 redesigned the ModelOpt linear
+# methods around one generic `ModelOptLinearMethod` built by
+# `build_linear_method()`, and removed the per-format classes
+# (`ModelOptNvFp4LinearMethod`, `ModelOptFp8LinearMethod`, ...) together with
+# the `LinearMethodCls` attributes they were reached through. This block
+# therefore resolves its target dynamically and self-extinguishes when only
+# the generic class exists, because that PWAL now REJECTS any NaN
+# weight_scale (#52501: `KNvfp4Static` creates `weight_scale` as
+# `torch.full(shape, nan)` and raises "... was never loaded (still NaN)" on
+# any surviving NaN). A NaN byte in weight_scale can come from that
+# unloaded-scale sentinel or from ModelOpt 0.44's FP32→FP8 E4M3 cast
+# overflow, and the two are byte-identical; a fused projection's
+# weight_scale is also written slice-by-slice by the weight loader, so
+# "some values are finite" does not prove the tensor was fully loaded.
+# Clamping would silently serve a partially loaded model, so the override is
+# retired there instead — a corrupt checkpoint gets a loud load-time
+# RuntimeError instead of the `!!!!` decode-time collapse.
+#
+# SELF-EXTINGUISH (heuristic, legacy pins only): `_already_patched_upstream`
+# heuristically detects when vLLM's own PWAL contains an in-place
+# `masked_fill_` against `weight_scale` / `isnan` — the structure the
+# upstream fix was expected to take. NOTE: the heuristic only matches
+# "vLLM PWAL clamps NaN in-place with `masked_fill_`"; if an upstream fix
+# lands as `nan_to_num_` or as a clamp before the FP32→FP8 cast, the check
+# won't fire and this override stays active. The override is idempotent so
+# the overlap is a warning log, not a correctness issue.
 #
 # ORDERING: the clamp must run BEFORE the original PWAL. The non-Blackwell
 # Marlin fallback (sm_<100) casts weight_scale FP8 -> bf16/fp16 and permutes
@@ -179,34 +198,77 @@ def _clamp_nvfp4_weight_scale_nans(layer) -> int:

 # Module-level defaults so downstream code (and tests) can import these names
 # without guarding for the import-failure / escape-hatch branches below.
-# `_already_patched_upstream` = upstream PWAL contains its own NaN clamp.
+# `_already_patched_upstream` = upstream's own PWAL already handles NaN
+#                               weight_scale by itself (its own clamp, or — on
+#                               the redesigned ModelOpt path — a hard
+#                               rejection), so we deliberately install nothing.
 # `_clamp_installed`         = our wrapper was installed on the upstream class.
 # These are independent: the env-var escape hatch and the import-failure path
-# both leave the wrapper uninstalled WITHOUT upstream being patched, so the
+# both leave the wrapper uninstalled WITHOUT upstream handling the case, so the
 # right check for "we own NaN-clamp behavior" is `_clamp_installed`.
 _already_patched_upstream = False
 _clamp_installed = False

+# Resolve the clamp target dynamically. A literal
+# `from ...modelopt import ModelOptNvFp4LinearMethod` is not an option any more:
+# the class was removed upstream (#49381), so the import is both a hard
+# ImportError and a static-check failure, and dynamic resolution is also what
+# lets us tell "no target" apart from "target replaced by the generic method".
+_MODELOPT_MODULE = "vllm.model_executor.layers.quantization.modelopt"
+_LEGACY_NVFP4_LINEAR_METHOD = "ModelOptNvFp4LinearMethod"
+_GENERIC_LINEAR_METHOD = "ModelOptLinearMethod"
+
 try:
    # Escape hatch — set VLLM_OMNI_SKIP_NVFP4_NAN_CLAMP=1 to skip installing
    # the patch (e.g. to confirm a `!!!!` failure is the NaN-byte case).
    # The escape-hatch deliberately raises ImportError so the not-installed
-    # warning below logs through the same path a real ImportError would.
+    # warning below logs through the same path a real ImportError would, and it
+    # short-circuits BEFORE any upstream probing so both module flags stay
+    # False on this path.
    # Use the repo-wide bool-env idiom so values like `0`, `false`, `no`,
    # `off` correctly mean "do not skip" rather than tripping naive
    # truthiness on the non-empty string.
    if os.environ.get("VLLM_OMNI_SKIP_NVFP4_NAN_CLAMP", "").lower() in ("1", "true", "yes", "on"):
        raise ImportError("VLLM_OMNI_SKIP_NVFP4_NAN_CLAMP is set; skipping NaN-clamp install")
-    from vllm.model_executor.layers.quantization.modelopt import (
-        ModelOptNvFp4LinearMethod as _OriginalModelOptNvFp4LinearMethod,
-    )
+    _modelopt = importlib.import_module(_MODELOPT_MODULE)
+    _legacy_nvfp4_linear_method = getattr(_modelopt, _LEGACY_NVFP4_LINEAR_METHOD, None)
+    _generic_linear_method = getattr(_modelopt, _GENERIC_LINEAR_METHOD, None)
 except ImportError as _nan_clamp_import_err:
    _PATCH_LOGGER.warning(
        "NVFP4 weight_scale NaN-clamp patch could NOT install: %s. NVFP4 W4A4 "
        "checkpoints with NaN bytes in per-block weight_scale will serve `!!!!`.",
        _nan_clamp_import_err,
    )
+    _legacy_nvfp4_linear_method = None
+    _generic_linear_method = None
 else:
+    if _legacy_nvfp4_linear_method is None and _generic_linear_method is not None:
+        # Redesigned upstream (#49381): only the generic ModelOptLinearMethod
+        # exists, and its PWAL rejects any NaN weight_scale (#52501). Retire the
+        # override rather than mask that check — see the SELF-EXTINGUISH
+        # (structural) note above for why the two NaN origins cannot be told
+        # apart. `_already_patched_upstream` records that upstream handles the
+        # case itself, so an absent `_clamp_installed` is the expected state.
+        _already_patched_upstream = True
+        _PATCH_LOGGER.info(
+            "NVFP4 W4A4 weight_scale NaN-clamp: skipped — upstream serves ModelOpt "
+            "linears through the generic %s and rejects any NaN weight_scale "
+            "(unloaded-scale sentinel), so a load-time error replaces the "
+            "`!!!!` decode-time collapse.",
+            _GENERIC_LINEAR_METHOD,
+        )
+    elif _legacy_nvfp4_linear_method is None:
+        # Unrecognised pin: neither the legacy per-format class nor the generic
+        # replacement is present. Nothing to install and nothing to assume.
+        _PATCH_LOGGER.warning(
+            "NVFP4 weight_scale NaN-clamp patch could NOT install: neither %s nor %s is present in %s.",
+            _LEGACY_NVFP4_LINEAR_METHOD,
+            _GENERIC_LINEAR_METHOD,
+            _MODELOPT_MODULE,
+        )
+
+if _legacy_nvfp4_linear_method is not None:
+    _OriginalModelOptNvFp4LinearMethod = _legacy_nvfp4_linear_method
    _current_nvfp4_pwal = _OriginalModelOptNvFp4LinearMethod.process_weights_after_loading
    # Reload idempotency: on a module reload (importlib.reload in a test, or a
    # second import path) the class attribute already holds OUR wrapper, so
@@ -449,4 +449,12 @@ class CudaOmniPlatform(OmniPlatform, CudaPlatformBase):
        if envs.VLLM_USE_OINK_OPS:
            rms_norm = ["oink"] + default

-        return IrOpPriorityConfig.with_default(default, rms_norm=rms_norm, fused_add_rms_norm=rms_norm)
+        # Mirrors upstream CudaPlatformBase defaults: `gelu_and_mul_sparse` is
+        # implemented by `triton` and `native` only, so it must not fall back to
+        # `default` (which contains `vllm_c`) via IrOpPriorityConfig.with_default.
+        return IrOpPriorityConfig.with_default(
+            default,
+            rms_norm=rms_norm,
+            fused_add_rms_norm=rms_norm,
+            gelu_and_mul_sparse=["triton", "native"],
+        )
@@ -1,5 +1,5 @@
 # SPDX-License-Identifier: Apache-2.0
-# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project

 import torch
 from vllm import envs
@@ -218,4 +218,12 @@ class RocmOmniPlatform(OmniPlatform, RocmPlatform):
        else:
            rms_norm = default

-        return IrOpPriorityConfig.with_default(default, rms_norm=rms_norm, fused_add_rms_norm=rms_norm)
+        # Mirrors upstream RocmPlatform defaults: `gelu_and_mul_sparse` has no
+        # ROCm-specific provider, so it must not fall back to `default` (which
+        # contains `vllm_c`) via IrOpPriorityConfig.with_default.
+        return IrOpPriorityConfig.with_default(
+            default,
+            rms_norm=rms_norm,
+            fused_add_rms_norm=rms_norm,
+            gelu_and_mul_sparse=["native"],
+        )
@@ -1,5 +1,5 @@
 # SPDX-License-Identifier: Apache-2.0
-# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project

 import os

@@ -155,4 +155,7 @@ class XPUOmniPlatform(OmniPlatform, XPUPlatform):
        using_inductor = cc.backend == "inductor" and cc.mode != CompilationMode.NONE
        default = ["native"] if using_inductor else ["vllm_c", "native"]

-        return IrOpPriorityConfig.with_default(default)
+        # Mirrors upstream XPUPlatform defaults: `gelu_and_mul_sparse` has no XPU
+        # provider, so it must not fall back to `default` (which contains
+        # `vllm_c`) via IrOpPriorityConfig.with_default.
+        return IrOpPriorityConfig.with_default(default, gelu_and_mul_sparse=["native"])
@@ -776,7 +776,7 @@ class GPUGenerationModelRunner(OmniGPUModelRunner, OmniConnectorModelRunnerMixin
        ):
            # Make sure padding doesn't exceed max_num_tokens
            assert num_tokens_padded <= self.max_num_tokens
-            model_kwargs = self._init_model_kwargs()
+            model_kwargs = self._init_model_kwargs(num_reqs=0)
            if self.supports_mm_inputs and not self.model_config.is_encoder_decoder:
                input_ids, inputs_embeds = self._prepare_mm_inputs(num_tokens_padded)

@@ -787,7 +787,7 @@ class GPUGenerationModelRunner(OmniGPUModelRunner, OmniConnectorModelRunnerMixin
            elif self.enable_prompt_embeds:
                input_ids = None
                inputs_embeds = self.inputs_embeds.gpu[:num_tokens_padded]
-                model_kwargs = self._init_model_kwargs()
+                model_kwargs = self._init_model_kwargs(num_reqs=0)
            elif getattr(getattr(self, "model", None), "has_preprocess", False):
                # Capture CUDA graph with inputs_embeds path so replay reads
                # from the same buffer that _preprocess writes into.
@@ -813,8 +813,6 @@ class GPUGenerationModelRunner(OmniGPUModelRunner, OmniConnectorModelRunnerMixin

            if self.uses_mrope:
                positions = self.mrope_positions.gpu[:, :num_tokens_padded]
-            elif getattr(self, "uses_xdrope_dim", 0) > 0:
-                positions = self.xdrope_positions.gpu[:, :num_tokens_padded]
            else:
                positions = self.positions[:num_tokens_padded]

@@ -685,14 +685,12 @@ class OmniGPUModelRunner(PrefixCacheRunnerMixin, GPUModelRunner):
                    if sampling_params.prompt_logprobs == -1
                    else sampling_params.prompt_logprobs
                )
-            # Only relevant for models using M-RoPE (e.g, Qwen2-VL)
+            # Only relevant for models using M-RoPE (e.g, Qwen2-VL).
+            # HunYuan-style XD-RoPE is unified into this path upstream
+            # (719284fe15): mrope_positions is sized (mrope_num_dims, ...).
            if self.uses_mrope:
                self._init_mrope_positions(req_state)

-            # Only relevant for models using XD-RoPE (e.g, HunYuan-VL)
-            if getattr(self, "uses_xdrope_dim", 0) > 0:
-                self._init_xdrope_positions(req_state)
-
            reqs_to_add.append(self.requests[req_id])
            # Track new requests for ngram_gpu full tensor copy
            if is_ngram_gpu:
@@ -1141,7 +1139,7 @@ class OmniGPUModelRunner(PrefixCacheRunnerMixin, GPUModelRunner):
        ):
            # Make sure padding doesn't exceed max_num_tokens
            assert num_tokens_padded <= self.max_num_tokens
-            model_kwargs = self._init_model_kwargs()
+            model_kwargs = self._init_model_kwargs(num_reqs=0)
            if self.supports_mm_inputs and not self.model_config.is_encoder_decoder:
                input_ids, inputs_embeds = self._prepare_mm_inputs(num_tokens_padded)

@@ -1152,7 +1150,7 @@ class OmniGPUModelRunner(PrefixCacheRunnerMixin, GPUModelRunner):
            elif self.enable_prompt_embeds:
                input_ids = None
                inputs_embeds = self.inputs_embeds.gpu[:num_tokens_padded]
-                model_kwargs = self._init_model_kwargs()
+                model_kwargs = self._init_model_kwargs(num_reqs=0)
            elif getattr(getattr(self, "model", None), "has_preprocess", False):
                # Capture CUDA graph with inputs_embeds path so replay reads
                # from the same buffer that _preprocess writes into.
@@ -1164,8 +1162,6 @@ class OmniGPUModelRunner(PrefixCacheRunnerMixin, GPUModelRunner):

            if self.uses_mrope:
                positions = self.mrope_positions.gpu[:, :num_tokens_padded]
-            elif getattr(self, "uses_xdrope_dim", 0) > 0:
-                positions = self.xdrope_positions.gpu[:, :num_tokens_padded]
            else:
                positions = self.positions[:num_tokens_padded]

@@ -1701,8 +1697,6 @@ class OmniGPUModelRunner(PrefixCacheRunnerMixin, GPUModelRunner):

        if self.uses_mrope:
            positions = self.mrope_positions.gpu[:, :num_input_tokens]
-        elif getattr(self, "uses_xdrope_dim", 0) > 0:
-            positions = self.xdrope_positions.gpu[:, :num_input_tokens]
        else:
            positions = self.positions[:num_input_tokens]
            if num_input_tokens > num_scheduled_tokens:
@@ -74,7 +74,7 @@ def _make_safe_get_rope(orig_get_rope):
            return None
        if not hasattr(mdl, "get_mrope_input_positions"):
            mdl.get_mrope_input_positions = types.MethodType(_default_mrope_positions, mdl)
-        return RopeState(num_dims=3, has_delta=True, **kwargs)
+        return RopeState(num_dims=3, **kwargs)

    return _safe_get_rope

@@ -117,6 +117,7 @@ class OmniARModelRunner(OmniGPUModelRunner):
        skip_attn_for_dummy_run: bool = False,
        is_profile: bool = False,
        context_len: int = 0,
+        valid_dummy_state_slots: bool = False,
    ) -> Any:
        if not dummy_run:
            self._handle_kv_transfer_pre(scheduler_output)
@@ -127,6 +128,7 @@ class OmniARModelRunner(OmniGPUModelRunner):
            skip_attn_for_dummy_run=skip_attn_for_dummy_run,
            is_profile=is_profile,
            context_len=context_len,
+            valid_dummy_state_slots=valid_dummy_state_slots,
        )

    # ------------------------------------------------------------------
@@ -756,7 +758,7 @@ class OmniAsyncOutput(AsyncModelRunnerOutput):
            del token_ids[num_tokens:]
        self.model_runner_output.sampled_token_ids = sampled_token_ids
        if self.sampling_mask_tensors is not None:
-            self.model_runner_output.sampling_masks = self.sampling_mask_tensors.tolists(self.num_sampled_tokens_np)
+            self.model_runner_output.sampling_masks = self.sampling_mask_tensors.tolists()
        if self.routed_experts_cpu is not None:
            self.model_runner_output.routed_experts = self.routed_experts_cpu.tolists()
        self.model_runner_output.sampled_token_ids_materialized = True
@@ -290,6 +290,7 @@ class OmniGenerationModelRunner(OmniGPUModelRunner):
        skip_attn_for_dummy_run: bool = False,
        is_profile: bool = False,
        context_len: int = 0,
+        valid_dummy_state_slots: bool = False,
    ) -> ModelRunnerOutput | IntermediateTensors | None:
        if not dummy_run:
            self._prepare_native_data_plane(scheduler_output)
@@ -428,6 +429,7 @@ class OmniGenerationModelRunner(OmniGPUModelRunner):
            dp_sync=dp_sync,
            ec_connector_output=ec_connector_output,
            routed_experts=None,
+            cudagraph_stats=None,
        )
        return None

@@ -402,6 +402,7 @@ class OmniGPUModelRunner(GPUModelRunner):
        skip_attn_for_dummy_run: bool = False,
        is_profile: bool = False,
        context_len: int = 0,
+        valid_dummy_state_slots: bool = False,
    ) -> Any:
        if not dummy_run:
            self._prepare_native_data_plane(scheduler_output)
@@ -478,7 +479,7 @@ class OmniGPUModelRunner(GPUModelRunner):
                max_query_len=batch_desc.max_query_len,
            )
            if not skip_attn_for_dummy_run:
-                block_tables, slot_mappings = self.prepare_dummy_attn(input_batch)
+                block_tables, slot_mappings = self.prepare_dummy_attn(input_batch, valid_dummy_state_slots)
                if context_len:
                    set_dummy_context(
                        input_batch,
@@ -608,6 +609,7 @@ class OmniGPUModelRunner(GPUModelRunner):
            dp_sync=dp_sync,
            ec_connector_output=ec_connector_output,
            routed_experts=routed_experts,
+            cudagraph_stats=None,
        )

        assert isinstance(hidden_states, torch.Tensor)