cherry-pick pr3067 (#3087 )

Signed-off-by: lyj-jjj <liuyingjun5@huawei.com>
[release v0.18.0.post1] Fix default sampling params for /v1/videos (#3021 )
2026-04-24 09:56:49 +08:00 · 2026-04-23 11:29:54 +08:00 · 2026-04-22 11:09:27 +08:00 · 2026-04-21 20:36:08 +08:00 · 2026-04-20 19:22:11 +08:00 · 2026-04-20 14:51:38 +08:00
155 changed files with 6657 additions and 5243 deletions
@@ -1,20 +0,0 @@
-steps:
-  - label: "Build and upload wheel"
-    key: "build-wheel"
-    agents:
-      queue: cpu_queue_release
-    commands:
-      - "curl -LsSf https://astral.sh/uv/install.sh | sh"
-      - 'export PATH="$HOME/.local/bin:$PATH"'
-      - "uv venv --python=3.12 && source .venv/bin/activate"
-      - "uv pip install --upgrade build"
-      - "python3 -m build"
-      - "bash .buildkite/scripts/upload-nightly-wheels.sh"
-
-  - label: "Generate and upload wheel indices"
-    depends_on: "build-wheel"
-    allow_dependency_failure: true
-    agents:
-      queue: small_cpu_queue_release
-    commands:
-      - "bash .buildkite/scripts/generate-and-upload-nightly-index.sh"
@@ -1,87 +0,0 @@
-#!/usr/bin/env bash
-
-set -ex
-
-# Generate and upload wheel indices for all vllm-omni wheels in the commit directory.
-# This script should run once after all wheels have been built and uploaded.
-# All paths are under the omni/ prefix in the vllm-wheels S3 bucket.
-
-# ======== setup ========
-
-BUCKET="vllm-wheels"
-INDICES_OUTPUT_DIR="indices"
-PYTHON="${PYTHON_PROG:-python3}"
-SUBPATH="omni/$BUILDKITE_COMMIT"
-S3_COMMIT_PREFIX="s3://$BUCKET/$SUBPATH/"
-
-# detect if python3.12+ is available
-has_new_python=$($PYTHON -c "print(1 if __import__('sys').version_info >= (3,12) else 0)")
-if [[ "$has_new_python" -eq 0 ]]; then
-    # use new python from docker
-    docker pull python:3-slim
-    PYTHON="docker run --rm -v $(pwd):/app -w /app python:3-slim python3"
-fi
-
-echo "Using python interpreter: $PYTHON"
-echo "Python version: $($PYTHON --version)"
-
-# ======== generate and upload indices ========
-
-# list all wheels in the commit directory
-echo "Existing wheels on S3:"
-aws s3 ls "$S3_COMMIT_PREFIX" || echo "(no objects found)"
-obj_json="objects.json"
-aws s3api list-objects-v2 --bucket "$BUCKET" --prefix "$SUBPATH/" --delimiter / --output json > "$obj_json"
-mkdir -p "$INDICES_OUTPUT_DIR"
-
-# HACK: we do not need regex module here, but it is required by pre-commit hook
-# To avoid any external dependency, we simply replace it back to the stdlib re module
-sed -i 's/import regex as re/import re/g' .buildkite/scripts/generate-nightly-index.py
-
-# Generate indices -- the version is just the commit hash (not omni/{commit})
-# because relative paths are computed between the index and wheel directories,
-# both of which live under the omni/ prefix in S3.
-$PYTHON .buildkite/scripts/generate-nightly-index.py \
-    --version "$BUILDKITE_COMMIT" \
-    --current-objects "$obj_json" \
-    --output-dir "$INDICES_OUTPUT_DIR" \
-    --comment "commit $BUILDKITE_COMMIT"
-
-# copy indices to /omni/{commit}/ unconditionally
-echo "Uploading indices to $S3_COMMIT_PREFIX"
-aws s3 cp --recursive "$INDICES_OUTPUT_DIR/" "$S3_COMMIT_PREFIX"
-
-# copy to /omni/nightly/ when NIGHTLY=1
-if [[ "${NIGHTLY:-}" == "1" ]]; then
-    echo "Uploading indices to overwrite /omni/nightly/"
-    aws s3 cp --recursive "$INDICES_OUTPUT_DIR/" "s3://$BUCKET/omni/nightly/"
-fi
-
-# detect version from any wheel in the commit directory
-first_wheel_key=$($PYTHON -c "import json; obj=json.load(open('$obj_json')); print(next((c['Key'] for c in obj.get('Contents', []) if c['Key'].endswith('.whl')), ''))")
-if [[ -z "$first_wheel_key" ]]; then
-    echo "Error: No wheels found in $S3_COMMIT_PREFIX"
-    exit 1
-fi
-first_wheel=$(basename "$first_wheel_key")
-aws s3 cp "s3://$BUCKET/${first_wheel_key}" "/tmp/${first_wheel}"
-version=$(unzip -p "/tmp/${first_wheel}" '**/METADATA' | grep '^Version: ' | cut -d' ' -f2)
-rm -f "/tmp/${first_wheel}"
-echo "Version in wheel: $version"
-pure_version="${version%%+*}"
-echo "Pure version (without variant): $pure_version"
-
-# re-generate and copy to /omni/{version}/ only if it does not have "dev" in the version
-if [[ "$version" != *"dev"* ]]; then
-    echo "Re-generating indices for /omni/$pure_version/"
-    rm -rf "${INDICES_OUTPUT_DIR:?}"
-    mkdir -p "$INDICES_OUTPUT_DIR"
-    # wheel-dir is overridden to be the commit directory, so that the indices point to the correct wheel path
-    $PYTHON .buildkite/scripts/generate-nightly-index.py \
-        --version "$pure_version" \
-        --wheel-dir "$BUILDKITE_COMMIT" \
-        --current-objects "$obj_json" \
-        --output-dir "$INDICES_OUTPUT_DIR" \
-        --comment "version $pure_version"
-    aws s3 cp --recursive "$INDICES_OUTPUT_DIR/" "s3://$BUCKET/omni/$pure_version/"
-fi
@@ -1,193 +0,0 @@
-#!/usr/bin/env python3
-# SPDX-License-Identifier: Apache-2.0
-# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
-
-import argparse
-import json
-import sys
-from dataclasses import asdict, dataclass
-from datetime import datetime
-from pathlib import Path
-from typing import Any
-from urllib.parse import quote
-
-import regex as re
-
-
-def normalize_package_name(name: str) -> str:
-    """Normalize package name per PEP 503."""
-    return re.sub(r"[-_.]+", "-", name).lower()
-
-
-if not sys.version_info >= (3, 12):
-    raise RuntimeError("This script requires Python 3.12 or higher.")
-
-INDEX_HTML_TEMPLATE = """<!DOCTYPE html>
-<html>
-  <!-- {comment} -->
-  <meta name="pypi:repository-version" content="1.0">
-  <body>
-{items}
-  </body>
-</html>
-"""
-
-
-@dataclass
-class WheelFileInfo:
-    package_name: str
-    version: str
-    build_tag: str | None
-    python_tag: str
-    abi_tag: str
-    platform_tag: str
-    filename: str
-
-
-def parse_from_filename(file: str) -> WheelFileInfo:
-    """
-    Parse wheel filename per PEP 427:
-        {package_name}-{version}(-{build_tag})?-{python_tag}-{abi_tag}-{platform_tag}.whl
-    """
-    wheel_file_re = re.compile(
-        r"^(?P<package_name>.+)-(?P<version>[^-]+?)(-(?P<build_tag>[^-]+))?-(?P<python_tag>[^-]+)-(?P<abi_tag>[^-]+)-(?P<platform_tag>[^-]+)\.whl$"
-    )
-    match = wheel_file_re.match(file)
-    if not match:
-        raise ValueError(f"Invalid wheel file name: {file}")
-
-    return WheelFileInfo(
-        package_name=match.group("package_name"),
-        version=match.group("version"),
-        build_tag=match.group("build_tag"),
-        python_tag=match.group("python_tag"),
-        abi_tag=match.group("abi_tag"),
-        platform_tag=match.group("platform_tag"),
-        filename=file,
-    )
-
-
-def generate_project_list(package_names: list[str], comment: str = "") -> str:
-    """Generate top-level PEP 503 project list HTML."""
-    href_tags = []
-    for name in sorted(package_names):
-        href_tags.append(f'    <a href="{name}/">{name}/</a><br/>')
-    return INDEX_HTML_TEMPLATE.format(items="\n".join(href_tags), comment=comment)
-
-
-def generate_package_index(
-    wheel_files: list[WheelFileInfo],
-    wheel_base_dir: Path,
-    index_base_dir: Path,
-    comment: str = "",
-) -> tuple[str, str]:
-    """Generate package index HTML and metadata JSON linking to wheel files."""
-    href_tags = []
-    metadata = []
-    for file in sorted(wheel_files, key=lambda x: x.filename):
-        relative_path = wheel_base_dir.relative_to(index_base_dir, walk_up=True) / file.filename
-        # handle '+' in URL; avoid double-encoding '/' and '%2B' (AWS S3 behavior)
-        file_path_quoted = quote(relative_path.as_posix(), safe=":%/")
-        href_tags.append(f'    <a href="{file_path_quoted}">{file.filename}</a><br/>')
-        file_meta = asdict(file)
-        file_meta["path"] = file_path_quoted
-        metadata.append(file_meta)
-    index_str = INDEX_HTML_TEMPLATE.format(items="\n".join(href_tags), comment=comment)
-    metadata_str = json.dumps(metadata, indent=2)
-    return index_str, metadata_str
-
-
-def generate_index(
-    whl_files: list[str],
-    wheel_base_dir: Path,
-    index_base_dir: Path,
-    comment: str = "",
-):
-    """
-    Generate PEP 503 index for all wheel files.
-
-    Output structure:
-        index_base_dir/
-            index.html          # project list linking to vllm-omni/
-            vllm-omni/
-                index.html      # package index linking to wheel files
-                metadata.json   # machine-readable metadata
-    """
-    parsed_files = [parse_from_filename(f) for f in whl_files]
-
-    if not parsed_files:
-        print("No wheel files found, skipping index generation.")
-        return
-
-    comment_str = f" ({comment})" if comment else ""
-    comment_tmpl = f"Generated on {datetime.now().isoformat()}{comment_str}"
-
-    # Group by normalized package name
-    packages: dict[str, list[WheelFileInfo]] = {}
-    for file in parsed_files:
-        name = normalize_package_name(file.package_name)
-        packages.setdefault(name, []).append(file)
-
-    print(f"Found packages: {list(packages.keys())}")
-
-    # Generate per-package index
-    for package, files in packages.items():
-        package_dir = index_base_dir / package
-        package_dir.mkdir(parents=True, exist_ok=True)
-        index_str, metadata_str = generate_package_index(files, wheel_base_dir, package_dir, comment)
-        with open(package_dir / "index.html", "w") as f:
-            f.write(index_str)
-        with open(package_dir / "metadata.json", "w") as f:
-            f.write(metadata_str)
-
-    # Generate top-level project list
-    project_list_str = generate_project_list(sorted(packages.keys()), comment_tmpl)
-    with open(index_base_dir / "index.html", "w") as f:
-        f.write(project_list_str)
-
-
-if __name__ == "__main__":
-    parser = argparse.ArgumentParser(description="Generate PEP 503 wheel index from S3 object listing.")
-    parser.add_argument("--version", type=str, required=True, help="Version string (e.g., commit hash)")
-    parser.add_argument("--current-objects", type=str, required=True, help="Path to JSON from S3 list-objects-v2")
-    parser.add_argument("--output-dir", type=str, required=True, help="Directory to write index files")
-    parser.add_argument("--wheel-dir", type=str, default=None, help="Wheel directory (defaults to --version)")
-    parser.add_argument("--comment", type=str, default="", help="Comment for generated HTML")
-
-    args = parser.parse_args()
-
-    version = args.version
-    if "\\" in version or "/" in version:
-        raise ValueError("Version string must not contain slashes or backslashes.")
-
-    output_dir = Path(args.output_dir)
-    output_dir.mkdir(parents=True, exist_ok=True)
-
-    with open(args.current_objects) as f:
-        current_objects: dict[str, list[dict[str, Any]]] = json.load(f)
-
-    wheel_files = [
-        item["Key"].split("/")[-1] for item in current_objects.get("Contents", []) if item["Key"].endswith(".whl")
-    ]
-
-    print(f"Found {len(wheel_files)} wheel files for version {version}: {wheel_files}")
-
-    # For release versions, filter to only matching non-dev wheels
-    PY_VERSION_RE = re.compile(r"^\d+\.\d+\.\d+([a-zA-Z0-9.+-]*)?$")
-    if PY_VERSION_RE.match(version):
-        wheel_files = [f for f in wheel_files if version in f and "dev" not in f]
-        print(f"Non-nightly version detected, wheel files used: {wheel_files}")
-    else:
-        print("Nightly version detected, keeping all wheel files.")
-
-    wheel_dir = (args.wheel_dir or version).strip().rstrip("/")
-    wheel_base_dir = Path(output_dir).parent / wheel_dir
-    index_base_dir = Path(output_dir)
-
-    generate_index(
-        whl_files=wheel_files,
-        wheel_base_dir=wheel_base_dir,
-        index_base_dir=index_base_dir,
-        comment=args.comment.strip(),
-    )
-    print(f"Successfully generated index in {output_dir}")
@@ -64,22 +64,15 @@ while true; do
 done

 echo "--- Pulling container"
-## Temporary change to use AMD Docker Hub to store the vllm-ci image
-# to bypass the rate limit issue with ECR Public Gallery.
-# TODO: @tjtanaa point back to ECR Public Gallery
-# once the amd agents are configured to use ECR Public Gallery.
-# image_name="public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:${BUILDKITE_COMMIT}-rocm-omni"
-image_name="rocm/vllm-ci:${BUILDKITE_COMMIT}-rocm-omni"
+image_name="public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:${BUILDKITE_COMMIT}-rocm-omni"
 container_name="rocm_${BUILDKITE_COMMIT}_$(tr -dc A-Za-z0-9 < /dev/urandom | head -c 10; echo)"

-# TODO: @tjtanaa uncomment this once the amd agents are configured to use ECR Public Gallery.
-# # Install AWS CLI to authenticate to ECR Public Gallery to get higher rate limit for pulling images
-# sudo apt-get update && sudo apt-get install -y awscli
-##  Use safe docker login helper to prevent race conditions
-# source "$(dirname "${BASH_SOURCE[0]}")/../docker_login_ecr_public.sh"
-# safe_docker_login_ecr_public
-
-## Pull the container from AMD Docker Hub
+# Install AWS CLI to authenticate to ECR Public Gallery to get higher rate limit for pulling images
+sudo apt-get update && sudo apt-get install -y awscli
+# Use safe docker login helper to prevent race conditions
+source "$(dirname "${BASH_SOURCE[0]}")/../docker_login_ecr_public.sh"
+safe_docker_login_ecr_public
+# Pull the container from ECR Public Gallery

 docker pull "${image_name}"

@@ -133,6 +126,8 @@ if [[ $commands == *"--shard-id="* ]]; then
        -e VLLM_ROCM_USE_AITER=0 \
        -e HIP_VISIBLE_DEVICES="${GPU}" \
        -e HF_TOKEN \
+        -e AWS_ACCESS_KEY_ID \
+        -e AWS_SECRET_ACCESS_KEY \
        -v "${HF_CACHE}:${HF_MOUNT}" \
        -e "HF_HOME=${HF_MOUNT}" \
        -e "PYTHONPATH=${MYPYTHONPATH}" \
@@ -165,6 +160,8 @@ else
          -e MIOPEN_DEBUG_CONV_GEMM=0 \
          -e VLLM_ROCM_USE_AITER=0 \
          -e HF_TOKEN \
+          -e AWS_ACCESS_KEY_ID \
+          -e AWS_SECRET_ACCESS_KEY \
          -v "${HF_CACHE}:${HF_MOUNT}" \
          -e "HF_HOME=${HF_MOUNT}" \
          -e "PYTHONPATH=${MYPYTHONPATH}" \
@@ -1,33 +0,0 @@
-#!/usr/bin/env bash
-
-set -ex
-
-# Upload a single wheel to S3 under the omni/ prefix.
-# Index generation is handled separately by generate-and-upload-nightly-index.sh.
-
-BUCKET="vllm-wheels"
-SUBPATH="omni/$BUILDKITE_COMMIT"
-S3_COMMIT_PREFIX="s3://$BUCKET/$SUBPATH/"
-
-# ========= collect & upload the wheel ==========
-
-# python3 -m build outputs to dist/ by default
-wheel_files=(dist/*.whl)
-
-# Check that exactly one wheel is found
-if [[ ${#wheel_files[@]} -ne 1 ]]; then
-  echo "Error: Expected exactly one wheel file in dist/, but found ${#wheel_files[@]}"
-  exit 1
-fi
-wheel="${wheel_files[0]}"
-
-echo "Uploading wheel: $wheel"
-
-# Extract the version from the wheel
-version=$(unzip -p "$wheel" '**/METADATA' | grep '^Version: ' | cut -d' ' -f2)
-echo "Version in wheel: $version"
-
-# Upload wheel to S3
-aws s3 cp "$wheel" "$S3_COMMIT_PREFIX"
-
-echo "Wheel uploaded to $S3_COMMIT_PREFIX. Index generation is handled by a separate step."
@@ -134,33 +134,15 @@ steps:
    - timeout 10m pytest -s -v tests/e2e/offline_inference/test_qwen3_omni.py
    - timeout 20m pytest -s -v tests/e2e/online_serving/test_qwen3_omni.py -m "advanced_model" --run-level "advanced_model"

- label: "Qwen3-TTS CustomVoice E2E Test"
+- label: "Qwen3-TTS E2E Test"
  agent_pool: mi325_2
  depends_on: amd-build
  mirror_hardwares: [amdproduction]
  grade: Blocking
  commands:
-    - |
-      timeout 20m bash -c '
-        export VLLM_LOGGING_LEVEL=DEBUG
-        export VLLM_WORKER_MULTIPROC_METHOD=spawn
-        export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
-        pytest -s -v tests/e2e/online_serving/test_qwen3_tts_customvoice.py -m "advanced_model" --run-level "advanced_model" && pytest -s -v tests/e2e/offline_inference/test_qwen3_tts_customvoice.py
-      '
-
- label: "Qwen3-TTS Base E2E Test"
-  agent_pool: mi325_2
-  depends_on: amd-build
-  mirror_hardwares: [amdproduction]
-  grade: Blocking
-  commands:
-    - |
-      timeout 20m bash -c '
-        export VLLM_LOGGING_LEVEL=DEBUG
-        export VLLM_WORKER_MULTIPROC_METHOD=spawn
-        export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
-        pytest -s -v tests/e2e/online_serving/test_qwen3_tts_base.py -m "advanced_model" --run-level "advanced_model" && pytest -s -v tests/e2e/offline_inference/test_qwen3_tts_base.py
-      '
+    - export VLLM_LOGGING_LEVEL=DEBUG
+    - export VLLM_WORKER_MULTIPROC_METHOD=spawn
+    - timeout 20m pytest -s -v tests/e2e/online_serving/test_qwen3_tts.py

 - label: "Diffusion Image Edit Test"
  agent_pool: mi325_1
@@ -124,8 +124,7 @@ steps:
  commands:
    - export VLLM_LOGGING_LEVEL=DEBUG
    - export VLLM_WORKER_MULTIPROC_METHOD=spawn
-    - export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
-    - timeout 20m pytest -s -v tests/e2e/online_serving/test_qwen3_tts_customvoice.py -m "core_model" --run-level "core_model"
+    - timeout 20m pytest -s -v tests/e2e/online_serving/test_qwen3_tts.py

 - label: "Diffusion Image Edit Test"
  agent_pool: mi325_1
@@ -208,52 +208,6 @@ steps:
          volumes:
            - "/fsx/hf_cache:/fsx/hf_cache"

-  - label: "Qwen3-TTS CustomVoice E2E Test"
-    depends_on: upload-merge-pipeline
-    commands:
-      - |
-        timeout 20m bash -c '
-          export VLLM_LOGGING_LEVEL=DEBUG
-          export VLLM_WORKER_MULTIPROC_METHOD=spawn
-          export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
-          pytest -s -v tests/e2e/online_serving/test_qwen3_tts_customvoice.py -m "advanced_model" --run-level "advanced_model" && pytest -s -v tests/e2e/offline_inference/test_qwen3_tts_customvoice.py
-        '
-    agents:
-      queue: "gpu_1_queue"
-    plugins:
-      - docker#v5.2.0:
-          image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
-          always-pull: true
-          propagate-environment: true
-          shm-size: "8gb"
-          environment:
-            - "HF_HOME=/fsx/hf_cache"
-          volumes:
-            - "/fsx/hf_cache:/fsx/hf_cache"
-
-  - label: "Qwen3-TTS Base E2E Test"
-    depends_on: upload-merge-pipeline
-    commands:
-      - |
-        timeout 20m bash -c '
-          export VLLM_LOGGING_LEVEL=DEBUG
-          export VLLM_WORKER_MULTIPROC_METHOD=spawn
-          export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
-          pytest -s -v tests/e2e/online_serving/test_qwen3_tts_base.py -m "advanced_model" --run-level "advanced_model" && pytest -s -v tests/e2e/offline_inference/test_qwen3_tts_base.py
-        '
-    agents:
-      queue: "gpu_1_queue"
-    plugins:
-      - docker#v5.2.0:
-          image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
-          always-pull: true
-          propagate-environment: true
-          shm-size: "8gb"
-          environment:
-            - "HF_HOME=/fsx/hf_cache"
-          volumes:
-            - "/fsx/hf_cache:/fsx/hf_cache"
-
  - label: "Omni Model Test with H100"
    timeout_in_minutes: 30
    depends_on: upload-merge-pipeline
@@ -50,15 +50,13 @@ steps:
                  path: /mnt/hf-cache
                  type: DirectoryOrCreate

-  - label: ":full_moon: Omni Model Test with L4"
-    timeout_in_minutes: 90
+  - label: ":full_moon: Omni Model Test"
+    timeout_in_minutes: 60
    depends_on: upload-nightly-pipeline
    if: build.env("NIGHTLY") == "1" || build.pull_request.labels includes "nightly-test"
    commands:
      - export VLLM_WORKER_MULTIPROC_METHOD=spawn
-      - export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
-      - pytest -s -v tests/examples/ -m "advanced_model and L4 and omni" --run-level "advanced_model"
-      - pytest -s -v tests/e2e/online_serving/test_*_expansion.py -m "advanced_model and L4 and omni" --run-level "advanced_model"
+      - pytest -s -v tests/examples/online_serving/test_qwen2_5_omni.py -m "advanced_model" --run-level "advanced_model"
    agents:
      queue: "gpu_4_queue" # g6.12xlarge instance on AWS, has 4 L4 GPU
    plugins:
@@ -219,6 +217,29 @@ steps:
                  path: /mnt/hf-cache
                  type: DirectoryOrCreate

+  - label: ":full_moon: Qwen3-TTS Non-Async-Chunk E2E Test"
+    timeout_in_minutes: 30
+    depends_on: upload-nightly-pipeline
+    if: build.env("NIGHTLY") == "1" || build.pull_request.labels includes "nightly-test"
+    commands:
+      - |
+        huggingface-cli download Qwen/Qwen3-TTS-12Hz-0.6B-CustomVoice
+        export VLLM_WORKER_MULTIPROC_METHOD=spawn
+        pytest -s -v tests/e2e/online_serving/test_qwen3_tts.py::TestQwen3TTSNoAsyncChunk
+    agents:
+      queue: "gpu_4_queue"
+    plugins:
+      - docker#v5.2.0:
+          image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
+          always-pull: true
+          propagate-environment: true
+          shm-size: "8gb"
+          environment:
+            - "HF_HOME=/fsx/hf_cache"
+            - "HF_TOKEN"
+          volumes:
+            - "/fsx/hf_cache:/fsx/hf_cache"
+
  - label: ":full_moon: Omni Model Perf Test & Test Case Statistics"
    key: nightly-omni-performance
    timeout_in_minutes: 180
@@ -227,7 +248,6 @@ steps:
    commands:
      - export VLLM_WORKER_MULTIPROC_METHOD=spawn
      - export BENCHMARK_DIR=tests/dfx/perf/results
-      - export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
      - pytest -s -v tests/dfx/perf/scripts/run_benchmark.py
      - buildkite-agent artifact upload "tests/dfx/perf/results/*.json"
      - python tools/nightly/buildkite_testcase_statistics.py -o tests/dfx/perf/results/buildkite_testcase_statistics.html
@@ -303,13 +303,13 @@ steps:
    commands:
      - |
        timeout 20m bash -c '
+          huggingface-cli download Qwen/Qwen3-TTS-12Hz-0.6B-CustomVoice
          export VLLM_LOGGING_LEVEL=DEBUG
          export VLLM_WORKER_MULTIPROC_METHOD=spawn
-          export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
-          pytest -s -v tests/e2e/online_serving/test_qwen3_tts_customvoice.py -m "core_model" --run-level "core_model"
+          pytest -s -v tests/e2e/online_serving/test_qwen3_tts.py -k "not NoAsyncChunk"
        '
    agents:
-      queue: "gpu_1_queue"
+      queue: "gpu_4_queue"
    plugins:
      - docker#v5.2.0:
          image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
@@ -3,7 +3,7 @@
   Last synced: 2025-12-15
   Modifications: Removed unused CUDA/NVIDIA logic, keeping only AMD tests
 #}
-{% set docker_image_amd = "rocm/vllm-ci:$BUILDKITE_COMMIT-rocm-omni" %}
+{% set docker_image_amd = "public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT-rocm-omni" %}
 {% set default_working_dir = "/app/vllm-omni" %}

  - group: "AMD Tests"
@@ -13,6 +13,7 @@
        depends_on: ~
        soft_fail: false
        commands:
+          - "source .buildkite/scripts/docker_login_ecr_public.sh && safe_docker_login_ecr_public"
          - "docker build -f docker/Dockerfile.rocm -t {{ docker_image_amd }} --target test --build-arg ARG_PYTORCH_ROCM_ARCH=gfx942 --progress plain ."
          - "docker push {{ docker_image_amd }}"
        key: "amd-build"
@@ -21,15 +22,13 @@
        retry:
          automatic:
            - exit_status: -1  # Agent was lost
-              limit: 2
+              limit: 1
            - exit_status: -10  # Agent was lost
-              limit: 2
-            - exit_status: 128  # Git connectivity issues
-              limit: 2
+              limit: 1
            - exit_status: 1  # Machine occasionally fail
              limit: 1
        agents:
-          queue: amd-cpu
+          queue: cpu_queue_premerge

    {% for step in steps %}
    {% if step.mirror_hardwares and mirror_hw in step.mirror_hardwares %}
@@ -54,28 +54,21 @@ nav:
  - Features:
    - Sleep Mode: features/sleep_mode.md
    - Diffusion Features:
-      - Overview: user_guide/diffusion_features.md
-      - Feature Compatibility: user_guide/feature_compatibility.md
-      - Cache Acceleration:
-        - TeaCache: user_guide/diffusion/cache_acceleration/teacache.md
-        - Cache-DiT: user_guide/diffusion/cache_acceleration/cache_dit.md
+      - Acceleration Overview: user_guide/diffusion_acceleration.md
+      - TeaCache: user_guide/diffusion/teacache.md
+      - Cache-DiT: user_guide/diffusion/cache_dit_acceleration.md
      - Quantization:
        - Overview: user_guide/diffusion/quantization/overview.md
        - FP8: user_guide/diffusion/quantization/fp8.md
        - Int8: user_guide/diffusion/quantization/int8.md
        - GGUF: user_guide/diffusion/quantization/gguf.md
-      - Parallelism:
-        - Overview: user_guide/diffusion/parallelism/overview.md
-        - CFG Parallel: user_guide/diffusion/parallelism/cfg_parallel.md
-        - Expert Parallel: user_guide/diffusion/parallelism/expert_parallel.md
-        - Hybrid Sharded Data Parallel: user_guide/diffusion/parallelism/hsdp.md
-        - Sequence Parallel: user_guide/diffusion/parallelism/sequence_parallel.md
-        - Tensor Parallel: user_guide/diffusion/parallelism/tensor_parallel.md
-        - VAE Patch Parallel: user_guide/diffusion/parallelism/vae_patch_parallel.md
+      - Frame Interpolation: user_guide/diffusion/frame_interpolation.md
+      - Step Execution: user_guide/diffusion/step_execution.md
+      - Parallelism Acceleration: user_guide/diffusion/parallelism_acceleration.md
      - CPU Offloading: user_guide/diffusion/cpu_offload_diffusion.md
      - LoRA: user_guide/diffusion/lora.md
+      - Hybrid Sharded Data Parallel: design/feature/hsdp.md
      - Custom Pipeline: features/custom_pipeline.md
-      - Step Execution: user_guide/diffusion/step_execution.md
    - ComfyUI: features/comfyui.md
 - Developer Guide:
  - General:
@@ -100,8 +93,6 @@ nav:
      - design/feature/cfg_parallel.md
      - design/feature/sequence_parallel.md
      - design/feature/tensor_parallel.md
-      - design/feature/vae_parallel.md
-      - design/feature/hsdp.md
      - design/feature/cache_dit.md
      - design/feature/teacache.md
      - design/feature/async_chunk_design.md
@@ -20,4 +20,7 @@ For introduction, please check [Introduction for stage config](./stage_configs.m

 ## Optimization Features

- **[Diffusion Features Overview](../user_guide/diffusion_features.md)** - Complete overview of all diffusion model features and supported models
+- **[TeaCache Configuration](../user_guide/diffusion/teacache.md)** - Enable TeaCache adaptive caching for DiT models to achieve 1.5x-2.0x speedup with minimal quality loss
+- **[Cache-DiT Configuration](../user_guide/diffusion/cache_dit_acceleration.md)** - Enable Cache-DiT as cache acceleration backends for DiT models
+- **[Parallelism Configuration](../user_guide/diffusion/parallelism_acceleration.md)** - Enable parallelism (e.g., sequence parallelism) for for DiT models
+- **[CPU Offloading](../user_guide/diffusion/cpu_offload_diffusion.md)** - Enable CPU offloading (model-level and layerwise) for for DiT models
@@ -548,19 +548,103 @@ L4 level testing is a comprehensive quality audit before a version release. It e
 -   ***Execution Environment***: ***GPU*** server clusters to meet the resource demands of performance testing.
 -   ***Script Example***:

-??? example "Test Examples: Documentation Example Tests"
+???+ example "Test Examples"

-    --8<-- "docs/contributing/ci/test_examples/l4_doc_example_tests.inc.md"
+    When adding L4-level ***documentation example Tests***, please pay attention to the following guides.

-??? example "Test Examples: Performance Tests"
+    --8<-- "docs/contributing/ci/test_examples/doc_example_tests.inc.md"

-    --8<-- "docs/contributing/ci/test_examples/l4_performance_tests.inc.md"
+    When you want to add L4-level ***performance test*** cases, you can refer to the following format for case addition in tests/dfx/perf/tests/test.json:

-??? example "Test Examples: Functionality Tests"
+    ```JSON
+    {
+        "test_name": "test_qwen3_omni",
+        "server_params": {
+            "model": "Qwen/Qwen3-Omni-30B-A3B-Instruct",
+            "stage_config_name": "qwen3_omni.yaml"
+        },
+        "benchmark_params": [
+            {
+                "dataset_name": "random",
+                "num_prompts": [10, 20],
+                "request_rate": [0.5, 1],
+                "random_input_len": 2500,
+                "random_output_len": 900,
+                "ignore_eos": true,
+                "percentile-metrics": "ttft,tpot,itl,e2el,audio_rtf,audio_ttfp,audio_duration",
+                "baseline": {
+                    "mean_ttft_ms": 100000,
+                    "mean_audio_ttfp_ms": 100000,
+                    "mean_audio_rtf": 100000
+                }
+            }
+        ]
+    }
+    ```

-    --8<-- "docs/contributing/ci/test_examples/l4_functionality_tests.inc.md"
+    **Parameter Explanation**

-   ***Run Command***: (Specific commands would depend on the performance testing tool and configuration defined in `nightly.json`).
+    *Overview*
+
+    | Field            | Required | Description                                                     |
+    | ---------------- | -------- | --------------------------------------------------------------- |
+    | test_name        | Yes      | Unique identifier for the test case                             |
+    | server_params    | Yes      | Server-side configuration parameters                            |
+    | benchmark_params | Yes      | Benchmark running parameters (supports multiple configurations) |
+
+    **server_params Configuration**
+
+    *Basic Parameters*
+
+    | Parameter         | Required | Example                            | Description                   |
+    | ----------------- | -------- | ---------------------------------- | ----------------------------- |
+    | model             | Yes      | "Qwen/Qwen3-Omni-30B-A3B-Instruct" | Model name or path            |
+    | stage_config_name | Yes      | "qwen3_omni.yaml"                  | Stage configuration file name |
+
+    *Dynamic Configuration (update/delete)*
+
+    Supports incremental modifications based on the basic configuration:
+
+    | Operation | Description                          |
+    | --------- | ------------------------------------ |
+    | update    | Update or add configuration items    |
+    | delete    | Delete specified configuration items |
+
+    ***Example***:
+
+    ```
+    "update": {
+        "async_chunk": true,  // Enable asynchronous chunk processing
+        "stage_args": {
+            "0": {
+                "engine_args.custom_process_next_stage_input_func": "vllm_omni.model_executor.stage_input_processors.qwen3_omni.thinker2talker_async_chunk"
+            }
+        }
+    },
+    "delete": {
+        "stage_args": {
+            "2": ["custom_process_input_func"]  // Delete this configuration for stage 2
+        }
+    }
+    ```
+
+    **benchmark_params Configuration**
+
+    You can add any benchmark running parameters you need here. For all optional parameters, refer to the [benchmark documentation](https://github.com/vllm-project/vllm-omni/blob/main/docs/cli/bench/serve.md). General modifications are as follows:
+
+    1.  Change the ---xxx-xx-xx running parameters to xxx_xx_xx format and fill them as keys in the JSON file.
+    2.  For boolean variables in the running parameters, modify them to forms such as ignore_eos: true/false and fill them into the JSON file.
+    3.  Add the baseline parameter to specify the required validation values, ensuring the validation metric names match those in the result.json generated by the benchmark.
+    4.  The qps and concurrency modes are mutually exclusive. For detailed explanations, see the table below:
+
+    | Parameter       | Type        | Required | Example/Values  | Description                                                                                                                                                                                                                                                          |
+    | --------------- | ----------- | -------- | --------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+    | num_prompts     | int / array | Yes      | 10,[10, 20, 30] | Number of requests. Supports single values or arrays. If a single value is used, it will be automatically expanded to match the number of qps or max_concurrency, e.g., [10,10,10]. If an array is used, its length must match the number of qps or max_concurrency. |
+    | request_rate    | int / array | No       | 1, [1, 2, 3]    | Queries per second. Supports single values or arrays. If a single value is used, it will be automatically expanded to match the number of num_prompts, e.g., [1,1,1]. If an array is used, its length must match the number of num_prompts.                          |
+    | max_concurrency | int / array | No       | 1, [1, 2, 3]    | Queries per second. Supports single values or arrays. If a single value is used, it will be automatically expanded to match the number of num_prompts, e.g., [1,1,1]. If an array is used, its length must match the number of num_prompts.                          |
+    </details>
+
+    -   -   ***Run Command***: (Specific commands would depend on the performance testing tool and configuration defined in `nightly.json`).

 ## Chapter 4: L5 Level Testing - Stability and Reliability Testing

@@ -1,106 +1,10 @@
 # CI Failures

-## Overview of CI Checks
+What should I do when a CI job fails on my PR, but I don't think my PR caused the failure?

-When you open a PR against vLLM-Omni, several CI checks run automatically:

-| Check | Platform | What it does |
-| ----- | -------- | ------------ |
-| **pre-commit** | GitHub Actions | Runs linting (Ruff), formatting, spell-checking (typos), and YAML validation. |
-| **Build Wheel** | GitHub Actions | Builds Python wheels for Python 3.11 and 3.12 on Ubuntu. Skipped for docs-only or Markdown-only changes (controlled by `paths-ignore` in the workflow). |
-| **DCO** | GitHub | Verifies every commit has a `Signed-off-by` line. |
-| **docs/readthedocs.org:vllm-omni** | Read the Docs | Builds the MkDocs documentation site. |
-| **buildkite/vllm-omni** | Buildkite | Runs GPU-based tests on NVIDIA CUDA hardware (L4, H100). |
-| **buildkite/vllm-omni-amd** | Buildkite | Runs GPU-based tests on AMD ROCm hardware (MI325). |
-| **buildkite/vllm-omni-intel** | Buildkite | Runs GPU-based tests on Intel XPU hardware (Intel Arc BMG). |
+## Common Case 1:

-## Step 1: Identify the Failing Check
+🚨 Error: The command was interrupted by a signal: signal: terminated

-Click the **Details** link next to the failing check on your PR to open the build log. The most common failures fall into these categories:
-
-### pre-commit failures
-
-These are typically formatting or linting issues introduced by your PR. Fix them locally:
-
-```bash
-uv pip install pre-commit
-pre-commit run --all-files
-```
-
-Then commit the fixes and push.
-
-### DCO failures
-
-Every commit must include a `Signed-off-by` line. If you forgot, amend your commits:
-
-```bash
-git commit --amend -s
-git push --force-with-lease
-```
-
-For multiple commits, use an interactive rebase to add the sign-off to each one.
-
-### Read the Docs failures
-
-The documentation build uses MkDocs with `fail_on_warning: true`, so even a minor warning (not just errors) will cause the build to fail. To reproduce locally:
-
-```bash
-uv pip install -e ".[docs]"
-mkdocs build --strict
-```
-
-Common causes include broken cross-references, invalid admonition syntax, or missing files referenced by `--8<--` includes.
-
-### Buildkite failures
-
-Buildkite runs GPU tests in Docker containers. These are the most complex checks and can fail for reasons unrelated to your PR (infrastructure issues, flaky tests, etc.). See the sections below for how to investigate.
-
-## Step 2: Check if the Failure Is a Known Issue
-
-Before spending time debugging, check whether the failure already exists on the `main` branch:
-
-1. **Look at the Buildkite build log** — the test name and error message are usually enough to identify the issue.
-2. **Check recent CI runs on `main`** — if the same test is failing there, the failure is not caused by your PR.
-3. **Search existing issues** — look for open issues in the [vllm-omni issue tracker](https://github.com/vllm-project/vllm-omni/issues) with the test name or error message.
-
-If the failure is already tracked, leave a comment on your PR noting that the failure is pre-existing and link the issue.
-
-## Step 3: Investigate the Failure
-
-If the failure appears to be new, investigate whether your changes caused it.
-
-### Reading Buildkite Logs
-
-1. Click **Details** next to the Buildkite check on your PR.
-2. Find the failing step in the pipeline (e.g., "Diffusion Model Test", "Simple Unit Test").
-3. Expand the step to see the full test output with the traceback.
-
-### Running Tests Locally
-
-For instructions on running tests locally (including specific test files, functions, and markers), see the [Running Tests](https://docs.vllm.ai/projects/vllm-omni/en/latest/contributing/ci/test_guide/#running-tests) section in the Test Guide.
-
-## Step 4: Raise an Issue or Fix It
-
-### If the failure is pre-existing (not caused by your PR)
-
-1. **Raise a new issue** if one doesn't already exist, using the title format:
-   `[CI Failure]: [job-name] - [test-path]`
-2. Include the error message, relevant log excerpts, and the commit hash where the failure occurs (e.g., "Still failing on main as of commit `abc1234`").
-3. Leave a comment on your PR linking to the issue and noting that the failure is unrelated to your changes.
-
-### If the failure is caused by your PR
-
-1. Fix the issue in your branch and push the update.
-2. If the fix is non-trivial, consider adding a test to prevent regression.
-
-## Common Failure Patterns
-
-| Symptom | Likely Cause | Fix |
-| ------- | ------------ | --- |
-| `ruff` or formatting errors | Code style violation | Run `pre-commit run --all-files` |
-| `Signed-off-by` missing | DCO check | Amend commits with `git commit --amend -s` |
-| MkDocs build warning | Broken docs reference | Run `mkdocs build --strict` locally |
-| `OOM` or `CUDA out of memory` | Test exceeds GPU memory | Check if your changes increased memory usage; use `--vae_use_slicing` / `--vae_use_tiling` for diffusion tests |
-| Import errors | Missing or changed dependency | Check `pyproject.toml` and make sure dependencies are correct |
-| Timeout (step exceeded N minutes) | Test is too slow or hangs | Profile the test; check for infinite loops or deadlocks |
-| `Agent lost` in Buildkite | Infrastructure issue (not your fault) | Re-trigger the build; comment on your PR |
+Reason: The test is terminated due to exceed the time limits. Sometimes rebuild the test will pass.
@@ -1,46 +0,0 @@
-**Scope**
-
-For diffusion models, the L4 functionality test covers all or common *diffusion features* that are supported by this model, including several [parallelism acceleration methods](https://docs.vllm.ai/projects/vllm-omni/en/latest/user_guide/diffusion/parallelism_acceleration/), [CPU offloading](https://docs.vllm.ai/projects/vllm-omni/en/latest/user_guide/diffusion/cpu_offload_diffusion/), [TeaCache](https://docs.vllm.ai/projects/vllm-omni/en/latest/user_guide/diffusion/teacache/) and [Cache-DiT](https://docs.vllm.ai/projects/vllm-omni/en/latest/user_guide/diffusion/cache_dit_acceleration/) cache backends, [quantization methods](https://docs.vllm.ai/projects/vllm-omni/en/latest/user_guide/diffusion/quantization/overview/).
-
-**Test Case Design**
-
-For a *high priority* model (currently listed in [issue #1832](https://github.com/vllm-project/vllm-omni/issues/1832)), we use several test cases, each with multiple features turned on, so that each supported feature is tested in at least one test case. This is to relieve the GPU workload on the CI machine. The suggested test case combination is as follows:
-
- If the model can fit into 4 L4 GPU (with quantization and tensor parallel always on) (20GB RAM each)
-    - (1 GPU) TeaCache + Layerwise CPU offloading + GGUF
-    - (4 GPUs) CacheDiT + Ulysses=2 + TP=2 + VAE=2 + FP8
-    - (4 GPUs) CacheDiT + Ring=2 + HSDP=2 + VAE=2 + GGUF
-    - (4 GPUs) TeaCache + CFG=2 + TP=2 + VAE=2 + FP8
- Otherwise, consider 2 H100 GPU environment (80GB RAM each) with the following tests
-    - (1 GPU) TeaCache + Layerwise CPU offloading + GGUF
-    - (2 GPUs) CacheDiT + Ulysses=2 + FP8
-    - (2 GPUs) CacheDiT + Ring=2 + GGUF
-    - (2 GPUs) TeaCache + CFG=2 + FP8
-    - (2 GPUs) CacheDiT + TP=2 + VAE=2 + FP8
-    - (2 GPUs) CacheDiT + HSDP=2 + VAE=2 + GGUF
- If 2 H100 GPU cannot handle the model either (e.g., HunyuanImage 3.0)
-    - Still design tests and feature combinations that can best fit real-world scenario.
-    - Do not include it in CI (or exclude it from the CI's filtering criteria). Instead, relevant PR authors are suggested to run these tests locally.
- Fallback plan
-    - If the model does not support layerwise CPU offloading, replace the corresponding test case with module-wise offloading
-    - If the model only supports specific or no caching feature, use this option or remove this option in all test cases.
-    - If the model only supports specific or no quantization feature, use this option or remove this option in all test cases.
-    - If the model does not support certain other features, remove this option from that test case. If, consequently, the coverage of this modified test case completely overlaps with others, remove this test case.
-
-For a *normal priority* model, further reduce the number of test cases.
-
- Only write one or two test cases for the most common feature combinations.
- The author can explore themselves to see which feature combination balances output quality and performance. Alternatively, the author can refer to any example code in the PR that adds the model, or the example code in the PR that adds a feature (if the code involves this model of interest).
-
-Currently all the features are available in online serving mode. Hence, only need to add `tests/e2e/online_serving/test_{model}_expansion.py`.
-
-**Code Style**
-
- Validation: test that the multimodal output files of your model have the correct shapes. `OpenAIClientHandler.send_diffusion_request` should have taken care of this.
- Test marks: always add `advanced_model` and `diffusion`. Add GPU-related marks if needed. Ref: [Markers for Tests](https://docs.vllm.ai/projects/vllm-omni/en/latest/contributing/ci/tests_markers/).
- To maximize code reuse, you may refer to
-    - `tests/conftest.py` for `omni_server` (running server in subprocess) and `openai_client` fixtures (sending requests and validating output), `generate_synthetic_image` and `assert_XXX_valid` helper.
-    - `tests/utils.py` for `@hardware_test(...)` and `hardware_marks`.
-    - [Parametrizing tests (pytest doc)](https://docs.pytest.org/en/stable/example/parametrize.html) to reuse test function implementation for different cases.
- Doc: add a concise docstring for each test function.
- Reference L4 test implementation: [tests/e2e/online_serving/test_qwen_image_edit_expansion.py](https://github.com/vllm-project/vllm-omni/blob/main/tests/e2e/online_serving/test_qwen_image_edit_expansion.py).
@@ -1,89 +0,0 @@
-When you want to add L4-level ***performance test*** cases, you can refer to the following format for case addition in tests/dfx/perf/tests/test.json:
-
-```JSON
-{
-    "test_name": "test_qwen3_omni",
-    "server_params": {
-        "model": "Qwen/Qwen3-Omni-30B-A3B-Instruct",
-        "stage_config_name": "qwen3_omni.yaml"
-    },
-    "benchmark_params": [
-        {
-            "dataset_name": "random",
-            "num_prompts": [10, 20],
-            "max_concurrency": [1, 4],
-            "random_input_len": 2500,
-            "random_output_len": 900,
-            "ignore_eos": true,
-            "percentile-metrics": "ttft,tpot,itl,e2el,audio_rtf,audio_ttfp,audio_duration",
-            "baseline": {
-                "mean_ttft_ms": [500, 800],
-                "mean_audio_ttfp_ms": [2000, 3500],
-                "mean_audio_rtf": [0.25, 0.35]
-            }
-        }
-    ]
-}
-```
-
-**Parameter Explanation**
-
-*Overview*
-
-| Field            | Required | Description                                                     |
-| ---------------- | -------- | --------------------------------------------------------------- |
-| test_name        | Yes      | Unique identifier for the test case                             |
-| server_params    | Yes      | Server-side configuration parameters                            |
-| benchmark_params | Yes      | Benchmark running parameters (supports multiple configurations) |
-
-**`server_params` Configuration**
-
-*Basic Parameters*
-
-| Parameter         | Required | Example                            | Description                   |
-| ----------------- | -------- | ---------------------------------- | ----------------------------- |
-| model             | Yes      | "Qwen/Qwen3-Omni-30B-A3B-Instruct" | Model name or path            |
-| stage_config_name | Yes      | "qwen3_omni.yaml"                  | Stage configuration file name |
-
-*Dynamic Configuration (update/delete)*
-
-Supports incremental modifications based on the basic configuration:
-
-| Operation | Description                          |
-| --------- | ------------------------------------ |
-| update    | Update or add configuration items    |
-| delete    | Delete specified configuration items |
-
-**Example**:
-
-```
-"update": {
-    "async_chunk": true,  // Enable asynchronous chunk processing
-    "stage_args": {
-        "0": {
-            "engine_args.custom_process_next_stage_input_func": "vllm_omni.model_executor.stage_input_processors.qwen3_omni.thinker2talker_async_chunk"
-        }
-    }
-},
-"delete": {
-    "stage_args": {
-        "2": ["custom_process_input_func"]  // Delete this configuration for stage 2
-    }
-}
-```
-
-**`benchmark_params` Configuration**
-
-You can add any benchmark running parameters you need here. For all optional parameters, refer to the [benchmark documentation](https://github.com/vllm-project/vllm-omni/blob/main/docs/cli/bench/serve.md). General modifications are as follows:
-
-1.  Change the --xxx-xx-xx running parameters to xxx_xx_xx format and fill them as keys in the JSON file.
-2.  For boolean variables in the running parameters, modify them to forms such as ignore_eos: true/false and fill them into the JSON file.
-3.  Optionally add a `baseline` object (see **Baseline thresholds** below). If you omit `baseline` or leave it empty, the performance test still runs but does not assert metric thresholds from this field.
-4.  The qps and concurrency modes are recommended to be mutually exclusive. For detailed explanations, see the table below:
-
-| Parameter       | Type        | Required | Example/Values  | Description                                                                                                                                                                                                                                                          |
-| --------------- | ----------- | -------- | --------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| num_prompts     | int / array | Yes      | 10,[10, 20, 30] | Number of requests. Supports single values or arrays. If a single value is used, it will be automatically expanded to match the number of qps or max_concurrency, e.g., [10,10,10]. If an array is used, its length must match the number of qps or max_concurrency. |
-| request_rate    | float / array | No  | 0.5, [0.5, 1, inf] | Queries per second. Supports single values or arrays. If a single value is used, it will be automatically expanded to match the number of num_prompts, e.g., [1,1,1]. If an array is used, its length must match the number of num_prompts.                          |
-| max_concurrency | int / array | No       | 1, [1, 2, 3]    | Maximum concurrent in-flight requests. Same array / expansion rules as `request_rate` (mutually exclusive with QPS mode).                                                                                                                                                                                                             |
-| baseline        | object      | No       | see above       | Optional per-metric thresholds; keys must match benchmark output fields. Scalar, list (per sweep step), or object (keyed by concurrency or QPS string).  
@@ -653,7 +653,25 @@ For a fair comparison, keep the same **prompt**, **seed**, **resolution**, **num

 To ensure project maintainability and sustainable development, please submit test code (unit tests, system tests, or end-to-end tests) alongside their code changes.

-For comprehensive testing guidelines and the definition of test levels (L1-L5), please refer to the [Multi-Level Automated Testing System Documentation](../ci/CI_5levels.md). You are at least required to add an L4 *functionality* test described in that document.
+For comprehensive testing guidelines and the definition of test levels (L1-L5), please refer to the [Test File Structure and Style Guide](../ci/tests_style.md).
+The following tests are required to add:
+
+- L4 test of the model's full *functionality* (i.e., all the *diffusion features* that are supported by this model), including several [parallelism acceleration methods](https://docs.vllm.ai/projects/vllm-omni/en/latest/user_guide/diffusion/parallelism_acceleration/), [CPU offloading](https://docs.vllm.ai/projects/vllm-omni/en/latest/user_guide/diffusion/cpu_offload_diffusion/), [TeaCache](https://docs.vllm.ai/projects/vllm-omni/en/latest/user_guide/diffusion/teacache/) and [Cache-DiT](https://docs.vllm.ai/projects/vllm-omni/en/latest/user_guide/diffusion/cache_dit_acceleration/) cache backends, [quantization methods](https://docs.vllm.ai/projects/vllm-omni/en/latest/user_guide/diffusion/quantization/overview/).
+  - Test cases: Currently all the features are available in online serving mode. Hence, only need to add `tests/e2e/online_serving/test_{model}_expansion.py`. The following test cases shall cover all features:
+    - 1 GPU: TeaCache & GGUF (or fallback to FP8, or disable it) & Layer-wise CPU offloading (or fallback to Module-wise)
+    - 2 GPUs: Cache-DiT & FP8 (or fallback to GGUF, or disable it) & Ulysses = 2
+    - 2 GPUs: Cache-DiT & GGUF (or fallback to FP8, or disable it) & Ring = 2
+    - 2 GPUs: TeaCache & FP8 (or fallback to GGUF, or disable it) & CFG Parallel = 2
+    - 2 GPUs: Cache-DiT & FP8 (or fallback to GGUF, or disable it) & Tensor Parallel = 2 & VAE Patch Parallel = 2
+    - 2 GPUs: Cache-DiT & GGUF (or fallback to FP8, or disable it) & HSDP = 2 & VAE Patch Parallel = 2
+  - Validation: test that the multimodal output files of your model have the correct shapes.
+  - Test marks: always add `advanced_model` and `diffusion`. Add `parallel` and GPU-related marks if needed. Ref: [Markers for Tests](https://docs.vllm.ai/projects/vllm-omni/en/latest/contributing/ci/tests_markers/)
+  - To maximize code reuse, you may refer to
+    - `tests/conftest.py` for `omni_server` and `openai_client` fixtures, `generate_synthetic_image` and `assert_XXX_valid` helper.
+    - `tests/utils.py` for `@hardware_test(...)` and `hardware_marks`.
+    - [Parametrizing tests (pytest doc)](https://docs.pytest.org/en/stable/example/parametrize.html) to reuse test function implementation for different cases.
+  - Doc: add a concise dostring for each test function.
+  - Reference L4 test implementation: [tests/e2e/online_serving/test_qwen_image_edit_expansion.py](https://github.com/vllm-project/vllm-omni/blob/main/tests/e2e/online_serving/test_qwen_image_edit_expansion.py).

 ---

@@ -256,7 +256,7 @@ cache_config={
 }
 ```

-Check the [user guide for cache_dit](../../user_guide/diffusion/cache_acceleration/cache_dit.md) for more adjustable parameters.
+Check the [user guide for cache_dit](../../user_guide/diffusion/cache_dit_acceleration.md) for more adjustable parameters.

 ---

@@ -34,7 +34,7 @@ vLLM-omni provides `CFGParallelMixin` that encapsulates all CFG parallel logic.
 | Method | Purpose | Automatic Behavior |
 |--------|---------|-------------------|
 | [`predict_noise_maybe_with_cfg()`](https://docs.vllm.ai/projects/vllm-omni/en/latest/api/vllm_omni/diffusion/distributed/cfg_parallel/) | Predict noise with CFG | Detects parallel mode, distributes computation, gathers results |
-| [`scheduler_step_maybe_with_cfg()`](https://docs.vllm.ai/projects/vllm-omni/en/latest/api/vllm_omni/diffusion/distributed/cfg_parallel/) | Step scheduler | All ranks step locally (no broadcast needed) |
+| [`scheduler_step_maybe_with_cfg()`](https://docs.vllm.ai/projects/vllm-omni/en/latest/api/vllm_omni/diffusion/distributed/cfg_parallel/) | Step scheduler with sync | Rank 0 steps, broadcasts latents to all ranks |
 | [`combine_cfg_noise()`](https://docs.vllm.ai/projects/vllm-omni/en/latest/api/vllm_omni/diffusion/distributed/cfg_parallel/) | Combine positive/negative | Applies CFG formula with optional normalization |
 | [`predict_noise()`](https://docs.vllm.ai/projects/vllm-omni/en/latest/api/vllm_omni/diffusion/distributed/cfg_parallel/) | Forward pass wrapper | Override for custom transformer calls |
 | [`cfg_normalize_function()`](https://docs.vllm.ai/projects/vllm-omni/en/latest/api/vllm_omni/diffusion/distributed/cfg_parallel/) | Normalize CFG output | Override for custom normalization |
@@ -47,7 +47,7 @@ vLLM-omni provides `CFGParallelMixin` that encapsulates all CFG parallel logic.
  - Rank 0 computes positive prompt prediction
  - Rank 1 computes negative prompt prediction
  - Results are gathered via `all_gather()`
-  - All ranks compute CFG combine locally (deterministic, identical results)
+  - Combined on rank 0 using CFG formula

 - **Sequential mode** (when `cfg_world_size == 1`):
  - Single rank computes both positive and negative predictions
@@ -55,7 +55,14 @@ vLLM-omni provides `CFGParallelMixin` that encapsulates all CFG parallel logic.

 `scheduler_step_maybe_with_cfg()` ensures consistent latent states across all ranks:

- All ranks compute the scheduler step locally — no broadcast needed because `predict_noise_maybe_with_cfg` already ensures all ranks have identical noise predictions after `all_gather` + local combine.
+- **CFG-Parallel mode**:
+  - Only rank 0 performs the scheduler step (applies noise prediction to update latents)
+  - Updated latents are broadcast to all other ranks via `broadcast()`
+  - All ranks maintain synchronized latent states for the next iteration
+
+- **Sequential mode**:
+  - Single rank directly performs the scheduler step
+  - No synchronization needed

 ---

@@ -170,70 +177,6 @@ class LongCatImagePipeline(nn.Module, CFGParallelMixin):
        # return noise_pred
 ```

-
-### Override `combine_cfg_noise()` for Multi-Output Models
-
-When `predict_noise()` returns a tuple (e.g., video + audio), the default `combine_cfg_noise()` applies CFG to every element. Override it to apply different logic per element — for example, CFG on video but positive-only on audio:
-
-```python
-class MyVideoAudioPipeline(nn.Module, CFGParallelMixin):
-    def combine_cfg_noise(self, positive_noise_pred, negative_noise_pred, scale, normalize):
-        (video_pos, audio_pos) = positive_noise_pred
-        (video_neg, audio_neg) = negative_noise_pred
-        video_combined = super().combine_cfg_noise(video_pos, video_neg, scale, normalize)
-        return (video_combined, audio_pos)  # audio: positive only, no CFG
-```
-
-This also requires `predict_noise()` to return a tuple (see [Override predict_noise](#override-predict_noise-for-custom-transformer-calls) above).
-
-### Implement a Composite Scheduler for Multi-Output Models
-
-When each output has its own denoising schedule, implement a composite scheduler that dispatches to per-output schedulers. Assign it to `self.scheduler` so the default `scheduler_step()` works without override.
-
-**Complete example (video + audio with separate schedulers and diffuse loop):**
-
-```python
-class VideoAudioScheduler:
-    """Composite scheduler dispatching to video and audio schedulers."""
-    def __init__(self, video_scheduler, audio_scheduler):
-        self.video_scheduler = video_scheduler
-        self.audio_scheduler = audio_scheduler
-
-    def step(self, noise_pred, t, latents, return_dict=False, generator=None):
-        video_out = self.video_scheduler.step(noise_pred[0], t[0], latents[0], return_dict=False, generator=generator)[0]
-        audio_out = self.audio_scheduler.step(noise_pred[1], t[1], latents[1], return_dict=False, generator=generator)[0]
-        return ((video_out, audio_out),)
-
-class MyVideoAudioPipeline(nn.Module, CFGParallelMixin):
-    def __init__(self, ...):
-        self.scheduler = VideoAudioScheduler(video_sched, audio_sched)
-
-    def predict_noise(self, **kwargs):
-        video_pred, audio_pred = self.transformer(**kwargs)
-        return (video_pred, audio_pred)
-
-    def combine_cfg_noise(self, positive_noise_pred, negative_noise_pred, scale, normalize):
-        # ... (as above)
-
-    def diffuse(self, video_latents, audio_latents, timesteps_video, timesteps_audio, ...):
-        for t_v, t_a in zip(timesteps_video, timesteps_audio):
-            positive_kwargs = {...}  
-            negative_kwargs = {...} if do_true_cfg else None
-
-            video_pred, audio_pred = self.predict_noise_maybe_with_cfg(
-                do_true_cfg=do_true_cfg, true_cfg_scale=self.guidance_scale,
-                positive_kwargs=positive_kwargs, negative_kwargs=negative_kwargs,
-            )
-            video_latents, audio_latents = self.scheduler_step_maybe_with_cfg(
-                (video_pred, audio_pred), (t_v, t_a),
-                (video_latents, audio_latents), do_true_cfg=do_true_cfg,
-                generator=generator,
-            )
-        return video_latents, audio_latents
-```
-
-> **Note:** If you use a non-deterministic scheduler, e.g., DDPM, please set `self.scheduler_step_maybe_with_cfg(..., generator=torch.Generator(device).manual_seed(seed))` explicitly to control the randomness of scheduler step among ranks.
-
 ---

 ## Testing
@@ -369,7 +369,7 @@ images = omni.generate(
 2. **Compare performance** - Measure speedup vs baseline (expect 1.5x-2.0x)
 3. **Verify output quality** - Visually compare cached vs uncached outputs (should be nearly identical)

-See more detailed examples in [user guide for teacache](../../user_guide/diffusion/cache_acceleration/teacache.md).
+See more detailed examples in [user guide for teacache](../../user_guide/diffusion/teacache.md).

 ---

@@ -1,14 +1,15 @@
 # VAE Patch Parallelism

 This document describes how to add **VAE Patch Parallelism** support to a diffusion model.
-We use **Qwen-Image** as the reference implementation.
+We use **Qwen-Image** as the reference implementation for decode parallel, and **Wan2.2** for encode parallel.

 ---

 ## Table of Contents

 - [Overview](#overview)
- [Step-by-Step Implementation](#step-by-step-implementation)
+- [Step-by-Step Implementation (Decode)](#step-by-step-implementation-decode)
+- [Encode Parallel Implementation](#encode-parallel-implementation)
 - [Testing](#testing)
 - [Reference Implementations](#reference-implementations)
 - [Summary](#summary)
@@ -19,13 +20,13 @@ We use **Qwen-Image** as the reference implementation.

 ### What is Vae Patch parallel?

-**VAE Patch Parallelism** is a decoding acceleration technique. Instead of decoding the entire latent tensor at once, the latent tensor is:
+**VAE Patch Parallelism** is an acceleration technique for both **encoding** and **decoding**. Instead of processing the entire tensor at once, the tensor is:

 + Split into multiple spatial tiles

 + Distributed across multiple ranks

-+ Decoded in parallel
+ Encoded/Decoded in parallel

 + Merged to reconstruct the final output

@@ -35,10 +36,17 @@ This approach:

 + Reduces peak memory usage per device

-+ Accelerates decoding latency
+ Accelerates encoding/decoding latency
+
+### When to Use Encode vs Decode Parallel
+
+| Operation | Use Case | Example |
+|-----------|----------|---------|
+| **Decode Parallel** | Text-to-Image, Text-to-Video | Latent → Image/Video |
+| **Encode Parallel** | Image-to-Video (I2V) | Image → Latent (for conditioning) |

 ### Architecture
-We introduce **DistributedVaeExecutor** as the core component responsible for distributed VAE decoding.
+We introduce **DistributedVaeExecutor** as the core component responsible for distributed VAE encoding/decoding.

 The executor is model-agnostic and accepts three function parameters:

@@ -84,7 +92,7 @@ Therefore:

 + Merge must perform blending to avoid seams

-## Step-by-Step Implementation
+## Step-by-Step Implementation (Decode)

 ### Step 1: Implement DistributedAutoencoderKLQwenImage
 `QwenImagePipeline` use `AutoencoderKLQwenImage` for vae, so implement a distributed version:
@@ -205,14 +213,14 @@ def tile_merge(self, coord_tensor_map: dict[tuple[int, ...], torch.Tensor], grid
 We need to override tiled_decode, the main logic is:
 + check distributed is enabled
 + select split/exec/merge
-+ Invoke self.distributed_decoder.execute to decode
+ Invoke self.distributed_executor.execute to decode
 ```
 def tiled_decode(self, z: torch.Tensor, return_dict: bool = True):
    if not self.is_distributed_enabled():
        return super().tiled_decode(z, return_dict=return_dict)

    logger.info("Decode run with distributed executor")
-    result = self.distributed_decoder.execute(
+    result = self.distributed_executor.execute(
        z,
        DistributedOperator(split=self.tile_split, exec=self.tile_exec, merge=self.tile_merge),
        broadcast_result=True,
@@ -243,6 +251,166 @@ class YourModelPipeline(nn.Module):
 +       ).to(self.device)
 ```

+## Encode Parallel Implementation
+
+For models that require VAE encoding (e.g., Image-to-Video), you can also parallelize the encode operation. We use **Wan2.2** as the reference implementation.
+
+### Step 1: Implement encode_tile_split
+
+Similar to decode, split the input tensor into tiles. Key considerations:
+
+ **Patchify handling**: If the model uses `patch_size`, scale tile parameters accordingly
+ **Temporal chunking**: Video VAEs may have temporal compression (e.g., 4x)
+
+```python
+def encode_tile_split(self, x: torch.Tensor) -> tuple[list[TileTask], GridSpec]:
+    _, _, num_frames, height, width = x.shape
+    encode_spatial_compression_ratio = self.spatial_compression_ratio
+
+    # Scale tile parameters for patchified coordinate system
+    tile_sample_min_height = self.tile_sample_min_height
+    tile_sample_min_width = self.tile_sample_min_width
+    tile_sample_stride_height = self.tile_sample_stride_height
+    tile_sample_stride_width = self.tile_sample_stride_width
+
+    if self.config.patch_size is not None:
+        # When input is patchified, scale tile parameters accordingly
+        encode_spatial_compression_ratio = self.spatial_compression_ratio // self.config.patch_size
+        tile_sample_min_height = tile_sample_min_height // self.config.patch_size
+        tile_sample_min_width = tile_sample_min_width // self.config.patch_size
+        tile_sample_stride_height = tile_sample_stride_height // self.config.patch_size
+        tile_sample_stride_width = tile_sample_stride_width // self.config.patch_size
+
+    latent_height = height // encode_spatial_compression_ratio
+    latent_width = width // encode_spatial_compression_ratio
+
+    tile_latent_min_height = tile_sample_min_height // encode_spatial_compression_ratio
+    tile_latent_min_width = tile_sample_min_width // encode_spatial_compression_ratio
+    tile_latent_stride_height = tile_sample_stride_height // encode_spatial_compression_ratio
+    tile_latent_stride_width = tile_sample_stride_width // encode_spatial_compression_ratio
+
+    blend_height = tile_latent_min_height - tile_latent_stride_height
+    blend_width = tile_latent_min_width - tile_latent_stride_width
+
+    tiletask_list = []
+    # Use temporal compression ratio from config instead of hardcoding
+    temporal_compression = self.config.scale_factor_temporal
+
+    for i in range(0, height, tile_sample_stride_height):
+        for j in range(0, width, tile_sample_stride_width):
+            time_list = []
+            frame_range = 1 + (num_frames - 1) // temporal_compression
+            for k in range(frame_range):
+                if k == 0:
+                    tile = x[:, :, :1, i : i + tile_sample_min_height, j : j + tile_sample_min_width]
+                else:
+                    tile = x[
+                        :, :,
+                        1 + temporal_compression * (k - 1) : 1 + temporal_compression * k,
+                        i : i + tile_sample_min_height,
+                        j : j + tile_sample_min_width,
+                    ]
+                time_list.append(tile)
+            tiletask_list.append(
+                TileTask(len(tiletask_list), (i // tile_sample_stride_height, j // tile_sample_stride_width),
+                         time_list, workload=time_list[0].shape[3] * time_list[0].shape[4])
+            )
+
+    grid_spec = GridSpec(
+        split_dims=(3, 4),
+        grid_shape=(tiletask_list[-1].grid_coord[0] + 1, tiletask_list[-1].grid_coord[1] + 1),
+        tile_spec={
+            "latent_height": latent_height, "latent_width": latent_width,
+            "blend_height": blend_height, "blend_width": blend_width,
+            "tile_latent_stride_height": tile_latent_stride_height,
+            "tile_latent_stride_width": tile_latent_stride_width,
+        },
+        output_dtype=self.dtype,
+    )
+    return tiletask_list, grid_spec
+```
+
+### Step 2: Implement encode_tile_exec
+
+```python
+def encode_tile_exec(self, task: TileTask) -> torch.Tensor:
+    """Encode a single sample tile into latent space."""
+    self.clear_cache()
+    time = []
+    for k, tile in enumerate(task.tensor):
+        self._enc_conv_idx = [0]
+        encoded = self.encoder(tile, feat_cache=self._enc_feat_map, feat_idx=self._enc_conv_idx)
+        encoded = self.quant_conv(encoded)
+        time.append(encoded)
+    result = torch.cat(time, dim=2)
+    self.clear_cache()
+    return result
+```
+
+### Step 3: Implement encode_tile_merge
+
+```python
+def encode_tile_merge(
+    self, coord_tensor_map: dict[tuple[int, ...], torch.Tensor], grid_spec: GridSpec
+) -> torch.Tensor:
+    """Merge encoded tiles into a full latent tensor."""
+    grid_h, grid_w = grid_spec.grid_shape
+    result_rows = []
+    for i in range(grid_h):
+        result_row = []
+        for j in range(grid_w):
+            tile = coord_tensor_map[(i, j)]
+            if i > 0:
+                tile = self.blend_v(coord_tensor_map[(i - 1, j)], tile, grid_spec.tile_spec["blend_height"])
+            if j > 0:
+                tile = self.blend_h(coord_tensor_map[(i, j - 1)], tile, grid_spec.tile_spec["blend_width"])
+            result_row.append(tile[:, :, :,
+                : grid_spec.tile_spec["tile_latent_stride_height"],
+                : grid_spec.tile_spec["tile_latent_stride_width"]])
+        result_rows.append(torch.cat(result_row, dim=-1))
+
+    enc = torch.cat(result_rows, dim=3)[
+        :, :, :, : grid_spec.tile_spec["latent_height"], : grid_spec.tile_spec["latent_width"]
+    ]
+    return enc
+```
+
+### Step 4: Override tiled_encode method
+
+Override `tiled_encode` instead of `encode`. The parent's `_encode()` handles patchify before calling `tiled_encode()`, so input `x` is already patchified.
+
+```python
+def tiled_encode(self, x: torch.Tensor) -> torch.Tensor:
+    """
+    Encode using distributed VAE executor.
+
+    Note: x is already patchified by parent's _encode() before calling this method.
+    """
+    if not self.is_distributed_enabled():
+        return super().tiled_encode(x)
+
+    self.clear_cache()
+    result = self.distributed_executor.execute(
+        x,
+        DistributedOperator(
+            split=self.encode_tile_split,
+            exec=self.encode_tile_exec,
+            merge=self.encode_tile_merge,
+        ),
+        broadcast_result=True,  # Latents needed by all ranks for diffusion
+    )
+    self.clear_cache()
+    return result
+```
+
+**Key differences from decode parallel:**
+
+| Aspect | Decode Parallel | Encode Parallel |
+|--------|-----------------|-----------------|
+| `broadcast_result` | Often `False` (only rank 0 needs output) | `True` (all ranks need latents for diffusion) |
+| Patchify | Applied in merge (unpatchify) | Handled by parent `_encode()` before `tiled_encode()` |
+| Temporal chunking | Frame-by-frame | Chunk-based (e.g., 1 + 4n frames) |
+
 ## Testing
 Verify numerical consistency between:
 + vae_patch_parallel_size = 1
@@ -272,18 +440,20 @@ When vae_patch_parallel_size is larger than the DiT world size, it will automati

 Complete examples in the codebase:

-| Model | Path | Notes |
-|-------|------|-------|
-| **Z-Image** | `vllm_omni/diffusion/distributed/autoencoders/autoencoder_kl.py` | Distributed AutoencoderKL |
-| **Wan2.2** | `vllm_omni/diffusion/distributed/autoencoders/autoencoder_kl_wan.py` | Distributed AutoencoderKLWan |
-| **Qwen-Image** | `vllm_omni/diffusion/distributed/autoencoders/autoencoder_kl_qwenimage.py` | Distributed AutoencoderKLQwenImage |
+| Model | Path | Decode Parallel | Encode Parallel |
+|-------|------|-----------------|-----------------|
+| **Z-Image** | `vllm_omni/diffusion/distributed/autoencoders/autoencoder_kl.py` | ✅ | ❌ |
+| **Wan2.2** | `vllm_omni/diffusion/distributed/autoencoders/autoencoder_kl_wan.py` | ✅ | ✅ |
+| **Qwen-Image** | `vllm_omni/diffusion/distributed/autoencoders/autoencoder_kl_qwenimage.py` | ✅ | ❌ |

 ---

 ## Summary

-Adding Vae Patch Parallel support to diffusion model:
+Adding VAE Patch Parallel support to diffusion model:

-1. **Implement Distributed Vae** - mainly copy from `diffusers` tiled_decode, and refactor into split/exec/merge
-2. **Change vae model in pipeline to Distributed Vae**
-3. **Test** - Verify with `tensor_parallel_size=N` quality
+1. **Implement Distributed VAE** - Inherit from base VAE class and `DistributedVaeMixin`
+2. **Decode Parallel** - Refactor `tiled_decode` into `tile_split`/`tile_exec`/`tile_merge`
+3. **Encode Parallel** (optional) - Implement `encode_tile_split`/`encode_tile_exec`/`encode_tile_merge` for I2V models
+4. **Change VAE model in pipeline** - Use the distributed version
+5. **Test** - Verify numerical consistency with `vae_patch_parallel_size=1` vs `N`
@@ -29,11 +29,6 @@ uv pip install vllm --torch-backend=auto
 uv pip install vllm-omni
 ```

-To run Gradio demos, also install the optional extras:
-```bash
-uv pip install 'vllm-omni[demo]'
-```
-
 # --8<-- [end:pre-built-wheels]

 # --8<-- [start:build-wheel-from-source]
@@ -55,11 +50,6 @@ cd vllm-omni
 uv pip install -e .
 ```

-To run Gradio demos, install with optional extras:
-```bash
-uv pip install -e '.[demo]'
-```
-
 <details><summary>(Optional) Installation of vLLM from source</summary>
 If you want to check, modify or debug with source code of vLLM, install the library from source with the following instructions:

@@ -1,285 +0,0 @@
-# Cache-DiT Guide
-
-
-## Table of Content
-
- [Overview](#overview)
- [Quick Start](#quick-start)
- [Example Script](#example-script)
- [Acceleration Methods](#acceleration-methods)
- [Configuration Parameters](#configuration-parameters)
- [Best Practices](#best-practices)
- [Troubleshooting](#troubleshooting)
- [Summary](#summary)
- [Additional Resources](#additional-resources)
-
---
-
-## Overview
-
-Cache-DiT accelerates diffusion transformer models through intelligent caching mechanisms, providing significant speedup with minimal quality loss. It supports multiple acceleration techniques that can be combined for optimal performance:
-
- **DBCache**: Dual Block Cache for reducing redundant computations
- **TaylorSeer**: Taylor expansion-based forecasting for faster inference
- **SCM**: Step Computation Masking for selective step computation
-
-See supported models list in [Supported Models](../../diffusion_features.md#supported-models).
-
---
-
-## Quick Start
-
-### Basic Usage
-
-Enable cache-dit acceleration by simply setting `cache_backend="cache_dit"`:
-
-```python
-from vllm_omni import Omni
-from vllm_omni.inputs.data import OmniDiffusionSamplingParams
-
-omni = Omni(
-    model="Qwen/Qwen-Image",
-    cache_backend="cache_dit",  # Enable Cache-DiT with defaults
-)
-
-outputs = omni.generate(
-    "a beautiful landscape",
-    OmniDiffusionSamplingParams(num_inference_steps=50),
-)
-```
-
-**Note**: When `cache_config` is not provided, Cache-DiT uses optimized default values. See the [Configuration Parameters](#configuration-parameters) section for details.
-
-### Custom Configuration
-
-To customize cache-dit settings, provide a `cache_config` dictionary, for example:
-
-```python
-omni = Omni(
-    model="Qwen/Qwen-Image",
-    cache_backend="cache_dit",
-    cache_config={
-        "Fn_compute_blocks": 1,
-        "Bn_compute_blocks": 0,
-        "max_warmup_steps": 4,
-        "residual_diff_threshold": 0.12,
-    },
-)
-```
-
---
-
-## Example Script
-
-### Offline Inference
-
-Use the example script under `examples/offline_inference/text_to_image`:
-
-```bash
-cd examples/offline_inference/text_to_image
-python text_to_image.py \
-    --model Qwen/Qwen-Image \
-    --prompt "a cup of coffee on the table" \
-    --cache-backend cache_dit \
-    --num-inference-steps 50
-```
-
-See the [text_to_image.py](https://github.com/vllm-project/vllm-omni/blob/main/examples/offline_inference/text_to_image/text_to_image.py) for detailed configuration options.
-
-The script uses cache-dit acceleration with a hybrid configuration combining DBCache, SCM, and TaylorSeer:
-
-```python
-omni = Omni(
-    model="Qwen/Qwen-Image",
-    cache_backend="cache_dit",
-    cache_config={
-        # Scheme: Hybrid DBCache + SCM + TaylorSeer
-        "Fn_compute_blocks": 1,  # Optimized for single-transformer models
-        "Bn_compute_blocks": 0,  # Number of backward compute blocks
-        "max_warmup_steps": 4,  # Maximum warmup steps (works for few-step models)
-        "residual_diff_threshold": 0.24,  # Higher threshold for more aggressive caching
-        "max_continuous_cached_steps": 3,  # Limit to prevent precision degradation
-        # TaylorSeer parameters [cache-dit only]
-        "enable_taylorseer": False,  # Disabled by default (not suitable for few-step models)
-        "taylorseer_order": 1,  # TaylorSeer polynomial order
-        # SCM (Step Computation Masking) parameters [cache-dit only]
-        "scm_steps_mask_policy": None,  # SCM mask policy: None (disabled), "slow", "medium", "fast", "ultra"
-        "scm_steps_policy": "dynamic",  # SCM steps policy: "dynamic" or "static"
-    }
-)
-```
-
-You can customize the configuration by modifying the `cache_config` dictionary to use only specific methods (e.g., DBCache only, DBCache + SCM, etc.) based on your quality and speed requirements.
-
-For image-to-image tasks, use the example script under `examples/offline_inference/image_to_image`:
-
-```bash
-cd examples/offline_inference/image_to_image
-python image_edit.py \
-    --model Qwen/Qwen-Image-Edit \
-    --prompt "make the sky more colorful" \
-    --image path/to/input/image.jpg \
-    --cache-backend cache_dit \
-    --num-inference-steps 50 \
-    --cache-dit-max-continuous-cached-steps 3 \
-    --cache-dit-residual-diff-threshold 0.24 \
-    --cache-dit-enable-taylorseer
-```
-
-See the [image_edit.py](https://github.com/vllm-project/vllm-omni/blob/main/examples/offline_inference/image_to_image/image_edit.py) for detailed configuration options.
-
-### Online Serving
-
-```bash
-# Default configuration (recommended)
-vllm serve Qwen/Qwen-Image --omni --port 8091 --cache-backend cache_dit
-
-# Custom configuration
-vllm serve Qwen/Qwen-Image --omni --port 8091 \
-  --cache-backend cache_dit \
-  --cache-config '{"Fn_compute_blocks": 1, "residual_diff_threshold": 0.12}'
-```
-
---
-
-## Acceleration Methods
-
-For comprehensive illustration, please view Cache-DiT [User Guide](https://cache-dit.readthedocs.io/en/latest/user_guide/OVERVIEWS/).
-
-### 1. DBCache (Dual Block Cache)
-
-DBCache intelligently caches intermediate transformer block outputs when the residual differences between consecutive steps are small, reducing redundant computations without sacrificing quality.
-
-**Example Configuration**:
-
-```python
-cache_config={
-    "Fn_compute_blocks": 8,           # Use first 8 blocks for difference computation
-    "Bn_compute_blocks": 0,           # No additional fusion blocks
-    "max_warmup_steps": 8,            # Cache after 8 warmup steps
-    "residual_diff_threshold": 0.12,  # Lower threshold for faster inference
-    "max_cached_steps": -1,           # No limit on cached steps
-}
-```
-
-**Performance Tips**:
-
- Default `Fn_compute_blocks=1` works well for most cases. Some models (e.g., [FLUX.2-klein](https://github.com/wtomin/vllm-omni/blob/main/vllm_omni/diffusion/cache/cache_dit_backend.py#L363)) use a larger value for `Fn_compute_blocks` for a balanced performance.
- Increase `residual_diff_threshold` (e.g., 0.12-0.15) for faster inference with slight quality trade-off, or decrease from default 0.24 for higher quality.
- Default `max_warmup_steps=4` is optimized for few-step models. Increase to 6-8 for more steps if needed.
-
-### 2. TaylorSeer
-
-TaylorSeer uses Taylor expansion to forecast future hidden states, allowing the model to skip some computation steps while maintaining quality.
-
-**Example Configuration**:
-
-```python
-cache_config={
-    "enable_taylorseer": True,
-    "taylorseer_order": 1,  # First-order Taylor expansion
-}
-```
-
-**Performance Tips**:
-
- TaylorSeer is **not suitable for few-step distilled models**.
- Use `taylorseer_order=1` for most cases (good balance of speed and quality).
- Combine with DBCache for maximum acceleration.
- Higher orders (2-3) may improve quality but reduce speed gains.
-
-### 3. SCM (Step Computation Masking)
-
-SCM allows you to specify which steps must be computed and which can use cached results, similar to LeMiCa/EasyCache style acceleration.
-
-`scm_steps_mask_policy` options (number of compute steps out of 28):
-
-| Policy | Compute Steps | Speed | Quality |
-|--------|--------------|-------|---------|
-| `None` (default) | All | Baseline | Best |
-| `"slow"` | 18 / 28 | Moderate | High |
-| `"medium"` | 15 / 28 | Balanced | Good |
-| `"fast"` | 11 / 28 | Fast | Moderate |
-| `"ultra"` | 8 / 28 | Fastest | Lower |
-
-**Example Configuration**:
-
-```python
-cache_config={
-    "scm_steps_mask_policy": "medium",  # Balanced speed/quality
-    "scm_steps_policy": "dynamic",      # Use dynamic cache
-}
-```
-
-**Performance Tips**:
-
- SCM is disabled by default. Enable it by setting a policy value if you need additional acceleration.
- Start with `"medium"` policy and adjust based on quality requirements.
- Use `"fast"` or `"ultra"` for maximum speed when quality can be slightly compromised.
- `"dynamic"` policy generally provides better quality than `"static"`.
- SCM mask is automatically regenerated when `num_inference_steps` changes during inference.
-
---
-
-## Configuration Parameters
-
-In `cache_config` passed to `Omni` constructor, it accepts the arguments of `DBCacheConfig` ([Cache-DiT API Reference](https://cache-dit.readthedocs.io/en/latest/user_guide/CACHE_API/)). Key parameters are listed below:
-
-| Parameter | Type | Default | Description |
-|-----------|------|---------|-------------|
-| `Fn_compute_blocks` | int | 1 | First n blocks for difference computation (optimized for single-transformer models) |
-| `Bn_compute_blocks` | int | 0 | Last n blocks for fusion |
-| `max_warmup_steps` | int | 4 | Steps before caching starts (optimized for few-step distilled models) |
-| `max_cached_steps` | int | -1 | Max cached steps (-1 = unlimited) |
-| `max_continuous_cached_steps` | int | 3 | Max consecutive cached steps (prevents precision degradation) |
-| `residual_diff_threshold` | float | 0.24 | Residual difference threshold (higher for more aggressive caching) |
-| `num_inference_steps` | int \| None | None | Initial inference steps for SCM mask generation (optional, auto-refreshed during inference) |
-| `enable_taylorseer` | bool | False | Enable TaylorSeer acceleration (not suitable for few-step distilled models) |
-| `taylorseer_order` | int | 1 | Taylor expansion order |
-| `scm_steps_mask_policy` | str \| None | None | SCM mask policy (None, "slow", "medium", "fast", "ultra") |
-| `scm_steps_policy` | str | "dynamic" | SCM computation policy ("dynamic" or "static") |
-
---
-
-## Best Practices
-
-### When to Use
-
-**Good for:**
-
- Production deployments requiring fast inference
- Diffusion transformer models (DiT architecture)
- Scenarios where 1.5x-3x speedup is valuable
-
-**Not for:**
-
- Non-DiT architectures (use model-specific acceleration instead)
- Models already using few-step distillation (< 10 steps)
-
---
-
-## Troubleshooting
-
-### Common Issue 1: Quality Degradation
-
-**Symptoms**: Generated images have visible artifacts or lower quality
-
-**Solution**:
-```python
-# Reduce aggressiveness - use more conservative settings
-cache_config={
-    "residual_diff_threshold": 0.20,  # Lower threshold (closer to default 0.24)
-    "Fn_compute_blocks": 8,            # Use more blocks for better decisions
-    "max_warmup_steps": 6,             # Longer warmup
-    "scm_steps_mask_policy": "slow",   # More compute steps
-}
-```
-
---
-
-## Summary
-
-Using Cache-DiT acceleration:
-
-1. ✅ **Enable Cache-DiT** - Set `cache_backend="cache_dit"` to get 1.5x-3x speedup with optimized defaults
-2. ✅ **(Optional) Customize** - Adjust `cache_config` parameters for specific speed/quality trade-offs
@@ -1,194 +0,0 @@
-# TeaCache Guide
-
-
-## Table of Content
-
- [Overview](#overview)
- [Quick Start](#quick-start)
- [Example Script](#example-script)
- [Configuration Parameters](#configuration-parameters)
- [Best Practices](#best-practices)
- [Troubleshooting](#troubleshooting)
- [Summary](#summary)
-
---
-
-## Overview
-
-TeaCache accelerates diffusion model inference by caching transformer computations when consecutive timesteps are similar, providing **1.5x-2.0x speedup** with minimal quality loss. It dynamically decides whether to reuse cached outputs based on input similarity, making it ideal for production deployments where inference speed matters without sacrificing generation quality.
-
-See supported models list in [Supported Models](../../diffusion_features.md#supported-models).
-
---
-
-## Quick Start
-
-
-
-### Basic Usage
-
-
-```python
-from vllm_omni import Omni
-from vllm_omni.inputs.data import OmniDiffusionSamplingParams
-
-omni = Omni(
-    model="Qwen/Qwen-Image",
-    cache_backend="tea_cache",
-)
-
-outputs = omni.generate(
-    "A cat sitting on a windowsill",
-    OmniDiffusionSamplingParams(num_inference_steps=50),
-)
-```
-
-### Custom Configuration
-
-```python
-omni = Omni(
-    model="Qwen/Qwen-Image",
-    cache_backend="tea_cache",
-    cache_config={
-        "rel_l1_thresh": 0.2,  # Controls speed/quality tradeoff
-    },
-)
-```
-
-### Using Environment Variable
-
-You can also enable TeaCache via environment variable:
-
-```bash
-export DIFFUSION_CACHE_BACKEND=tea_cache
-```
-
-Then initialize without explicitly setting `cache_backend`:
-
-```python
-from vllm_omni import Omni
-
-omni = Omni(
-    model="Qwen/Qwen-Image",
-    cache_config={"rel_l1_thresh": 0.2}
-)
-```
-
---
-
-## Example Script
-
-### Offline Inference
-
-Use python script under `examples/offline_inference/text_to_image/` or `examples/offline_inference/image_to_image/` with CLI:
-
-```bash
-# Text-to-image example
-python examples/offline_inference/text_to_image/text_to_image.py \
-  --model Qwen/Qwen-Image \
-  --cache-backend tea_cache
-
-# Image-to-image example
-python examples/offline_inference/image_to_image/image_edit.py \
-  --model Qwen/Qwen-Image-Edit \
-  --image input.png \
-  --prompt "Edit description" \
-  --cache-backend tea_cache \
-  --tea-cache-rel-l1-thresh 0.25
-```
-
-See the [text_to_image.py](https://github.com/vllm-project/vllm-omni/blob/main/examples/offline_inference/text_to_image/text_to_image.py) or [image_edit.py](https://github.com/vllm-project/vllm-omni/blob/main/examples/offline_inference/image_to_image/image_edit.py) for detailed configuration options.
-
-### Online Serving
-
-```bash
-# Default configuration
-vllm serve Qwen/Qwen-Image --omni --port 8091 --cache-backend tea_cache
-
-# Custom configuration
-vllm serve Qwen/Qwen-Image --omni --port 8091 \
-  --cache-backend tea_cache \
-  --cache-config '{"rel_l1_thresh": 0.2}'
-```
-
---
-
-## Configuration Parameters
-
-In `OmniDiffusionConfig`
-
-| Parameter | Type | Default | Description |
-|-----------|------|---------|-------------|
-| `rel_l1_thresh` | float | `0.2` | Similarity threshold for cache reuse. Lower values prioritize quality (less caching), higher values prioritize speed (more caching). Suggested range: 0.1-0.8 |
-| `coefficients` | list[float] \| None | `None` | Polynomial coefficients for rescaling L1 distance. Must contain exactly 5 elements if provided. If `None`, uses model-specific defaults based on transformer type. |
-
-Users can find the default model coefficients in [`vllm_omni/diffusion/cache/teacache/config.py`](https://github.com/vllm-project/vllm-omni/blob/main/vllm_omni/diffusion/cache/teacache/config.py), for example:
-
-```python
-_MODEL_COEFFICIENTS = {
-    # Qwen-Image transformer coefficients from ComfyUI-TeaCache
-    # Tuned specifically for Qwen's dual-stream transformer architecture
-    # Used for all Qwen-Image Family pipelines, in general
-    "QwenImageTransformer2DModel": [
-        -4.50000000e02,
-        2.80000000e02,
-        -4.50000000e01,
-        3.20000000e00,
-        -2.00000000e-02,
-    ],
-    ...
-}
-```
-
---
-
-## Best Practices
-
-### When to Use
-
-**Good for:**
-
- Production deployments requiring faster inference, tolerant of minimal quality loss
- Scenarios where 1.5-2x speedup is valuable
- Useful for single-card acceleration
-
-**Not for:**
-
- Maximum quality requirements where no degradation is acceptable
- Very short inference runs (< 20 steps) where caching overhead may outweigh benefits
-
-
---
-
-## Troubleshooting
-
-### Common Issue 1: Quality Degradation
-
-**Symptoms**: Generated images show artifacts, reduced detail, or inconsistent quality compared to non-cached results
-
-**Solution**:
-
-```python
-# Lower the threshold for more conservative caching
-cache_config={"rel_l1_thresh": 0.1}
-```
-
-### Common Issue 2: Limited Speedup
-
-**Symptoms**: Actual speedup is less than expected (< 1.3x)
-
-**Solutions**:
-1. Increase the threshold to enable more aggressive caching:
-   ```python
-   cache_config={"rel_l1_thresh": 0.8}
-   ```
-2. Ensure you're using sufficient inference steps (35+ recommended)
-3. Check that your model architecture is supported (see Supported Models section)
-
---
-
-
-## Summary
-
-1. ✅ **Enable TeaCache** - Set `cache_backend="tea_cache"` to get 1.5x-2.0x speedup with optimized defaults
-2. ✅ **(Optional) Customize** - Adjust thresholds and polynomial coefficients for specific speed/quality trade-offs
@@ -0,0 +1,228 @@
+# Cache-DiT Acceleration Guide
+
+This guide explains how to use cache-dit acceleration in vLLM-Omni to speed up diffusion model inference.
+
+## Overview
+
+Cache-dit is a library that accelerates diffusion transformer models through intelligent caching mechanisms. It supports multiple acceleration techniques that can be combined for optimal performance:
+
+- **DBCache**: Dual Block Cache for reducing redundant computations
+- **TaylorSeer**: Taylor expansion-based forecasting for faster inference
+- **SCM**: Step Computation Masking for selective step computation
+
+## Quick Start
+
+### Basic Usage
+
+Enable cache-dit acceleration by simply setting `cache_backend="cache_dit"`. Cache-dit will use its recommended default parameters:
+
+```python
+from vllm_omni.entrypoints.omni import Omni
+from vllm_omni.inputs.data import OmniDiffusionSamplingParams
+
+# Simplest way: just enable cache-dit with default parameters
+omni = Omni(
+    model="Qwen/Qwen-Image",
+    cache_backend="cache_dit",
+)
+
+images = omni.generate(
+    "a beautiful landscape",
+    OmniDiffusionSamplingParams(num_inference_steps=50),
+)
+```
+
+**Default Parameters**: When `cache_config` is not provided, cache-dit uses optimized default values. See the [Configuration Reference](#configuration-reference) section for a complete list of all parameters and their default values.
+
+### Custom Configuration
+
+To customize cache-dit settings, provide a `cache_config` dictionary, for example:
+
+```python
+omni = Omni(
+    model="Qwen/Qwen-Image",
+    cache_backend="cache_dit",
+    cache_config={
+        "Fn_compute_blocks": 1,
+        "Bn_compute_blocks": 0,
+        "max_warmup_steps": 4,
+        "residual_diff_threshold": 0.12,
+    },
+)
+```
+
+## Online Serving (OpenAI-Compatible)
+
+Enable Cache-DiT for online serving by passing `--cache-backend cache_dit` when starting the server:
+
+```bash
+# Use Cache-DiT default (recommended) parameters
+vllm serve Qwen/Qwen-Image --omni --port 8091 --cache-backend cache_dit
+```
+
+To customize Cache-DiT settings for online serving, pass a JSON string via `--cache-config`:
+
+```bash
+vllm serve Qwen/Qwen-Image --omni --port 8091 \
+  --cache-backend cache_dit \
+  --cache-config '{"Fn_compute_blocks": 1, "Bn_compute_blocks": 0, "max_warmup_steps": 4, "residual_diff_threshold": 0.12}'
+```
+
+## Acceleration Methods
+
+For comprehensive illustration, please view cache-dit [User_Guide](https://cache-dit.readthedocs.io/en/latest/user_guide/OVERVIEWS/)
+
+### 1. DBCache (Dual Block Cache)
+
+DBCache intelligently caches intermediate transformer block outputs when the residual differences between consecutive steps are small, reducing redundant computations without sacrificing quality.
+
+**Key Parameters**:
+
+- `Fn_compute_blocks` (int, default: 1): Number of **first n** transformer blocks used to compute stable feature differences. Higher values provide more accurate caching decisions but increase computation.
+- `Bn_compute_blocks` (int, default: 0): Number of **last n** transformer blocks used for additional fusion. These blocks act as an auto-scaler for approximate hidden states.
+- `max_warmup_steps` (int, default: 4): Number of initial steps where caching is disabled to ensure the model learns sufficient features before caching begins. Optimized for few-step distilled models.
+- `residual_diff_threshold` (float, default: 0.24): Threshold for residual difference. Higher values lead to faster performance but may reduce precision. Default uses a relatively higher threshold for more aggressive caching.
+- `max_cached_steps` (int, default: -1): Maximum number of cached steps. Set to -1 for unlimited caching.
+- `max_continuous_cached_steps` (int, default: 3): Maximum number of consecutive cached steps. Limits consecutive caching to prevent precision degradation.
+
+**Example Configuration**:
+
+```python
+cache_config={
+    "Fn_compute_blocks": 8,      # Use first 8 blocks for difference computation
+    "Bn_compute_blocks": 0,       # No additional fusion blocks
+    "max_warmup_steps": 8,        # Cache after 8 warmup steps
+    "residual_diff_threshold": 0.12,  # Higher threshold for faster inference
+    "max_cached_steps": -1,        # No limit on cached steps
+}
+```
+
+**Performance Tips**:
+
+- Default `Fn_compute_blocks=1` works well for most cases. Increase to 8-12 for larger models or when more accuracy is needed
+- Increase `residual_diff_threshold` (e.g., 0.12-0.15) for faster inference with slight quality trade-off, or decrease from default 0.24 for higher quality
+- Default `max_warmup_steps=4` is optimized for few-step models. Increase to 6-8 for more steps if needed
+
+### 2. TaylorSeer
+
+TaylorSeer uses Taylor expansion to forecast future hidden states, allowing the model to skip some computation steps while maintaining quality.
+
+**Key Parameters**:
+
+- `enable_taylorseer` (bool, default: False): Enable TaylorSeer acceleration
+- `taylorseer_order` (int, default: 1): Order of Taylor expansion. Higher orders provide better accuracy but require more computation.
+
+**Example Configuration**:
+
+```python
+cache_config={
+    "enable_taylorseer": True,
+    "taylorseer_order": 1,  # First-order Taylor expansion
+}
+```
+
+**Performance Tips**:
+
+- Use `taylorseer_order=1` for most cases (good balance of speed and quality)
+- Combine with DBCache for maximum acceleration
+- Higher orders (2-3) may improve quality but reduce speed gains
+
+### 3. SCM (Step Computation Masking)
+
+SCM allows you to specify which steps must be computed and which can use cached results, similar to LeMiCa/EasyCache style acceleration.
+
+**Key Parameters**:
+
+- `scm_steps_mask_policy` (str | None, default: None): Predefined mask policy. Options:
+  - `None`: SCM disabled (default)
+  - `"slow"`: More compute steps, higher quality (18 compute steps out of 28)
+  - `"medium"`: Balanced (15 compute steps out of 28)
+  - `"fast"`: More cache steps, faster inference (11 compute steps out of 28)
+  - `"ultra"`: Maximum speed (8 compute steps out of 28)
+- `scm_steps_policy` (str, default: "dynamic"): Policy for cached steps:
+  - `"dynamic"`: Use dynamic cache for masked steps (recommended)
+  - `"static"`: Use static cache for masked steps
+
+**Example Configuration**:
+
+```python
+cache_config={
+    "scm_steps_mask_policy": "medium",  # Balanced speed/quality
+    "scm_steps_policy": "dynamic",      # Use dynamic cache
+}
+```
+
+**Performance Tips**:
+
+- SCM is disabled by default (`scm_steps_mask_policy=None`). Enable it by setting a policy value if you need additional acceleration
+- Start with `"medium"` policy and adjust based on quality requirements
+- Use `"fast"` or `"ultra"` for maximum speed when quality can be slightly compromised
+- `"dynamic"` policy generally provides better quality than `"static"`
+- SCM mask is automatically regenerated when `num_inference_steps` changes during inference
+
+## Configuration Reference
+
+### DiffusionCacheConfig Parameters
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `Fn_compute_blocks` | int | 1 | First n blocks for difference computation (optimized for single-transformer models) |
+| `Bn_compute_blocks` | int | 0 | Last n blocks for fusion |
+| `max_warmup_steps` | int | 4 | Steps before caching starts (optimized for few-step distilled models) |
+| `max_cached_steps` | int | -1 | Max cached steps (-1 = unlimited) |
+| `max_continuous_cached_steps` | int | 3 | Max consecutive cached steps (prevents precision degradation) |
+| `residual_diff_threshold` | float | 0.24 | Residual difference threshold (higher for more aggressive caching) |
+| `num_inference_steps` | int \| None | None | Initial inference steps for SCM mask generation (optional, auto-refreshed during inference) |
+| `enable_taylorseer` | bool | False | Enable TaylorSeer acceleration (not suitable for few-step distilled models) |
+| `taylorseer_order` | int | 1 | Taylor expansion order |
+| `scm_steps_mask_policy` | str \| None | None | SCM mask policy (None, "slow", "medium", "fast", "ultra") |
+| `scm_steps_policy` | str | "dynamic" | SCM computation policy ("dynamic" or "static") |
+
+## Example: Accelerate Text-to-Image Generation with CacheDiT
+
+See `examples/offline_inference/text_to_image/text_to_image.py` for a complete working example with cache-dit acceleration.
+
+```bash
+# Enable cache-dit with hybrid acceleration
+cd examples/offline_inference/text_to_image
+python text_to_image.py \
+    --model Qwen/Qwen-Image \
+    --prompt "a cup of coffee on the table" \
+    --cache-backend cache_dit \
+    --num-inference-steps 50
+```
+
+
+The script uses cache-dit acceleration with a hybrid configuration combining DBCache, SCM, and TaylorSeer:
+
+```python
+omni = Omni(
+    model="Qwen/Qwen-Image",
+    cache_backend="cache_dit",
+    cache_config={
+        # Scheme: Hybrid DBCache + SCM + TaylorSeer
+        # DBCache
+        "Fn_compute_blocks": 8,
+        "Bn_compute_blocks": 0,
+        "max_warmup_steps": 4,
+        "residual_diff_threshold": 0.12,
+        # TaylorSeer
+        "enable_taylorseer": True,
+        "taylorseer_order": 1,
+        # SCM
+        "scm_steps_mask_policy": "fast",  # Set to None to disable SCM
+        "scm_steps_policy": "dynamic",
+    },
+)
+```
+
+You can customize the configuration by modifying the `cache_config` dictionary to use only specific methods (e.g., DBCache only, DBCache + SCM, etc.) based on your quality and speed requirements.
+
+To test another model, you can modify `--model` with the target model identifier like `Tongyi-MAI/Z-Image-Turbo` and update `cache_config` according the model architecture (e.g., number of transformer blocks).
+
+
+## Additional Resources
+
+- [Cache-DiT User Guide](https://cache-dit.readthedocs.io/en/latest/user_guide/OVERVIEWS/)
+- [Cache-DiT Benchmark](https://cache-dit.readthedocs.io/en/latest/benchmark/HYBRID_CACHE/)
+- [DBCache Technical Details](https://cache-dit.readthedocs.io/en/latest/user_guide/CACHE_API/)
@@ -0,0 +1,92 @@
+# Frame Interpolation
+
+## Overview
+
+vLLM-Omni supports post-generation frame interpolation for supported video
+diffusion pipelines. This feature inserts synthesized intermediate frames
+between adjacent generated frames to improve temporal smoothness without
+rerunning the diffusion denoising loop.
+
+Frame interpolation runs in the diffusion worker post-processing path instead
+of the API server encoding path. This allows the interpolation step to reuse
+the worker's current accelerator device and keeps the FastAPI event loop free
+from heavy synchronous PyTorch work.
+
+For an input video with `N` generated frames and interpolation exponent `exp`,
+the output frame count is:
+
+```text
+(N - 1) * 2**exp + 1
+```
+
+The output FPS is multiplied by `2**exp` so the clip duration remains close to
+the original generated video.
+
+## Supported Pipelines
+
+Frame interpolation is currently supported for:
+
+- `WanPipeline` (Wan2.2 text-to-video)
+- `WanImageToVideoPipeline`
+- `Wan22TI2VPipeline`
+
+## Request Parameters
+
+The video APIs `/v1/videos` and `/v1/videos/sync` accept:
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `enable_frame_interpolation` | bool | `false` | Enable post-generation frame interpolation |
+| `frame_interpolation_exp` | int | `1` | Interpolation exponent. `1=2x`, `2=4x`, etc. |
+| `frame_interpolation_scale` | float | `1.0` | RIFE inference scale |
+| `frame_interpolation_model_path` | str | `None` | Local directory or Hugging Face repo ID containing `flownet.pkl` |
+
+## Execution Flow
+
+For supported Wan2.2 pipelines, the execution order is:
+
+1. Diffusion worker finishes denoising and decodes the raw video tensor.
+2. Worker-side model-specific post-processing runs.
+3. If frame interpolation is enabled, RIFE interpolates the decoded video
+   tensor on the worker side and records a FPS multiplier in `custom_output`.
+4. The API server receives the already-interpolated video and only performs
+   MP4 export.
+
+This design keeps interpolation close to the generated tensor and avoids
+introducing another heavyweight GPU context in the API server process.
+
+## Example
+
+Start the server:
+
+```bash
+vllm serve Wan-AI/Wan2.2-T2V-A14B-Diffusers --omni --port 8091
+```
+
+Run a sync request with interpolation enabled:
+
+```bash
+curl -X POST http://localhost:8091/v1/videos/sync \
+  -F "prompt=A dog running through a park" \
+  -F "num_frames=81" \
+  -F "width=832" \
+  -F "height=480" \
+  -F "fps=16" \
+  -F "num_inference_steps=40" \
+  -F "guidance_scale=1.0" \
+  -F "guidance_scale_2=1.0" \
+  -F "enable_frame_interpolation=true" \
+  -F "frame_interpolation_exp=1" \
+  -F "frame_interpolation_scale=1.0" \
+  -F "seed=42" \
+  -o sync_t2v_interpolated.mp4
+```
+
+## Notes
+
+- This is a post-processing feature. It does not modify the diffusion denoising
+  schedule.
+- Higher interpolation exponents increase post-processing time and memory usage.
+- If the interpolation model weights are not available locally,
+  `frame_interpolation_model_path` may point to a Hugging Face repo containing
+  `flownet.pkl`.
@@ -1,169 +0,0 @@
-# CFG-Parallel Guide
-
-
-## Table of Content
-
- [Overview](#overview)
- [Quick Start](#quick-start)
- [Example Script](#example-script)
- [Configuration Parameters](#configuration-parameters)
- [Best Practices](#best-practices)
- [Troubleshooting](#troubleshooting)
- [Summary](#summary)
-
---
-
-## Overview
-
-CFG-Parallel accelerates diffusion models by distributing positive and negative classifier-free guidance (CFG) passes across different GPUs, providing ~1.8x speedup when CFG is enabled. It's ideal for image editing tasks that require guidance scales greater than 1.0.
-
-See supported models list in [Supported Models](../../diffusion_features.md#supported-models).
-
---
-
-## Quick Start
-
-### Basic Usage
-
-Simplest working example:
-
-```python
-from vllm_omni import Omni
-from vllm_omni.diffusion.data import DiffusionParallelConfig
-from vllm_omni.inputs.data import OmniDiffusionSamplingParams
-from PIL import Image
-
-omni = Omni(
-    model="Qwen/Qwen-Image-Edit",
-    parallel_config=DiffusionParallelConfig(cfg_parallel_size=2),  # Enable CFG-Parallel
-)
-
-input_image = Image.open("input.png").convert("RGB")
-outputs = omni.generate(
-    {
-        "prompt": "turn this cat to a dog",
-        "negative_prompt": "low quality, blurry",
-        "multi_modal_data": {"image": input_image},
-    },
-    OmniDiffusionSamplingParams(
-        true_cfg_scale=4.0,
-        num_inference_steps=50,
-    ),
-)
-```
-
---
-
-## Example Script
-
-### Offline Inference
-
-Use python script under `examples/offline_inference/image_to_image/image_edit.py`:
-
-```bash
-cd examples/offline_inference/image_to_image/
-python image_edit.py \
-  --model "Qwen/Qwen-Image-Edit" \
-  --image "input.png" \
-  --prompt "turn this cat to a dog" \
-  --negative-prompt "low quality, blurry" \
-  --cfg-scale 4.0 \
-  --output "edited_image.png" \
-  --cfg-parallel-size 2
-```
-
-### Online Serving
-
-Enable CFG-Parallel in online serving:
-
-```bash
-# Default configuration
-vllm serve Qwen/Qwen-Image-Edit --omni --port 8091 --cfg-parallel-size 2
-
-```
-
---
-
-## Configuration Parameters
-
-In `DiffusionParallelConfig`
-
-| Parameter | Type | Default | Description |
-|-----------|------|---------|-------------|
-| `cfg_parallel_size` | int | 1 | Number of GPUs for CFG parallelism. Set to 2 to enable CFG-Parallel (rank 0 for positive, rank 1 for negative branch) |
-
-
-!!! info
-    Most models support `cfg_parallel_size=2` (positive branch on rank 0, negative branch on rank 1). **Bagel** is an exception: it supports `cfg_parallel_size=3`, which adds a third branch on rank 2 for full three-way CFG parallelism.
-
-
---
-
-## Best Practices
-
-### When to Use
-
-**Good for:**
-
- Tasks requiring classifier-free guidance
- Multi-GPU setups (at least 2 GPUs available)
- Combining with other parallelism methods (sequence/tensor parallel)
-
-**Not for:**
-
- Single GPU setups
- Models that don't support CFG-Parallel (check [supported models](../../diffusion_features.md#supported-models))
- Workloads without negative prompts or classifier-free guidance
- Very short inference runs (< 10 steps) where parallelism overhead may outweigh benefits
-
-### Expected Performance
-
-| Configuration | Speedup | Quality | Use Case |
-|--------------|---------|---------|----------|
-| CFG-Parallel (2 GPUs) | 1.5~1.8x | No degradation | Large model, VRAM limited |
-
---
-
-## Troubleshooting
-
-### Common Issue 1: No Speedup with CFG-Parallel
-
-**Symptoms**: CFG-Parallel enabled but no performance improvement
-
-**Solutions**:
-
-1. **Ensure CFG scale is set correctly:**
-```python
-# Bad: No CFG effect
-sampling_params = OmniDiffusionSamplingParams(num_inference_steps=50)
-
-# Good: CFG-Parallel will work
-sampling_params = OmniDiffusionSamplingParams(
-    num_inference_steps=50,
-    true_cfg_scale=4.0  # Must be > 1.0
-)
-```
-
-2. **Add negative prompt:**
-```python
-outputs = omni.generate(
-    {
-        "prompt": "beautiful landscape",
-        "negative_prompt": "low quality, blurry",  # Required for best results
-        "multi_modal_data": {"image": input_image}
-    },
-    sampling_params
-)
-```
-
-3. **Check model support:**
-   - Verify your model in [supported models](../../diffusion_features.md#supported-models)
-   - Some models don't support CFG-Parallel
-
---
-
-## Summary
-
-1. ✅ **Enable CFG-Parallel** - Set `cfg_parallel_size=2` in `DiffusionParallelConfig` to get speedup when using CFG
-2. ✅ **Set CFG Scale** - Ensure `true_cfg_scale > 1.0` in `OmniDiffusionSamplingParams` for CFG-Parallel to take effect
-3. ✅ **Check Model Support** - Verify your model supports CFG-Parallel in [supported models](../../diffusion_features.md#supported-models)
@@ -1,87 +0,0 @@
-# Expert Parallelism Guide
-
-
-## Table of Content
-
- [Overview](#overview)
- [Quick Start](#quick-start)
- [Configuration Parameters](#configuration-parameters)
- [Best Practices](#best-practices)
- [Summary](#summary)
-
---
-
-## Overview
-
-Unlike Tensor Parallelism which shards every layer's weights, Expert Parallelism (EP) only shards the MoE expert MLP blocks. This significantly reduces the memory footprint of MoE models (e.g., HunyuanImage3.0) while maintaining constant dense-equivalent compute efficiency.
-
-During the forward pass, a gating mechanism routes tokens to their designated experts, requiring all-to-all communication to dispatch tokens to the correct ranks and combine results.
-
-See supported models list in [Supported Models](../../diffusion_features.md#supported-models).
-
-!!! note "EP Size Constraint"
-    The effective EP size equals `tp × sp × cfg × dp`. At least one of TP/SP/CFG/DP must be set when EP is enabled.
-
---
-
-## Quick Start
-
-### Basic Usage
-
-```python
-from vllm_omni import Omni
-from vllm_omni.inputs.data import OmniDiffusionSamplingParams
-from vllm_omni.diffusion.data import DiffusionParallelConfig
-
-omni = Omni(
-    model="tencent/HunyuanImage-3.0",
-    parallel_config=DiffusionParallelConfig(
-        tensor_parallel_size=8,
-        enable_expert_parallel=True,
-    ),
-)
-
-outputs = omni.generate(
-    "A brown and white dog is running on the grass",
-    OmniDiffusionSamplingParams(
-        num_inference_steps=50,
-        width=1024,
-        height=1024,
-    ),
-)
-```
-
---
-
-## Configuration Parameters
-
-In `DiffusionParallelConfig`:
-
-| Parameter | Type | Default | Description |
-|-----------|------|---------|-------------|
-| `enable_expert_parallel` | bool | False | Enable Expert Parallelism for MoE models |
-
-EP size is derived automatically as `tp × sp × cfg × dp` — configure at least one of those to set the EP degree.
-
---
-
-## Best Practices
-
-### When to Use
-
-**Good for:**
-
- MoE models (e.g., HunyuanImage3.0) with numbers of experts
- Memory-constrained multi-GPU setups where only expert blocks need sharding
-
-**Not for:**
-
- Dense models (no MoE layers) — EP has no effect
- Single GPU setups
-
---
-
-## Summary
-
-1. ✅ **Enable EP** - Set `enable_expert_parallel=True` in `DiffusionParallelConfig` for MoE models
-2. ✅ **Set parallelism degree** - At least one of `tensor_parallel_size` / `ulysses_degree` / `cfg_parallel_size` must be > 1 to define the EP size
@@ -1,149 +0,0 @@
-# HSDP Guide
-
-
-## Table of Content
-
- [Overview](#overview)
- [Quick Start](#quick-start)
- [Example Script](#example-script)
- [Configuration Parameters](#configuration-parameters)
- [Best Practices](#best-practices)
- [Summary](#summary)
-
---
-
-## Overview
-
-HSDP (Hybrid Sharded Data Parallel) shards model weights across GPUs to reduce per-GPU memory usage. This enables inference of large models (e.g., Wan2.2 14B) on GPUs with limited memory.
-
-Unlike Tensor Parallelism which splits computation, HSDP uses PyTorch's FSDP2 to shard and redistribute weights at runtime. Each GPU only holds a fraction of the model weights, and weights are gathered on-demand during forward passes.
-
-See supported models list in [Supported Models](../../diffusion_features.md#supported-models).
-
-**Operating Modes:**
-
- **Standalone Mode**: HSDP alone without other parallelism. Must specify `hsdp_shard_size` explicitly.
- **Combined Mode**: HSDP overlays on top of other parallelism (Ulysses-SP, CFG-Parallel). HSDP dimensions must match world_size.
-
---
-
-## Quick Start
-
-### Basic Usage
-
-Simplest working example (standalone HSDP, shard across 4 GPUs):
-
-```python
-from vllm_omni import Omni
-from vllm_omni.inputs.data import OmniDiffusionSamplingParams
-from vllm_omni.diffusion.data import DiffusionParallelConfig
-
-omni = Omni(
-    model="Wan-AI/Wan2.2-T2V-A14B-Diffusers",
-    parallel_config=DiffusionParallelConfig(
-        use_hsdp=True,
-        hsdp_shard_size=4,  # Shard across 4 GPUs
-    ),
-)
-
-outputs = omni.generate(
-    "A cat playing piano",
-    OmniDiffusionSamplingParams(num_inference_steps=50),
-)
-```
-
-### Combined with Sequence Parallel
-
-```python
-omni = Omni(
-    model="Wan-AI/Wan2.2-T2V-A14B-Diffusers",
-    parallel_config=DiffusionParallelConfig(
-        ulysses_degree=4,  # Sequence parallel
-        use_hsdp=True,     # HSDP overlays on SP
-    ),
-)
-```
-
---
-
-## Example Script
-
-### Offline Inference
-
-Use Python script under `examples/offline_inference/image_to_video/`:
-
-```bash
-# Standalone HSDP: shard across 4 GPUs
-python examples/offline_inference/image_to_video/image_to_video.py \
-    --model Wan-AI/Wan2.2-T2V-A14B-Diffusers \
-    --use-hsdp \
-    --hsdp-shard-size 4
-
-# Combined HSDP + Sequence Parallel
-python examples/offline_inference/image_to_video/image_to_video.py \
-    --model Wan-AI/Wan2.2-T2V-A14B-Diffusers \
-    --ulysses-degree 4 \
-    --use-hsdp
-```
-
-### Online Serving
-
-**Standalone HSDP** (shard model across 4 GPUs):
-
-```bash
-vllm serve Wan-AI/Wan2.2-T2V-A14B-Diffusers --omni --port 8091 \
-    --use-hsdp --hsdp-shard-size 4
-```
-
-**Combined with Sequence Parallel**:
-
-```bash
-vllm serve Wan-AI/Wan2.2-T2V-A14B-Diffusers --omni --port 8091 \
-    --use-hsdp --usp 4
-```
-
---
-
-## Configuration Parameters
-
-In `DiffusionParallelConfig`:
-
-| Parameter | Type | Default | Description |
-|-----------|------|---------|-------------|
-| `use_hsdp` | bool | False | Enable HSDP |
-| `hsdp_shard_size` | int | -1 | Number of GPUs to shard weights across. `-1` = auto (requires other parallelism > 1) |
-| `hsdp_replicate_size` | int | 1 | Number of replica groups. Each group holds a full sharded copy |
-
-**Constraints:**
-
- `hsdp_replicate_size × hsdp_shard_size == world_size`
- HSDP cannot be used with Tensor Parallelism (`tensor_parallel_size` must be 1)
-
---
-
-## Best Practices
-
-### When to Use
-
-**Good for:**
-
- Very large models (e.g., Wan2.2 14B)
- Multi-GPU setups where memory reduction is the primary goal
- Combining with Sequence Parallelism for large video models
-
-**Not for:**
-
- Models that fit comfortably in single-GPU memory
- Use cases requiring Tensor Parallelism (HSDP and TP are mutually exclusive)
-
-### Adding HSDP Support to New Models
-
-For detailed instructions on adding HSDP support to new models, see the [HSDP Contributing Guide](../../../design/feature/hsdp.md).
-
---
-
-## Summary
-
-1. ✅ **Enable HSDP** - Set `use_hsdp=True` and `hsdp_shard_size` to reduce per-GPU memory for large models
-2. ✅ **Combine with SP** - Use together with `ulysses_degree` for video models requiring both memory reduction and sequence parallelism
-3. ⚠️ **Incompatible with TP** - `tensor_parallel_size` must be 1 when HSDP is enabled
@@ -1,16 +0,0 @@
-# Parallelism Acceleration Guide
-
-This guide covers the parallelism methods in vLLM-Omni for speeding up diffusion model inference and reducing per-device memory requirements.
-
-## Supported Methods
-
-| Method | Description |
-|--------|-------------|
-| **[Tensor Parallelism](tensor_parallel.md)** | Shards DiT weights across GPUs to reduce per-GPU memory |
-| **[Sequence Parallelism](sequence_parallel.md)** | Splits sequence dimension across GPUs (Ulysses-SP, Ring-Attention, or hybrid) for high-resolution images and videos |
-| **[CFG-Parallel](cfg_parallel.md)** | Runs CFG positive/negative branches on separate GPUs for ~1.8x speedup on guided generation |
-| **[VAE Patch Parallelism](vae_patch_parallel.md)** | Distributes VAE decode spatially across GPUs to reduce peak VAE memory |
-| **[HSDP](hsdp.md)** | Shards full model weights via PyTorch FSDP2 to enable large-model inference on memory-constrained GPUs |
-| **[Expert Parallelism](expert_parallel.md)** | Shards MoE expert blocks across GPUs for MoE models (e.g. HunyuanImage3.0) |
-
-See [Supported Models](../../diffusion_features.md#supported-models) for per-model compatibility.
@@ -1,233 +0,0 @@
-# Sequence Parallelism Guide
-
-
-## Table of Content
-
- [Overview](#overview)
- [Quick Start](#quick-start)
- [Example Script](#example-script)
- [Best Practices](#best-practices)
- [Troubleshooting](#troubleshooting)
- [Summary](#summary)
-
---
-
-## Overview
-
-Sequence parallelism splits the input along the sequence dimension across multiple GPUs, allowing each device to process only a portion of the sequence. vLLM-Omni provides 1.5x-3.6x speedup for large images and videos using DeepSpeed Ulysses, Ring-Attention, or hybrid approaches. Use sequence parallelism when generating high-resolution images/videos that don't fit on a single GPU or require faster inference.
-
-See supported models list in [Diffusion Features - Supported Models](../../diffusion_features.md#supported-models).
-
-**Supported Methods:**
-
- **DeepSpeed Ulysses Sequence Parallel (Ulysses-SP)** ([paper](https://arxiv.org/pdf/2309.14509)): Uses all-to-all communication for subset of attention heads per device
- **Ring-Attention** ([paper](https://arxiv.org/abs/2310.01889)): Uses ring-based P2P communication with sharded sequence dimension throughout
- **Hybrid Ulysses + Ring**: Combines both for larger scale parallelism (`ulysses_degree × ring_degree`)
-
---
-
-## Quick Start
-
-### Basic Usage - Ulysses-SP
-
-Simplest working example with Ulysses Sequence Parallel:
-
-```python
-from vllm_omni import Omni
-from vllm_omni.inputs.data import OmniDiffusionSamplingParams
-from vllm_omni.diffusion.data import DiffusionParallelConfig
-
-omni = Omni(
-    model="Qwen/Qwen-Image",
-    parallel_config=DiffusionParallelConfig(ulysses_degree=2)  # Enable Ulysses-SP
-)
-
-outputs = omni.generate(
-    "A cat sitting on a windowsill",
-    OmniDiffusionSamplingParams(num_inference_steps=50, width=1024, height=1024),
-)
-```
-
-!!! note "Experimental UAA mode"
-    `ulysses_mode="advanced_uaa"` is an experimental extension to Ulysses-SP. It lets Ulysses attention handle arbitrary sequence lengths and arbitrary attention head counts without relying on `attention_mask`-based token padding.
-
-    In hybrid Ulysses + Ring mode, Ring still requires every rank in the same ring group to observe the same post-Ulysses sequence length. If that condition is not met, vLLM-Omni raises a validation error instead of entering the ring kernel with inconsistent shapes.
-
-To enable the experimental UAA mode, use a model/configuration that requires it. For example, `Tongyi-MAI/Z-Image-Turbo` has 30 attention heads, so `ulysses_degree=4` requires UAA because 30 is not divisible by 4:
-
-```python
-omni = Omni(
-    model="Tongyi-MAI/Z-Image-Turbo",
-    parallel_config=DiffusionParallelConfig(
-        ulysses_degree=4,
-        ulysses_mode="advanced_uaa",
-    ),
-)
-```
-
-### Alternative Methods
-
-**Ring-Attention** (better for very long sequences):
-
-```python
-omni = Omni(
-    model="Qwen/Qwen-Image",
-    parallel_config=DiffusionParallelConfig(ring_degree=2)  # Enable Ring-Attention
-)
-```
-
-**Hybrid Ulysses + Ring** (for larger scale):
-
-```python
-omni = Omni(
-    model="Qwen/Qwen-Image",
-    parallel_config=DiffusionParallelConfig(ulysses_degree=2, ring_degree=2)  # 4 GPUs total
-)
-```
-
---
-
-## Example Script
-
-### Offline Inference
-
-Use Python script under `examples/offline_inference/text_to_image/text_to_image.py`:
-
-**Ulysses-SP:**
-
-```bash
-python examples/offline_inference/text_to_image/text_to_image.py \
-    --model Qwen/Qwen-Image \
-    --prompt "A cat sitting on a windowsill" \
-    --ulysses-degree 2 \
-    --width 1024 --height 1024
-```
-
-**Ring-Attention:**
-
-```bash
-python examples/offline_inference/text_to_image/text_to_image.py \
-    --model Qwen/Qwen-Image \
-    --prompt "A cat sitting on a windowsill" \
-    --ring-degree 2 \
-    --width 1024 --height 1024
-```
-
-**Hybrid Ulysses + Ring:**
-
-```bash
-# Hybrid: 2 Ulysses × 2 Ring = 4 GPUs total
-python examples/offline_inference/text_to_image/text_to_image.py \
-    --model Qwen/Qwen-Image \
-    --prompt "A cat sitting on a windowsill" \
-    --ulysses-degree 2 --ring-degree 2 \
-    --width 1024 --height 1024
-```
-
-### Online Serving
-
-**Ulysses-SP:**
-
-```bash
-# Text-to-image (requires >= 2 GPUs)
-vllm serve Qwen/Qwen-Image --omni --port 8091 --usp 2
-```
-
-**Ulysses-SP with UAA mode** (for models with non-divisible head counts):
-
-```bash
-vllm serve Tongyi-MAI/Z-Image-Turbo --omni --port 8091 --usp 4 --ulysses-mode advanced_uaa
-```
-
-**Ring-Attention:**
-
-```bash
-# Text-to-image (requires >= 2 GPUs)
-vllm serve Qwen/Qwen-Image --omni --port 8091 --ring 2
-```
-
-**Hybrid Ulysses + Ring:**
-
-```bash
-# Text-to-image (requires >= 4 GPUs)
-vllm serve Qwen/Qwen-Image --omni --port 8091 --usp 2 --ring 2
-```
-
---
-
-## Configuration Parameters
-
-In `DiffusionParallelConfig`:
-
-| Parameter | Type | Default | Description |
-|-----------|------|---------|-------------|
-| `ulysses_degree` | int | 1 | Number of GPUs for Ulysses-SP. Uses all-to-all communication. |
-| `ring_degree` | int | 1 | Number of GPUs for Ring-Attention. Uses P2P ring communication. |
-| `ulysses_mode` | str | `"default"` | Ulysses attention mode. Set to `"advanced_uaa"` to handle arbitrary sequence lengths and head counts without padding. |
-
-**Notes:**
- Total sequence parallel size equals to `ulysses_degree × ring_degree`
- Degrees must evenly divide the sequence length for optimal performance (or use `ulysses_mode="advanced_uaa"` for Ulysses-SP)
-
-
-## Best Practices
-
-### When to Use
-
-**Good for:**
-
- Large images (1024x1024 or higher) or videos
- Fast inter-GPU communication, larger bandwidth (e.g., NVLink)
-
-**Not for:**
-
- Small images (<1024px) - overhead exceeds benefit, use single GPU with cache instead
-
-
---
-
-## Troubleshooting
-
-### Common Issue 1: Performance Not Scaling
-
-**Symptoms**: Adding GPUs doesn't improve speed proportionally, or higher parallelism degree is slower
-
-**Diagnosis:**
-```bash
-# Check GPU topology
-nvidia-smi topo -m
-
-```
-
-**Solutions:**
-
-1. Check inter-GPU communication - NVLink is better than PCIe
-2. Reduce parallelism degree if over-parallelized:
-```python
-# If 4 GPUs is slower than 2
-parallel_config=DiffusionParallelConfig(ulysses_degree=2)
-```
-3. Try to switch between Ring-Attention and Ulysses-SP
-
- Ring-Attention has advantages, like communication-computation overlap, but the block-wise loop overhead is relatively higher, especially for short sequences
- Ulysses-SP: can benefit from larger bandwidth (such as NVLink), with two major constraints, the sequence length should be divisible by usp size, and the number of heads should be divisible by usp size (or use `ulysses_mode="advanced_uaa"`)
-
-
-### Common Issue 2: Out of Memory (OOM)
-
-**Symptoms**: CUDA OOM errors or process crashes with memory errors
-
-**Solutions:**
-
-1. Increase parallelism degree to split sequence more:
-```python
-parallel_config=DiffusionParallelConfig(ulysses_degree=4)  # From 2
-```
-2. Combine with other parallelism method, e.g., tensor parallel, and memory optimization methods, e.g., cpu offloading.
-
-
-## Summary
-
-1. ✅ **Enable Sequence Parallelism** - Set `ulysses_degree` or `ring_degree` for long sequence generation
-2. ✅ **UAA mode** - Use `ulysses_mode="advanced_uaa"` when head count is not divisible by `ulysses_degree`
-3. ✅ **Troubleshooting** - Check GPU topology with `nvidia-smi topo -m`, reduce degree if performance doesn't scale
@@ -1,151 +0,0 @@
-# Tensor Parallelism Guide
-
-
-## Table of Content
-
- [Overview](#overview)
- [Quick Start](#quick-start)
- [Example Script](#example-script)
- [Configuration Parameters](#configuration-parameters)
- [Best Practices](#best-practices)
- [Troubleshooting](#troubleshooting)
- [Summary](#summary)
-
---
-
-## Overview
-
-Tensor Parallelism (TP) shards some model weights across multiple GPUs, usually the Linear layers. This enables running large models that don't fit on a single GPU. It's essential for memory-constrained setups or very large models.
-
-See supported models list in [Supported Models](../../diffusion_features.md#supported-models).
-
-!!! note "TP Limitations for Diffusion Models"
-    We currently implement Tensor Parallelism (TP) only for the DiT (Diffusion Transformer) blocks. This is because the `text_encoder` component in vLLM-Omni uses the original Transformers implementation, which does not yet support TP.
-
-    - Good news: The text_encoder typically has minimal impact on overall inference performance.
-    - Bad news: When TP is enabled, every TP process retains a full copy of the text_encoder weights, leading to significant GPU memory waste.
-
-    We are actively refactoring this design to address this. For details and progress, please refer to [Issue #771](https://github.com/vllm-project/vllm-omni/issues/771).
-
---
-
-## Quick Start
-
-
-### Basic Usage
-
-Simplest working example:
-
-```python
-from vllm_omni import Omni
-from vllm_omni.inputs.data import OmniDiffusionSamplingParams
-from vllm_omni.diffusion.data import DiffusionParallelConfig
-
-omni = Omni(
-    model="Tongyi-MAI/Z-Image-Turbo",
-    parallel_config=DiffusionParallelConfig(tensor_parallel_size=2),  # Enable TP
-)
-
-outputs = omni.generate(
-    "a cat reading a book",
-    OmniDiffusionSamplingParams(num_inference_steps=9),
-)
-```
-
---
-
-## Example Script
-
-### Offline Inference
-
-Use Python script under `examples/offline_inference`, and enable TP:
-
-```bash
-# Text-to-Image with Qwen-Image
-python examples/offline_inference/text_to_image/text_to_image.py \
-    --model Qwen/Qwen-Image \
-    --tensor-parallel-size 2
-
-# Image Editing with Qwen-Image-Edit
-python examples/offline_inference/image_to_image/image_edit.py \
-    --model Qwen/Qwen-Image-Edit \
-    --image input.png \
-    --prompt "Edit description" \
-    --tensor-parallel-size 2
-```
-
-### Online Serving
-
-You can enable tensor parallelism in online serving via `--tensor-parallel-size`:
-
-```bash
-# Text-to-Image with Qwen-Image on 2 GPUs
-vllm serve Qwen/Qwen-Image --omni --port 8091 \
-    --tensor-parallel-size 2
-
-# Text-to-Image with Z-Image (TP=2 only)
-vllm serve Tongyi-MAI/Z-Image-Turbo --omni --port 8091 \
-    --tensor-parallel-size 2
-```
-
---
-
-## Configuration Parameters
-
-In `DiffusionParallelConfig`:
-
-| Parameter | Type | Default | Description |
-|-----------|------|---------|-------------|
-| `tensor_parallel_size` | int | 1 | Number of GPUs to shard model weights across. Must divide number of heads. |
-
-
---
-
-## Best Practices
-
-### When to Use
-
-**Good for:**
-
- Large models that don't fit on a single GPU, especially for models with large DiT blocks (transformer layers)
- Memory-constrained environments
-
-**Not for:**
-
- When maximum throughput is needed and memory is sufficient
- Models with incompatible dimensions (e.g., Z-Image `num_heads=30`, which now supports `tensor_parallel_size=2`)
-
-
-## Troubleshooting
-
-### Common Issue 1: Out of Memory (OOM)
-
-**Symptoms**: CUDA OOM errors during model loading or inference, process crashes with memory errors
-
-**Solution**:
-```python
-# Step 1: Enable TP with smallest degree
-parallel_config=DiffusionParallelConfig(tensor_parallel_size=2)
-
-# Step 2: If still OOM, increase TP degree
-parallel_config=DiffusionParallelConfig(tensor_parallel_size=4)
-
-```
-
-### Common Issue 2: Divisibility Error
-
-**Symptoms**: Error like "Model dimension X not divisible by tensor_parallel_size Y"
-
-**Solutions**:
-1. Check model-specific constraints (e.g., Z-Image only supports TP=2)
-2. Use a smaller TP size that divides model dimensions
-3. Consult [Supported Models](../../diffusion_features.md#supported-models) for compatible TP sizes
-
-
---
-
-## Summary
-
-1. ✅ **Enable TP** - Set `--tensor-parallel-size` to reduce per-GPU memory
-2. ✅ **Increase TP size** - Only increase if OOM persists
-3. ⚠️ **Text encoder not sharded** - Known limitation
@@ -1,200 +0,0 @@
-# VAE Patch Parallelism Guide
-
-
-## Table of Content
-
- [Overview](#overview)
- [Quick Start](#quick-start)
- [Example Script](#example-script)
- [Configuration Parameters](#configuration-parameters)
- [Best Practices](#best-practices)
- [Troubleshooting](#troubleshooting)
- [Summary](#summary)
-
---
-
-## Overview
-
-VAE Patch Parallelism distributes the VAE (Variational AutoEncoder) decode/encode computation across multiple GPUs by splitting the latent space into spatial tiles or patches. Each GPU processes a subset of tiles in parallel, significantly reducing peak memory consumption during the VAE decode stage while maintaining output quality.
-
-This is particularly useful for:
- **High-resolution image generation** where VAE decode can become a memory bottleneck
- **Memory-constrained environments** where the VAE decode activation peak exceeds available VRAM
- **Multi-GPU setups** where you want to leverage distributed resources for the VAE stage
-
-See supported models list in [Supported Models](../../diffusion_features.md#supported-models).
-
-
-VAE Patch Parallelism uses two strategies based on image size:
-
-| Strategy | Use Case | How It Works | Overlap Handling | Output Quality |
-|----------|----------|--------------|------------------|----------------|
-| **Tiled Decode** | Large images (triggers VAE tiling) | Distributes existing VAE tiling computation across ranks. Each rank decodes a subset of overlapping tiles. | Uses VAE's native `blend_v` and `blend_h` functions to seamlessly merge overlapping regions | Bit-identical (same logic as single-GPU tiling) |
-| **Patch Decode** | Small images (no VAE tiling) | Splits latent into spatial patches with halos. Each rank decodes one patch with boundary context. | Halo regions provide edge context; core regions are directly stitched without blending | Near-identical (diff < 0.5%, visually imperceptible) |
-
-
-VAE Patch Parallelism **reuses the DiT process group** (`dit_group`) and does not initialize a separate ProcessGroup. This means:
-
- **Shared ranks**: VAE patch parallelism uses the same GPU ranks as DiT parallelism (Tensor Parallel, Sequence Parallel, etc.)
- **Combined usage**: VAE patch parallelism is typically used together with other parallelism methods
- **Configuration alignment**: The `vae_patch_parallel_size` should be no greater than the size of your DiT process group
-
---
-
-## Quick Start
-
-### Basic Usage
-
-Simplest working example:
-
-```python
-from vllm_omni import Omni
-from vllm_omni.inputs.data import OmniDiffusionSamplingParams
-from vllm_omni.diffusion.data import DiffusionParallelConfig
-
-# TP=2 for DiT, VAE patch parallel also uses these 2 GPUs
-omni = Omni(
-    model="Tongyi-MAI/Z-Image-Turbo",
-    parallel_config=DiffusionParallelConfig(
-        tensor_parallel_size=2,          # Enable tensor parallelism for DiT
-        vae_patch_parallel_size=2,       # Enable VAE patch parallelism
-    ),
-    vae_use_tiling=True,  # Required for VAE patch parallelism
-)
-
-outputs = omni.generate(
-    "a futuristic city at sunset, high resolution, 8k",
-    OmniDiffusionSamplingParams(
-        num_inference_steps=9,
-        height=1152,  # High resolution benefits from VAE patch parallel
-        width=1152,
-    ),
-)
-```
-
---
-
-## Example Script
-
-### Offline Inference
-
-Use Python script under `examples/offline_inference/text_to_image/`:
-
-```bash
-# Text-to-Image with Z-Image
-python examples/offline_inference/text_to_image/text_to_image.py \
-    --model Tongyi-MAI/Z-Image-Turbo \
-    --prompt "a futuristic city at sunset" \
-    --height 1152 \
-    --width 1152 \
-    --tensor-parallel-size 2 \
-    --vae-patch-parallel-size 2 \
-    --vae-use-tiling
-```
-
-### Online Serving
-
-You can enable VAE patch parallelism in online serving via `--vae-patch-parallel-size`:
-
-```bash
-# Text-to-Image with Z-Image, TP=2 + VAE patch parallel=2
-vllm serve Tongyi-MAI/Z-Image-Turbo --omni --port 8091 \
-    --tensor-parallel-size 2 \
-    --vae-patch-parallel-size 2 \
-    --vae-use-tiling
-```
-
---
-
-## Configuration Parameters
-
-In `DiffusionParallelConfig`:
-
-| Parameter | Type | Default | Description |
-|-----------|------|---------|-------------|
-| `vae_patch_parallel_size` | int | 1 | Number of GPUs for VAE patch/tile parallelism. Set to 2 or higher to enable. Should typically match `tensor_parallel_size` as they share the same process group. |
-
-Additional requirements:
-
-| Parameter | Type | Default | Description |
-|-----------|------|---------|-------------|
-| `vae_use_tiling` | bool | False | Must be set to `True` when using VAE patch parallelism. |
-
-!!! note "Automatic VAE Tiling"
-    When `vae_patch_parallel_size > 1` and the model has a distributed VAE (`DistributedVaeMixin`), the system automatically sets `vae_use_tiling=True` if not already enabled.
-
---
-
-## Best Practices
-
-### When to Use
-
-**Good for:**
-
- High-resolution image generation and long video generation
- Memory-constrained setups where VAE decode causes OOM
- Multi-GPU environments
-
-**Not for:**
-
- Low-resolution images/videos where VAE decode is not a bottleneck
- Single GPU setups should use vae tiling decode, but not parallel vae tiling decode
- Models that do not support vae patch parallel
-
---
-
-## Troubleshooting
-
-### Common Issue 1: Model Not Support VAE Patch Parallel
-
-**Symptoms**:
-```
-WARNING: vae_patch_parallel_size=2 is set but VAE patch parallelism is NOT enabled for xxxPipeline; ignoring.
-```
-
-**Root Cause**: VAE Patch Parallelism requires the model's VAE to implement `DistributedVaeMixin`. At startup, `vllm_omni/diffusion/registry.py` checks whether the instantiated pipeline has a `.vae` attribute that is an instance of `DistributedVaeMixin`. If it does not, the setting is silently ignored:
-
-```python
-vae_pp_size = od_config.parallel_config.vae_patch_parallel_size
-is_distributed_vae = hasattr(model, "vae") and isinstance(model.vae, DistributedVaeMixin)
-if vae_pp_size > 1 and not is_distributed_vae:
-    logger.warning(
-        "vae_patch_parallel_size=%d is set but VAE patch parallelism is NOT enabled for %s; ignoring.",
-        vae_pp_size,
-        od_config.model_class_name,
-    )
-```
-
-**Solutions**:
-
-1. **Use a supported model** (recommended): check [Supported Models](../../diffusion_features.md#supported-models) for the VAE-Patch-Parallel column.
-
-2. To add support for a new model, implement `DistributedVaeMixin` on its VAE class (contributions are welcome).
-
-
-### Common Issue 2: `vae_patch_parallel_size` Exceeds DiT Process Group Size
-
-**Symptoms**: Shows warning message, and vae patch parallel size is resized to DiT process group size
-
-**Root Cause**: VAE Patch Parallelism reuses the DiT process group.
-
-**Recommendation**: Always set `vae_patch_parallel_size` to be no greater than your DiT process group size.
-
-Note that the size of DiT process group size equals to:
-```text
-dit_parallel_size = data_parallel_size
-                  × cfg_parallel_size
-                  × sequence_parallel_size
-                  × pipeline_parallel_size
-                  × tensor_parallel_size
-
-```
-_sequence_parallel_size = ulysses_degree × ring_degree_
-
---
-
-## Summary
-
-1. ✅ **Enable VAE Patch Parallelism** - Set `vae_patch_parallel_size`， `vae_use_tiling=True` in `DiffusionParallelConfig` to reduce VAE decode peak memory
-2. ✅ **Use Long Sequence** - VAE patch parallelism benefits are most apparent at long sequence decoding
-3. ✅ **Combine with other parallelism methods** - Suggest to use together with Tensor Parallel or CFG-Parallel for maximum memory savings
@@ -0,0 +1,477 @@
+# Parallelism Acceleration Guide
+
+This guide includes how to use parallelism methods in vLLM-Omni to speed up diffusion model inference as well as reduce the memory requirement on each device.
+
+## Overview
+
+The following parallelism methods are currently supported in vLLM-Omni:
+
+1. DeepSpeed Ulysses Sequence Parallel (DeepSpeed Ulysses-SP) ([arxiv paper](https://arxiv.org/pdf/2309.14509)): Ulysses-SP splits the input along the sequence dimension and uses all-to-all communication to allow each device to compute only a subset of attention heads.
+
+2. [Ring-Attention](#ring-attention) - splits the input along the sequence dimension and uses ring-based P2P communication to accumulate attention results, keeping the sequence dimension sharded
+
+3. Classifier-Free-Guidance Parallel (CFG-Parallel): CFG-Parallel runs the positive/negative prompts of classifier-free guidance (CFG) on different devices, then merges on a single device to perform the scheduler step.
+
+4. [Tensor Parallelism](#tensor-parallelism): Tensor parallelism shards model weights across devices. This can reduce per-GPU memory usage. Note that for diffusion models we currently shard the majority of layers within the DiT.
+
+5. [VAE Patch Parallelism](#vae-patch-parallelism): VAE patch parallelism shards VAE decode spatially across ranks. This can reduce the peak memory of VAE decode and (depending on resolution and communication overhead) speed up VAE decode.
+
+6. [HSDP](#hsdp): Hybrid Sharded Data Parallel shards model weights across GPUs using PyTorch FSDP2. This reduces per-GPU memory usage, enabling inference of large models on GPUs with limited memory.
+
+7. [Expert Parallel](#expert-parallelism): Expert Parallelism shards the Experts of a Mixture-of-Experts (MoE) layer across multiple devices. During the forward, a gating mechanism routes tokens to their designated experts, necessitating cross-cards communication(all-to-all) to dispatch tokens to the correct ranks and combine the results. This parallelism allows for massive scaling of model parameters without a proportional increase in the computational load per device.
+
+The following table shows which models are currently supported by parallelism method:
+
+### ImageGen
+
+| Model                    | Model Identifier                     | Ulysses-SP | Ring-SP | CFG-Parallel | Tensor-Parallel | VAE-Patch-Parallel | Expert-Parallel | HSDP |
+|--------------------------|--------------------------------------|:----------:|:-------:|:------------:|:---------------:|:------------------:|:---------------:|:----:|
+| **LongCat-Image**        | `meituan-longcat/LongCat-Image`      |     ✅      |    ✅    |      ❌       |        ✅        |         ❌          |      N/A        | ❌   |
+| **LongCat-Image-Edit**   | `meituan-longcat/LongCat-Image-Edit` |     ✅      |    ✅    |      ❌       |        ✅        |         ❌          |      N/A        | ❌   |
+| **Ovis-Image**           | `OvisAI/Ovis-Image`                  |     ❌      |    ❌    |      ❌       |        ❌        |         ❌          |      N/A        | ❌   |
+| **Qwen-Image**           | `Qwen/Qwen-Image`                    |     ✅      |    ✅    |      ✅       |        ✅        |         ✅          |      N/A        | ❌   |
+| **Qwen-Image-Edit**      | `Qwen/Qwen-Image-Edit`               |     ✅      |    ✅    |      ✅       |        ✅        |         ❌          |      N/A        | ❌   |
+| **Qwen-Image-Edit-2509** | `Qwen/Qwen-Image-Edit-2509`          |     ✅      |    ✅    |      ✅       |        ✅        |         ❌          |      N/A        | ❌   |
+| **Qwen-Image-Layered**   | `Qwen/Qwen-Image-Layered`            |     ✅      |    ✅    |      ✅       |        ✅        |         ❌          |      N/A        | ❌   |
+| **Z-Image**              | `Tongyi-MAI/Z-Image-Turbo`           |     ✅      |    ✅    |      ❌       |  ✅ (TP=2 only)  |         ✅          |      N/A        | ❌   |
+| **Stable-Diffusion3.5**  | `stabilityai/stable-diffusion-3.5`   |     ❌      |    ❌    |      ❌       |        ✅        |         ✅          |      N/A        | ❌   |
+| **FLUX.2-klein**         | `black-forest-labs/FLUX.2-klein-4B`  |     ✅      |    ✅    |      ❌       |        ✅        |         ❌          |      N/A        | ✅   |
+| **FLUX.1-dev**           | `black-forest-labs/FLUX.1-dev`       |     ❌      |    ❌    |      ✅       |        ✅        |         ❌          |      N/A        | ✅   |
+| **FLUX.2-dev**           | `black-forest-labs/FLUX.2-dev`       |     ❌      |    ❌    |      ❌       |        ✅        |         ❌          |      N/A        | ✅   |
+| **HunyuanImage3.0**      | `tencent/HunyuanImage-3.0`, `tencent/HunyuanImage-3.0-Instruct` |     ❌      |    ❌    |      ❌       |        ✅        |         ❌          |      ✅        | ❌   |
+| **Bagel**                  | `ByteDance-Seed/BAGEL-7B-MoT` |     ✅      |    ✅    |      ✅       |        ✅        |         ❌          |      N/A        | ❌   |
+| **DreamID-Omni**           | `XuGuo699/DreamID-Omni`       |     ❌      |    ❌    |      ✅       |        ❌        |         ❌          |      N/A        | ❌   |
+| **FLUX.1-Kontext-dev**   | `black-forest-labs/FLUX.1-Kontext-dev`       |     ❌      |    ❌    |      ❌       |        ✅        |         ❌          |      N/A        | ✅   |
+| **OmniGen2**               | `OmniGen2/OmniGen2`           |     ❌      |    ❌    |      ❌       |        ✅        |         ❌          |      N/A        | ❌   |
+
+!!! note "TP Limitations for Diffusion Models"
+    We currently implement Tensor Parallelism (TP) only for the DiT (Diffusion Transformer) blocks. This is because the `text_encoder` component in vLLM-Omni uses the original Transformers implementation, which does not yet support TP.
+
+    - Good news: The text_encoder typically has minimal impact on overall inference performance.
+    - Bad news: When TP is enabled, every TP process retains a full copy of the text_encoder weights, leading to significant GPU memory waste.
+
+    We are actively refactoring this design to address this. For details and progress, please refer to [Issue #771](https://github.com/vllm-project/vllm-omni/issues/771).
+
+
+!!! note "Why Z-Image is TP=2 only"
+    Z-Image Turbo is currently limited to `tensor_parallel_size` of **1 or 2** due to model shape divisibility constraints.
+    For example, the model has `n_heads=30` and a final projection out dimension of `64`, so valid TP sizes must divide both 30 and 64; the only common divisors are **1 and 2**.
+
+### VideoGen
+
+| Model | Model Identifier | Ulysses-SP | Ring-Attention | Tensor-Parallel | HSDP | VAE-Patch-Parallel |
+|-------|------------------|:----------:|:--------------:|:---------------:|:----:| :----:|
+| **Wan2.1** | `Wan-AI/Wan2.1-T2V-1.3B-Diffusers` | ✅ | ✅ | ✅ | ✅ | ✅ |
+| **Wan2.1** | `Wan-AI/Wan2.1-T2V-14B-Diffusers` | ✅ | ✅ | ✅ | ✅ | ✅ |
+| **Wan2.2** | `Wan-AI/Wan2.2-T2V-A14B-Diffusers` | ✅ | ✅ | ✅ | ✅ | ✅ |
+| **LTX-2** | `Lightricks/LTX-2` | ✅ | ✅ | ✅ | ❌ | ❌ |
+
+### Tensor Parallelism
+
+Tensor parallelism splits model parameters across GPUs. In vLLM-Omni, tensor parallelism is configured via `DiffusionParallelConfig.tensor_parallel_size`.
+
+#### Offline Inference
+
+```python
+from vllm_omni import Omni
+from vllm_omni.diffusion.data import DiffusionParallelConfig
+
+omni = Omni(
+    model="Tongyi-MAI/Z-Image-Turbo",
+    parallel_config=DiffusionParallelConfig(tensor_parallel_size=2),
+)
+
+outputs = omni.generate(
+    "a cat reading a book",
+    OmniDiffusionSamplingParams(
+        num_inference_steps=9,
+        width=512,
+        height=512,
+    ),
+)
+```
+
+### VAE Patch Parallelism
+
+VAE patch parallelism distributes the VAE decode workload across multiple ranks by splitting the latent spatially. It is configured via `DiffusionParallelConfig.vae_patch_parallel_size` and can be combined with other parallelism methods (e.g., TP).
+
+!!! note "Enablement and feature gate"
+    - VAE patch parallelism is currently **enabled only for validated pipelines** (check [ImageGen](#imagegen) and [VideoGen](#videogen) for more information).
+    - If `vae_patch_parallel_size > 1` is set for a validated pipeline, vLLM-Omni will automatically enable `vae_use_tiling` as a safety gate. (We use `vae_use_tiling` because it indicates the VAE supports diffusers tiling parameters like `tile_latent_min_size` and `tile_overlap_factor`.)
+
+#### Offline Inference
+
+```python
+from vllm_omni import Omni
+from vllm_omni.diffusion.data import DiffusionParallelConfig
+
+omni = Omni(
+    model="Tongyi-MAI/Z-Image-Turbo",
+    parallel_config=DiffusionParallelConfig(
+        tensor_parallel_size=2,
+        vae_patch_parallel_size=2,
+    ),
+    vae_use_tiling=True,
+)
+
+outputs = omni.generate(
+    prompt="a cat reading a book",
+    num_inference_steps=9,
+    width=1024,
+    height=1024,
+)
+```
+
+#### How it works (method selection)
+
+VAE patch parallelism automatically selects between two internal decode methods based on whether diffusers tiling would kick in:
+
+- `_distributed_tiled_decode`: Used when the latent spatial size exceeds `vae.tile_latent_min_size` (i.e., diffusers tiled decode). Each rank decodes a subset of tiles; rank0 gathers and runs the same overlap+blend+stitch logic as diffusers. This matches the single-rank diffusers tiled output.
+
+- `_distributed_patch_decode`: Used when diffusers tiling would not kick in. Each rank decodes a grid patch expanded with a latent-space halo; then rank0 gathers the cropped core patches and stitches them into the full image. This path has no blending and can introduce small numerical differences compared to the non-parallel decode.
+
+### Sequence Parallelism
+
+#### Ulysses-SP
+
+!!! note "Experimental UAA mode"
+    `ulysses_mode="advanced_uaa"` is an experimental extension to Ulysses-SP. It lets Ulysses attention handle arbitrary sequence lengths and arbitrary attention head counts without relying on `attention_mask`-based token padding.
+
+    In hybrid Ulysses + Ring mode, Ring still requires every rank in the same ring group to observe the same post-Ulysses sequence length. If that condition is not met, vLLM-Omni raises a validation error instead of entering the ring kernel with inconsistent shapes.
+
+##### Offline Inference
+
+An example of offline inference script using [Ulysses-SP](https://arxiv.org/pdf/2309.14509) is shown below:
+```python
+from vllm_omni import Omni
+from vllm_omni.inputs.data import OmniDiffusionSamplingParams
+from vllm_omni.diffusion.data import DiffusionParallelConfig
+ulysses_degree = 2
+
+omni = Omni(
+    model="Qwen/Qwen-Image",
+    parallel_config=DiffusionParallelConfig(ulysses_degree=2)
+)
+
+outputs = omni.generate(
+    "A cat sitting on a windowsill",
+    OmniDiffusionSamplingParams(num_inference_steps=50, width=2048, height=2048),
+)
+```
+
+See `examples/offline_inference/text_to_image/text_to_image.py` for a complete working example.
+
+To enable the experimental UAA mode explicitly, use a model/configuration that actually requires it. For example, `Tongyi-MAI/Z-Image-Turbo` has 30 attention heads, so `ulysses_degree=4` requires UAA because 30 is not divisible by 4:
+
+```python
+omni = Omni(
+    model="Tongyi-MAI/Z-Image-Turbo",
+    parallel_config=DiffusionParallelConfig(
+        ulysses_degree=4,
+        ulysses_mode="advanced_uaa",
+    ),
+)
+```
+
+##### Online Serving
+
+You can enable Ulysses-SP in online serving for diffusion models via `--usp`:
+
+```bash
+# Text-to-image (requires >= 2 GPUs)
+vllm serve Qwen/Qwen-Image --omni --port 8091 --usp 2
+
+# Experimental UAA mode for a model with 30 attention heads
+vllm serve Tongyi-MAI/Z-Image-Turbo --omni --port 8091 --usp 4 --ulysses-mode advanced_uaa
+```
+
+##### Benchmarks
+!!! note "Benchmark Disclaimer"
+    These benchmarks are provided for **general reference only**. The configurations shown use default or common parameter settings and have not been exhaustively optimized for maximum performance. Actual performance may vary based on:
+
+    - Specific model and use case
+    - Hardware configuration
+    - Careful parameter tuning
+    - Different inference settings (e.g., number of steps, image resolution)
+
+
+To measure the parallelism methods, we run benchmarks with **Qwen/Qwen-Image** model generating images (**2048x2048** as long sequence input) with 50 inference steps. The hardware devices are NVIDIA H800 GPUs. `sdpa` is the attention backends.
+
+| Configuration | Ulysses degree |Generation Time | Speedup |
+|---------------|----------------|---------|---------|
+| **Baseline (diffusers)** | - | 112.5s | 1.0x |
+| Ulysses-SP  |  2  |  65.2s | 1.73x |
+| Ulysses-SP  |  4  | 39.6s | 2.84x |
+| Ulysses-SP  |  8  | 30.8s | 3.65x |
+
+#### Ring-Attention
+
+Ring-Attention ([arxiv paper](https://arxiv.org/abs/2310.01889)) splits the input along the sequence dimension and uses ring-based P2P communication to accumulate attention results. Unlike Ulysses-SP which uses all-to-all communication, Ring-Attention keeps the sequence dimension sharded throughout the computation and circulates Key/Value blocks through a ring topology.
+
+##### Offline Inference
+
+An example of offline inference script using Ring-Attention is shown below:
+```python
+from vllm_omni import Omni
+from vllm_omni.inputs.data import OmniDiffusionSamplingParams
+from vllm_omni.diffusion.data import DiffusionParallelConfig
+ring_degree = 2
+
+omni = Omni(
+    model="Qwen/Qwen-Image",
+    parallel_config=DiffusionParallelConfig(ring_degree=2)
+)
+
+outputs = omni.generate(
+    "A cat sitting on a windowsill",
+    OmniDiffusionSamplingParams(num_inference_steps=50, width=2048, height=2048),
+)
+```
+
+See `examples/offline_inference/text_to_image/text_to_image.py` for a complete working example.
+
+
+##### Online Serving
+
+You can enable Ring-Attention in online serving for diffusion models via `--ring`:
+
+```bash
+# Text-to-image (requires >= 2 GPUs)
+vllm serve Qwen/Qwen-Image --omni --port 8091 --ring 2
+```
+
+##### Benchmarks
+!!! note "Benchmark Disclaimer"
+    These benchmarks are provided for **general reference only**. The configurations shown use default or common parameter settings and have not been exhaustively optimized for maximum performance. Actual performance may vary based on:
+
+    - Specific model and use case
+    - Hardware configuration
+    - Careful parameter tuning
+    - Different inference settings (e.g., number of steps, image resolution)
+
+
+To measure the parallelism methods, we run benchmarks with **Qwen/Qwen-Image** model generating images (**1024x1024** as long sequence input) with 50 inference steps. The hardware devices are NVIDIA A100 GPUs. `flash_attn` is the attention backends.
+
+| Configuration | Ring degree |Generation Time | Speedup |
+|---------------|----------------|---------|---------|
+| **Baseline (diffusers)** | - | 45.2s | 1.0x |
+| Ring-Attention  |  2  |  29.9s | 1.51x |
+| Ring-Attention  |  4  | 23.3s | 1.94x |
+
+
+#### Hybrid Ulysses + Ring
+
+You can combine both Ulysses-SP and Ring-Attention for larger scale parallelism. The total sequence parallel size equals `ulysses_degree × ring_degree`.
+
+!!! note "Experimental UAA in hybrid mode"
+    `ulysses_mode="advanced_uaa"` can also be used with hybrid Ulysses + Ring, but this does not remove Ring's shape requirement. Every rank in the same ring group must still have the same post-Ulysses sequence length.
+
+##### Offline Inference
+
+```python
+from vllm_omni import Omni
+from vllm_omni.inputs.data import OmniDiffusionSamplingParams
+from vllm_omni.diffusion.data import DiffusionParallelConfig
+
+# Hybrid: 2 Ulysses × 2 Ring = 4 GPUs total
+omni = Omni(
+    model="Qwen/Qwen-Image",
+    parallel_config=DiffusionParallelConfig(
+        ulysses_degree=2,
+        ring_degree=2,
+    )
+)
+
+outputs = omni.generate(
+    "A cat sitting on a windowsill",
+    OmniDiffusionSamplingParams(num_inference_steps=50, width=2048, height=2048),
+)
+```
+
+##### Online Serving
+
+```bash
+# Text-to-image (requires >= 4 GPUs)
+vllm serve Qwen/Qwen-Image --omni --port 8091 --usp 2 --ring 2
+```
+
+##### Benchmarks
+!!! note "Benchmark Disclaimer"
+    These benchmarks are provided for **general reference only**. The configurations shown use default or common parameter settings and have not been exhaustively optimized for maximum performance. Actual performance may vary based on:
+
+    - Specific model and use case
+    - Hardware configuration
+    - Careful parameter tuning
+    - Different inference settings (e.g., number of steps, image resolution)
+
+
+To measure the parallelism methods, we run benchmarks with **Qwen/Qwen-Image** model generating images (**1024x1024** as long sequence input) with 50 inference steps. The hardware devices are NVIDIA A100 GPUs. `flash_attn` is the attention backends.
+
+| Configuration | Ulysses degree | Ring degree | Generation Time | Speedup |
+|---------------|----------------|-------------|-----------------|---------|
+| **Baseline (diffusers)** | - | - | 45.2s | 1.0x |
+| Hybrid Ulysses + Ring  |  2  |  2  |  24.3s | 1.87x |
+
+
+### CFG-Parallel
+
+#### Offline Inference
+
+CFG-Parallel is enabled through `DiffusionParallelConfig(cfg_parallel_size=2)`, which runs one rank for the positive branch and one rank for the negative branch.
+
+An example of offline inference using CFG-Parallel (image-to-image) is shown below:
+
+```python
+from vllm_omni import Omni
+from vllm_omni.diffusion.data import DiffusionParallelConfig
+
+image_path = "path_to_image.png"
+omni = Omni(
+    model="Qwen/Qwen-Image-Edit",
+    parallel_config=DiffusionParallelConfig(cfg_parallel_size=2),
+)
+input_image = Image.open(image_path).convert("RGB")
+
+outputs = omni.generate(
+    {
+        "prompt": "turn this cat to a dog",
+        "negative_prompt": "low quality, blurry",
+        "multi_modal_data": {"image": input_image},
+    },
+    OmniDiffusionSamplingParams(
+        true_cfg_scale=4.0,
+        num_inference_steps=50,
+    ),
+)
+```
+
+Notes:
+
+- CFG-Parallel is only effective when a `negative_prompt` is provided AND a guidance scale (or `cfg_scale`) is greater than 1.
+
+See `examples/offline_inference/image_to_image/image_edit.py` for a complete working example.
+```bash
+cd examples/offline_inference/image_to_image/
+python image_edit.py \
+  --model "Qwen/Qwen-Image-Edit" \
+  --image "qwen_image_output.png" \
+  --prompt "turn this cat to a dog" \
+  --negative-prompt "low quality, blurry" \
+  --cfg-scale 4.0 \
+  --output "edited_image.png" \
+  --cfg-parallel-size 2
+```
+
+#### Online Serving
+
+You can enable CFG-Parallel in online serving for diffusion models via `--cfg-parallel-size`:
+
+```bash
+vllm serve Qwen/Qwen-Image-Edit --omni --port 8091 --cfg-parallel-size 2
+```
+
+### HSDP
+
+HSDP (Hybrid Sharded Data Parallel) shards model weights across GPUs to reduce per-GPU memory usage. This enables inference of large models (e.g., Wan2.2 14B) on GPUs with limited memory.
+
+Unlike Tensor Parallelism which splits computation, HSDP uses PyTorch's FSDP2 to shard and redistribute weights at runtime. Each GPU only holds a fraction of the model weights, and weights are gathered on-demand during forward passes.
+
+#### Configuration
+
+HSDP is configured via `DiffusionParallelConfig`:
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `use_hsdp` | bool | False | Enable HSDP |
+| `hsdp_shard_size` | int | -1 | Number of GPUs to shard weights across. -1 = auto (requires other parallelism > 1) |
+| `hsdp_replicate_size` | int | 1 | Number of replica groups. Each group holds a full sharded copy |
+
+**Constraints:**
+
+- `hsdp_replicate_size × hsdp_shard_size == world_size`
+- HSDP cannot be used with Tensor Parallelism (`tensor_parallel_size` must be 1)
+
+#### Operating Modes
+
+HSDP can work in two modes:
+
+- **Standalone Mode**: HSDP alone without other parallelism. Must specify `hsdp_shard_size` explicitly.
+- **Combined Mode**: HSDP overlays on top of other parallelism (Ulysses Sequence Parallel, CFG Parallel). HSDP dimensions must match world_size.
+
+#### Offline Inference
+
+**Standalone HSDP** (shard across 4 GPUs, no other parallelism):
+
+```python
+from vllm_omni import Omni
+from vllm_omni.inputs.data import OmniDiffusionSamplingParams
+from vllm_omni.diffusion.data import DiffusionParallelConfig
+
+omni = Omni(
+    model="Wan-AI/Wan2.2-T2V-A14B-Diffusers",
+    parallel_config=DiffusionParallelConfig(
+        use_hsdp=True,
+        hsdp_shard_size=4,  # Shard across 4 GPUs
+    ),
+)
+
+outputs = omni.generate(
+    "A cat playing piano",
+    OmniDiffusionSamplingParams(num_inference_steps=50),
+)
+```
+
+**Combined HSDP + Sequence Parallel**:
+
+```python
+omni = Omni(
+    model="Wan-AI/Wan2.2-T2V-A14B-Diffusers",
+    parallel_config=DiffusionParallelConfig(
+        ulysses_degree=4,  # Sequence parallel
+        use_hsdp=True,     # HSDP overlays on SP
+    ),
+)
+```
+
+#### Online Serving
+
+**Standalone HSDP** (shard model across 4 GPUs):
+
+```bash
+vllm serve Wan-AI/Wan2.2-T2V-A14B-Diffusers --omni --port 8091 --use-hsdp --hsdp-shard-size 4
+```
+
+**Combined with Sequence Parallel**:
+
+```bash
+vllm serve Wan-AI/Wan2.2-T2V-A14B-Diffusers --omni --port 8091 --use-hsdp --usp 4
+```
+
+#### Adding HSDP Support to New Models
+
+For detailed instructions on adding HSDP support to new models, see the [HSDP Contributing Guide](../../design/feature/hsdp.md).
+
+### Expert Parallelism
+
+Unlike Tensor Parallelism which shards every layer's weights, EP only shards the MoE expert MLP blocks. This significantly reduces the memory footprint of MoE models (e.g., HunyuanImage3.0) while maintaining constant dense-equivalent compute efficiency. Expert Parallelism is enabled via `DiffusionParallelConfig.enable_expert_parallel`. And `self.ep = tp * sp * cfg * dp` for now, so at least one of TP/SP/CFG/DP should set when EP enabled.
+
+#### Offline Inference
+
+```python
+from vllm_omni import Omni
+from vllm_omni.diffusion.data import DiffusionParallelConfig
+
+omni = Omni(
+    model="tencent/HunyuanImage-3.0",
+    parallel_config=DiffusionParallelConfig(tensor_parallel_size=8, enable_expert_parallel=True),
+)
+
+outputs = omni.generate(
+    "A brown and white dog is running on the grass",
+    OmniDiffusionSamplingParams(
+        num_inference_steps=50,
+        width=1024,
+        height=1024,
+    ),
+)
+```
@@ -65,7 +65,6 @@ The available `ignored_layers` names depend on the model architecture (e.g., `to
 | Flux | `black-forest-labs/FLUX.1-dev` | All layers | None |
 | HunyuanImage-3 | `tencent/HunyuanImage3` | All layers | None |
 | HunyuanVideo-1.5 | `hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-480p_t2v`, `720p_t2v`, `480p_i2v` | All layers | None |
-| Helios | `BestWishYsh/Helios-Base`, `BestWishYsh/Helios-Mid`, `BestWishYsh/Helios-Distilled` | All layers | None |

 ## Combining with Other Features

@@ -46,7 +46,7 @@ its stepwise request state machine. For normal diffusion inference, leave it
 disabled unless your workflow depends on this mode.

 If you are looking for general diffusion speedups, see
-[Diffusion Features Overview](../diffusion_features.md).
+[Diffusion Acceleration Overview](../diffusion_acceleration.md).

 ## Troubleshooting

@@ -0,0 +1,145 @@
+# TeaCache Configuration Guide
+
+TeaCache speeds up diffusion model inference by caching transformer computations when consecutive timesteps are similar. This typically provides **1.5x-2.0x speedup** with minimal quality loss.
+
+## Quick Start
+
+Enable TeaCache by setting `cache_backend` to `"tea_cache"`:
+
+```python
+from vllm_omni import Omni
+from vllm_omni.inputs.data import OmniDiffusionSamplingParams
+
+# Simple configuration - model_type is automatically extracted from pipeline.__class__.__name__
+omni = Omni(
+    model="Qwen/Qwen-Image",
+    cache_backend="tea_cache",
+    cache_config={
+        "rel_l1_thresh": 0.2  # Optional, defaults to 0.2
+    }
+)
+outputs = omni.generate(
+    "A cat sitting on a windowsill",
+    OmniDiffusionSamplingParams(
+        num_inference_steps=50,
+    ),
+)
+```
+
+### Using Environment Variable
+
+You can also enable TeaCache via environment variable:
+
+```bash
+export DIFFUSION_CACHE_BACKEND=tea_cache
+```
+
+Then initialize without explicitly setting `cache_backend`:
+
+```python
+from vllm_omni import Omni
+
+omni = Omni(
+    model="Qwen/Qwen-Image",
+    cache_config={"rel_l1_thresh": 0.2}  # Optional
+)
+```
+
+## Online Serving (OpenAI-Compatible)
+
+Enable TeaCache for online serving by passing `--cache-backend tea_cache` when starting the server:
+
+```bash
+vllm serve Qwen/Qwen-Image --omni --port 8091 \
+  --cache-backend tea_cache \
+  --cache-config '{"rel_l1_thresh": 0.2}'
+```
+
+## Configuration Parameters
+
+### `rel_l1_thresh` (float, default: `0.2`)
+
+Controls the balance between speed and quality. Lower values prioritize quality, higher values prioritize speed.
+
+**Recommended values:**
+
+- `0.2` - **~1.5x speedup** with minimal quality loss (recommended)
+- `0.4` - **~1.8x speedup** with slight quality loss
+- `0.6` - **~2.0x speedup** with noticeable quality loss
+- `0.8` - **~2.25x speedup** with significant quality loss
+
+## Examples
+
+### Python API
+
+```python
+from vllm_omni import Omni
+from vllm_omni.inputs.data import OmniDiffusionSamplingParams
+
+omni = Omni(
+    model="Qwen/Qwen-Image",
+    cache_backend="tea_cache",
+    cache_config={"rel_l1_thresh": 0.2}
+)
+outputs = omni.generate(
+    "A cat sitting on a windowsill",
+    OmniDiffusionSamplingParams(
+        num_inference_steps=50,
+    ),
+)
+```
+
+## Performance Tuning
+
+Start with the default `rel_l1_thresh=0.2` and adjust based on your needs:
+
+- **Maximum quality**: Use `0.1-0.2`
+- **Balanced**: Use `0.2-0.4` (recommended)
+- **Maximum speed**: Use `0.6-0.8` (may reduce quality)
+
+## Troubleshooting
+
+### Quality Degradation
+
+If you notice quality issues, lower the threshold:
+
+```python
+cache_config={"rel_l1_thresh": 0.1}  # More conservative caching
+```
+
+## Supported Models
+
+### ImageGen
+
+<style>
+th {
+  white-space: nowrap;
+  min-width: 0 !important;
+}
+</style>
+
+| Architecture | Models | Example HF Models |
+|--------------|--------|-------------------|
+| `QwenImagePipeline` | Qwen-Image | `Qwen/Qwen-Image` |
+| `QwenImageEditPipeline` | Qwen-Image-Edit | `Qwen/Qwen-Image-Edit` |
+| `QwenImageEditPlusPipeline` | Qwen-Image-Edit-2509 | `Qwen/Qwen-Image-Edit-2509` |
+| `QwenImageLayeredPipeline` | Qwen-Image-Layered | `Qwen/Qwen-Image-Layered` |
+| `BagelForConditionalGeneration` | BAGEL (DiT-only) | `ByteDance-Seed/BAGEL-7B-MoT` |
+
+### VideoGen
+
+No VideoGen models are supported by TeaCache yet.
+
+### Coming Soon
+
+<style>
+th {
+  white-space: nowrap;
+  min-width: 0 !important;
+}
+</style>
+
+| Architecture | Models | Example HF Models |
+|--------------|--------|-------------------|
+| `FluxPipeline` | Flux | - |
+| `CogVideoXPipeline` | CogVideoX | - |
@@ -0,0 +1,380 @@
+# Diffusion Acceleration Overview
+
+vLLM-Omni supports various acceleration methods to speed up diffusion model inference with minimal quality degradation. These include **cache methods** that intelligently cache intermediate computations to avoid redundant work across diffusion timesteps, **parallelism methods** that distribute the computation across multiple devices, and **quantization methods** that reduce memory footprint while preserving accuracy.
+
+## Supported Acceleration Methods
+
+vLLM-Omni currently supports two main cache acceleration backends:
+
+1. **[TeaCache](diffusion/teacache.md)** - Hook-based adaptive caching that caches transformer computations when consecutive timesteps are similar
+2. **[Cache-DiT](diffusion/cache_dit_acceleration.md)** - Library-based acceleration using multiple techniques:
+    - **DBCache** (Dual Block Cache): Caches intermediate transformer block outputs based on residual differences
+    - **TaylorSeer**: Uses Taylor expansion-based forecasting for faster inference
+    - **SCM** (Step Computation Masking): Selectively computes steps based on adaptive masking
+
+Both methods can provide significant speedups (typically **1.5x-2.0x**) while maintaining high output quality.
+
+vLLM-Omni also supports quantization methods:
+
+3. **[Quantization](diffusion/quantization/overview.md)** - Reduces DiT linear layers from BF16 to FP8 or Int8, providing ~1.28x speedup with minimal quality loss. Supports per-layer skip for sensitive layers.
+
+vLLM-Omni also supports parallelism methods for diffusion models, including:
+
+1. [Ulysses-SP](diffusion/parallelism_acceleration.md#ulysses-sp) - splits the input along the sequence dimension and uses all-to-all communication to allow each device to compute only a subset of attention heads.
+
+2. [Ring-Attention](diffusion/parallelism_acceleration.md#ring-attention) - splits the input along the sequence dimension and uses ring-based P2P communication to accumulate attention results, keeping the sequence dimension sharded.
+
+3. [CFG-Parallel](diffusion/parallelism_acceleration.md#cfg-parallel) - runs the positive/negative prompts of classifier-free guidance (CFG) on different devices, then merges on a single device to perform the scheduler step.
+
+4. [Tensor Parallelism](diffusion/parallelism_acceleration.md#tensor-parallelism) - shards DiT weights across devices to reduce per-GPU memory usage.
+
+5. [VAE Patch Parallelism](diffusion/parallelism_acceleration.md#vae-patch-parallelism) - shards VAE decode/encode spatially across ranks to reduce VAE peak memory (and can speed up VAE decode).
+
+6. [HSDP](diffusion/parallelism_acceleration.md#hsdp) - Hybrid Sharded Data Parallel shards model weights across GPUs to reduce per-GPU memory usage, enabling inference of large models on limited GPU memory.
+
+## Quick Comparison
+
+### Cache Methods
+
+| Method | Configuration | Description | Best For |
+|--------|--------------|-------------|----------|
+| **TeaCache** | `cache_backend="tea_cache"` | Simple, adaptive caching with minimal configuration | Quick setup, balanced speed/quality |
+| **Cache-DiT** | `cache_backend="cache_dit"` | Advanced caching with multiple techniques (DBCache, TaylorSeer, SCM) | Maximum acceleration, fine-grained control |
+
+### Quantization Methods
+
+| Method | Configuration | Description | Best For |
+|--------|--------------|-------------|----------|
+| **FP8** | `quantization="fp8"` | FP8 W8A8 on Ada/Hopper, weight-only on older GPUs | Memory reduction, inference speedup |
+| **Int8** | `quantization="int8"` | Int8 W8A8 | Memory reduction, inference speedup |
+
+## Supported Models
+
+The following table shows which models are currently supported by each acceleration method:
+
+### ImageGen
+
+| Model | Model Identifier | TeaCache | Cache-DiT | Ulysses-SP | Ring-Attention | CFG-Parallel | Tensor-Parallel | VAE-Patch-Parallel |
+|-------|------------------|:--------:|:---------:|:----------:|:--------------:|:------------:|:---------------:|:------------------:|
+| **LongCat-Image** | `meituan-longcat/LongCat-Image` | ❌ | ✅ | ❌ | ❌ | ✅ | ✅ | ❌ |
+| **LongCat-Image-Edit** | `meituan-longcat/LongCat-Image-Edit` | ❌ | ✅ | ❌ | ❌ | ✅ | ✅ | ❌ |
+| **Ovis-Image** | `OvisAI/Ovis-Image` | ❌ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ |
+| **Qwen-Image** | `Qwen/Qwen-Image` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| **Qwen-Image-2512** | `Qwen/Qwen-Image-2512` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| **Qwen-Image-Edit** | `Qwen/Qwen-Image-Edit` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ |
+| **Qwen-Image-Edit-2509** | `Qwen/Qwen-Image-Edit-2509` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ |
+| **Qwen-Image-Layered** | `Qwen/Qwen-Image-Layered` | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ |
+| **Z-Image** | `Tongyi-MAI/Z-Image-Turbo` | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ (TP=2 only) | ✅ |
+| **Stable-Diffusion3.5** | `stabilityai/stable-diffusion-3.5` | ❌ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ |
+| **Bagel** | `ByteDance-Seed/BAGEL-7B-MoT` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ |
+| **FLUX.1-dev** | `black-forest-labs/FLUX.1-dev` | ❌ | ✅ | ❌ | ❌ | ✅ | ✅ | ❌ |
+| **NextStep-1.1** | `stepfun-ai/NextStep-1.1` | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ❌ |
+| **FLUX.2-klein** | `black-forest-labs/FLUX.2-klein-4B` | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ❌ |
+| **FLUX.2-dev** | `black-forest-labs/FLUX.2-dev` | ❌ | ✅ | ❌ | ❌ | ❌ | ✅ | ❌ |
+| **FLUX.1-Kontext-dev** | `black-forest-labs/FLUX.1-Kontext-dev` | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ |
+| **GLM-Image** | `zai-org/GLM-Image` | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ❌ |
+
+### VideoGen
+
+| Model | Model Identifier | TeaCache | Cache-DiT | Ulysses-SP | Ring-Attention | CFG-Parallel | HSDP | VAE-Patch-Parallel |
+|-------|------------------|:--------:|:---------:|:----------:|:--------------:|:------------:|:----:|:----:|
+| **Wan2.1-T2V** | `Wan-AI/Wan2.1-T2V-1.3B-Diffusers` | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| **Wan2.1-T2V** | `Wan-AI/Wan2.1-T2V-14B-Diffusers` | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| **Wan2.2** | `Wan-AI/Wan2.2-T2V-A14B-Diffusers` | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
+| **LTX-2** | `Lightricks/LTX-2` | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
+| **DreamID-Omni** | `XuGuo699/DreamID-Omni` | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ |
+
+### Quantization
+
+| Model | Model Identifier | FP8 | Int8 |
+|-------|------------------|:---:|:---:|
+| **Qwen-Image** | `Qwen/Qwen-Image` | ✅ | ✅ |
+| **Qwen-Image-2512** | `Qwen/Qwen-Image-2512` | ✅ | ✅ |
+| **Z-Image** | `Tongyi-MAI/Z-Image-Turbo` | ✅ | ✅ |
+
+
+### AudioGen
+
+| Model                  | Model Identifier                         | TeaCache | Cache-DiT | Ulysses-SP | Ring-Attention | CFG-Parallel |
+|------------------------|------------------------------------------|:--------:|:---------:|:----------:|:--------------:|:------------:|
+| **Stable-Audio-Open**  | `stabilityai/stable-audio-open-1.0`      |    ✅    |           |            |                |              |
+
+## Performance Benchmarks
+
+The following benchmarks were measured on **Qwen/Qwen-Image** and **Qwen/Qwen-Image-Edit** models generating 1024x1024 images with 50 inference steps:
+
+!!! note "Benchmark Disclaimer"
+    These benchmarks are provided for **general reference only**. The configurations shown use default or common parameter settings and have not been exhaustively optimized for maximum performance. Actual performance may vary based on:
+
+    - Specific model and use case
+    - Hardware configuration
+    - Careful parameter tuning
+    - Different inference settings (e.g., number of steps, image resolution)
+
+    For optimal performance in your specific scenario, we recommend experimenting with different parameter configurations as described in the detailed guides below.
+
+| Model | Cache Backend | Cache Config | Generation Time | Speedup | Notes |
+|-------|---------------|--------------|----------------|---------|-------|
+| **Qwen/Qwen-Image** | None  | None | 20.0s | 1.0x | Baseline (diffusers) |
+| **Qwen/Qwen-Image** | TeaCache | `rel_l1_thresh=0.2` | 10.47s | **1.91x** | Recommended default setting |
+| **Qwen/Qwen-Image** | Cache-DiT | DBCache + TaylorSeer (Fn=1, Bn=0, W=8, TaylorSeer order=1) | 10.8s | **1.85x** | - |
+| **Qwen/Qwen-Image** | Cache-DiT | DBCache + TaylorSeer + SCM (Fn=8, Bn=0, W=4, TaylorSeer order=1, SCM fast) | 14.0s | **1.43x** | - |
+| **Qwen/Qwen-Image-Edit** | None | No acceleration | 51.5s | 1.0x | Baseline (diffusers) |
+| **Qwen/Qwen-Image-Edit** | Cache-DiT | Default (Fn=1, Bn=0, W=4, TaylorSeer disabled, SCM disabled) | 21.6s | **2.38x** | - |
+
+To measure the parallelism methods, we run benchmarks with **Qwen/Qwen-Image** model generating images (**2048x2048** as long sequence input) with 50 inference steps. The hardware devices are NVIDIA H800 GPUs. `sdpa` is the attention backends.
+
+| Configuration | Ulysses degree |Generation Time | Speedup |
+|---------------|----------------|---------|---------|
+| **Baseline (diffusers)** | - | 112.5s | 1.0x |
+| Ulysses-SP  |  2  |  65.2s | 1.73x |
+| Ulysses-SP  |  4  | 39.6s | 2.84x |
+| Ulysses-SP  |  8  | 30.8s | 3.65x |
+
+## Quick Start
+
+### Using TeaCache
+
+```python
+from vllm_omni import Omni
+from vllm_omni.inputs.data import OmniDiffusionSamplingParams
+
+omni = Omni(
+    model="Qwen/Qwen-Image",
+    cache_backend="tea_cache",
+    cache_config={"rel_l1_thresh": 0.2}  # Optional, defaults to 0.2
+)
+
+outputs = omni.generate(
+    "A cat sitting on a windowsill",
+    OmniDiffusionSamplingParams(
+        num_inference_steps=50,
+    ),
+)
+```
+
+### Using Cache-DiT
+
+```python
+from vllm_omni import Omni
+from vllm_omni.inputs.data import OmniDiffusionSamplingParams
+
+omni = Omni(
+    model="Qwen/Qwen-Image",
+    cache_backend="cache_dit",
+    cache_config={
+        "Fn_compute_blocks": 1,
+        "Bn_compute_blocks": 0,
+        "max_warmup_steps": 8,
+        "enable_taylorseer": True,
+        "taylorseer_order": 1,
+    }
+)
+
+outputs = omni.generate(
+    "A cat sitting on a windowsill",
+    OmniDiffusionSamplingParams(
+        num_inference_steps=50,
+    ),
+)
+```
+
+### Using Ulysses-SP
+
+Run text-to-image:
+```python
+from vllm_omni import Omni
+from vllm_omni.inputs.data import OmniDiffusionSamplingParams
+from vllm_omni.diffusion.data import DiffusionParallelConfig
+ulysses_degree = 2
+
+omni = Omni(
+    model="Qwen/Qwen-Image",
+    parallel_config=DiffusionParallelConfig(ulysses_degree=ulysses_degree)
+)
+
+outputs = omni.generate(
+    "A cat sitting on a windowsill",
+    OmniDiffusionSamplingParams(num_inference_steps=50, width=2048, height=2048),
+)
+```
+
+
+Run image-to-image:
+```python
+from vllm_omni import Omni
+from vllm_omni.inputs.data import OmniDiffusionSamplingParams
+from vllm_omni.diffusion.data import DiffusionParallelConfig
+ulysses_degree = 2
+
+omni = Omni(
+    model="Qwen/Qwen-Image-Edit",
+    parallel_config=DiffusionParallelConfig(ulysses_degree=ulysses_degree)
+)
+
+outputs = omni.generate(
+    {
+        "prompt": "turn this cat to a dog",
+        "multi_modal_data": {"image": input_image}
+    },
+    OmniDiffusionSamplingParams(num_inference_steps=50),
+)
+```
+
+### Using Ring-Attention
+
+Run text-to-image:
+```python
+from vllm_omni import Omni
+from vllm_omni.inputs.data import OmniDiffusionSamplingParams
+from vllm_omni.diffusion.data import DiffusionParallelConfig
+ring_degree = 2
+
+omni = Omni(
+    model="Qwen/Qwen-Image",
+    parallel_config=DiffusionParallelConfig(ring_degree=2)
+)
+
+outputs = omni.generate(
+    "A cat sitting on a windowsill",
+    OmniDiffusionSamplingParams(num_inference_steps=50, width=2048, height=2048),
+)
+```
+
+### Using Tensor Parallelism
+
+```python
+from vllm_omni import Omni
+from vllm_omni.diffusion.data import DiffusionParallelConfig
+
+omni = Omni(
+    model="Tongyi-MAI/Z-Image-Turbo",
+    parallel_config=DiffusionParallelConfig(tensor_parallel_size=2),
+)
+
+outputs = omni.generate(
+    prompt="a cat reading a book",
+    num_inference_steps=9,
+    width=512,
+    height=512,
+)
+```
+
+### Using VAE Patch Parallelism
+
+```python
+from vllm_omni import Omni
+from vllm_omni.diffusion.data import DiffusionParallelConfig
+
+omni = Omni(
+    model="Tongyi-MAI/Z-Image-Turbo",
+    parallel_config=DiffusionParallelConfig(
+        tensor_parallel_size=2,
+        vae_patch_parallel_size=2,
+    ),
+)
+
+outputs = omni.generate(
+    prompt="a cat reading a book",
+    num_inference_steps=9,
+    width=1024,
+    height=1024,
+)
+```
+
+### Using HSDP
+
+HSDP (Hybrid Sharded Data Parallel) shards model weights across GPUs to reduce per-GPU memory usage. This enables inference of large models (e.g., Wan2.2 14B) on GPUs with limited memory.
+
+```python
+from vllm_omni import Omni
+from vllm_omni.inputs.data import OmniDiffusionSamplingParams
+from vllm_omni.diffusion.data import DiffusionParallelConfig
+
+omni = Omni(
+    model="Wan-AI/Wan2.2-T2V-A14B-Diffusers",
+    parallel_config=DiffusionParallelConfig(
+        use_hsdp=True,
+        hsdp_replicate_size=1,  # Number of replica groups
+        hsdp_shard_size=8,      # Number of GPUs to shard across (or -1 for auto)
+    ),
+)
+
+outputs = omni.generate(
+    "A cat playing piano",
+    OmniDiffusionSamplingParams(num_inference_steps=50),
+)
+```
+
+### Using CFG-Parallel
+
+Run image-to-image:
+
+CFG-Parallel splits the CFG positive/negative branches across GPUs. Use it when you set a non-trivial `true_cfg_scale`.
+
+```python
+from vllm_omni import Omni
+from vllm_omni.inputs.data import OmniDiffusionSamplingParams
+from vllm_omni.diffusion.data import DiffusionParallelConfig
+cfg_parallel_size = 2
+
+omni = Omni(
+    model="Qwen/Qwen-Image-Edit",
+    parallel_config=DiffusionParallelConfig(cfg_parallel_size=cfg_parallel_size)
+)
+
+outputs = omni.generate(
+    {
+        "prompt": "turn this cat to a dog",
+        "multi_modal_data": {"image": input_image}
+    },
+    OmniDiffusionSamplingParams(num_inference_steps=50, true_cfg_scale=4.0),
+)
+```
+
+### Using FP8 Quantization
+
+```python
+from vllm_omni import Omni
+from vllm_omni.inputs.data import OmniDiffusionSamplingParams
+
+omni = Omni(
+    model="<your-model>",
+    quantization="fp8",
+)
+
+outputs = omni.generate(
+    "A cat sitting on a windowsill",
+    OmniDiffusionSamplingParams(num_inference_steps=50),
+)
+```
+
+### Using Int8 Quantization
+
+```python
+from vllm_omni import Omni
+from vllm_omni.inputs.data import OmniDiffusionSamplingParams
+
+omni = Omni(
+    model="<your-model>",
+    quantization="int8",
+)
+
+outputs = omni.generate(
+    "A cat sitting on a windowsill",
+    OmniDiffusionSamplingParams(num_inference_steps=50),
+)
+```
+
+## Documentation
+
+For detailed information on each acceleration method:
+
+- **[TeaCache Guide](diffusion/teacache.md)** - Complete TeaCache documentation, configuration options, and best practices
+- **[Cache-DiT Acceleration Guide](diffusion/cache_dit_acceleration.md)** - Comprehensive Cache-DiT guide covering DBCache, TaylorSeer, SCM, and configuration parameters
+- **[Quantization Guide](diffusion/quantization/overview.md)** - Quantization for DiT models with per-layer control
+- **[Tensor Parallelism](diffusion/parallelism_acceleration.md#tensor-parallelism)** - Guidance on how to enable TP for diffusion models.
+- **[Sequence Parallelism](diffusion/parallelism_acceleration.md#sequence-parallelism)** - Guidance on how to set sequence parallelism with configuration.
+- **[CFG-Parallel](diffusion/parallelism_acceleration.md#cfg-parallel)** - Guidance on how to set CFG-Parallel to run positive/negative branches across ranks.
+- **[VAE Patch Parallelism](diffusion/parallelism_acceleration.md#vae-patch-parallelism)** - Guidance on how to reduce VAE memory via patch/tile parallelism.
+- **[HSDP](diffusion/parallelism_acceleration.md#hsdp)** - Hybrid Sharded Data Parallel for memory-efficient inference of large models.
@@ -1,190 +0,0 @@
-# Diffusion Advanced Features
-
-## Table of Contents
-
- [Overview](#overview)
- [Supported Features](#supported-features)
- [Supported Models](#supported-models)
- [Feature Compatibility](#feature-compatibility)
- [Learn More](#learn-more)
-
-## Overview
-
-vLLM-Omni supports various advanced features for diffusion models:
-
- Acceleration: **cache methods**, **parallelism methods**
- Memory optimization: **cpu offloading**, **quantization**
- Extensions: **LoRA inference**
-
-## Supported Features
-
-### Acceleration
-
-#### Lossy Acceleration
-
-Cache methods trade minimal quality for significant speedup. Quality loss is typically imperceptible with proper tuning.
-
-| Method | Description | Best For |
-|--------|-------------|----------|
-| **[TeaCache](diffusion/cache_acceleration/teacache.md)** | Adaptive caching using modulated inputs | Quick setup, balanced quality/speed on single GPU |
-| **[Cache-DiT](diffusion/cache_acceleration/cache_dit.md)** | Multiple caching techniques: DBCache, TaylorSeer, SCM | Fine-grained control, tunable quality-speed tradeoff |
-
-
-#### Lossless Acceleration
-
-Parallelism methods distribute computation across GPUs without quality loss (mathematically equivalent to single-GPU).
-
-| Method | Description | Best For |
-|--------|-------------|----------|
-| **[Ulysses-SP](diffusion/parallelism/sequence_parallel.md)** | Sequence parallelism via all-to-all communication | High-resolution images (>1536px) or long videos with 2-8 GPUs |
-| **[Ring-Attention](diffusion/parallelism/sequence_parallel.md)** | Sequence parallelism via ring-based communication | Videos, very long sequences, memory-constrained, with 2-8 GPUs |
-| **[CFG-Parallel](diffusion/parallelism/cfg_parallel.md)** | Splits CFG positive/negative branches across devices | Image editing with CFG guidance (true_cfg_scale > 1) on 2 GPUs |
-| **[Tensor Parallelism](diffusion/parallelism/tensor_parallel.md)** | Shards model weights across devices | Large models that don't fit in single GPU, with 2+ GPUs |
-| **[HSDP](diffusion/parallelism/hsdp.md)** | Weight sharding via FSDP2, redistributed on-demand at runtime | Very large models (14B+) on limited VRAM, combinable with SP |
-| **[Expert Parallelism](diffusion/parallelism/expert_parallel.md)** | Shards MoE expert MLP blocks across devices | MoE diffusion models (e.g., HunyuanImage3.0) |
-
-**Note:** Some acceleration methods can be combined together for optimized performance. See [Feature Compatibility Table](#feature-compatibility) and [Feature Compatibility Tutorial](feature_compatibility.md) for detailed configuration examples.
-
-### Memory Optimization
-
-Memory optimization methods help reduce GPU memory usage, enabling inference on resource-constrained hardware or larger models.
-
-| Method | Description | Best For |
-|--------|-------------|----------|
-| **[CPU Offload](diffusion/cpu_offload_diffusion.md)** | Offloads model components to CPU memory | Limited VRAM, large models on consumer GPUs |
-| **[Quantization](diffusion/quantization/overview.md)** | Reduces DiT layers from BF16 to FP8/INT8/etc. | Limited VRAM, minimal accuracy loss    |
-| **[VAE Patch Parallelism](diffusion/parallelism/vae_patch_parallel.md)** | Distributes VAE decode tiling across GPUs | High-resolution generation with reduced VAE memory peak |
-
-### Extensions
-
-Extension methods add specialized capabilities to diffusion models beyond standard inference.
-
-| Method | Description | Best For |
-|--------|-------------|----------|
-| **[LoRA Inference](diffusion/lora.md)** | Enables inference with Low-Rank Adaptation (LoRA) adapters weights | Reinforcement learning extensions |
-
-
-### Quantization Methods
-
-| Method | Configuration | Description | Best For |
-|--------|--------------|-------------|----------|
-| **[FP8](diffusion/quantization/fp8.md)** | `quantization="fp8"` | FP8 W8A8 on Ada/Hopper, weight-only on older GPUs | Memory reduction, inference speedup |
-| **[INT8](diffusion/quantization/int8.md)** | `quantization="int8"` | INT8 weight-only, no calibration or pre-quantized checkpoint needed | Memory reduction, broad GPU compatibility |
-| **[GGUF](diffusion/quantization/gguf.md)** | `quantization="gguf"` | Native GGUF transformer-only weights (Q4, Q8, etc.) | Memory reduction on consumer GPUs |
-
-## Supported Models
-
-The following tables show which models support each feature:
-
- **🔀SP (Ulysses & Ring)**: Includes both Ulysses-SP and Ring-Attention methods
- ✅ = Fully supported
- ❌ = Not supported
-
-> Notes:
-
-> 1. CPU Offload has two methods: Module-wise (default for models with DiT + text encoder) and Layerwise. The tables below show **Layerwise support** only.
-> 2. The **💾Quantization** column is collapsed for readability. See [Quantization Overview](diffusion/quantization/overview.md) for per-method (FP8, GGUF, …) and per-model support details.
-
-### ImageGen
-
-| Model | ⚡TeaCache | ⚡Cache-DiT | 🔀SP (Ulysses & Ring) | 🔀CFG-Parallel | 🔀Tensor-Parallel | 🔀HSDP | 💾CPU Offload (Layerwise) | 💾VAE-Patch-Parallel | 💾Quantization |
-|-------|:----------:|:-----------:|:---------------------:|:--------------:|:-----------------:|:------:|:------------------------:|:--------------------:|:--------------:|
-| **Bagel** | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ |
-| **FLUX.1-dev** | ❌ | ✅ | ❌ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ |
-| **FLUX.2-klein** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ |
-| **FLUX.1-Kontext-dev** | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
-| **FLUX.2-dev** | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
-| **GLM-Image** | ❌ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ |
-| **HunyuanImage3** | ❌ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ✅ | ❌ |
-| **LongCat-Image** | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ |
-| **LongCat-Image-Edit** | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ |
-| **MammothModa2(T2I)** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
-| **Nextstep_1(T2I)** | ❓ | ❓ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ |
-| **OmniGen2** | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
-| **Ovis-Image** | ❌ | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ |
-| **Qwen-Image** | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ |
-| **Qwen-Image-2512** | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ |
-| **Qwen-Image-Edit** | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ |
-| **Qwen-Image-Edit-2509** | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ |
-| **Qwen-Image-Layered** | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ |
-| **Stable-Diffusion3.5** | ❌ | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ |
-| **Z-Image** | ✅ | ✅ | ✅ | ❓ | ✅ (TP=2 only) | ❌ | ❌ | ✅ | ✅ |
-
-> Notes:
-> 1. Nextstep_1(T2I) does not support cache acceleration methods such as TeaCache or Cache-DiT.
-> 2. `Tongyi-MAI/Z-Image-Turbo` is a distilled model with minimal NFEs; CFG-Parallel is not necessary.
-
-### VideoGen
-
-| Model | ⚡TeaCache | ⚡Cache-DiT | 🔀SP (Ulysses & Ring) | 🔀CFG-Parallel | 🔀Tensor-Parallel | 🔀HSDP | 💾CPU Offload (Layerwise) | 💾VAE-Patch-Parallel | 💾Quantization |
-|-------|:----------:|:-----------:|:---------------------:|:--------------:|:-----------------:|:------:|:------------------------:|:--------------------:|:--------------:|
-| **Wan2.2** | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ |
-| **LTX-2** | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ |
-| **Helios** | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
-| **HunyuanVideo-1.5 T2V I2V** | ❌ | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
-| **DreamID-Omni** | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ |
-
-### AudioGen
-
-| Model | ⚡TeaCache | ⚡Cache-DiT | 🔀SP (Ulysses & Ring) | 🔀CFG-Parallel | 🔀Tensor-Parallel | 🔀HSDP | 💾CPU Offload (Layerwise) | 💾VAE-Patch-Parallel | 💾Quantization |
-|-------|:----------:|:-----------:|:---------------------:|:--------------:|:-----------------:|:------:|:------------------------:|:--------------------:|:--------------:|
-| **Stable-Audio-Open** | ❌ | ❌ | ❓ | ❓ | ❌ | ❌ | ❌ | ❌ | ✅ |
-
-
-## Feature Compatibility
-
-**Legend:**
-
- ✅: Functionality is supported
- ❌: No support plan
- ❓: Not verified yet and Not Recommended
-
-|  | ⚡TeaCache | ⚡Cache-DiT | 🔀Ulysses-SP | 🔀Ring-Attn | 🔀CFG-Parallel | 🔀Tensor Parallel | 🔀HSDP | 🔀Expert Parallel | 💾CPU Offloading (Layerwise) | 💾CPU Offloading (Module-wise) | 💾VAE Patch Parallel | 💾FP8 Quant | 🔧LoRA Inference |
-|---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
-| **⚡TeaCache** | | | | | | | | | | | | | |
-| **⚡Cache-DiT** | ❌ | | | | | | | | | | | | |
-| **🔀Ulysses-SP** | ✅ | ✅ | | | | | | | | | | | |
-| **🔀Ring-Attn** | ✅ | ✅ | ✅ | | | | | | | | | | |
-| **🔀CFG-Parallel** | ✅ | ✅ | ✅ | ✅ | | | | | | | | | |
-| **🔀Tensor Parallel** | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | |
-| **🔀HSDP** | ❓ | ❓ | ❓ | ❓ | ❓ | ❌ | | | | | | | |
-| **🔀Expert Parallel** | ❓ | ❓ | ❓ | ❓ | ❓ | ❓ | ❓ | | | | | | |
-| **💾CPU Offloading (Layerwise)** | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | | | | | |
-| **💾CPU Offloading (Module-wise)** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❓ | ❓ | ❌ | | | | |
-| **💾VAE Patch Parallel** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | | | |
-| **💾FP8 Quant** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❓ | ❓ | ✅ | ✅ | ✅ | | |
-| **🔧LoRA Inference** | ❓ | ❓ | ❓ | ❓ | ❓ | ❓ | ❓ | ❓ | ❓ | ❓ | ❓ | ❓ | |
-
-!!! info
-
-    1. Tensor Parallel and HSDP are not compatible.
-    2. TeaCache and Cache-DiT are not compatible.
-    3. CPU Offloading (Layerwise) and CPU Offloading (Module-wise) are not compatible.
-    4. CPU Offloading (Layerwise) supports single-card for now.
-    5. Using FP8-Quant as an example of qunatization methods.
-
-
-## Learn More
-
-**Cache Acceleration:**
-
- **[TeaCache Configuration Guide](diffusion/cache_acceleration/teacache.md)** - Parameter tuning, performance tips, troubleshooting
- **[Cache-DiT Advanced Guide](diffusion/cache_acceleration/cache_dit.md)** - DBCache, TaylorSeer, SCM techniques and optimization
-
-**Parallelism Methods:**
-
- **[Parallelism Overview](diffusion/parallelism/overview.md)** - Tensor Parallelism, Sequence Parallelism, CFG Parallelism, HSDP, and Expert Parallelism
-
-**Memory Optimization:**
-
- **[CPU Offload Guide](diffusion/cpu_offload_diffusion.md)** - Offload model components to CPU, reduce GPU memory usage
- **[VAE Patch Parallelism Guide](diffusion/parallelism/vae_patch_parallel.md)** - Distribute VAE decode tiling across GPUs for high-resolution images
- **[Quantization Overview](diffusion/quantization/overview.md)** - Overview of quantization methods for diffusion models
-
-**Extensions:**
-
- **[LoRA Inference Guide](diffusion/lora.md)** - Low-Rank Adaptation for style customization and fine-tuning
-
-**Advanced Topics:**
-
- **[Feature Compatibility](feature_compatibility.md)** - How to combine multiple features for maximum performance
@@ -256,15 +256,6 @@ lora_adapter/

 ## Web UI Demo

-!!! note "Gradio is an optional dependency"
-    The Gradio demo requires the `[demo]` extras. Install them first:
-
-    ```bash
-    pip install 'vllm-omni[demo]'
-    ```
-
-    Or, if installing from source: `pip install -e '.[demo]'`
-
 Launch the Gradio demo:

 ```bash
@@ -11,15 +11,6 @@ Source <https://github.com/vllm-project/vllm-omni/tree/main/examples/online_serv

 ## Gradio Demo

-!!! note "Gradio is an optional dependency"
-    The Gradio demo requires the `[demo]` extras. Install them first:
-
-    ```bash
-    pip install 'vllm-omni[demo]'
-    ```
-
-    Or, if installing from source: `pip install -e '.[demo]'`
-
 An interactive Gradio demo is available with text-to-speech synthesis, voice cloning, and streaming support.

 ```bash
@@ -122,15 +122,6 @@ python openai_chat_client.py --input input1.png input2.png --prompt "Combine the

 ### Method 4: Using Gradio Demo

-!!! note "Gradio is an optional dependency"
-    The Gradio demo requires the `[demo]` extras. Install them first:
-
-    ```bash
-    pip install 'vllm-omni[demo]'
-    ```
-
-    Or, if installing from source: `pip install -e '.[demo]'`
-
 ```bash
 python gradio_demo.py
 # Visit http://localhost:7861
@@ -149,15 +149,6 @@ python openai_chat_completion_client_for_multimodal_generation.py \

 ## Run Local Web UI Demo

-!!! note "Gradio is an optional dependency"
-    The Gradio demo requires the `[demo]` extras. Install them first:
-
-    ```bash
-    pip install 'vllm-omni[demo]'
-    ```
-
-    Or, if installing from source: `pip install -e '.[demo]'`
-
 This Web UI demo allows users to interact with the model through a web browser.

 ### Running Gradio Demo
@@ -154,7 +154,7 @@ print(response.choices[1].message.audio)    # Audio response
 ```

 ## Streaming Output
-If you want to enable streaming output, please set the argument as below. The final output will be obtained just after generated by corresponding stage. We support both text streaming output and audio streaming output. Other modalities can output normally.
+If you want to enable streaming output, please set the argument as below. The final output will be obtained just after generated by corresponding stage. Now we only support text streaming output. Other modalities can output normally.
 ```bash
 python openai_chat_completion_client_for_multimodal_generation.py \
    --query-type use_image \
@@ -163,15 +163,6 @@ python openai_chat_completion_client_for_multimodal_generation.py \

 ## Run Local Web UI Demo

-!!! note "Gradio is an optional dependency"
-    The Gradio demo requires the `[demo]` extras. Install them first:
-
-    ```bash
-    pip install 'vllm-omni[demo]'
-    ```
-
-    Or, if installing from source: `pip install -e '.[demo]'`
-
 This Web UI demo allows users to interact with the model through a web browser.

 ### Running Gradio Demo
@@ -19,15 +19,6 @@ Please refer to [README.md](https://github.com/vllm-project/vllm-omni/tree/main/

 ## Gradio Demo

-!!! note "Gradio is an optional dependency"
-    The Gradio demo requires the `[demo]` extras. Install them first:
-
-    ```bash
-    pip install 'vllm-omni[demo]'
-    ```
-
-    Or, if installing from source: `pip install -e '.[demo]'`
-
 Two interactive Gradio demos are available, both supporting all 3 task types:

 | Demo     | File                     | Transport    | Streaming Quality                                  |
@@ -105,15 +105,6 @@ python openai_chat_client.py --prompt "A beautiful landscape painting" --output

 ### Method 4: Using Gradio Demo

-!!! note "Gradio is an optional dependency"
-    The Gradio demo requires the `[demo]` extras. Install them first:
-
-    ```bash
-    pip install 'vllm-omni[demo]'
-    ```
-
-    Or, if installing from source: `pip install -e '.[demo]'`
-
 ```bash
 python gradio_demo.py
 # Visit http://localhost:7860
@@ -123,6 +123,9 @@ curl -X POST http://localhost:8091/v1/videos \
  -F "guidance_scale_2=4.0" \
  -F "boundary_ratio=0.875" \
  -F "flow_shift=5.0" \
+  -F "enable_frame_interpolation=true" \
+  -F "frame_interpolation_exp=1" \
+  -F "frame_interpolation_scale=1.0" \
  -F "seed=42"
 ```

@@ -145,6 +148,35 @@ curl -X POST http://localhost:8091/v1/videos \
 | `flow_shift`          | float  | None    | Scheduler flow shift (Wan2.2)                    |
 | `seed`                | int    | None    | Random seed (reproducible)                       |
 | `lora`                | object | None    | LoRA configuration                               |
+| `enable_frame_interpolation` | bool | false | Enable RIFE frame interpolation before MP4 encoding |
+| `frame_interpolation_exp` | int | 1 | Interpolation exponent; 1=2x temporal resolution, 2=4x |
+| `frame_interpolation_scale` | float | 1.0 | RIFE inference scale; use 0.5 for high-resolution inputs |
+| `frame_interpolation_model_path` | str | None | Local directory or Hugging Face repo ID with `flownet.pkl`; defaults to `elfgum/RIFE-4.22.lite` |
+
+## Frame Interpolation
+
+Frame interpolation is an optional post-processing step for `/v1/videos` and
+`/v1/videos/sync`. It synthesizes intermediate frames between generated frames
+without rerunning the diffusion model. If the generated video has `N` frames,
+the interpolated output frame count is `(N - 1) * 2**exp + 1`. The encoder FPS
+is multiplied by `2**exp` so the output duration remains close to the original.
+
+Frame interpolation runs in the diffusion worker post-processing path instead of
+the API server encoding path, so it can reuse the worker's current accelerator
+device without blocking the FastAPI event loop.
+
+Example: generate 5 frames and interpolate to 9 frames:
+
+```bash
+curl -X POST http://localhost:8091/v1/videos/sync \
+  -F "prompt=A dog running through a park" \
+  -F "num_frames=5" \
+  -F "fps=8" \
+  -F "enable_frame_interpolation=true" \
+  -F "frame_interpolation_exp=1" \
+  -F "frame_interpolation_scale=1.0" \
+  -o sync_t2v_interpolated.mp4
+```

 ## Create Response Format

@@ -1,218 +0,0 @@
-# Feature Compatibility
-
-This guide explains the compatibility matrix of different diffusion features in vLLM-Omni. You can use cache methods together with parallelism methods and other features to achieve optimal speed and efficiency.
-
-## Overview
-
-vLLM-Omni supports combining:
-
- **Cache methods** (TeaCache, Cache-DiT) with **Parallelism methods** (Ulysses-SP, Ring-Attention, CFG-Parallel, Tensor Parallelism)
- **Multiple parallelism methods** together (e.g., Ulysses-SP + Ring-Attention, CFG-Parallel + Sequence Parallelism)
- **LoRA adapters** with most acceleration features
- **CPU offloading** with other memory optimization features
-
-See the feature compatibility matrix in [Table](diffusion_features.md#feature-compatibility)
-
-## Common Combinations
-
-### 1. Cache + Sequence Parallelism (Recommended)
-
-Best for: **Large images (>1536px) or videos**
-
-Combines cache acceleration with sequence parallelism for maximum speedup on single-device-challenging workloads.
-
-**Using TeaCache + Ulysses-SP:**
-
-```bash
-python examples/offline_inference/text_to_image/text_to_image.py \
-  --model Qwen/Qwen-Image \
-  --prompt "A beautiful mountain landscape" \
-  --cache-backend tea_cache \
-  --ulysses-degree 2
-```
-
-**Using Cache-DiT + Ring-Attention:**
-
-```bash
-python examples/offline_inference/text_to_image/text_to_image.py \
-  --model Qwen/Qwen-Image \
-  --prompt "A futuristic city" \
-  --cache-backend cache_dit \
-  --ring-degree 2
-```
-
-### 2. Cache + CFG-Parallel
-
-Best for: **Image editing with Classifier-Free Guidance**
-
-Accelerates both the diffusion process and CFG computation.
-
-```bash
-python examples/offline_inference/image_to_image/image_edit.py \
-  --model Qwen/Qwen-Image-Edit \
-  --prompt "make it sunset" \
-  --negative-prompt "low quality, blurry" \
-  --image input.png \
-  --cache-backend cache_dit \
-  --cfg-parallel-size 2 \
-  --cfg-scale 4.0
-```
-
-### 3. CFG-Parallel + Sequence Parallelism
-
-Best for: **Large resolution image editing with CFG**
-
-Combines both CFG branch splitting and sequence parallelism for maximum GPU utilization.
-
-**CFG-Parallel + Ulysses-SP:**
-
-```bash
-python examples/offline_inference/image_to_image/image_edit.py \
-  --model Qwen/Qwen-Image-Edit \
-  --prompt "transform into autumn scene" \
-  --negative-prompt "low quality" \
-  --image input.png \
-  --cache-backend cache_dit \
-  --cfg-parallel-size 2 \
-  --ulysses-degree 2 \
-  --cfg-scale 4.0
-```
-
-### 4. Hybrid Ulysses + Ring + Vae tiling
-
-Best for: **Very large images or videos on multiple devices**
-
-Combines Ulysses-SP (all-to-all) with Ring-Attention (ring P2P) for scalable parallelism.
-
-```bash
-python examples/offline_inference/text_to_image/text_to_image.py \
-  --model Qwen/Qwen-Image \
-  --prompt "Epic fantasy landscape" \
-  --cache-backend cache_dit \
-  --ulysses-degree 2 \
-  --ring-degree 2 \
-  --num-inference-steps 50 \
-  --width 2048 \
-  --height 2048 \
-  --vae-use-tiling
-```
-
-### 5. Cache + Tensor Parallelism
-
-Best for: **Large models that don't fit in single GPU memory**
-
-Reduces per-GPU memory usage while maintaining cache acceleration.
-
-```bash
-python examples/offline_inference/text_to_image/text_to_image.py \
-  --model Tongyi-MAI/Z-Image-Turbo \
-  --prompt "A cat reading a book" \
-  --cache-backend tea_cache \
-  --tensor-parallel-size 2 \
-  --num-inference-steps 9 \
-```
-
-## Online Serving
-
-### Cache + Sequence Parallelism
-
-```bash
-# TeaCache + Ulysses-SP
-vllm serve Qwen/Qwen-Image --omni --port 8091 \
-  --cache-backend tea_cache \
-  --cache-config '{"rel_l1_thresh": 0.2}' \
-  --usp 2
-
-# Cache-DiT + Ring-Attention
-vllm serve Qwen/Qwen-Image --omni --port 8091 \
-  --cache-backend cache_dit \
-  --cache-config '{"Fn_compute_blocks": 1, "max_warmup_steps": 8}' \
-  --ring 2
-```
-
-### Cache + CFG-Parallel
-
-```bash
-vllm serve Qwen/Qwen-Image-Edit --omni --port 8091 \
-  --cache-backend cache_dit \
-  --cfg-parallel-size 2
-```
-
-### Multiple Parallelism Methods
-
-```bash
-# CFG-Parallel + Ulysses-SP (4 GPUs total)
-vllm serve Qwen/Qwen-Image-Edit --omni --port 8091 \
-  --cache-backend cache_dit \
-  --cfg-parallel-size 2 \
-  --usp 2
-
-# Hybrid Ulysses + Ring (4 GPUs total)
-vllm serve Qwen/Qwen-Image --omni --port 8091 \
-  --cache-backend cache_dit \
-  --usp 2 \
-  --ring 2
-```
-
-
-## Limitations
-
-### Incompatibilities
-
- **TeaCache + Cache-DiT**: These two cache methods cannot be used together. Only one cache backend can be active at a time. Attempting to enable both will result in an error.
-
-### Partial Support
-
- **Tensor Parallelism — Text Encoder Not Sharded**: TP currently only shards the DiT blocks. Each TP rank retains a **full copy of the text encoder weights**, leading to significant GPU memory overhead proportional to TP degree. Tracked in [Issue #771](https://github.com/vllm-project/vllm-omni/issues/771).
-
- **CPU Offloading — Two Modes Are Mutually Exclusive**: Model-level offload (`enable_cpu_offload`) and layerwise offload (`enable_layerwise_offload`) cannot be used simultaneously. If both are set, layerwise takes priority and model-level is silently ignored.
-
- **CPU Offloading — VAE stays on GPU**: Both offloading strategies keep the VAE on GPU at all times. For high-resolution generation, VAE decode can still cause OOM. Mitigate by combining with `vae_use_tiling=True` or VAE Patch Parallelism.
-
- **VAE Patch Parallelism — DistributedVaeExecutor Required**: VAE Patch Parallelism is only enabled for models that have `DistributedVaeExecutor`. Unsupported models will silently ignore `vae_patch_parallel_size`, and use sequential vae tiling instead.
-
-### Configuration Constraints
-
- **GPU Count Must Match Parallel Degrees**: Total GPU count must satisfy:
-  ```
-  total_gpus = ulysses_degree × ring_degree × cfg_parallel_size × tensor_parallel_size
-  ```
-  Any mismatch will cause a configuration error at startup.
-
- **VAE Patch Parallel Size ≤ DiT Process Group Size**: `vae_patch_parallel_size` reuses the DiT process group and cannot exceed it. Larger values are automatically clamped with a warning.
-
- **Model-Specific TP Constraints**: Some models impose divisibility constraints on TP size. For example, Z-Image Turbo (`num_heads=30`) only supports `tensor_parallel_size=2`. Check [Supported Models](diffusion_features.md#supported-models) for per-model constraints.
-
-## Troubleshooting
-
-### Performance Not Scaling
-
-**Symptoms:** Adding more GPUs doesn't improve speed proportionally
-
-**Solutions:**
-1. Check GPU communication bandwidth (use `nvidia-smi topo -m`)
-2. Reduce parallelism degree if communication overhead is high
-3. For very long sequences, prefer Ring-Attention over Ulysses-SP
-4. Ensure batch size is large enough to saturate GPUs
-
-### Out of Memory with Parallelism
-
-**Symptoms:** OOM errors when combining methods
-
-**Solutions:**
-1. Enable Tensor Parallelism to shard weights
-2. Reduce resolution or batch size
-3. Combine with memory efficient methods, such as cpu offloading
-
-### Configuration Errors
-
-**Symptoms:** Errors about invalid parallel configuration
-
-**Solutions:**
-1. Verify total GPU count matches: `ulysses × ring × cfg × tp`
-2. Check model supports all enabled methods
-3. Ensure divisibility constraints (e.g., Z-Image TP=1 or 2 only)
-
-## See Also
-
- [Diffusion Acceleration Overview](diffusion_features.md) - Main acceleration guide
@@ -129,17 +129,8 @@ The first run loads model weights. Subsequent runs are faster:
 --stage-init-timeout 900  # Increase timeout for slow storage
 ```

-### `Transformers does not recognize this architecture` Error
-
-Your have to upgrade `transformers` package to `5.3.0` or above:
-
-```
-pip install --upgrade transformers
-```
-
 ## Requirements

 - vLLM-Omni with GLM-Image support
 - CUDA-capable GPU (recommended: H100/A100 with 80GB)
 - GLM-Image model weights
- `transformers` v5.3.0 or above
@@ -178,12 +178,6 @@ def parse_args() -> argparse.Namespace:
        action="store_true",
        help="Enable diffusion pipeline profiler to display stage durations.",
    )
-    parser.add_argument(
-        "--quantization",
-        type=str,
-        default=None,
-        help="Quantization method (e.g. 'fp8').",
-    )

    # Memory & parallelism
    parser.add_argument("--vae-use-slicing", action="store_true", help="Enable VAE slicing.")
@@ -219,7 +213,6 @@ def main():
        parallel_config=parallel_config,
        enforce_eager=args.enforce_eager,
        enable_diffusion_pipeline_profiler=args.enable_diffusion_pipeline_profiler,
-        quantization=args.quantization,
    )

    # Validate I2V / V2V arguments
@@ -1,10 +1,7 @@
 import argparse
 from functools import lru_cache

-try:
-    import gradio as gr
-except ImportError:
-    raise ImportError("gradio is required to run this demo. Install it with: pip install 'vllm-omni[demo]'") from None
+import gradio as gr
 import torch

 from vllm_omni.entrypoints.omni import Omni
@@ -64,8 +64,8 @@ def parse_args() -> argparse.Namespace:
    parser.add_argument(
        "--guidance-scale",
        type=float,
-        default=4.0,
-        help="Classifier-free guidance scale. HunyuanImage3 recommends 4.0-5.0.",
+        default=1.0,
+        help="Classifier-free guidance scale.",
    )
    parser.add_argument("--height", type=int, default=1024, help="Height of generated image.")
    parser.add_argument("--width", type=int, default=1024, help="Width of generated image.")
@@ -16,10 +16,7 @@ import argparse
 import base64
 import io

-try:
-    import gradio as gr
-except ImportError:
-    raise ImportError("gradio is required to run this demo. Install it with: pip install 'vllm-omni[demo]'") from None
+import gradio as gr
 import httpx
 import numpy as np
 import soundfile as sf
@@ -196,9 +196,3 @@ gpu_memory_utilization: 0.5
 ```

 - The first request may be slow due to model warmup. Subsequent requests will be faster.
-
- If you encounter `Transformers does not recognize this architecture` error, your have to upgrade `transformers` package to `5.3.0` or above:
-
-```
-pip install --upgrade transformers
-```
@@ -10,10 +10,7 @@ import argparse
 import base64
 from io import BytesIO

-try:
-    import gradio as gr
-except ImportError:
-    raise ImportError("gradio is required to run this demo. Install it with: pip install 'vllm-omni[demo]'") from None
+import gradio as gr
 import requests
 from PIL import Image

@@ -6,10 +6,7 @@ import random
 from pathlib import Path
 from typing import Any

-try:
-    import gradio as gr
-except ImportError:
-    raise ImportError("gradio is required to run this demo. Install it with: pip install 'vllm-omni[demo]'") from None
+import gradio as gr
 import numpy as np
 import soundfile as sf
 import torch
@@ -211,7 +211,7 @@ print(response.choices[1].message.audio)
 Supported speaker names depend on the model (e.g. `Ethan`, `Chelsie`, `Aiden`). Omit `speaker` to use the default.

 ## Streaming Output
-If you want to enable streaming output, please set the argument as below. The final output will be obtained just after generated by corresponding stage. We support both text streaming output and audio streaming output. Other modalities can output normally.
+If you want to enable streaming output, please set the argument as below. The final output will be obtained just after generated by corresponding stage. Now we only support text streaming output. Other modalities can output normally.
 ```bash
 python examples/online_serving/openai_chat_completion_client_for_multimodal_generation.py \
    --query-type use_image \
@@ -6,10 +6,7 @@ import random
 from pathlib import Path
 from typing import Any

-try:
-    import gradio as gr
-except ImportError:
-    raise ImportError("gradio is required to run this demo. Install it with: pip install 'vllm-omni[demo]'") from None
+import gradio as gr
 import numpy as np
 import soundfile as sf
 import torch
@@ -24,10 +24,7 @@ import io
 import json
 import logging

-try:
-    import gradio as gr
-except ImportError:
-    raise ImportError("gradio is required to run this demo. Install it with: pip install 'vllm-omni[demo]'") from None
+import gradio as gr
 import httpx
 import numpy as np
 import soundfile as sf
@@ -3,10 +3,7 @@
 import base64
 import io

-try:
-    import gradio as gr
-except ImportError:
-    raise ImportError("gradio is required to run this demo. Install it with: pip install 'vllm-omni[demo]'") from None
+import gradio as gr
 import httpx
 import numpy as np
 import soundfile as sf
@@ -10,10 +10,7 @@ import argparse
 import base64
 from io import BytesIO

-try:
-    import gradio as gr
-except ImportError:
-    raise ImportError("gradio is required to run this demo. Install it with: pip install 'vllm-omni[demo]'") from None
+import gradio as gr
 import requests
 from PIL import Image

@@ -20,10 +20,7 @@ from datetime import datetime
 from pathlib import Path
 from typing import Any

-try:
-    import gradio as gr
-except ImportError:
-    raise ImportError("gradio is required to run this demo. Install it with: pip install 'vllm-omni[demo]'") from None
+import gradio as gr
 import httpx
 import numpy as np
 import soundfile as sf
@@ -53,14 +53,9 @@ dev = [
    "av", # for ComfyUI tests
    "openpyxl>=3.0.0", # for nightly CI
    "pyttsx3>=2.99",
-    "opencc>=1.2.0",
    "mistune>=3.2.0", # for example tests
 ]

-demo = [
-    "gradio>=6.7.0",
-]
-
 docs = [
    "mkdocs>=1.5.0",
    "mkdocs-api-autonav",
@@ -4,6 +4,7 @@ librosa>=0.11.0
 resampy>=0.4.3
 diffusers>=0.36.0
 accelerate==1.12.0
+gradio==5.50
 soundfile>=0.13.1
 cache-dit==1.3.0
 tqdm>=4.66.0
@@ -17,4 +18,3 @@ prettytable>=3.8.0
 aenum==3.1.16
 pyzmq>=25.0.0
 janus>=1.0.0
-pydub
@@ -5,7 +5,6 @@ import json
 import math
 import os
 import random
-import re
 import tempfile

 import requests
@@ -23,7 +22,6 @@ import subprocess
 import sys
 import threading
 import time
-import uuid
 from collections.abc import Generator
 from dataclasses import dataclass
 from io import BytesIO
@@ -39,7 +37,6 @@ import torch
 import yaml
 from openai import OpenAI, omit
 from PIL import Image
-from transformers import pipeline
 from vllm import TextPrompt
 from vllm.distributed.parallel_state import cleanup_dist_env_and_memory
 from vllm.logger import init_logger
@@ -55,13 +52,6 @@ PromptAudioInput = list[tuple[Any, int]] | tuple[Any, int] | None
 PromptImageInput = list[Any] | Any | None
 PromptVideoInput = list[Any] | Any | None

-_GENDER_PIPELINE = None
-# transformers.Pipeline is not thread-safe; concurrent e2e requests must serialize inference.
-_GENDER_PIPELINE_LOCK = threading.Lock()
-
-# int16 mono PCM from /v1/audio/speech when response_format=pcm (Qwen3-TTS code2wav output rate).
-_PCM_SPEECH_SAMPLE_RATE_HZ = 24_000
-

 class OmniServerParams(NamedTuple):
    model: str
@@ -273,6 +263,7 @@ def assert_video_valid(

 def assert_audio_valid(path: Path, *, sample_rate: int, channels: int, duration_s: float) -> None:
    """Assert the WAV has the expected sample rate, channel count, and duration."""
+
    assert path.exists(), f"Audio not found: {path}"
    info = sf.info(str(path))
    assert info.samplerate == sample_rate, f"Expected sample_rate={sample_rate}, got {info.samplerate}"
@@ -475,6 +466,7 @@ def generate_synthetic_audio(
    """
    Generate TTS speech with pyttsx3 and return base64 string.
    """
+    import tempfile

    import pyttsx3
    import soundfile as sf
@@ -682,8 +674,8 @@ def generate_synthetic_audio(
    # Return result
    base64_audio = base64.b64encode(audio_bytes).decode("utf-8")
    result["base64"] = base64_audio
-    # Always include file_path to avoid KeyError in callers.
-    result["file_path"] = output_path if save_to_file and output_path else None
+    if save_to_file and output_path:
+        result["file_path"] = output_path

    return result

@@ -895,7 +887,7 @@ def generate_synthetic_image(width: int, height: int, save_to_file: bool = False


 def preprocess_text(text):
-    import opencc
+    import re

    word_to_num = {
        "zero": "0",
@@ -917,14 +909,6 @@ def preprocess_text(text):

    text = re.sub(r"[^\w\s]", "", text)
    text = re.sub(r"\s+", " ", text)
-    cc = opencc.OpenCC("t2s")
-    text = cc.convert(text)
-
-    # Special handling for spaces between Chinese characters:
-    # - Keep single spaces between English words/numbers
-    # - Remove spaces only when surrounded by Chinese characters on both sides to prevent incorrect word segmentation
-    text = re.sub(r"(?<=[\u4e00-\u9fff])\s+(?=[\u4e00-\u9fff])", "", text)
-
    return text.lower().strip()


@@ -936,7 +920,6 @@ def cosine_similarity_text(text1, text2, n: int = 3):

    text1 = preprocess_text(text1)
    text2 = preprocess_text(text2)
-    print(f"cosine similarity text1 is: {text1}, text2 is: {text2}")

    ngrams1 = [text1[i : i + n] for i in range(len(text1) - n + 1)]
    ngrams2 = [text2[i : i + n] for i in range(len(text2) - n + 1)]
@@ -962,7 +945,7 @@ def convert_audio_to_text(audio_data):
    Convert base64 encoded audio data to text using speech recognition.
    """
    audio_data = base64.b64decode(audio_data)
-    output_path = f"./test_{int(time.time())}.wav"
+    output_path = f"./test_{int(time.time())}"
    with open(output_path, "wb") as audio_file:
        audio_file.write(audio_data)

@@ -971,23 +954,11 @@ def convert_audio_to_text(audio_data):
    return text


-def _merge_base64_audio_to_segment(base64_list: list[str]):
-    """Merge a list of base64-encoded audio chunks into one pydub AudioSegment."""
-    from pydub import AudioSegment
-
-    merged = None
-    for b64 in base64_list:
-        raw = base64.b64decode(b64.split(",", 1)[-1])
-        seg = AudioSegment.from_file(io.BytesIO(raw))
-        merged = seg if merged is None else merged + seg
-    return merged
-
-
 def _whisper_transcribe_in_current_process(output_path: str) -> str:
    import whisper

-    # Keep Whisper on CPU to avoid consuming GPU memory in tests.
-    model = whisper.load_model("small", device="cpu")
+    device = "cuda" if torch.cuda.is_available() else "cpu"
+    model = whisper.load_model("small", device=device)
    try:
        text = model.transcribe(
            output_path,
@@ -996,8 +967,14 @@ def _whisper_transcribe_in_current_process(output_path: str) -> str:
            condition_on_previous_text=False,
        )["text"]
    finally:
+        # Sync GPU so in-flight ops finish before we free the model; otherwise
+        # freed memory may not show up until those ops complete.
+        if torch is not None and torch.cuda.is_available():
+            torch.cuda.synchronize()
        del model
        gc.collect()
+        if torch is not None and torch.cuda.is_available():
+            torch.cuda.empty_cache()

    return text or ""

@@ -1011,26 +988,22 @@ def convert_audio_file_to_text(output_path: str) -> str:
        return future.result()


-def convert_audio_bytes_to_text(raw_bytes: bytes) -> str:
-    """
-    Write container audio bytes (WAV, etc.) to a temp WAV file suitable for Whisper/ffmpeg.
-    Normalizes with soundfile to PCM_16 WAV when possible to avoid codec issues.
-    """
-    output_path = f"./test_{uuid.uuid4().hex}.wav"
-    data, samplerate = sf.read(io.BytesIO(raw_bytes))
-    sf.write(output_path, data, samplerate, format="WAV", subtype="PCM_16")
-    text = convert_audio_file_to_text(output_path)
-    return text
-
-
 def merge_base64_and_convert_to_text(base64_list):
    """
    Merge a list of base64 encoded audio data and convert to text.
    """
-    merged_audio = _merge_base64_audio_to_segment(base64_list)
-    output_path = f"./test_{uuid.uuid4().hex}.wav"
+    from pydub import AudioSegment
+
+    merged_audio = None
+    for base64_data in base64_list:
+        audio_data = base64.b64decode(base64_data.split(",", 1)[-1])
+        seg = AudioSegment.from_file(io.BytesIO(audio_data))
+        if merged_audio is None:
+            merged_audio = seg
+        else:
+            merged_audio += seg
+    output_path = f"./test_{int(time.time())}"
    merged_audio.export(output_path, format="wav")
-    print(f"audio data is saved: {output_path}")
    text = convert_audio_file_to_text(output_path)
    return text

@@ -1211,7 +1184,8 @@ def modify_stage_config(
                                break

                        if target_stage is None:
-                            continue
+                            available_ids = [s.get("stage_id") for s in stage_args if "stage_id" in s]
+                            raise KeyError(f"Stage ID {stage_id} not found, available: {available_ids}")

                        # Delete specified paths in this stage
                        for path in delete_paths:
@@ -1489,8 +1463,6 @@ class OmniResponse:
    text_content: str | None = None
    audio_data: list[str] | None = None
    audio_content: str | None = None
-    audio_format: str | None = None
-    audio_bytes: bytes | None = None
    similarity: float | None = None
    e2e_latency: float | None = None
    success: bool = False
@@ -1508,189 +1480,6 @@ class DiffusionResponse:
    error_message: str | None = None


-def _load_gender_pipeline():
-    """
-    Lazy-load a cached audio-classification pipeline for gender.
-
-    We prefer the pipeline wrapper because it encapsulates processor/model loading
-    and avoids direct AutoProcessor.from_pretrained call sites in this file.
-    """
-    global _GENDER_PIPELINE
-    if _GENDER_PIPELINE is not None:
-        return _GENDER_PIPELINE
-
-    model_name = "7wolf/wav2vec2-base-gender-classification"
-    try:
-        # device=-1 forces CPU for pipeline.
-        _GENDER_PIPELINE = pipeline(
-            task="audio-classification",
-            model=model_name,
-            device=-1,
-        )
-        return _GENDER_PIPELINE
-    except Exception as exc:  # pragma: no cover - best-effort fallback
-        print(f"Warning: failed to create gender pipeline '{model_name}': {exc}")
-        _GENDER_PIPELINE = None
-        return None
-
-
-def _median_pitch_hz_from_autocorr(mono: np.ndarray, sr: int) -> float | None:
-    """
-    Rough median F0 (Hz) over short-time frames. Used to debias wav2vec2 gender head on TTS,
-    which often labels lower-pitched synthetic speech as female under load or on clean signals.
-    Returns None if the clip is too short or mostly unvoiced.
-    """
-    x = np.asarray(mono, dtype=np.float64)
-    x = x - np.mean(x)
-    if x.size < int(0.15 * sr):
-        return None
-    frame_len = int(0.04 * sr)
-    hop = max(frame_len // 2, 1)
-    f0_min_hz, f0_max_hz = 70.0, 400.0
-    lag_min = max(1, int(sr / f0_max_hz))
-    lag_max = min(frame_len - 2, int(sr / f0_min_hz))
-    if lag_max <= lag_min:
-        return None
-    win = np.hamming(frame_len)
-    pitches: list[float] = []
-    for start in range(0, int(x.shape[0]) - frame_len, hop):
-        frame = x[start : start + frame_len] * win
-        frame = frame - np.mean(frame)
-        if float(np.sqrt(np.mean(frame**2))) < 1e-4:
-            continue
-        ac = np.correlate(frame, frame, mode="full")[frame_len - 1 :]
-        ac = ac / (float(ac[0]) + 1e-12)
-        region = ac[lag_min : lag_max + 1]
-        peak_rel = int(np.argmax(region))
-        peak_lag = peak_rel + lag_min
-        if peak_lag <= 0:
-            continue
-        f0 = float(sr) / float(peak_lag)
-        if f0_min_hz <= f0 <= f0_max_hz:
-            pitches.append(f0)
-    if len(pitches) < 4:
-        return None
-    return float(np.median(np.asarray(pitches, dtype=np.float64)))
-
-
-def _estimate_voice_gender_from_audio(audio_bytes: bytes) -> str:
-    """
-    Estimate voice gender from audio using a small pre-trained classification model.
-
-    Uses a cached `audio-classification` pipeline to classify the clip.
-    Returns 'male' / 'female' when the model confidence is >= 0.9 and the label
-    maps to one of these; otherwise returns 'unknown'. If the model is unavailable
-    or inference fails, returns 'unknown' to keep tests stable.
-
-    Under concurrent tests, a global lock serializes pipeline calls (the HF pipeline is not
-    thread-safe). A coarse F0 median can correct systematic "male -> female" errors on TTS audio.
-    """
-    data, sr = sf.read(io.BytesIO(audio_bytes), dtype="float32", always_2d=True)
-    if data.size == 0:
-        raise ValueError("Empty audio")
-    mono = np.mean(data, axis=1)
-
-    try:
-        target_sr = 16000
-        if int(sr) != target_sr and mono.size > 1:
-            src_len = int(mono.shape[0])
-            dst_len = max(1, int(round(src_len * float(target_sr) / float(sr))))
-            src_idx = np.arange(src_len, dtype=np.float32)
-            dst_idx = np.linspace(0, src_len - 1, dst_len, dtype=np.float32)
-            mono = np.interp(dst_idx, src_idx, mono.astype(np.float32, copy=False)).astype(np.float32)
-            sr = target_sr
-
-        median_f0 = _median_pitch_hz_from_autocorr(mono, sr)
-
-        clf = _load_gender_pipeline()
-        if clf is None:
-            print("gender model not available, returning 'unknown'")
-            return "unknown"
-
-        # transformers pipeline returns a list of {label, score} (highest score first).
-        with _GENDER_PIPELINE_LOCK:
-            outputs = clf(mono, sampling_rate=sr)
-        if not outputs:
-            return "unknown"
-
-        top = outputs[0]
-        label = str(top.get("label", "")).lower()
-        conf = float(top.get("score", 0.0))
-
-        if conf < 0.6:
-            gender = "unknown"
-        # Some models use non-English labels (e.g., Russian). Normalize to 'male'/'female'.
-        elif ("female" in label) or ("жен" in label):
-            gender = "female"
-        elif ("male" in label) or ("муж" in label):
-            gender = "male"
-        else:
-            gender = "unknown"
-
-        # Debias: wav2vec2 gender heads often call TTS / band-limited male speech "female".
-        # Low median F0 (~speech male range) + female label -> trust pitch when score is not overwhelming.
-        if gender == "female" and median_f0 is not None and median_f0 < 165.0 and conf < 0.88:
-            print(f"gender pitch assist: reclassifying female->male (median_f0={median_f0:.1f} Hz, conf={conf:.3f})")
-            gender = "male"
-        elif gender == "male" and median_f0 is not None and median_f0 > 230.0 and conf < 0.88:
-            print(f"gender pitch assist: reclassifying male->female (median_f0={median_f0:.1f} Hz, conf={conf:.3f})")
-            gender = "female"
-
-        print(
-            f"gender classifier: label={label}, conf={conf:.3f}, gender={gender}"
-            + (f", median_f0={median_f0:.1f}Hz" if median_f0 is not None else "")
-        )
-        return gender
-    except Exception as exc:  # pragma: no cover - best-effort fallback
-        print(f"Warning: gender classification failed, returning 'unknown': {exc}")
-        return "unknown"
-
-
-# Threshold aligned with _compute_pcm_hnr_db docstring (clean clone vs distorted).
-_MIN_PCM_SPEECH_HNR_DB = 1.0
-
-
-def _compute_pcm_hnr_db(pcm_samples: np.ndarray, sr: int = _PCM_SPEECH_SAMPLE_RATE_HZ) -> float:
-    """Compute mean Harmonic-to-Noise Ratio (dB) for speech quality.
-
-    Clean cloned speech has HNR > 1.2 dB; distorted speech (e.g. lost
-    ref_code decoder context) drops below 1.0 dB.
-    """
-    frame_len = int(0.03 * sr)  # 30ms frames
-    hop = frame_len // 2
-    hnr_values: list[float] = []
-
-    for start in range(0, len(pcm_samples) - frame_len, hop):
-        frame = pcm_samples[start : start + frame_len].astype(np.float32, copy=False)
-        frame = frame - np.mean(frame)
-        if np.max(np.abs(frame)) < 0.01:
-            continue
-        ac = np.correlate(frame, frame, mode="full")[len(frame) - 1 :]
-        ac = ac / (ac[0] + 1e-10)
-        min_lag = int(sr / 400)
-        max_lag = min(int(sr / 80), len(ac))
-        if min_lag >= max_lag:
-            continue
-        peak = float(np.max(ac[min_lag:max_lag]))
-        if 0 < peak < 1:
-            hnr_values.append(10 * np.log10(peak / (1 - peak + 1e-10)))
-
-    return float(np.mean(hnr_values)) if hnr_values else 0.0
-
-
-def _assert_pcm_int16_speech_hnr(audio_bytes: bytes) -> None:
-    """Validate harmonic-to-noise ratio on raw int16 PCM from /v1/audio/speech."""
-    assert audio_bytes is not None and len(audio_bytes) >= 2, "missing PCM bytes"
-    assert len(audio_bytes) % 2 == 0, "PCM byte length must be aligned to int16"
-    pcm_samples = np.frombuffer(audio_bytes, dtype=np.int16).astype(np.float32) / 32768.0
-    hnr = _compute_pcm_hnr_db(pcm_samples)
-    print(f"PCM speech HNR: {hnr:.2f} dB (threshold: {_MIN_PCM_SPEECH_HNR_DB} dB)")
-    assert hnr >= _MIN_PCM_SPEECH_HNR_DB, (
-        f"Audio distortion detected: HNR={hnr:.2f} dB < {_MIN_PCM_SPEECH_HNR_DB} dB. "
-        "Voice clone decoder may be losing ref_code speaker context on later chunks."
-    )
-
-
 def assert_omni_response(response: OmniResponse, request_config: dict[str, Any], run_level):
    """
    Validate response results.
@@ -1704,7 +1493,7 @@ def assert_omni_response(response: OmniResponse, request_config: dict[str, Any],
    assert response.success, "The request failed."
    e2e_latency = response.e2e_latency
    if e2e_latency is not None:
-        print(f"the e2e latency is: {e2e_latency}")
+        print(f"the avg e2e latency is: {e2e_latency}")

    modalities = request_config.get("modalities", ["text", "audio"])

@@ -1739,68 +1528,6 @@ def assert_omni_response(response: OmniResponse, request_config: dict[str, Any],
            print(f"similarity is: {response.similarity}")


-def assert_audio_speech_response(
-    response: OmniResponse,
-    request_config: dict[str, Any],
-    run_level: str,
-) -> None:
-    """
-    Validate /v1/audio/speech response: success, optional format check, transcription similarity
-    and gender (non-PCM only for advanced_model), and int16 PCM HNR when response_format is pcm.
-    """
-    assert response.success, "The request failed."
-
-    req_fmt = request_config.get("response_format")
-
-    if req_fmt == "pcm" and response.audio_bytes:
-        _assert_pcm_int16_speech_hnr(response.audio_bytes)
-        if response.audio_format:
-            assert "pcm" in response.audio_format.lower(), (
-                f"Expected audio/pcm content-type, got {response.audio_format!r}"
-            )
-
-    elif req_fmt == "wav" and response.audio_format:
-        assert req_fmt in response.audio_format, (
-            f"The response audio format {response.audio_format} don't match the request audio format {req_fmt}"
-        )
-
-    e2e_latency = response.e2e_latency
-    if e2e_latency is not None:
-        print(f"the avg e2e latency is: {e2e_latency}")
-
-    if run_level == "advanced_model" and req_fmt != "pcm":
-        # Text–audio semantic similarity check (skipped for raw PCM: no Whisper transcript).
-        expected_text = request_config.get("input")
-        if expected_text:
-            transcript = (response.audio_content or "").strip()
-            print(f"audio content is: {transcript}")
-            print(f"input text is: {expected_text}")
-            similarity = cosine_similarity_text(transcript.lower(), expected_text.lower())
-            print(f"Cosine similarity: {similarity:.3f}")
-            assert similarity > 0.9, (
-                f"Transcript doesn't match input: similarity={similarity:.2f}, transcript='{transcript}'"
-            )
-
-        # Voice gender consistency check:
-        # When the estimator returns 'unknown', we treat it as inconclusive and do NOT fail the test.
-        voice = (request_config.get("voice") or "").lower()
-        if voice and response.audio_bytes:
-            estimated_gender = _estimate_voice_gender_from_audio(response.audio_bytes)
-            voice_gender_map = {
-                # adjust this mapping to your actual voice names
-                "serena": "female",
-                "uncle_fu": "male",
-                "clone": "female",
-            }
-            expected_gender = voice_gender_map.get(voice)
-            if expected_gender is not None:
-                print(f"Estimated voice gender from audio: {estimated_gender} (voice='{voice}')")
-                if estimated_gender != "unknown":
-                    assert estimated_gender == expected_gender, (
-                        f"Voice '{voice}' is expected {expected_gender}, but estimated gender is '{estimated_gender}'"
-                    )
-
-
 def assert_diffusion_response(response: DiffusionResponse, request_config: dict[str, Any], run_level: str = None):
    """
    Validate diffusion response results.
@@ -2027,109 +1754,6 @@ class OpenAIClientHandler:

        return result

-    def _process_stream_audio_speech_response(self, response, *, response_format: str | None = None) -> OmniResponse:
-        """
-        Process streaming /v1/audio/speech responses into an OmniResponse.
-
-        This mirrors _process_stream_omni_response but operates on low-level
-        audio bytes and produces an OmniResponse with audio_content filled
-        from Whisper transcription.
-        """
-        result = OmniResponse()
-        start_time = time.perf_counter()
-
-        try:
-            # Aggregate all audio bytes from the streaming response.
-            data = bytearray()
-
-            # Preferred OpenAI helper.
-            if hasattr(response, "iter_bytes") and callable(getattr(response, "iter_bytes")):
-                for chunk in response.iter_bytes():
-                    if chunk:
-                        data.extend(chunk)
-            else:
-                # Generic iterable-of-bytes fallback (e.g., generator or list of chunks).
-                try:
-                    iterator = iter(response)
-                except TypeError:
-                    iterator = None
-
-                if iterator is not None:
-                    for chunk in iterator:
-                        if not chunk:
-                            continue
-                        if isinstance(chunk, (bytes, bytearray)):
-                            data.extend(chunk)
-                        elif hasattr(chunk, "data"):
-                            data.extend(chunk.data)  # type: ignore[arg-type]
-                        elif hasattr(chunk, "content"):
-                            data.extend(chunk.content)  # type: ignore[arg-type]
-                        else:
-                            raise TypeError(f"Unsupported stream chunk type: {type(chunk)}")
-                else:
-                    raise TypeError(f"Unsupported audio speech streaming response type: {type(response)}")
-
-            raw_bytes = bytes(data)
-            if response_format == "pcm":
-                transcript = None
-            else:
-                transcript = convert_audio_bytes_to_text(raw_bytes)
-
-            # Populate OmniResponse.
-            result.audio_bytes = raw_bytes
-            result.audio_content = transcript
-            result.e2e_latency = time.perf_counter() - start_time
-            result.success = True
-            result.audio_format = getattr(response, "response", None)
-            if result.audio_format is not None:
-                result.audio_format = result.audio_format.headers.get("content-type", "")
-
-        except Exception as e:
-            result.error_message = f"Audio speech stream processing error: {str(e)}"
-            print(f"Error: {result.error_message}")
-
-        return result
-
-    def _process_non_stream_audio_speech_response(
-        self, response, *, response_format: str | None = None
-    ) -> OmniResponse:
-        """
-        Process non-streaming /v1/audio/speech responses into an OmniResponse.
-
-        This mirrors _process_non_stream_omni_response but for the binary
-        audio payload returned by audio.speech.create.
-        """
-        result = OmniResponse()
-        start_time = time.perf_counter()
-
-        try:
-            # OpenAI non-streaming audio.speech.create returns HttpxBinaryResponseContent (.read() or .content)
-            if hasattr(response, "read") and callable(getattr(response, "read")):
-                raw_bytes = response.read()
-            elif hasattr(response, "content"):
-                raw_bytes = response.content  # type: ignore[assignment]
-            else:
-                raise TypeError(f"Unsupported audio speech response type: {type(response)}")
-
-            if response_format == "pcm":
-                transcript = None
-            else:
-                transcript = convert_audio_bytes_to_text(raw_bytes)
-
-            result.audio_bytes = raw_bytes
-            result.audio_content = transcript
-            result.e2e_latency = time.perf_counter() - start_time
-            result.success = True
-            result.audio_format = getattr(response, "response", None)
-            if result.audio_format is not None:
-                result.audio_format = result.audio_format.headers.get("content-type", "")
-
-        except Exception as e:
-            result.error_message = f"Audio speech non-stream processing error: {str(e)}"
-            print(f"Error: {result.error_message}")
-
-        return result
-
    def send_omni_request(self, request_config: dict[str, Any], request_num: int = 1) -> list[OmniResponse]:
        """
        Send OpenAI requests.
@@ -2164,138 +1788,35 @@ class OpenAIClientHandler:
            responses.append(response)

        else:
-            # Send concurrent requests: run create + process in worker so e2e_latency includes full round-trip.
-            def _one_omni_request():
-                start = time.perf_counter()
-                chat_completion = self.client.chat.completions.create(
-                    model=request_config.get("model"),
-                    messages=request_config.get("messages"),
-                    modalities=modalities,
-                    stream=stream,
-                )
-                if stream:
-                    response = self._process_stream_omni_response(chat_completion)
-                else:
-                    response = self._process_non_stream_omni_response(chat_completion)
-                response.e2e_latency = time.perf_counter() - start
-                return response
-
+            # Send concurrent requests
            with concurrent.futures.ThreadPoolExecutor(max_workers=request_num) as executor:
-                futures = [executor.submit(_one_omni_request) for _ in range(request_num)]
+                futures = []
+
+                # Submit all request tasks
+                for _ in range(request_num):
+                    future = executor.submit(
+                        self.client.chat.completions.create,
+                        model=request_config.get("model"),
+                        messages=request_config.get("messages"),
+                        modalities=modalities,
+                        stream=stream,
+                    )
+                    futures.append(future)
+
+                # Process completed tasks
                for future in concurrent.futures.as_completed(futures):
-                    response = future.result()
+                    chat_completion = future.result()
+
+                    if stream:
+                        response = self._process_stream_omni_response(chat_completion)
+                    else:
+                        response = self._process_non_stream_omni_response(chat_completion)
+
                    assert_omni_response(response, request_config, run_level=self.run_level)
                    responses.append(response)

        return responses

-    def send_audio_speech_request(self, request_config: dict[str, Any], request_num: int = 1) -> list[OmniResponse]:
-        """
-        Call the /v1/audio/speech endpoint using the same configuration-dict
-        style as send_omni_request, but via the OpenAI Python client's
-        audio.speech APIs.
-
-        Expected keys in request_config:
-          - model: model name/path (required)
-          - input: text to synthesize (required)
-          - response_format: audio format such as "wav" or "pcm" (optional)
-          - task_type, ref_text, ref_audio: TTS-specific extras (optional, passed via extra_body)
-          - timeout: request timeout in seconds (float, optional, default 120.0)
-          - stream: whether to use streaming API (bool, optional, default False)
-        """
-        timeout = float(request_config.get("timeout", 120.0))
-
-        model = request_config["model"]
-        text_input = request_config["input"]
-        stream = bool(request_config.get("stream", False))
-        voice = request_config.get("voice", None)
-
-        # Standard OpenAI param: use omit when not provided to keep default behavior.
-        response_format = request_config.get("response_format", omit)
-
-        # Qwen3-TTS custom fields, forwarded via extra_body.
-        extra_body: dict[str, Any] = {}
-        # Keep this list aligned with vllm_omni.entrypoints.openai.protocol.audio params.
-        for key in ("task_type", "ref_text", "ref_audio", "language", "max_new_tokens"):
-            if key in request_config:
-                extra_body[key] = request_config[key]
-
-        responses: list[OmniResponse] = []
-
-        speech_fmt: str | None = None if response_format is omit else str(response_format).lower()
-
-        if request_num == 1:
-            if stream:
-                # Use streaming response helper.
-                with self.client.audio.speech.with_streaming_response.create(
-                    model=model,
-                    input=text_input,
-                    response_format=response_format,
-                    extra_body=extra_body or None,
-                    timeout=timeout,
-                    voice=voice,
-                ) as resp:
-                    omni_resp = self._process_stream_audio_speech_response(resp, response_format=speech_fmt)
-            else:
-                # Non-streaming response.
-                resp = self.client.audio.speech.create(
-                    model=model,
-                    input=text_input,
-                    response_format=response_format,
-                    extra_body=extra_body or None,
-                    timeout=timeout,
-                    voice=voice,
-                )
-                omni_resp = self._process_non_stream_audio_speech_response(resp, response_format=speech_fmt)
-
-            assert_audio_speech_response(omni_resp, request_config, run_level=self.run_level)
-            responses.append(omni_resp)
-            return responses
-        else:
-            # request_num > 1: concurrent requests (use same params as single-request path)
-
-            if stream:
-
-                def _stream_task():
-                    with self.client.audio.speech.with_streaming_response.create(
-                        model=model,
-                        input=text_input,
-                        response_format=response_format,
-                        extra_body=extra_body or None,
-                        timeout=timeout,
-                        voice=voice,
-                    ) as resp:
-                        return self._process_stream_audio_speech_response(resp, response_format=speech_fmt)
-
-                with concurrent.futures.ThreadPoolExecutor(max_workers=request_num) as executor:
-                    futures = [executor.submit(_stream_task) for _ in range(request_num)]
-                    for future in concurrent.futures.as_completed(futures):
-                        omni_resp = future.result()
-                        assert_audio_speech_response(omni_resp, request_config, run_level=self.run_level)
-                        responses.append(omni_resp)
-            else:
-                with concurrent.futures.ThreadPoolExecutor(max_workers=request_num) as executor:
-                    futures = []
-                    for _ in range(request_num):
-                        future = executor.submit(
-                            self.client.audio.speech.create,
-                            model=model,
-                            input=text_input,
-                            response_format=response_format,
-                            extra_body=extra_body or None,
-                            timeout=timeout,
-                            voice=voice,
-                        )
-                        futures.append(future)
-
-                    for future in concurrent.futures.as_completed(futures):
-                        resp = future.result()
-                        omni_resp = self._process_non_stream_audio_speech_response(resp, response_format=speech_fmt)
-                        assert_audio_speech_response(omni_resp, request_config, run_level=self.run_level)
-                        responses.append(omni_resp)
-
-        return responses
-
    def send_diffusion_request(self, request_config: dict[str, Any], request_num: int = 1) -> list[OmniResponse]:
        """
        Send OpenAI requests for diffusion models.
@@ -2524,44 +2045,6 @@ class OmniRunner:
            **kwargs,
        )

-    def _estimate_prompt_len(
-        self,
-        additional_information: dict[str, Any],
-        model_name: str,
-        _cache: dict[str, Any] = {},
-    ) -> int:
-        """Estimate prompt_token_ids placeholder length for the Talker stage.
-
-        The AR Talker replaces all input embeddings via ``preprocess``, so the
-        placeholder values are irrelevant but the **length** must match the
-        embeddings that ``preprocess`` will produce.
-        """
-        try:
-            from vllm_omni.model_executor.models.qwen3_tts.configuration_qwen3_tts import Qwen3TTSConfig
-            from vllm_omni.model_executor.models.qwen3_tts.qwen3_tts_talker import (
-                Qwen3TTSTalkerForConditionalGeneration,
-            )
-
-            if model_name not in _cache:
-                from transformers import AutoTokenizer
-
-                tok = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True, padding_side="left")
-                cfg = Qwen3TTSConfig.from_pretrained(model_name, trust_remote_code=True)
-                _cache[model_name] = (tok, getattr(cfg, "talker_config", None))
-
-            tok, tcfg = _cache[model_name]
-            task_type = (additional_information.get("task_type") or ["CustomVoice"])[0]
-            return Qwen3TTSTalkerForConditionalGeneration.estimate_prompt_len_from_additional_information(
-                additional_information=additional_information,
-                task_type=task_type,
-                tokenize_prompt=lambda t: tok(t, padding=False)["input_ids"],
-                codec_language_id=getattr(tcfg, "codec_language_id", None),
-                spk_is_dialect=getattr(tcfg, "spk_is_dialect", None),
-            )
-        except Exception as exc:
-            logger.warning("Failed to estimate prompt length, using fallback 2048: %s", exc)
-            return 2048
-
    def get_default_sampling_params_list(self) -> list[OmniSamplingParams]:
        """
        Get a list of default sampling parameters for all stages.
@@ -2616,42 +2099,6 @@ class OmniRunner:
        if isinstance(prompts, str):
            prompts = [prompts]

-        # Qwen-TTS: follow examples/offline_inference/qwen3_tts/end2end.py style.
-        # Stage 0 expects token placeholders + additional_information (text/speaker/task_type/...),
-        # and Talker replaces embeddings in preprocess based on additional_information only.
-        is_tts_model = "Qwen3-TTS" in self.model_name or "qwen3_tts" in self.model_name.lower()
-        if is_tts_model and modalities == ["audio"]:
-            tts_kw = mm_processor_kwargs or {}
-            task_type = tts_kw.get("task_type", "CustomVoice")
-            speaker = tts_kw.get("speaker", "Vivian")
-            language = tts_kw.get("language", "Auto")
-            max_new_tokens = int(tts_kw.get("max_new_tokens", 2048))
-            ref_audio = tts_kw.get("ref_audio", None)
-            ref_text = tts_kw.get("ref_text", None)
-
-            omni_inputs: list[TextPrompt] = []
-            for prompt_text in prompts:
-                text_str = str(prompt_text).strip() or " "
-                additional_information: dict[str, Any] = {
-                    "task_type": [task_type],
-                    "text": [text_str],
-                    "language": [language],
-                    "speaker": [speaker],
-                    "max_new_tokens": [max_new_tokens],
-                }
-                if ref_audio is not None:
-                    additional_information["ref_audio"] = [ref_audio]
-                if ref_text is not None:
-                    additional_information["ref_text"] = [ref_text]
-                # Use official helper to get correct placeholder length
-                plen = self._estimate_prompt_len(additional_information, self.model_name)
-                input_dict: TextPrompt = {
-                    "prompt_token_ids": [0] * plen,
-                    "additional_information": additional_information,
-                }
-                omni_inputs.append(input_dict)
-            return omni_inputs
-
        def _normalize_mm_input(mm_input, num_prompts):
            if mm_input is None:
                return [None] * num_prompts
@@ -2919,84 +2366,9 @@ class OmniRunnerHandler:
            prompts=prompts, videos=videos, images=images, audios=audios, modalities=modalities
        )
        response = self._process_output(outputs)
-        assert_omni_response(response, request_config, run_level="core_model")
+        assert_omni_response(response, request_config, run_level="L2")
        return response

-    def send_audio_speech_request(
-        self,
-        request_config: dict[str, Any],
-    ) -> OmniResponse:
-        """
-        Offline TTS: text -> audio via generate_multimodal, then validate with assert_audio_speech_response.
-
-        request_config must contain:
-          - 'input' or 'prompts': text to synthesize.
-        Optional keys:
-          - 'voice'       -> speaker (CustomVoice)
-          - 'task_type'   -> task_type in additional_information (default: "CustomVoice")
-          - 'language'    -> language in additional_information (default: "Auto")
-          - 'max_new_tokens' -> max_new_tokens in additional_information (default: 2048)
-          - 'response_format' -> desired audio format (used only for assertion)
-        """
-        input_text = request_config.get("input") or request_config.get("prompts")
-        if input_text is None:
-            raise ValueError("request_config must contain 'input' or 'prompts' for TTS")
-        if isinstance(input_text, list):
-            input_text = input_text[0] if input_text else ""
-
-        # Build TTS-specific kwargs passed through to get_omni_inputs for Qwen3-TTS,
-        # matching examples/offline_inference/qwen3_tts/end2end.py.
-        mm_processor_kwargs: dict[str, Any] = {}
-        if "voice" in request_config:
-            mm_processor_kwargs["speaker"] = request_config["voice"]
-        if "task_type" in request_config:
-            mm_processor_kwargs["task_type"] = request_config["task_type"]
-        if "ref_audio" in request_config:
-            mm_processor_kwargs["ref_audio"] = request_config["ref_audio"]
-        if "ref_text" in request_config:
-            mm_processor_kwargs["ref_text"] = request_config["ref_text"]
-        if "language" in request_config:
-            mm_processor_kwargs["language"] = request_config["language"]
-        if "max_new_tokens" in request_config:
-            mm_processor_kwargs["max_new_tokens"] = request_config["max_new_tokens"]
-
-        outputs = self.runner.generate_multimodal(
-            prompts=input_text,
-            modalities=["audio"],
-            mm_processor_kwargs=mm_processor_kwargs or None,
-        )
-        mm_out: dict[str, Any] | None = None
-        for stage_out in outputs:
-            if getattr(stage_out, "final_output_type", None) == "audio":
-                mm_out = stage_out.request_output.outputs[0].multimodal_output
-                break
-        if mm_out is None:
-            result = OmniResponse(success=False, error_message="No audio output from pipeline")
-            assert result.success, result.error_message
-            return result
-
-        audio_data = mm_out.get("audio")
-        if audio_data is None:
-            result = OmniResponse(success=False, error_message="No audio tensor in multimodal output")
-            assert result.success, result.error_message
-            return result
-
-        sr_raw = mm_out.get("sr")
-        sr_val = sr_raw[-1] if isinstance(sr_raw, list) and sr_raw else sr_raw
-        sr = int(sr_val.item() if hasattr(sr_val, "item") else sr_val)
-        wav_tensor = torch.cat(audio_data, dim=-1) if isinstance(audio_data, list) else audio_data
-        wav_buf = io.BytesIO()
-        sf.write(
-            wav_buf,
-            wav_tensor.float().cpu().numpy().reshape(-1),
-            samplerate=sr,
-            format="WAV",
-            subtype="PCM_16",
-        )
-        result = OmniResponse(success=True, audio_bytes=wav_buf.getvalue(), audio_format="audio/wav")
-        assert_audio_speech_response(result, request_config, run_level="core_model")
-        return result
-
    def start_profile(
        self,
        profile_prefix: str | None = None,
@@ -118,6 +118,10 @@ def benchmark_params(request, omni_server):
    if param_index >= len(all_params):
        raise ValueError(f"No benchmark parameters found for index {param_index} in test: {test_name}")

+    if all_params[param_index]["dataset_name"] == "random-mm":
+        # TODO: Due to known issues, skip the random-mm dataset.
+        pytest.skip("Skipping parameter for random-mm dataset.")
+
    current = param_index + 1
    total = len(all_params)
    print(f"\n  Running benchmark {current}/{total} for {test_name}")
@@ -128,70 +132,15 @@ def benchmark_params(request, omni_server):
    }


-def _resolve_baseline_value(
-    baseline_raw: Any,
-    *,
-    sweep_index: int | None,
-    max_concurrency: Any = None,
-    request_rate: Any = None,
-) -> Any:
-    """Pick the baseline threshold for this sweep step.
-
-    Supported shapes per metric:
-    - **Scalar** — same threshold for every concurrency / QPS.
-    - **List** — aligned with ``max_concurrency`` / ``request_rate`` sweep order; use ``sweep_index``.
-    - **Dict** — keyed by concurrency or rate, e.g. ``{"1": 500, "4": 800}`` (keys are strings in JSON).
-
-    For dict lookup, ``max_concurrency`` is preferred when both are set (concurrency sweep).
-    """
-    if baseline_raw is None:
-        # If no baseline is set, the maximum value will be used.
-        return 100000
-    if isinstance(baseline_raw, dict):
-        if max_concurrency is not None:
-            for key in (max_concurrency, str(max_concurrency)):
-                if key in baseline_raw:
-                    return baseline_raw[key]
-        if request_rate is not None:
-            for key in (request_rate, str(request_rate)):
-                if key in baseline_raw:
-                    return baseline_raw[key]
-        raise KeyError(
-            f"baseline dict has no key for max_concurrency={max_concurrency!r} "
-            f"or request_rate={request_rate!r}; keys={list(baseline_raw.keys())!r}"
-        )
-    if isinstance(baseline_raw, (list, tuple)):
-        return baseline_raw[sweep_index]
-    return baseline_raw
-
-
-def assert_result(
-    result,
-    params,
-    num_prompt,
-    *,
-    sweep_index: int | None = None,
-    max_concurrency: Any = None,
-    request_rate: Any = None,
-) -> None:
+def assert_result(result, params, num_prompt):
    assert result["completed"] == num_prompt, "Request failures exist"
    baseline_data = params.get("baseline", {})
-    for metric_name, baseline_raw in baseline_data.items():
+    for metric_name, baseline_value in baseline_data.items():
        current_value = result[metric_name]
-        baseline_value = _resolve_baseline_value(
-            baseline_raw,
-            sweep_index=sweep_index,
-            max_concurrency=max_concurrency,
-            request_rate=request_rate,
-        )
        if "throughput" in metric_name:
-            if current_value <= baseline_value:
-                print(
-                    f"ERROR: Throughput test results were below baseline: {metric_name}: {current_value} > {baseline_value}"
-                )
+            assert current_value >= baseline_value, f"{metric_name}: {current_value} < {baseline_value}"
        else:
-            if current_value >= baseline_value:
-                print(f"ERROR: Test results exceeded baseline: {metric_name}: {current_value} < {baseline_value}")
+            assert current_value <= baseline_value, f"{metric_name}: {current_value} > {baseline_value}"


@pytest.mark.parametrize("omni_server", test_params, indirect=True)
@@ -246,8 +195,8 @@ def test_performance_benchmark(omni_server, benchmark_params):
        elif not isinstance(value, bool):
            args.extend([arg_name, str(value)])

-    # QPS test (sweep_index aligns with qps_list / num_prompt_list for this loop)
-    for i, (qps, num_prompt) in enumerate(zip(qps_list, num_prompt_list)):
+    # QPS test
+    for qps, num_prompt in zip(qps_list, num_prompt_list):
        args = args + ["--request-rate", str(qps), "--num-prompts", str(num_prompt)]
        result = run_benchmark(
            args=args,
@@ -256,16 +205,10 @@ def test_performance_benchmark(omni_server, benchmark_params):
            dataset_name=dataset_name,
            num_prompt=num_prompt,
        )
-        assert_result(
-            result,
-            params,
-            num_prompt=num_prompt,
-            sweep_index=i,
-            request_rate=qps,
-        )
+        assert_result(result, params, num_prompt=num_prompt)

-    # concurrency test (sweep_index aligns with max_concurrency_list for separate thresholds per concurrency)
-    for i, (concurrency, num_prompt) in enumerate(zip(max_concurrency_list, num_prompt_list)):
+    # concurrency test
+    for concurrency, num_prompt in zip(max_concurrency_list, num_prompt_list):
        args = args + ["--max-concurrency", str(concurrency), "--num-prompts", str(num_prompt), "--request-rate", "inf"]
        result = run_benchmark(
            args=args,
@@ -274,10 +217,4 @@ def test_performance_benchmark(omni_server, benchmark_params):
            dataset_name=dataset_name,
            num_prompt=num_prompt,
        )
-        assert_result(
-            result,
-            params,
-            num_prompt=num_prompt,
-            sweep_index=i,
-            max_concurrency=concurrency,
-        )
+        assert_result(result, params, num_prompt=num_prompt)
@@ -25,9 +25,9 @@
                "ignore_eos": true,
                "percentile-metrics": "ttft,tpot,itl,e2el,audio_rtf,audio_ttfp,audio_duration",
                "baseline": {
-                    "mean_ttft_ms": [1000, 3000, 5000],
-                    "mean_audio_ttfp_ms": [8000, 10000, 13000],
-                    "mean_audio_rtf": [0.2, 0.25, 0.45]
+                    "mean_ttft_ms": 100000,
+                    "mean_audio_ttfp_ms": 100000,
+                    "mean_audio_rtf": 100000
                }
            },
            {
@@ -39,10 +39,10 @@
                    40,
                    100
                ],
-                "request_rate": [
-                    0.1,
-                    0.3,
-                    0.5
+                "max_concurrency": [
+                    1,
+                    4,
+                    10
                ],
                "random_input_len": 100,
                "random_output_len": 100,
@@ -62,31 +62,9 @@
                },
                "percentile-metrics": "ttft,tpot,itl,e2el,audio_rtf,audio_ttfp,audio_duration",
                "baseline": {
-                    "mean_ttft_ms": [2000, 4000, 6000],
-                    "mean_audio_ttfp_ms": [10000, 13000, 15000],
-                    "mean_audio_rtf": [0.25, 0.35, 0.45]
-                }
-            },
-            {
-                "dataset_name": "random",
-                "backend": "openai-chat-omni",
-                "endpoint": "/v1/chat/completions",
-                "num_prompts": [
-                    4,
-                    16
-                ],
-                "max_concurrency": [
-                    1,
-                    4
-                ],
-                "random_input_len": 2500,
-                "random_output_len": 900,
-                "ignore_eos": true,
-                "percentile-metrics": "ttft,tpot,itl,e2el,audio_rtf,audio_ttfp,audio_duration",
-                "baseline": {
-                    "mean_ttft_ms": [1000, 3000],
-                    "mean_audio_ttfp_ms": [30000, 60000],
-                    "mean_audio_rtf": [0.35, 0.45]
+                    "mean_ttft_ms": 100000,
+                    "mean_audio_ttfp_ms": 100000,
+                    "mean_audio_rtf": 100000
                }
            }
        ]
@@ -135,9 +113,9 @@
                "ignore_eos": true,
                "percentile-metrics": "ttft,tpot,itl,e2el,audio_rtf,audio_ttfp,audio_duration",
                "baseline": {
-                    "mean_ttft_ms": [1000, 3000, 5000],
-                    "mean_audio_ttfp_ms": [1000, 3000, 5000],
-                    "mean_audio_rtf": [0.2, 0.35, 0.6]
+                    "mean_ttft_ms": 100000,
+                    "mean_audio_ttfp_ms": 100000,
+                    "mean_audio_rtf": 100000
                }
            },
            {
@@ -149,10 +127,10 @@
                    40,
                    100
                ],
-                "request_rate": [
-                    0.1,
-                    0.3,
-                    0.5
+                "max_concurrency": [
+                    1,
+                    4,
+                    10
                ],
                "random_input_len": 100,
                "random_output_len": 100,
@@ -172,31 +150,9 @@
                },
                "percentile-metrics": "ttft,tpot,itl,e2el,audio_rtf,audio_ttfp,audio_duration",
                "baseline": {
-                    "mean_ttft_ms": [2000, 4000, 6000],
-                    "mean_audio_ttfp_ms": [2000, 4000, 6000],
-                    "mean_audio_rtf": [0.25, 0.4, 0.7]
-                }
-            },
-            {
-                "dataset_name": "random",
-                "backend": "openai-chat-omni",
-                "endpoint": "/v1/chat/completions",
-                "num_prompts": [
-                    4,
-                    16
-                ],
-                "max_concurrency": [
-                    1,
-                    4
-                ],
-                "random_input_len": 2500,
-                "random_output_len": 900,
-                "ignore_eos": true,
-                "percentile-metrics": "ttft,tpot,itl,e2el,audio_rtf,audio_ttfp,audio_duration",
-                "baseline": {
-                    "mean_ttft_ms": [1000, 3000],
-                    "mean_audio_ttfp_ms": [1000, 3000],
-                    "mean_audio_rtf": [0.35, 0.45]
+                    "mean_ttft_ms": 100000,
+                    "mean_audio_ttfp_ms": 100000,
+                    "mean_audio_rtf": 100000
                }
            }
        ]
@@ -227,8 +183,8 @@
                },
                "percentile-metrics": "ttft,e2el,audio_rtf,audio_ttfp,audio_duration",
                "baseline": {
-                    "mean_audio_ttfp_ms": [6000, 6000],
-                    "mean_audio_rtf": [0.3, 0.3]
+                    "mean_audio_ttfp_ms": 100000,
+                    "mean_audio_rtf": 100000
                }
            }
        ]
@@ -0,0 +1,43 @@
+import pytest
+import torch
+
+from vllm_omni.diffusion.distributed.autoencoders import autoencoder_kl_wan as wan_vae_module
+from vllm_omni.diffusion.distributed.autoencoders.autoencoder_kl_wan import OmniAutoencoderKLWan
+
+pytestmark = [pytest.mark.core_model, pytest.mark.cpu]
+
+
+class _DummyOmniAutoencoderKLWan(OmniAutoencoderKLWan):
+    def __init__(self, *, dtype: torch.dtype):
+        torch.nn.Module.__init__(self)
+        self.register_parameter("dummy_weight", torch.nn.Parameter(torch.ones(1, dtype=dtype)))
+
+
+def test_wan_vae_execution_context_handles_fp32():
+    model = _DummyOmniAutoencoderKLWan(dtype=torch.float32)
+    with model._execution_context():
+        output = model.dummy_weight + 1
+    assert output.dtype == torch.float32
+
+
+def test_wan_vae_execution_context_handles_bf16():
+    model = _DummyOmniAutoencoderKLWan(dtype=torch.bfloat16)
+    with model._execution_context():
+        output = model.dummy_weight + 1
+    assert output.dtype == torch.bfloat16
+
+
+def test_wan_vae_execution_context_uses_platform_autocast(mocker):
+    sentinel = object()
+    platform = mocker.Mock()
+    platform.create_autocast_context.return_value = sentinel
+    mocker.patch.object(wan_vae_module, "current_omni_platform", platform)
+
+    model = _DummyOmniAutoencoderKLWan(dtype=torch.bfloat16)
+
+    assert model._execution_context() is sentinel
+    platform.create_autocast_context.assert_called_once_with(
+        device_type=model.dummy_weight.device.type,
+        dtype=torch.bfloat16,
+        enabled=True,
+    )
@@ -0,0 +1,273 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
+
+"""Unit tests for DistributedAutoencoderKLWan encode parallel (CPU-only)."""
+
+import pytest
+import torch
+
+pytestmark = [pytest.mark.cpu, pytest.mark.core_model]
+
+
+class _DummyConfig:
+    def __init__(self, patch_size=None, scale_factor_temporal=4):
+        self.patch_size = patch_size
+        self.scale_factor_temporal = scale_factor_temporal
+
+
+class _DummyWanVae:
+    """Minimal mock of DistributedAutoencoderKLWan for testing encode_tile_split."""
+
+    def __init__(
+        self,
+        config=None,
+        spatial_compression_ratio=8,
+        tile_sample_min_height=256,
+        tile_sample_min_width=256,
+        tile_sample_stride_height=192,
+        tile_sample_stride_width=192,
+    ):
+        self.config = config or _DummyConfig()
+        self.spatial_compression_ratio = spatial_compression_ratio
+        self.tile_sample_min_height = tile_sample_min_height
+        self.tile_sample_min_width = tile_sample_min_width
+        self.tile_sample_stride_height = tile_sample_stride_height
+        self.tile_sample_stride_width = tile_sample_stride_width
+        self.dtype = torch.float32
+
+        # Mock caches
+        self._enc_feat_map = None
+        self._enc_conv_idx = [0]
+
+    def clear_cache(self):
+        self._enc_feat_map = None
+        self._enc_conv_idx = [0]
+
+    def encoder(self, x, feat_cache=None, feat_idx=None):  # noqa: ARG002
+        # Simple mock: just return the input
+        return x
+
+    def quant_conv(self, x):
+        return x
+
+    def blend_v(self, _a, b, _blend_extent):
+        return b
+
+    def blend_h(self, _a, b, _blend_extent):
+        return b
+
+
+def _import_encode_tile_split():
+    """Import the encode_tile_split method from the module."""
+    from vllm_omni.diffusion.distributed.autoencoders.autoencoder_kl_wan import (
+        DistributedAutoencoderKLWan,
+    )
+
+    return DistributedAutoencoderKLWan.encode_tile_split
+
+
+def _import_encode_tile_exec():
+    from vllm_omni.diffusion.distributed.autoencoders.autoencoder_kl_wan import (
+        DistributedAutoencoderKLWan,
+    )
+
+    return DistributedAutoencoderKLWan.encode_tile_exec
+
+
+def _import_encode_tile_merge():
+    from vllm_omni.diffusion.distributed.autoencoders.autoencoder_kl_wan import (
+        DistributedAutoencoderKLWan,
+    )
+
+    return DistributedAutoencoderKLWan.encode_tile_merge
+
+
+class TestEncodeTileSplit:
+    """Tests for encode_tile_split method."""
+
+    def test_basic_split_without_patch_size(self):
+        """Test basic tile splitting without patch_size."""
+        encode_tile_split = _import_encode_tile_split()
+
+        vae = _DummyWanVae(
+            config=_DummyConfig(patch_size=None, scale_factor_temporal=4),
+            spatial_compression_ratio=8,
+            tile_sample_min_height=256,
+            tile_sample_min_width=256,
+            tile_sample_stride_height=192,
+            tile_sample_stride_width=192,
+        )
+
+        # Input: (B, C, T, H, W) = (1, 3, 5, 256, 256)
+        x = torch.randn(1, 3, 5, 256, 256)
+
+        tiletask_list, grid_spec = encode_tile_split(vae, x)
+
+        # With stride 192 and input size 256, we should get:
+        # Height: ceil(256/192) = 2 positions (0, 192) but 192+256 > 256, so only 1
+        # Actually for i in range(0, 256, 192): i = 0, 192 but 192 is out of bounds
+        # So we get 1x1 grid
+        assert len(tiletask_list) >= 1
+        assert grid_spec.grid_shape[0] >= 1
+        assert grid_spec.grid_shape[1] >= 1
+
+        # Check temporal chunking: 5 frames -> 1 + (5-1)//4 = 2 chunks
+        first_task = tiletask_list[0]
+        assert len(first_task.tensor) == 2  # 2 temporal chunks
+
+    def test_split_with_patch_size_scales_coordinates(self):
+        """Test that patch_size properly scales tile coordinates."""
+        encode_tile_split = _import_encode_tile_split()
+
+        # Without patch_size
+        vae_no_patch = _DummyWanVae(
+            config=_DummyConfig(patch_size=None, scale_factor_temporal=4),
+            spatial_compression_ratio=8,
+            tile_sample_min_height=256,
+            tile_sample_min_width=256,
+            tile_sample_stride_height=128,
+            tile_sample_stride_width=128,
+        )
+
+        # With patch_size=2 (simulating patchified input)
+        vae_with_patch = _DummyWanVae(
+            config=_DummyConfig(patch_size=2, scale_factor_temporal=4),
+            spatial_compression_ratio=8,
+            tile_sample_min_height=256,
+            tile_sample_min_width=256,
+            tile_sample_stride_height=128,
+            tile_sample_stride_width=128,
+        )
+
+        # Same patchified input size
+        x = torch.randn(1, 3, 5, 256, 256)
+
+        tasks_no_patch, _ = encode_tile_split(vae_no_patch, x)
+        tasks_with_patch, _ = encode_tile_split(vae_with_patch, x)
+
+        # With patch_size=2, stride becomes 128//2=64, so more tiles
+        assert len(tasks_with_patch) >= len(tasks_no_patch)
+
+    def test_temporal_compression_from_config(self):
+        """Test that temporal compression ratio is read from config."""
+        encode_tile_split = _import_encode_tile_split()
+
+        # temporal_compression=4 (default)
+        vae_4x = _DummyWanVae(
+            config=_DummyConfig(scale_factor_temporal=4),
+            tile_sample_min_height=512,
+            tile_sample_min_width=512,
+            tile_sample_stride_height=512,
+            tile_sample_stride_width=512,
+        )
+
+        # temporal_compression=2
+        vae_2x = _DummyWanVae(
+            config=_DummyConfig(scale_factor_temporal=2),
+            tile_sample_min_height=512,
+            tile_sample_min_width=512,
+            tile_sample_stride_height=512,
+            tile_sample_stride_width=512,
+        )
+
+        # 9 frames input
+        x = torch.randn(1, 3, 9, 512, 512)
+
+        tasks_4x, _ = encode_tile_split(vae_4x, x)
+        tasks_2x, _ = encode_tile_split(vae_2x, x)
+
+        # With 4x compression: 1 + (9-1)//4 = 3 chunks
+        assert len(tasks_4x[0].tensor) == 3
+
+        # With 2x compression: 1 + (9-1)//2 = 5 chunks
+        assert len(tasks_2x[0].tensor) == 5
+
+    def test_grid_spec_latent_dimensions(self):
+        """Test that grid_spec contains correct latent dimensions."""
+        encode_tile_split = _import_encode_tile_split()
+
+        vae = _DummyWanVae(
+            config=_DummyConfig(patch_size=None),
+            spatial_compression_ratio=8,
+            tile_sample_min_height=512,
+            tile_sample_min_width=512,
+            tile_sample_stride_height=512,
+            tile_sample_stride_width=512,
+        )
+
+        # Input: 512x512 with compression 8 -> 64x64 latent
+        x = torch.randn(1, 3, 5, 512, 512)
+
+        _, grid_spec = encode_tile_split(vae, x)
+
+        assert grid_spec.tile_spec["latent_height"] == 64
+        assert grid_spec.tile_spec["latent_width"] == 64
+
+
+class TestEncodeTileExec:
+    """Tests for encode_tile_exec method."""
+
+    def test_basic_exec(self):
+        """Test basic tile execution."""
+        encode_tile_exec = _import_encode_tile_exec()
+
+        vae = _DummyWanVae()
+
+        from vllm_omni.diffusion.distributed.autoencoders.distributed_vae_executor import (
+            TileTask,
+        )
+
+        # Create a simple task with 2 temporal chunks
+        tile1 = torch.randn(1, 3, 1, 32, 32)
+        tile2 = torch.randn(1, 3, 4, 32, 32)
+        task = TileTask(tile_id=0, grid_coord=(0, 0), tensor=[tile1, tile2])
+
+        result = encode_tile_exec(vae, task)
+
+        # Result should concatenate temporal dimension
+        assert result.shape[2] == 5  # 1 + 4 frames
+
+
+class TestEncodeTileMerge:
+    """Tests for encode_tile_merge method."""
+
+    def test_basic_merge(self):
+        """Test basic tile merging."""
+        encode_tile_merge = _import_encode_tile_merge()
+
+        vae = _DummyWanVae()
+
+        from vllm_omni.diffusion.distributed.autoencoders.distributed_vae_executor import (
+            GridSpec,
+        )
+
+        # Create 2x2 grid of tiles
+        tile_00 = torch.ones(1, 16, 2, 32, 32) * 0
+        tile_01 = torch.ones(1, 16, 2, 32, 32) * 1
+        tile_10 = torch.ones(1, 16, 2, 32, 32) * 2
+        tile_11 = torch.ones(1, 16, 2, 32, 32) * 3
+
+        coord_tensor_map = {
+            (0, 0): tile_00,
+            (0, 1): tile_01,
+            (1, 0): tile_10,
+            (1, 1): tile_11,
+        }
+
+        grid_spec = GridSpec(
+            split_dims=(3, 4),
+            grid_shape=(2, 2),
+            tile_spec={
+                "latent_height": 48,
+                "latent_width": 48,
+                "blend_height": 8,
+                "blend_width": 8,
+                "tile_latent_stride_height": 24,
+                "tile_latent_stride_width": 24,
+            },
+        )
+
+        result = encode_tile_merge(vae, coord_tensor_map, grid_spec)
+
+        # Output should be (1, 16, 2, 48, 48)
+        assert result.shape == (1, 16, 2, 48, 48)
@@ -209,9 +209,12 @@ def _test_cfg_parallel_worker(
            cfg_normalize=test_config["cfg_normalize"],
        )

-    # CFG parallel returns the combined prediction on every rank.
-    assert noise_pred is not None
-    result_queue.put((cfg_rank, noise_pred.cpu()))
+    # Only rank 0 has valid output in CFG parallel mode
+    if cfg_rank == 0:
+        assert noise_pred is not None
+        result_queue.put(noise_pred.cpu())
+    else:
+        assert noise_pred is None

    destroy_distributed_env()

@@ -345,18 +348,7 @@ def test_predict_noise_maybe_with_cfg(cfg_parallel_size: int, dtype: torch.dtype

    # Get results from queues
    baseline_output = baseline_queue.get()
-    cfg_parallel_outputs = [cfg_parallel_queue.get() for _ in range(cfg_parallel_size)]
-    cfg_parallel_outputs.sort(key=lambda item: item[0])
-    cfg_parallel_output = cfg_parallel_outputs[0][1]
-
-    for cfg_rank, rank_output in cfg_parallel_outputs[1:]:
-        torch.testing.assert_close(
-            rank_output,
-            cfg_parallel_output,
-            rtol=0,
-            atol=0,
-            msg=f"CFG parallel ranks produced different outputs (rank 0 vs rank {cfg_rank})",
-        )
+    cfg_parallel_output = cfg_parallel_queue.get()

    # Verify shapes match
    assert baseline_output.shape == cfg_parallel_output.shape, (
@@ -59,9 +59,9 @@ class E2EOperator:
 class DummyMixin(DistributedVaeMixin):
    def __init__(self):
        self.use_tiling = True
-        self.distributed_decoder = MagicMock()
-        self.distributed_decoder.parallel_size = 2
-        self.distributed_decoder.group = None
+        self.distributed_executor = MagicMock()
+        self.distributed_executor.parallel_size = 2
+        self.distributed_executor.group = None


@pytest.fixture(autouse=True)
@@ -7,49 +7,6 @@ import pytest
 pytestmark = [pytest.mark.core_model, pytest.mark.cpu]


-class TestSetForwardContextNumTokens:
-    """Test _set_forward_context_num_tokens defensive fix for vLLM 0.18.0."""
-
-    def test_sets_num_tokens_when_context_available(self, mocker):
-        """num_tokens should be set on ForwardContext when available."""
-        import vllm_omni.diffusion.models.hunyuan_image_3.hunyuan_fused_moe as hunyuan_moe
-
-        mock_ctx = mocker.MagicMock()
-        del mock_ctx.in_profile_run  # simulate missing attr
-        mocker.patch.object(hunyuan_moe._vllm_fc, "is_forward_context_available", return_value=True)
-        mocker.patch.object(hunyuan_moe._vllm_fc, "get_forward_context", return_value=mock_ctx)
-
-        hunyuan_moe._set_forward_context_num_tokens(1024)
-
-        assert mock_ctx.num_tokens == 1024
-        assert mock_ctx.in_profile_run is False
-
-    def test_sets_in_profile_run_only_if_missing(self, mocker):
-        """in_profile_run should not be overwritten if already set."""
-        import vllm_omni.diffusion.models.hunyuan_image_3.hunyuan_fused_moe as hunyuan_moe
-
-        mock_ctx = mocker.MagicMock()
-        mock_ctx.in_profile_run = True  # already set
-        mocker.patch.object(hunyuan_moe._vllm_fc, "is_forward_context_available", return_value=True)
-        mocker.patch.object(hunyuan_moe._vllm_fc, "get_forward_context", return_value=mock_ctx)
-
-        hunyuan_moe._set_forward_context_num_tokens(512)
-
-        assert mock_ctx.num_tokens == 512
-        assert mock_ctx.in_profile_run is True  # not overwritten
-
-    def test_noop_when_context_unavailable(self, mocker):
-        """Should do nothing when ForwardContext is not available."""
-        import vllm_omni.diffusion.models.hunyuan_image_3.hunyuan_fused_moe as hunyuan_moe
-
-        mocker.patch.object(hunyuan_moe._vllm_fc, "is_forward_context_available", return_value=False)
-        mock_get = mocker.patch.object(hunyuan_moe._vllm_fc, "get_forward_context")
-
-        hunyuan_moe._set_forward_context_num_tokens(256)
-
-        mock_get.assert_not_called()
-
-
 class TestHunyuanFusedMoEPlatformDispatch:
    """Test platform dispatch via platform qualname hooks."""

@@ -0,0 +1,38 @@
+# SPDX-License-Identifier: Apache-2.0
+
+import json
+from pathlib import Path
+from types import SimpleNamespace
+
+import numpy as np
+import pytest
+from PIL import Image
+
+from vllm_omni.diffusion.models.qwen_image.pipeline_qwen_image_edit_plus import (
+    get_qwen_image_edit_plus_pre_process_func,
+)
+
+pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, pytest.mark.cpu]
+
+
+def test_qwen_image_edit_plus_rejects_too_many_input_images(tmp_path: Path):
+    vae_dir = tmp_path / "vae"
+    vae_dir.mkdir()
+    # Keep the mock config intentionally minimal: this test only needs the
+    # fields touched during pre-process initialization.
+    (vae_dir / "config.json").write_text(json.dumps({"z_dim": 16}))
+
+    pre_process = get_qwen_image_edit_plus_pre_process_func(SimpleNamespace(model=str(tmp_path)))
+    image = Image.fromarray(np.zeros((32, 32, 3), dtype=np.uint8))
+    request = SimpleNamespace(
+        prompts=[
+            {
+                "prompt": "combine",
+                "multi_modal_data": {"image": [image, image, image, image, image]},
+            }
+        ],
+        sampling_params=SimpleNamespace(height=None, width=None),
+    )
+
+    with pytest.raises(ValueError, match=r"At most 4 images are supported by this model"):
+        pre_process(request)
@@ -0,0 +1,260 @@
+import inspect
+from types import SimpleNamespace
+
+import pytest
+import torch
+from torch import nn
+
+from vllm_omni.diffusion.models.qwen_image.pipeline_qwen_image import (
+    QwenImagePipeline,
+)
+from vllm_omni.diffusion.models.qwen_image.pipeline_qwen_image_edit import (
+    QwenImageEditPipeline,
+)
+from vllm_omni.diffusion.models.qwen_image.pipeline_qwen_image_edit_plus import (
+    QwenImageEditPlusPipeline,
+)
+from vllm_omni.diffusion.models.qwen_image.pipeline_qwen_image_layered import (
+    QwenImageLayeredPipeline,
+)
+
+pytestmark = [pytest.mark.core_model, pytest.mark.cpu]
+
+
+class _RejectingTextEncoder:
+    dtype = torch.float32
+
+    def __call__(self, *args, **kwargs):
+        raise AssertionError("text encoder should not run for prompts that exceed max_sequence_length")
+
+
+class _FakeModelInputs:
+    def __init__(self, total_sequence_length: int):
+        attention_mask = torch.ones((1, total_sequence_length), dtype=torch.long)
+        self.input_ids = attention_mask.clone()
+        self.attention_mask = attention_mask
+        self.pixel_values = None
+        self.image_grid_thw = None
+
+    def to(self, device):
+        return self
+
+
+class _FakeTokenizer:
+    def __init__(self, total_sequence_length: int | list[int]):
+        if isinstance(total_sequence_length, list):
+            self.total_sequence_lengths = list(total_sequence_length)
+        else:
+            self.total_sequence_lengths = [total_sequence_length]
+
+    def __call__(self, *args, **kwargs):
+        if len(self.total_sequence_lengths) > 1:
+            total_sequence_length = self.total_sequence_lengths.pop(0)
+        else:
+            total_sequence_length = self.total_sequence_lengths[0]
+        return _FakeModelInputs(total_sequence_length)
+
+
+class _FakeProcessor(_FakeTokenizer):
+    pass
+
+
+class _FakeScheduler:
+    def __init__(self):
+        self.begin_index = None
+
+    def set_begin_index(self, begin_index: int):
+        self.begin_index = begin_index
+
+
+PIPELINE_CASES = [
+    pytest.param(QwenImagePipeline, 34, "tokenizer", id="qwen-image"),
+    pytest.param(QwenImageLayeredPipeline, 34, "tokenizer", id="qwen-image-layered"),
+    pytest.param(QwenImageEditPipeline, 64, "processor", id="qwen-image-edit"),
+    pytest.param(QwenImageEditPlusPipeline, 64, "processor", id="qwen-image-edit-plus"),
+]
+
+
+def _make_pipeline(
+    pipeline_class: type,
+    *,
+    total_sequence_length: int,
+    drop_idx: int,
+    input_kind: str,
+):
+    pipeline = object.__new__(pipeline_class)
+    nn.Module.__init__(pipeline)
+    pipeline.device = torch.device("cpu")
+    pipeline.text_encoder = _RejectingTextEncoder()
+    pipeline.tokenizer_max_length = 1024
+    pipeline.prompt_template_encode = "{}"
+    pipeline.prompt_template_encode_start_idx = drop_idx
+    pipeline.tokenizer = _FakeTokenizer([total_sequence_length, 0])
+    if input_kind == "processor":
+        pipeline.processor = _FakeProcessor(total_sequence_length)
+    return pipeline
+
+
+@pytest.mark.parametrize(("pipeline_class", "drop_idx", "input_kind"), PIPELINE_CASES)
+def test_encode_prompt_rejects_prompt_longer_than_default_max_sequence_length(
+    pipeline_class: type,
+    drop_idx: int,
+    input_kind: str,
+):
+    pipeline = _make_pipeline(
+        pipeline_class,
+        total_sequence_length=1025,
+        drop_idx=drop_idx,
+        input_kind=input_kind,
+    )
+
+    with pytest.raises(ValueError, match=r"got 1025 tokens, but `max_sequence_length` is 1024"):
+        pipeline.encode_prompt(prompt="prompt")
+
+
+@pytest.mark.parametrize(("pipeline_class", "drop_idx", "input_kind"), PIPELINE_CASES)
+def test_encode_prompt_rejects_prompt_longer_than_explicit_max_sequence_length(
+    pipeline_class: type,
+    drop_idx: int,
+    input_kind: str,
+):
+    pipeline = _make_pipeline(
+        pipeline_class,
+        total_sequence_length=17,
+        drop_idx=drop_idx,
+        input_kind=input_kind,
+    )
+
+    with pytest.raises(ValueError, match=r"got 17 tokens, but `max_sequence_length` is 16"):
+        pipeline.encode_prompt(prompt="prompt", max_sequence_length=16)
+
+
+def test_prepare_encode_defaults_to_tokenizer_max_length():
+    pipeline = object.__new__(QwenImagePipeline)
+    nn.Module.__init__(pipeline)
+    pipeline.tokenizer_max_length = 1024
+    pipeline.vae_scale_factor = 8
+    pipeline.default_sample_size = 128
+    pipeline.scheduler = _FakeScheduler()
+    pipeline._extract_prompts = lambda prompts: (["prompt"], None)
+
+    captured = {}
+
+    def _fake_prepare_generation_context(**kwargs):
+        captured["max_sequence_length"] = kwargs["max_sequence_length"]
+        embeds = torch.ones((1, 1, 1))
+        mask = torch.ones((1, 1), dtype=torch.long)
+        return {
+            "prompt_embeds": embeds,
+            "prompt_embeds_mask": mask,
+            "negative_prompt_embeds": None,
+            "negative_prompt_embeds_mask": None,
+            "latents": embeds,
+            "timesteps": torch.tensor([1]),
+            "do_true_cfg": False,
+            "guidance": None,
+            "img_shapes": [[(1, 1, 1)]],
+            "txt_seq_lens": [1],
+            "negative_txt_seq_lens": None,
+        }
+
+    pipeline._prepare_generation_context = _fake_prepare_generation_context
+    state = SimpleNamespace(
+        prompts=["prompt"],
+        sampling=SimpleNamespace(
+            height=None,
+            width=None,
+            num_inference_steps=None,
+            sigmas=None,
+            guidance_scale_provided=False,
+            num_outputs_per_prompt=0,
+            generator=None,
+            true_cfg_scale=None,
+            max_sequence_length=None,
+        ),
+    )
+
+    pipeline.prepare_encode(state)
+
+    assert captured["max_sequence_length"] == 1024
+
+
+@pytest.mark.parametrize(
+    ("pipeline_class", "drop_idx"),
+    [
+        pytest.param(QwenImageEditPipeline, 64, id="qwen-image-edit"),
+        pytest.param(QwenImageEditPlusPipeline, 64, id="qwen-image-edit-plus"),
+    ],
+)
+def test_edit_pipelines_validate_text_prompt_length_before_image_token_expansion(
+    pipeline_class: type,
+    drop_idx: int,
+):
+    pipeline = object.__new__(pipeline_class)
+    nn.Module.__init__(pipeline)
+    pipeline.device = torch.device("cpu")
+    pipeline.text_encoder = _RejectingTextEncoder()
+    pipeline.tokenizer_max_length = 1024
+    pipeline.prompt_template_encode = "{}"
+    pipeline.prompt_template_encode_start_idx = drop_idx
+    pipeline.tokenizer = _FakeTokenizer([8, 0])
+    pipeline.processor = _FakeProcessor(drop_idx + 1500)
+
+    with pytest.raises(AssertionError, match="text encoder should not run"):
+        pipeline.encode_prompt(prompt="short prompt")
+
+
+@pytest.mark.parametrize(
+    "pipeline_class",
+    [
+        pytest.param(QwenImagePipeline, id="qwen-image"),
+        pytest.param(QwenImageLayeredPipeline, id="qwen-image-layered"),
+    ],
+)
+def test_qwen_generation_validator_excludes_template_suffix_from_budget(pipeline_class: type):
+    pipeline = object.__new__(pipeline_class)
+    nn.Module.__init__(pipeline)
+    pipeline.device = torch.device("cpu")
+    pipeline.text_encoder = _RejectingTextEncoder()
+    pipeline.tokenizer_max_length = 1024
+    pipeline.prompt_template_encode = "{}"
+    pipeline.prompt_template_encode_start_idx = 34
+    pipeline.tokenizer = _FakeTokenizer([1029, 5])
+
+    with pytest.raises(AssertionError, match="text encoder should not run"):
+        pipeline.encode_prompt(prompt="boundary prompt")
+
+
+@pytest.mark.parametrize(
+    "pipeline_class",
+    [
+        pytest.param(QwenImageEditPipeline, id="qwen-image-edit"),
+        pytest.param(QwenImageEditPlusPipeline, id="qwen-image-edit-plus"),
+    ],
+)
+def test_qwen_edit_validator_excludes_image_placeholders_from_budget(pipeline_class: type):
+    pipeline = object.__new__(pipeline_class)
+    nn.Module.__init__(pipeline)
+    pipeline.device = torch.device("cpu")
+    pipeline.text_encoder = _RejectingTextEncoder()
+    pipeline.tokenizer_max_length = 1024
+    pipeline.prompt_template_encode = "{}"
+    pipeline.prompt_template_encode_start_idx = 64
+    pipeline.tokenizer = _FakeTokenizer([30, 20])
+    pipeline.processor = _FakeProcessor(1500)
+
+    with pytest.raises(AssertionError, match="text encoder should not run"):
+        pipeline.encode_prompt(prompt="short prompt")
+
+
+@pytest.mark.parametrize(
+    "pipeline_class",
+    [
+        QwenImagePipeline,
+        QwenImageLayeredPipeline,
+        QwenImageEditPipeline,
+        QwenImageEditPlusPipeline,
+    ],
+)
+def test_forward_max_sequence_length_default_is_1024(pipeline_class: type):
+    assert inspect.signature(pipeline_class.forward).parameters["max_sequence_length"].default == 1024
@@ -0,0 +1,26 @@
+import pytest
+
+from vllm_omni.diffusion.utils.size_utils import (
+    normalize_min_aligned_size,
+)
+
+pytestmark = [pytest.mark.core_model, pytest.mark.cpu]
+
+
+@pytest.mark.parametrize(
+    ("height", "width", "expected"),
+    [
+        (1, 1, (16, 16)),
+        (15, 15, (16, 16)),
+        (17, 17, (16, 16)),
+        (31, 33, (16, 32)),
+        (64, 80, (64, 80)),
+    ],
+)
+def test_normalize_min_aligned_size_clamps_to_minimum_aligned_shape(height, width, expected):
+    assert normalize_min_aligned_size(height, width, alignment=16) == expected
+
+
+def test_normalize_min_aligned_size_rejects_invalid_alignment():
+    with pytest.raises(ValueError, match="positive alignment"):
+        normalize_min_aligned_size(16, 16, alignment=0)
@@ -6,7 +6,7 @@ from unittest.mock import Mock, patch

 import pytest

-from vllm_omni.diffusion.data import DiffusionOutput
+from vllm_omni.diffusion.data import DiffusionOutput, OmniRequestError, normalize_omni_error
 from vllm_omni.diffusion.diffusion_engine import DiffusionEngine
 from vllm_omni.diffusion.request import OmniDiffusionRequest
 from vllm_omni.diffusion.sched import (
@@ -245,6 +245,39 @@ class TestDiffusionEngine:
        assert output is expected
        engine.executor.add_req.assert_called_once_with(request)

+    def test_add_req_and_wait_for_response_wraps_executor_errors(self) -> None:
+        engine = DiffusionEngine.__new__(DiffusionEngine)
+        engine.scheduler = RequestScheduler()
+        engine.scheduler.initialize(Mock())
+        engine.executor = Mock()
+        engine.executor.add_req.side_effect = RuntimeError("worker oom")
+        engine._rpc_lock = threading.Lock()
+
+        request = _make_request("engine-error")
+
+        with pytest.raises(OmniRequestError, match="worker oom") as exc_info:
+            engine.add_req_and_wait_for_response(request)
+
+        assert exc_info.value.error_type == "RuntimeError"
+
+    def test_add_req_and_wait_for_response_raises_scheduler_error_when_empty_without_requests(self) -> None:
+        engine = DiffusionEngine.__new__(DiffusionEngine)
+        engine.scheduler = Mock()
+        engine.scheduler.add_request.return_value = "sched-1"
+        engine.scheduler.schedule.return_value = Mock(is_empty=True)
+        engine.scheduler.has_requests.return_value = False
+        engine.scheduler.pop_request_state.return_value = None
+        engine.executor = Mock()
+        engine._rpc_lock = threading.Lock()
+
+        request = _make_request("engine-empty")
+
+        with pytest.raises(OmniRequestError, match="no runnable requests") as exc_info:
+            engine.add_req_and_wait_for_response(request)
+
+        assert exc_info.value.error_type == "SchedulerError"
+        engine.scheduler.pop_request_state.assert_called_once_with("sched-1")
+
    def test_supports_scheduler_interface_injection(self) -> None:
        request = _make_request("engine_iface")
        expected = DiffusionOutput(output=None)
@@ -297,3 +330,15 @@ class TestDiffusionEngine:

        with pytest.raises(RuntimeError, match="Dummy run failed: boom"):
            engine._dummy_run()
+
+
+def test_normalize_omni_error_preserves_existing_metadata() -> None:
+    err = OmniRequestError("boom", status_code=409, error_type="Conflict")
+
+    normalized = normalize_omni_error(err, request_id="req-1", stage_id=3)
+
+    assert normalized is err
+    assert normalized.request_id == "req-1"
+    assert normalized.stage_id == 3
+    assert normalized.status_code == 409
+    assert normalized.error_type == "Conflict"
@@ -443,14 +443,42 @@ class TestWorker:
 class TestIPC:
    def test_pack_unpack_runner_output_shm(self):
        tensor = torch.zeros(300_000, dtype=torch.float32)
-        output = RunnerOutput(req_id="req-1", finished=True, result=DiffusionOutput(output=tensor))
+        log_probs = torch.ones(300_000, dtype=torch.float32)
+        output = RunnerOutput(
+            req_id="req-1",
+            finished=True,
+            result=DiffusionOutput(output=tensor, trajectory_log_probs=log_probs),
+        )

        packed = pack_diffusion_output_shm(output)
        assert isinstance(packed.result.output, dict)
        assert packed.result.output["__tensor_shm__"] is True
+        assert isinstance(packed.result.trajectory_log_probs, dict)
+        assert packed.result.trajectory_log_probs["__tensor_shm__"] is True

        unpacked = unpack_diffusion_output_shm(packed)
        assert isinstance(unpacked.result.output, torch.Tensor)
+        assert isinstance(unpacked.result.trajectory_log_probs, torch.Tensor)
+        torch.testing.assert_close(unpacked.result.output, tensor)
+        torch.testing.assert_close(unpacked.result.trajectory_log_probs, log_probs)
+
+    def test_pack_unpack_runner_output_shm_bfloat16(self):
+        tensor = torch.arange(600_000, dtype=torch.bfloat16)
+        output = RunnerOutput(
+            req_id="req-1",
+            finished=True,
+            result=DiffusionOutput(output=tensor),
+        )
+
+        packed = pack_diffusion_output_shm(output)
+        assert isinstance(packed.result.output, dict)
+        assert packed.result.output["__tensor_shm__"] is True
+        assert packed.result.output["torch_dtype"] == str(torch.bfloat16)
+        assert packed.result.output["numpy_dtype"] == "uint16"
+
+        unpacked = unpack_diffusion_output_shm(packed)
+        assert isinstance(unpacked.result.output, torch.Tensor)
+        assert unpacked.result.output.dtype == torch.bfloat16
        torch.testing.assert_close(unpacked.result.output, tensor)


@@ -0,0 +1,122 @@
+from __future__ import annotations
+
+import asyncio
+from unittest.mock import MagicMock, patch
+
+import pytest
+
+from vllm_omni.diffusion.data import OmniDiffusionConfig
+from vllm_omni.diffusion.inline_stage_diffusion_client import InlineStageDiffusionClient
+from vllm_omni.engine.stage_init_utils import StageMetadata
+from vllm_omni.inputs.data import OmniDiffusionSamplingParams
+from vllm_omni.outputs import OmniRequestOutput
+
+pytestmark = [pytest.mark.core_model, pytest.mark.cpu]
+
+
+@pytest.fixture
+def mock_engine():
+    with patch("vllm_omni.diffusion.inline_stage_diffusion_client.DiffusionEngine") as mock:
+        engine_instance = MagicMock()
+        mock.make_engine.return_value = engine_instance
+        yield engine_instance
+
+
+@pytest.fixture
+def client(mock_engine):
+    metadata = StageMetadata(
+        stage_id=0,
+        stage_type="diffusion",
+        engine_output_type="image",
+        is_comprehension=False,
+        requires_multimodal_data=False,
+        engine_input_source="prompt",
+        final_output=True,
+        final_output_type="image",
+        default_sampling_params={},
+        custom_process_input_func=None,
+        model_stage=None,
+        runtime_cfg=None,
+    )
+    with patch.object(InlineStageDiffusionClient, "_enrich_config"):
+        od_config = MagicMock(spec=OmniDiffusionConfig)
+        c = InlineStageDiffusionClient(model="test_model", od_config=od_config, metadata=metadata, batch_size=1)
+        yield c
+        c.shutdown()
+
+
+@pytest.mark.asyncio
+async def test_inline_dispatch_request_success(client, mock_engine):
+    mock_result = OmniRequestOutput.from_diffusion(request_id="req-1", images=[MagicMock()])
+    mock_engine.step.return_value = [mock_result]
+
+    sampling_params = OmniDiffusionSamplingParams()
+    await client.add_request_async("req-1", "A test prompt", sampling_params)
+
+    # Wait for the task to be processed
+    for _ in range(10):
+        output = client.get_diffusion_output_nowait()
+        if output is not None:
+            break
+        await asyncio.sleep(0.01)
+
+    assert output is not None
+    assert output.request_id == "req-1"
+    mock_engine.step.assert_called_once()
+    request = mock_engine.step.call_args.args[0]
+    assert request.prompts == ["A test prompt"]
+    assert request.request_ids == ["req-1"]
+
+
+@pytest.mark.asyncio
+async def test_inline_dispatch_batch_success(client, mock_engine):
+    mock_engine.step.return_value = [
+        OmniRequestOutput.from_diffusion(request_id="req-batch-0", images=[MagicMock()]),
+        OmniRequestOutput.from_diffusion(request_id="req-batch-1", images=[MagicMock()]),
+    ]
+
+    sampling_params = OmniDiffusionSamplingParams()
+    prompts = ["prompt-0", "prompt-1"]
+    await client.add_batch_request_async("req-batch", prompts, sampling_params)
+
+    for _ in range(10):
+        output = client.get_diffusion_output_nowait()
+        if output is not None:
+            break
+        await asyncio.sleep(0.01)
+
+    assert output is not None
+    assert output.request_id == "req-batch"
+    assert len(output.images) == 2
+    request = mock_engine.step.call_args.args[0]
+    assert request.prompts == prompts
+    assert request.request_ids == ["req-batch-0", "req-batch-1"]
+
+
+@pytest.mark.asyncio
+async def test_inline_dispatch_request_error(client, mock_engine):
+    mock_engine.step.side_effect = RuntimeError("Engine failure")
+
+    sampling_params = OmniDiffusionSamplingParams()
+    await client.add_request_async("req-err", "A test prompt", sampling_params)
+
+    for _ in range(10):
+        output = client.get_diffusion_output_nowait()
+        if output is not None:
+            break
+        await asyncio.sleep(0.01)
+
+    assert output is not None
+    assert output["request_id"] == "req-err"
+    assert output["error"] == "Engine failure"
+    assert output["error_type"] == "RuntimeError"
+
+
+def test_inline_shutdown(client, mock_engine):
+    assert not client._shutting_down
+
+    # Shutting down should cleanly cancel anything queued and close engine
+    client.shutdown()
+
+    assert client._shutting_down
+    mock_engine.close.assert_called_once()
@@ -1,83 +0,0 @@
-# SPDX-License-Identifier: Apache-2.0
-# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
-"""
-E2E offline tests for Qwen3-TTS Base model with text input and audio output.
-
-Async_chunk disable, cuda_graph disabled (no_async_chunk stage config).
-CUDA graph is disabled by setting engine_args.enforce_eager=true via modify_stage_config().
-Same structure as test_qwen3_omni (models, stage_configs, test_params, parametrize omni_runner).
-"""
-
-import os
-
-os.environ["VLLM_WORKER_MULTIPROC_METHOD"] = "spawn"
-os.environ["VLLM_TEST_CLEAN_GPU_MEMORY"] = "0"
-
-from pathlib import Path
-
-import pytest
-
-from tests.conftest import modify_stage_config
-from tests.utils import hardware_test
-
-MODEL = "Qwen/Qwen3-TTS-12Hz-0.6B-Base"
-REF_AUDIO_URL = "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-TTS-Repo/clone_2.wav"
-REF_TEXT = "Okay. Yeah. I resent you. I love you. I respect you. But you know what? You blew it! And thanks to you."
-
-
-def get_cuda_graph_config():
-    path = modify_stage_config(
-        get_stage_config(),
-        updates={
-            "stage_args": {
-                0: {
-                    "engine_args.enforce_eager": "true",
-                },
-                1: {"engine_args.enforce_eager": "true"},
-            },
-        },
-    )
-    return path
-
-
-def get_stage_config(name: str = "qwen3_tts_no_async_chunk.yaml"):
-    """Get the no_async_chunk stage config path (async_chunk disable, cuda_graph disabled)."""
-    return str(Path(__file__).parent.parent.parent.parent / "vllm_omni" / "model_executor" / "stage_configs" / name)
-
-
-# Same structure as test_qwen3_omni: models, stage_configs, test_params
-tts_server_params = [
-    pytest.param(
-        (MODEL, get_cuda_graph_config()),
-        id="no_cuda_graph",
-    )
-]
-
-
-def get_prompt():
-    """Text prompt for text-to-audio (same role as get_question in test_qwen3_omni)."""
-    return "Hello, this is a test for text to audio."
-
-
-@pytest.mark.advanced_model
-@pytest.mark.omni
-@hardware_test(res={"cuda": "L4"}, num_cards=1)
-@pytest.mark.parametrize("omni_runner", tts_server_params, indirect=True)
-def test_text_to_audio_001(omni_runner, omni_runner_handler) -> None:
-    """
-    Test text input processing and audio output via offline Omni runner.
-    Deploy Setting: qwen3_tts_no_async_chunk.yaml + enforce_eager=true
-    Input Modal: text
-    Output Modal: audio
-    Input Setting: stream=False
-    Extra Setting: task_type=Base, voice=clone, ref_audio/ref_text provided
-    Datasets: few requests
-    """
-    request_config = {
-        "input": get_prompt(),
-        "task_type": "Base",
-        "voice": "clone",
-        "ref_audio": REF_AUDIO_URL,
-        "ref_text": REF_TEXT,
-    }
-    omni_runner_handler.send_audio_speech_request(request_config)
@@ -1,74 +0,0 @@
-# SPDX-License-Identifier: Apache-2.0
-# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
-"""
-E2E offline tests for Qwen3-TTS CustomVoice model with text input and audio output.
-
-Async_chunk disable, cuda_graph disabled (no_async_chunk stage config).
-CUDA graph is disabled by setting engine_args.enforce_eager=true via modify_stage_config().
-Same structure as test_qwen3_omni (models, stage_configs, test_params, parametrize omni_runner).
-"""
-
-import os
-
-os.environ["VLLM_WORKER_MULTIPROC_METHOD"] = "spawn"
-os.environ["VLLM_TEST_CLEAN_GPU_MEMORY"] = "0"
-
-from pathlib import Path
-
-import pytest
-
-from tests.conftest import modify_stage_config
-from tests.utils import hardware_test
-
-MODEL = "Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice"
-
-
-def get_cuda_graph_config():
-    path = modify_stage_config(
-        get_stage_config(),
-        updates={
-            "stage_args": {
-                0: {
-                    "engine_args.enforce_eager": "true",
-                },
-                1: {"engine_args.enforce_eager": "true"},
-            },
-        },
-    )
-    return path
-
-
-def get_stage_config(name: str = "qwen3_tts_no_async_chunk.yaml"):
-    """Get the no_async_chunk stage config path (async_chunk disable, cuda_graph disabled)."""
-    return str(Path(__file__).parent.parent.parent.parent / "vllm_omni" / "model_executor" / "stage_configs" / name)
-
-
-# Same structure as test_qwen3_omni: models, stage_configs, test_params
-tts_server_params = [
-    pytest.param(
-        (MODEL, get_cuda_graph_config()),
-        id="no_cuda_graph",
-    )
-]
-
-
-def get_prompt():
-    """Text prompt for text-to-audio (same role as get_question in test_qwen3_omni)."""
-    return "Hello, this is a test for text to audio."
-
-
-@pytest.mark.advanced_model
-@pytest.mark.omni
-@hardware_test(res={"cuda": "L4"}, num_cards=1)
-@pytest.mark.parametrize("omni_runner", tts_server_params, indirect=True)
-def test_text_to_audio_001(omni_runner, omni_runner_handler) -> None:
-    """
-    Test text input processing and audio output via offline Omni runner.
-    Deploy Setting: qwen3_tts_no_async_chunk.yaml + enforce_eager=true
-    Input Modal: text
-    Output Modal: audio
-    Input Setting: stream=False
-    Datasets: few requests
-    """
-    request_config = {"input": get_prompt(), "voice": "vivian"}
-    omni_runner_handler.send_audio_speech_request(request_config)
@@ -0,0 +1,285 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
+"""
+E2E Online tests for Qwen3-TTS model with text input and audio output.
+
+These tests verify the /v1/audio/speech endpoint works correctly with
+actual model inference, not mocks.
+"""
+
+import os
+
+os.environ["VLLM_WORKER_MULTIPROC_METHOD"] = "spawn"
+os.environ["VLLM_TEST_CLEAN_GPU_MEMORY"] = "0"
+
+from pathlib import Path
+
+import httpx
+import pytest
+
+from tests.conftest import OmniServer
+from tests.utils import hardware_test
+
+MODEL = "Qwen/Qwen3-TTS-12Hz-0.6B-CustomVoice"
+
+
+def get_stage_config(name: str = "qwen3_tts.yaml"):
+    """Get the stage config path for Qwen3-TTS."""
+    return str(Path(__file__).parent.parent.parent.parent / "vllm_omni" / "model_executor" / "stage_configs" / name)
+
+
+@pytest.fixture(scope="class")
+def omni_server():
+    """Start vLLM-Omni server with CustomVoice model."""
+    stage_config_path = get_stage_config()
+
+    print(f"Starting OmniServer with model: {MODEL}")
+
+    with OmniServer(
+        MODEL,
+        [
+            "--stage-configs-path",
+            stage_config_path,
+            "--stage-init-timeout",
+            "120",
+            "--trust-remote-code",
+            "--enforce-eager",
+            "--disable-log-stats",
+        ],
+    ) as server:
+        print("OmniServer started successfully")
+        yield server
+        print("OmniServer stopping...")
+
+    print("OmniServer stopped")
+
+
+def make_speech_request(
+    host: str,
+    port: int,
+    text: str,
+    voice: str = "vivian",
+    language: str = "English",
+    task_type: str | None = None,
+    instructions: str | None = None,
+    timeout: float = 120.0,
+) -> httpx.Response:
+    """Make a request to the /v1/audio/speech endpoint."""
+    url = f"http://{host}:{port}/v1/audio/speech"
+    payload = {
+        "input": text,
+        "voice": voice,
+        "language": language,
+    }
+    if task_type:
+        payload["task_type"] = task_type
+    if instructions:
+        payload["instructions"] = instructions
+
+    with httpx.Client(timeout=timeout) as client:
+        return client.post(url, json=payload)
+
+
+def verify_wav_audio(content: bytes) -> bool:
+    """Verify that content is valid WAV audio data."""
+    # WAV files start with "RIFF" header
+    if len(content) < 44:  # Minimum WAV header size
+        return False
+    return content[:4] == b"RIFF" and content[8:12] == b"WAVE"
+
+
+# Minimum expected audio size for a short sentence (~1 second of 24kHz 16-bit mono WAV)
+MIN_AUDIO_BYTES = 10000
+
+
+class TestQwen3TTSCustomVoice:
+    """E2E tests for Qwen3-TTS CustomVoice model."""
+
+    @pytest.mark.core_model
+    @pytest.mark.omni
+    @hardware_test(res={"cuda": "L4"}, num_cards=4)
+    def test_speech_english_basic(self, omni_server) -> None:
+        """Test basic English TTS generation."""
+        response = make_speech_request(
+            host=omni_server.host,
+            port=omni_server.port,
+            text="Hello, how are you?",
+            voice="vivian",
+            language="English",
+        )
+
+        assert response.status_code == 200, f"Request failed: {response.text}"
+        assert response.headers.get("content-type") == "audio/wav"
+        assert verify_wav_audio(response.content), "Response is not valid WAV audio"
+        assert len(response.content) > MIN_AUDIO_BYTES, (
+            f"Audio content too small ({len(response.content)} bytes), expected at least {MIN_AUDIO_BYTES} bytes"
+        )
+
+    @pytest.mark.core_model
+    @pytest.mark.omni
+    @hardware_test(res={"cuda": "L4"}, num_cards=4)
+    def test_speech_chinese_basic(self, omni_server) -> None:
+        """Test basic Chinese TTS generation."""
+        response = make_speech_request(
+            host=omni_server.host,
+            port=omni_server.port,
+            text="你好，我是通义千问",
+            voice="vivian",
+            language="Chinese",
+        )
+
+        assert response.status_code == 200, f"Request failed: {response.text}"
+        assert response.headers.get("content-type") == "audio/wav"
+        assert verify_wav_audio(response.content), "Response is not valid WAV audio"
+        assert len(response.content) > MIN_AUDIO_BYTES, (
+            f"Audio content too small ({len(response.content)} bytes), expected at least {MIN_AUDIO_BYTES} bytes"
+        )
+
+    @pytest.mark.core_model
+    @pytest.mark.omni
+    @hardware_test(res={"cuda": "L4"}, num_cards=4)
+    def test_speech_different_voices(self, omni_server) -> None:
+        """Test TTS with different voice options."""
+        voices = ["vivian", "ryan"]
+        for voice in voices:
+            response = make_speech_request(
+                host=omni_server.host,
+                port=omni_server.port,
+                text="Testing voice selection.",
+                voice=voice,
+                language="English",
+            )
+
+            assert response.status_code == 200, f"Request failed for voice {voice}: {response.text}"
+            assert verify_wav_audio(response.content), f"Invalid WAV for voice {voice}"
+
+    @pytest.mark.core_model
+    @pytest.mark.omni
+    @hardware_test(res={"cuda": "L4"}, num_cards=4)
+    def test_speech_binary_response_not_utf8_error(self, omni_server) -> None:
+        """
+        Regression test: Verify binary audio is returned, not UTF-8 error.
+
+        This test ensures the multimodal_output property correctly retrieves
+        audio from completion outputs, preventing the "TTS model did not
+        produce audio output" error.
+        """
+        response = make_speech_request(
+            host=omni_server.host,
+            port=omni_server.port,
+            text="This should return binary audio, not a JSON error.",
+            voice="vivian",
+            language="English",
+        )
+
+        # Should NOT be a JSON error response
+        assert response.status_code == 200, f"Request failed: {response.text}"
+
+        # Verify it's binary audio, not JSON
+        try:
+            # If this succeeds and starts with {"error", it's a bug
+            text = response.content.decode("utf-8")
+            assert not text.startswith('{"error"'), f"Got error response instead of audio: {text}"
+        except UnicodeDecodeError:
+            # This is expected - binary audio can't be decoded as UTF-8
+            pass
+
+        assert verify_wav_audio(response.content), "Response is not valid WAV audio"
+
+
+class TestQwen3TTSAPIEndpoints:
+    """Test API endpoint functionality."""
+
+    @pytest.mark.core_model
+    @pytest.mark.omni
+    @hardware_test(res={"cuda": "L4"}, num_cards=4)
+    def test_list_voices_endpoint(self, omni_server) -> None:
+        """Test the /v1/audio/voices endpoint returns available voices."""
+        url = f"http://{omni_server.host}:{omni_server.port}/v1/audio/voices"
+
+        with httpx.Client(timeout=30.0) as client:
+            response = client.get(url)
+
+        assert response.status_code == 200
+        data = response.json()
+        assert "voices" in data
+        assert isinstance(data["voices"], list)
+        assert len(data["voices"]) > 0
+        # Check some expected voices are present
+        voices_lower = [v.lower() for v in data["voices"]]
+        assert "vivian" in voices_lower or "ryan" in voices_lower
+
+    @pytest.mark.core_model
+    @pytest.mark.omni
+    @hardware_test(res={"cuda": "L4"}, num_cards=4)
+    def test_models_endpoint(self, omni_server) -> None:
+        """Test the /v1/models endpoint returns loaded model."""
+        url = f"http://{omni_server.host}:{omni_server.port}/v1/models"
+
+        with httpx.Client(timeout=30.0) as client:
+            response = client.get(url)
+
+        assert response.status_code == 200
+        data = response.json()
+        assert "data" in data
+        assert len(data["data"]) > 0
+
+
+@pytest.fixture(scope="class")
+def omni_server_no_async_chunk():
+    """Start vLLM-Omni server with non-async-chunk config."""
+    stage_config_path = get_stage_config("qwen3_tts_no_async_chunk.yaml")
+
+    with OmniServer(
+        MODEL,
+        [
+            "--stage-configs-path",
+            stage_config_path,
+            "--stage-init-timeout",
+            "120",
+            "--trust-remote-code",
+            "--enforce-eager",
+            "--disable-log-stats",
+        ],
+    ) as server:
+        yield server
+
+
+class TestQwen3TTSNoAsyncChunk:
+    """E2E tests for Qwen3-TTS in non-async-chunk (full decode) mode."""
+
+    @pytest.mark.core_model
+    @pytest.mark.omni
+    @hardware_test(res={"cuda": "L4"}, num_cards=4)
+    def test_speech_english(self, omni_server_no_async_chunk) -> None:
+        """Test English TTS with non-async-chunk pipeline."""
+        response = make_speech_request(
+            host=omni_server_no_async_chunk.host,
+            port=omni_server_no_async_chunk.port,
+            text="Hello, how are you?",
+            voice="vivian",
+            language="English",
+        )
+
+        assert response.status_code == 200, f"Request failed: {response.text}"
+        assert response.headers.get("content-type") == "audio/wav"
+        assert verify_wav_audio(response.content), "Response is not valid WAV audio"
+        assert len(response.content) > MIN_AUDIO_BYTES
+
+    @pytest.mark.core_model
+    @pytest.mark.omni
+    @hardware_test(res={"cuda": "L4"}, num_cards=4)
+    def test_speech_chinese(self, omni_server_no_async_chunk) -> None:
+        """Test Chinese TTS with non-async-chunk pipeline."""
+        response = make_speech_request(
+            host=omni_server_no_async_chunk.host,
+            port=omni_server_no_async_chunk.port,
+            text="你好，我是通义千问",
+            voice="vivian",
+            language="Chinese",
+        )
+
+        assert response.status_code == 200, f"Request failed: {response.text}"
+        assert response.headers.get("content-type") == "audio/wav"
+        assert verify_wav_audio(response.content), "Response is not valid WAV audio"
+        assert len(response.content) > MIN_AUDIO_BYTES
@@ -1,10 +1,10 @@
 # SPDX-License-Identifier: Apache-2.0
 # SPDX-FileCopyrightText: Copyright contributors to the vLLM project
 """
-E2E Online tests for Qwen3-TTS model with text input and audio output.
+E2E tests for Qwen3-TTS Base voice-clone model.

-These tests verify the /v1/audio/speech endpoint works correctly with
-actual model inference, not mocks.
+Regression test for #1663: speech tokenizer loaded in bfloat16 produced
+all-silence PCM due to NaN overflow in SnakeBeta activation.
 """

 import os
@@ -12,99 +12,212 @@ import os
 os.environ["VLLM_WORKER_MULTIPROC_METHOD"] = "spawn"
 os.environ["VLLM_TEST_CLEAN_GPU_MEMORY"] = "0"

+import struct
+import tempfile
 from pathlib import Path

+import httpx
+import numpy as np
 import pytest

-from tests.conftest import OmniServerParams
+from tests.conftest import (
+    OmniServer,
+    convert_audio_file_to_text,
+    cosine_similarity_text,
+)
 from tests.utils import hardware_test

-MODEL = "Qwen/Qwen3-TTS-12Hz-0.6B-Base"
+MODEL = "Qwen/Qwen3-TTS-12Hz-1.7B-Base"

+# Official Qwen3-TTS reference audio/text from examples/offline_inference/qwen3_tts/end2end.py
 REF_AUDIO_URL = "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-TTS-Repo/clone_2.wav"
 REF_TEXT = "Okay. Yeah. I resent you. I love you. I respect you. But you know what? You blew it! And thanks to you."
+SYN_TEXT = "Good one. Okay, fine, I'm just gonna leave this sock monkey here. Goodbye."


-def get_stage_config(name: str = "qwen3_tts.yaml"):
-    """Get the stage config path from vllm_omni model_executor stage_configs."""
-    return str(Path(__file__).parent.parent.parent.parent / "vllm_omni" / "model_executor" / "stage_configs" / name)
-
-
-def get_prompt(prompt_type="text"):
-    """Text prompt for text-to-audio tests (same as test_qwen3_omni - beijing test case)."""
-    prompts = {
-        "text": "The weather is nice today, perfect for a walk in the park.",
-    }
-    return prompts.get(prompt_type, prompts["text"])
-
-
-def get_max_batch_size(size_type="few"):
-    """Batch size for concurrent requests (same as test_qwen3_omni)."""
-    batch_sizes = {"few": 5, "medium": 100, "large": 256}
-    return batch_sizes.get(size_type, 5)
-
-
-tts_server_params = [
-    pytest.param(
-        OmniServerParams(
-            model=MODEL,
-            stage_config_path=get_stage_config("qwen3_tts.yaml"),
-            server_args=["--trust-remote-code", "--disable-log-stats"],
-        ),
-        id="async_chunk",
+def get_stage_config():
+    return str(
+        Path(__file__).parent.parent.parent.parent / "vllm_omni" / "model_executor" / "stage_configs" / "qwen3_tts.yaml"
    )
-]


-@pytest.mark.advanced_model
-@pytest.mark.core_model
-@pytest.mark.omni
-@hardware_test(res={"cuda": "L4"}, num_cards=1)
-@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
-def test_text_to_audio_001(omni_server, openai_client) -> None:
-    """
-    Test text input processing and audio output via OpenAI API.
-    Deploy Setting: default yaml
-    Input Modal: text
-    Output Modal: audio
-    Input Setting: stream=False
-    Datasets: few requests
-    """
+@pytest.fixture(scope="module")
+def omni_server():
+    """Start vLLM-Omni server with Base voice-clone model."""
+    stage_config_path = get_stage_config()

-    request_config = {
-        "model": omni_server.model,
-        "input": get_prompt(),
-        "stream": False,
-        "response_format": "wav",
+    with OmniServer(
+        MODEL,
+        [
+            "--stage-configs-path",
+            stage_config_path,
+            "--stage-init-timeout",
+            "120",
+            "--trust-remote-code",
+            "--enforce-eager",
+            "--disable-log-stats",
+        ],
+    ) as server:
+        yield server
+
+
+def make_base_speech_request(
+    host: str,
+    port: int,
+    text: str = SYN_TEXT,
+    ref_text: str = REF_TEXT,
+    ref_audio: str = REF_AUDIO_URL,
+    response_format: str = "wav",
+    timeout: float = 120.0,
+) -> httpx.Response:
+    url = f"http://{host}:{port}/v1/audio/speech"
+    payload = {
+        "model": MODEL,
+        "input": text,
        "task_type": "Base",
-        "voice": "clone",
-        "ref_audio": REF_AUDIO_URL,
-        "ref_text": REF_TEXT,
+        "ref_text": ref_text,
+        "ref_audio": ref_audio,
+        "response_format": response_format,
    }
-    openai_client.send_audio_speech_request(request_config, request_num=get_max_batch_size("few"))
+    with httpx.Client(timeout=timeout) as client:
+        return client.post(url, json=payload)


-@pytest.mark.advanced_model
-@pytest.mark.omni
-@hardware_test(res={"cuda": "L4"}, num_cards=1)
-@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
-def test_text_to_audio_002(omni_server, openai_client) -> None:
+def verify_wav_audio(content: bytes) -> bool:
+    if len(content) < 44:
+        return False
+    return content[:4] == b"RIFF" and content[8:12] == b"WAVE"
+
+
+def assert_not_silence(pcm_bytes: bytes):
+    """Assert PCM16 samples are not all identical (e.g. all-silence)."""
+    samples = struct.unpack(f"<{len(pcm_bytes) // 2}h", pcm_bytes)
+    unique = set(samples)
+    assert len(unique) > 1, (
+        f"All-silence detected: {len(samples)} samples, unique values: {unique}. "
+        "See https://github.com/vllm-project/vllm-omni/issues/1663"
+    )
+
+
+MIN_AUDIO_BYTES = 10000
+MIN_HNR_DB = 1.2  # Clean voice clone > 1.2 dB; distorted < 1.0 dB
+
+
+def compute_hnr_db(pcm_samples: np.ndarray, sr: int = 24000) -> float:
+    """Compute mean Harmonic-to-Noise Ratio (dB) for speech quality.
+
+    Clean cloned speech has HNR > 1.2 dB; distorted speech (e.g. lost
+    ref_code decoder context) drops below 1.0 dB.
    """
-    Test text input processing and audio output via OpenAI API.
-    Deploy Setting: default yaml
-    Input Modal: text
-    Output Modal: audio
-    Input Setting: stream=True
-    Datasets: single request
-    """
-    request_config = {
-        "model": omni_server.model,
-        "input": get_prompt(),
-        "stream": True,
-        "response_format": "wav",
-        "task_type": "Base",
-        "voice": "clone",
-        "ref_audio": REF_AUDIO_URL,
-        "ref_text": REF_TEXT,
-    }
-    openai_client.send_audio_speech_request(request_config)
+    frame_len = int(0.03 * sr)  # 30ms frames
+    hop = frame_len // 2
+    hnr_values: list[float] = []
+
+    for start in range(0, len(pcm_samples) - frame_len, hop):
+        frame = pcm_samples[start : start + frame_len]
+        if np.max(np.abs(frame)) < 0.01:
+            continue
+        ac = np.correlate(frame, frame, mode="full")[len(frame) - 1 :]
+        ac = ac / (ac[0] + 1e-10)
+        min_lag = int(sr / 400)
+        max_lag = min(int(sr / 80), len(ac))
+        if min_lag >= max_lag:
+            continue
+        peak = float(np.max(ac[min_lag:max_lag]))
+        if 0 < peak < 1:
+            hnr_values.append(10 * np.log10(peak / (1 - peak + 1e-10)))
+
+    return float(np.mean(hnr_values)) if hnr_values else 0.0
+
+
+class TestQwen3TTSBaseVoiceClone:
+    """Regression tests for Base voice-clone (fix #1663)."""
+
+    @pytest.mark.core_model
+    @pytest.mark.omni
+    @hardware_test(res={"cuda": "L4"}, num_cards=4)
+    def test_base_voice_clone_not_silence(self, omni_server) -> None:
+        """PCM output must contain real audio, not all-silence."""
+        response = make_base_speech_request(
+            host=omni_server.host,
+            port=omni_server.port,
+            response_format="pcm",
+        )
+
+        assert response.status_code == 200, f"Request failed: {response.text}"
+        assert len(response.content) > MIN_AUDIO_BYTES, f"Audio too small: {len(response.content)} bytes"
+        assert_not_silence(response.content)
+
+    @pytest.mark.core_model
+    @pytest.mark.omni
+    @hardware_test(res={"cuda": "L4"}, num_cards=4)
+    def test_base_voice_clone_whisper_transcription(self, omni_server) -> None:
+        """Whisper must transcribe the output as intelligible speech."""
+        response = make_base_speech_request(
+            host=omni_server.host,
+            port=omni_server.port,
+            response_format="wav",
+        )
+
+        assert response.status_code == 200, f"Request failed: {response.text}"
+        assert verify_wav_audio(response.content), "Response is not valid WAV"
+        assert len(response.content) > MIN_AUDIO_BYTES
+
+        with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as f:
+            f.write(response.content)
+            wav_path = f.name
+
+        try:
+            transcript = convert_audio_file_to_text(wav_path)
+            print(f"Whisper transcript: {transcript}")
+            assert len(transcript.strip()) > 0, "Whisper returned empty transcript — audio is likely silence"
+            similarity = cosine_similarity_text(transcript.lower(), SYN_TEXT.lower())
+            print(f"Cosine similarity: {similarity:.3f}")
+            assert similarity > 0.9, (
+                f"Transcript doesn't match input: similarity={similarity:.2f}, transcript='{transcript}'"
+            )
+        finally:
+            os.unlink(wav_path)
+
+    @pytest.mark.core_model
+    @pytest.mark.omni
+    @hardware_test(res={"cuda": "L4"}, num_cards=4)
+    def test_base_voice_clone_wav_format(self, omni_server) -> None:
+        """WAV response must have valid headers and sufficient size."""
+        response = make_base_speech_request(
+            host=omni_server.host,
+            port=omni_server.port,
+            response_format="wav",
+        )
+
+        assert response.status_code == 200, f"Request failed: {response.text}"
+        assert response.headers.get("content-type") == "audio/wav"
+        assert verify_wav_audio(response.content), "Response is not valid WAV"
+        assert len(response.content) > MIN_AUDIO_BYTES
+
+    @pytest.mark.core_model
+    @pytest.mark.omni
+    @hardware_test(res={"cuda": "L4"}, num_cards=4)
+    def test_base_voice_clone_no_distortion(self, omni_server) -> None:
+        """Audio must not be distorted (regression for ref_code context loss).
+
+        When the decoder loses ref_code speaker context on later streaming
+        chunks, HNR drops below 1.0 dB.  Clean voice clone should be > 1.2 dB.
+        See https://github.com/vllm-project/vllm-omni/issues/1944
+        """
+        response = make_base_speech_request(
+            host=omni_server.host,
+            port=omni_server.port,
+            response_format="pcm",
+        )
+
+        assert response.status_code == 200, f"Request failed: {response.text}"
+        assert len(response.content) > MIN_AUDIO_BYTES
+
+        pcm_samples = np.frombuffer(response.content, dtype=np.int16).astype(np.float32) / 32768.0
+        hnr = compute_hnr_db(pcm_samples)
+        print(f"Voice clone HNR: {hnr:.2f} dB (threshold: {MIN_HNR_DB} dB)")
+        assert hnr >= MIN_HNR_DB, (
+            f"Audio distortion detected: HNR={hnr:.2f} dB < {MIN_HNR_DB} dB. "
+            "Voice clone decoder may be losing ref_code speaker context on later chunks."
+        )
@@ -1,119 +0,0 @@
-# SPDX-License-Identifier: Apache-2.0
-# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
-"""
-E2E Online tests for Qwen3-TTS model with text input and audio output.
-
-These tests verify the /v1/audio/speech endpoint works correctly with
-actual model inference, not mocks.
-"""
-
-import os
-
-os.environ["VLLM_WORKER_MULTIPROC_METHOD"] = "spawn"
-os.environ["VLLM_TEST_CLEAN_GPU_MEMORY"] = "0"
-
-from pathlib import Path
-
-import pytest
-
-from tests.conftest import OmniServerParams
-from tests.utils import hardware_test
-
-MODEL = "Qwen/Qwen3-TTS-12Hz-0.6B-Base"
-
-REF_AUDIO_URL = "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-TTS-Repo/clone_2.wav"
-REF_TEXT = "Okay. Yeah. I resent you. I love you. I respect you. But you know what? You blew it! And thanks to you."
-
-
-def get_stage_config(name: str = "qwen3_tts.yaml"):
-    """Get the stage config path from vllm_omni model_executor stage_configs."""
-    return str(Path(__file__).parent.parent.parent.parent / "vllm_omni" / "model_executor" / "stage_configs" / name)
-
-
-def get_prompt(prompt_type="text"):
-    """Text prompt for text-to-audio tests (same as test_qwen3_omni - beijing test case)."""
-    prompts = {
-        "text": "The weather is nice today, perfect for a walk in the park.",
-    }
-    return prompts.get(prompt_type, prompts["text"])
-
-
-def get_max_batch_size(size_type="few"):
-    """Batch size for concurrent requests (same as test_qwen3_omni)."""
-    batch_sizes = {"few": 5, "medium": 100, "large": 256}
-    return batch_sizes.get(size_type, 5)
-
-
-tts_server_params = [
-    pytest.param(
-        OmniServerParams(
-            model=MODEL,
-            stage_config_path=get_stage_config("qwen3_tts.yaml"),
-            server_args=["--trust-remote-code", "--disable-log-stats"],
-        ),
-        id="async_chunk",
-    ),
-    pytest.param(
-        OmniServerParams(
-            model=MODEL,
-            stage_config_path=get_stage_config("qwen3_tts_no_async_chunk.yaml"),
-            server_args=["--trust-remote-code", "--disable-log-stats"],
-        ),
-        id="no_async_chunk",
-    ),
-]
-
-
-@pytest.mark.advanced_model
-@pytest.mark.core_model
-@pytest.mark.omni
-@hardware_test(res={"cuda": "L4"}, num_cards=1)
-@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
-def test_voice_clone_streaming_001(omni_server, openai_client) -> None:
-    """
-    Test text input processing and audio output via OpenAI API.
-    Deploy Setting: default yaml
-    Input Modal: text
-    Output Modal: audio
-    Input Setting: stream=True
-    Datasets: few requests
-    """
-
-    request_config = {
-        "model": omni_server.model,
-        "input": get_prompt(),
-        "stream": True,
-        "response_format": "wav",
-        "task_type": "Base",
-        "voice": "clone",
-        "ref_audio": REF_AUDIO_URL,
-        "ref_text": REF_TEXT,
-    }
-    openai_client.send_audio_speech_request(request_config, request_num=get_max_batch_size("few"))
-
-
-@pytest.mark.advanced_model
-@pytest.mark.core_model
-@pytest.mark.omni
-@hardware_test(res={"cuda": "L4"}, num_cards=1)
-@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
-def test_response_format_001(omni_server, openai_client) -> None:
-    """
-    Test text input processing and audio output via OpenAI API.
-    Deploy Setting: default yaml
-    Input Modal: text
-    Output Modal: audio
-    Input Setting: non-stream PCM
-    Datasets: few requests
-    """
-
-    request_config = {
-        "model": omni_server.model,
-        "input": get_prompt(),
-        "response_format": "pcm",
-        "task_type": "Base",
-        "voice": "clone",
-        "ref_audio": REF_AUDIO_URL,
-        "ref_text": REF_TEXT,
-    }
-    openai_client.send_audio_speech_request(request_config)
@@ -1,105 +0,0 @@
-# SPDX-License-Identifier: Apache-2.0
-# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
-"""
-E2E Online tests for Qwen3-TTS model with text input and audio output.
-
-These tests verify the /v1/audio/speech endpoint works correctly with
-actual model inference, not mocks.
-"""
-
-import os
-
-os.environ["VLLM_WORKER_MULTIPROC_METHOD"] = "spawn"
-os.environ["VLLM_TEST_CLEAN_GPU_MEMORY"] = "0"
-
-from pathlib import Path
-
-import pytest
-
-from tests.conftest import OmniServerParams
-from tests.utils import hardware_test
-
-MODEL = "Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice"
-
-
-def get_stage_config(name: str = "qwen3_tts.yaml"):
-    """Get the stage config path from vllm_omni model_executor stage_configs."""
-    return str(Path(__file__).parent.parent.parent.parent / "vllm_omni" / "model_executor" / "stage_configs" / name)
-
-
-def get_prompt(prompt_type="text"):
-    """Text prompt for text-to-audio tests (same as test_qwen3_omni - beijing test case)."""
-    prompts = {
-        "text": "Beijing, China's capital, blends ancient wonders like the Great Wall with modern marvels. This vibrant metropolis offers rich culture, delicious Peking duck, and endless exploration opportunities.",
-    }
-    return prompts.get(prompt_type, prompts["text"])
-
-
-def get_max_batch_size(size_type="few"):
-    """Batch size for concurrent requests (same as test_qwen3_omni)."""
-    batch_sizes = {"few": 5, "medium": 100, "large": 256}
-    return batch_sizes.get(size_type, 5)
-
-
-tts_server_params = [
-    pytest.param(
-        OmniServerParams(
-            model=MODEL,
-            stage_config_path=get_stage_config("qwen3_tts.yaml"),
-            server_args=["--trust-remote-code", "--disable-log-stats"],
-        ),
-        id="async_chunk",
-    )
-]
-
-
-@pytest.mark.core_model
-@pytest.mark.advanced_model
-@pytest.mark.omni
-@hardware_test(res={"cuda": "L4"}, num_cards=1)
-@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
-def test_text_to_audio_001(omni_server, openai_client) -> None:
-    """
-    Test text input processing and audio output via OpenAI API.
-    Deploy Setting: default yaml
-    Input Modal: text
-    Output Modal: audio
-    Input Setting: stream=False
-    Datasets: few requests
-    """
-    request_config = {
-        "model": omni_server.model,
-        "input": get_prompt(),
-        "stream": False,
-        "response_format": "wav",
-        "task_type": "CustomVoice",
-        "voice": "vivian",
-    }
-
-    openai_client.send_audio_speech_request(request_config, request_num=get_max_batch_size())
-
-
-@pytest.mark.core_model
-@pytest.mark.advanced_model
-@pytest.mark.omni
-@hardware_test(res={"cuda": "L4"}, num_cards=1)
-@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
-def test_text_to_audio_002(omni_server, openai_client) -> None:
-    """
-    Test text input processing and audio output via OpenAI API.
-    Deploy Setting: default yaml
-    Input Modal: text
-    Output Modal: audio
-    Input Setting: stream=True
-    Datasets: single request
-    """
-    request_config = {
-        "model": omni_server.model,
-        "input": get_prompt(),
-        "stream": True,
-        "response_format": "wav",
-        "task_type": "CustomVoice",
-        "voice": "vivian",
-    }
-
-    openai_client.send_audio_speech_request(request_config)
@@ -1,145 +0,0 @@
-# SPDX-License-Identifier: Apache-2.0
-# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
-"""
-E2E Online tests for Qwen3-TTS model with text input and audio output.
-
-These tests verify the /v1/audio/speech endpoint works correctly with
-actual model inference, not mocks.
-"""
-
-import os
-
-os.environ["VLLM_WORKER_MULTIPROC_METHOD"] = "spawn"
-os.environ["VLLM_TEST_CLEAN_GPU_MEMORY"] = "0"
-
-from pathlib import Path
-
-import pytest
-
-from tests.conftest import OmniServerParams
-from tests.utils import hardware_test
-
-MODEL = "Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice"
-
-
-def get_stage_config(name: str = "qwen3_tts.yaml"):
-    """Get the stage config path from vllm_omni model_executor stage_configs."""
-    return str(Path(__file__).parent.parent.parent.parent / "vllm_omni" / "model_executor" / "stage_configs" / name)
-
-
-def get_prompt(prompt_type="english"):
-    """Text prompt for text-to-audio tests (same as test_qwen3_omni - beijing test case)."""
-    prompts = {
-        "english": "Paris is a city in France. It has a tall tower by the river. The streets are old and small. You can walk and see art and food. The sky is light and soft. It is a calm place to rest.",
-        "chinese": "北京，中国的首都，是一座融合了长城等历史地点与现代建筑的国际化大都市，充满了独特的文化与活力",
-    }
-    return prompts.get(prompt_type, prompts["english"])
-
-
-def get_max_batch_size(size_type="few"):
-    """Batch size for concurrent requests (same as test_qwen3_omni)."""
-    batch_sizes = {"few": 3, "medium": 100, "large": 256}
-    return batch_sizes.get(size_type)
-
-
-tts_server_params = [
-    pytest.param(
-        OmniServerParams(
-            model=MODEL,
-            stage_config_path=get_stage_config("qwen3_tts.yaml"),
-            server_args=["--trust-remote-code", "--disable-log-stats"],
-        ),
-        id="async_chunk",
-    ),
-    pytest.param(
-        OmniServerParams(
-            model=MODEL,
-            stage_config_path=get_stage_config("qwen3_tts_no_async_chunk.yaml"),
-            server_args=["--trust-remote-code", "--disable-log-stats"],
-        ),
-        id="no_async_chunk",
-    ),
-]
-
-
-@pytest.mark.advanced_model
-@pytest.mark.omni
-@hardware_test(res={"cuda": "L4"}, num_cards=1)
-@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
-def test_voice_001(omni_server, openai_client) -> None:
-    """
-    Test text input processing and audio output via OpenAI API.
-    Deploy Setting: default yaml
-    Input Modal: text
-    Output Modal: audio
-    Input Setting: stream=False, voice=eric
-    Datasets: few requests
-    """
-    request_config = {
-        "model": omni_server.model,
-        "input": get_prompt(),
-        "stream": True,
-        "response_format": "wav",
-        "task_type": "CustomVoice",
-        "voice": "uncle_fu",
-    }
-
-    # Retry once on assertion failures from transcript similarity / gender checks (flaky ASR or estimators).
-    for attempt in range(2):
-        try:
-            openai_client.send_audio_speech_request(request_config, request_num=get_max_batch_size())
-            break
-        except AssertionError:
-            if attempt == 0:
-                continue
-            raise
-
-
-@pytest.mark.advanced_model
-@pytest.mark.omni
-@hardware_test(res={"cuda": "L4"}, num_cards=1)
-@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
-def test_voice_002(omni_server, openai_client) -> None:
-    """
-    Test text input processing and audio output via OpenAI API.
-    Deploy Setting: default yaml
-    Input Modal: text
-    Output Modal: audio
-    Input Setting: stream=False, language=chinese
-    Datasets: few requests
-    """
-    request_config = {
-        "model": omni_server.model,
-        "input": get_prompt(),
-        "stream": False,
-        "response_format": "wav",
-        "task_type": "CustomVoice",
-        "voice": "Serena",
-    }
-
-    openai_client.send_audio_speech_request(request_config)
-
-
-@pytest.mark.advanced_model
-@pytest.mark.omni
-@hardware_test(res={"cuda": "L4"}, num_cards=1)
-@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
-def test_language_001(omni_server, openai_client) -> None:
-    """
-    Test text input processing and audio output via OpenAI API.
-    Deploy Setting: default yaml
-    Input Modal: text
-    Output Modal: audio
-    Input Setting: stream=False, language=chinese
-    Datasets: few requests
-    """
-    request_config = {
-        "model": omni_server.model,
-        "input": get_prompt("chinese"),
-        "stream": False,
-        "response_format": "wav",
-        "task_type": "CustomVoice",
-        "voice": "vivian",
-    }
-
-    openai_client.send_audio_speech_request(request_config)
@@ -71,6 +71,67 @@ def test_initialize_stages_restores_device_visibility_after_diffusion_init(monke
            os.environ[env_var] = old_env


+def test_initialize_stages_uses_inline_diffusion_client_for_single_stage(monkeypatch):
+    """Single-stage diffusion init should request the inline client path."""
+    import vllm_omni.engine.async_omni_engine as engine_mod
+    from vllm_omni.platforms import current_omni_platform
+
+    engine = object.__new__(AsyncOmniEngine)
+    engine.model = "dummy-model"
+    engine.config_path = "dummy-config"
+    engine.num_stages = 1
+    engine.async_chunk = False
+    engine.diffusion_batch_size = 1
+    engine.stage_configs = [types.SimpleNamespace(stage_id=0, stage_type="diffusion")]
+
+    env_var = current_omni_platform.device_control_env_var
+    old_env = os.environ.get(env_var)
+    os.environ[env_var] = "0"
+
+    metadata = types.SimpleNamespace(
+        stage_id=0,
+        stage_type="diffusion",
+        runtime_cfg={"devices": "0"},
+        prompt_expand_func=None,
+    )
+    captured_use_inline = None
+    diffusion_client = types.SimpleNamespace(is_comprehension=False)
+
+    monkeypatch.setattr(engine_mod, "prepare_engine_environment", lambda: None)
+    monkeypatch.setattr(engine_mod, "load_omni_transfer_config_for_model", lambda *_: None)
+    monkeypatch.setattr(engine_mod, "extract_stage_metadata", lambda _cfg: metadata)
+    monkeypatch.setattr(engine_mod, "get_stage_connector_spec", lambda **_: {})
+    monkeypatch.setattr(engine_mod, "resolve_omni_kv_config_for_stage", lambda *_: (None, None, None))
+    monkeypatch.setattr(engine_mod, "setup_stage_devices", lambda *_: None)
+    monkeypatch.setattr(engine_mod, "_inject_kv_stage_info", lambda *_: None)
+
+    def _fake_initialize_diffusion_stage(*args, **kwargs):
+        nonlocal captured_use_inline
+        captured_use_inline = kwargs.get("use_inline")
+        return diffusion_client
+
+    monkeypatch.setattr(engine_mod, "initialize_diffusion_stage", _fake_initialize_diffusion_stage)
+    monkeypatch.setattr(
+        engine_mod,
+        "finalize_initialized_stages",
+        lambda stage_clients, _input_processor: (
+            stage_clients,
+            [types.SimpleNamespace()],
+            [{"final_output_type": "image"}],
+        ),
+    )
+
+    try:
+        engine._initialize_stages(stage_init_timeout=1)
+    finally:
+        if old_env is None:
+            os.environ.pop(env_var, None)
+        else:
+            os.environ[env_var] = old_env
+
+    assert captured_use_inline is True
+
+
 def test_attach_llm_stage_uses_omni_input_preprocessor(monkeypatch):
    """Regression test for GLM-Image t2i preprocessing path.

@@ -169,16 +169,10 @@ def test_client(mock_async_diffusion):
    app.state.engine_client = mock_async_diffusion
    app.state.diffusion_engine = mock_async_diffusion  # Also set for health endpoint
    app.state.stage_configs = [SimpleNamespace(stage_type="diffusion")]
-    from vllm.entrypoints.openai.models.protocol import BaseModelPath
-
-    from vllm_omni.entrypoints.openai.api_server import _DiffusionServingModels
-
-    app.state.openai_serving_models = _DiffusionServingModels(
-        [BaseModelPath(name="Qwen/Qwen-Image", model_path="Qwen/Qwen-Image")]
-    )
+    app.state.diffusion_model_name = "Qwen/Qwen-Image"  # For models endpoint
    app.state.args = Namespace(
-        default_sampling_params='{"0": {"num_inference_steps":4, "guidance_scale":7.5}}',
-        max_generated_image_size=4096,  # 64*64
+        default_sampling_params='{"0": {"num_inference_steps":4, "guidance_scale":7.5, "generator_device":"cpu"}}',
+        max_generated_image_size=1024 * 1792,
    )

    return TestClient(app)
@@ -200,7 +194,7 @@ def async_omni_test_client():
        SimpleNamespace(stage_type="diffusion"),
    ]
    app.state.args = Namespace(
-        default_sampling_params='{"1": {"num_inference_steps":4, "guidance_scale":7.5}}',
+        default_sampling_params='{"1": {"num_inference_steps":4, "guidance_scale":7.5, "generator_device":"cpu"}}',
        max_generated_image_size=1048576,  # 1024*1024 to support resolution tests
    )
    return TestClient(app)
@@ -222,7 +216,7 @@ def async_omni_rgba_test_client():
        SimpleNamespace(stage_type="diffusion"),
    ]
    app.state.args = Namespace(
-        default_sampling_params='{"1": {"num_inference_steps":4, "guidance_scale":7.5}}',
+        default_sampling_params='{"1": {"num_inference_steps":4, "guidance_scale":7.5, "generator_device":"cpu"}}',
        max_generated_image_size=1048576,
    )
    return TestClient(app)
@@ -244,8 +238,8 @@ def async_omni_stage_configs_only_client():
    # Intentionally do not populate app.state.stage_configs. Refactored
    # AsyncOmni exposes stage_configs on the engine instance.
    app.state.args = Namespace(
-        default_sampling_params='{"1": {"num_inference_steps":4, "guidance_scale":7.5}}',
-        max_generated_image_size=4096,  # 64*64
+        default_sampling_params='{"1": {"num_inference_steps":4, "guidance_scale":7.5, "generator_device":"cpu"}}',
+        max_generated_image_size=1024 * 1792,
    )
    return TestClient(app)

@@ -392,6 +386,18 @@ def test_image_edits_async_omni_stage_configs_only(async_omni_stage_configs_only
    assert len(captured) == 2


+def test_generate_images_max_size_rejected(async_omni_test_client):
+    """Test that a size exceeding max_generated_image_size returns 400."""
+    response = async_omni_test_client.post(
+        "/v1/images/generations",
+        json={
+            "prompt": "a cat",
+            "size": "2048x2048",  # 4,194,304 pixels > max_generated_image_size (1,048,576)
+        },
+    )
+    assert response.status_code == 400
+
+
 def test_generate_multiple_images(test_client):
    """Test generating multiple images"""
    response = test_client.post(
@@ -770,6 +776,63 @@ def test_image_edit_rejects_multiple_images_when_model_does_not_support_them(asy
    assert engine.captured_prompt is None


+def test_image_edit_rejects_too_many_images_for_qwen_image_edit_2511(async_omni_test_client):
+    engine = async_omni_test_client.app.state.engine_client
+    engine.get_diffusion_od_config = lambda: SimpleNamespace(
+        supports_multimodal_inputs=True,
+        max_multimodal_image_inputs=4,
+    )
+
+    response = async_omni_test_client.post(
+        "/v1/images/edits",
+        files=[
+            ("image", make_test_image_bytes((16, 16))),
+            ("image", make_test_image_bytes((16, 16))),
+            ("image", make_test_image_bytes((16, 16))),
+            ("image", make_test_image_bytes((16, 16))),
+            ("image", make_test_image_bytes((16, 16))),
+        ],
+        data={"prompt": "hello world."},
+    )
+
+    assert response.status_code == 400
+    assert response.json()["detail"] == "Received 5 input images. At most 4 images are supported by this model."
+    assert engine.captured_prompt is None
+
+
+def test_image_edit_rejects_too_many_images_for_qwen_image_edit_2511_before_loading(
+    async_omni_test_client, monkeypatch: pytest.MonkeyPatch
+):
+    import vllm_omni.entrypoints.openai.api_server as api_server_module
+
+    engine = async_omni_test_client.app.state.engine_client
+    engine.get_diffusion_od_config = lambda: SimpleNamespace(
+        supports_multimodal_inputs=True,
+        max_multimodal_image_inputs=4,
+    )
+
+    def _fail_load(*args, **kwargs):
+        raise AssertionError("_load_input_images should not run for over-limit requests")
+
+    monkeypatch.setattr(api_server_module, "_load_input_images", _fail_load)
+
+    response = async_omni_test_client.post(
+        "/v1/images/edits",
+        files=[
+            ("image", make_test_image_bytes((16, 16))),
+            ("image", make_test_image_bytes((16, 16))),
+            ("image", make_test_image_bytes((16, 16))),
+            ("image", make_test_image_bytes((16, 16))),
+            ("image", make_test_image_bytes((16, 16))),
+        ],
+        data={"prompt": "hello world."},
+    )
+
+    assert response.status_code == 400
+    assert response.json()["detail"] == "Received 5 input images. At most 4 images are supported by this model."
+    assert engine.captured_prompt is None
+
+
 def test_image_edit_parameter_pass(async_omni_test_client):
    img_bytes_1 = make_test_image_bytes((16, 16))

@@ -948,6 +1011,7 @@ def test_image_edit_parameter_default(async_omni_test_client):
    assert captured_sampling_params.num_outputs_per_prompt == 1
    assert captured_sampling_params.num_inference_steps == 4
    assert captured_sampling_params.guidance_scale == 7.5
+    assert captured_sampling_params.generator_device == "cpu"

    # Test that a size exceeding max_generated_image_size returns 400
    response = async_omni_test_client.post(
@@ -981,13 +1045,15 @@ def test_image_edit_parameter_default_single_stage(test_client):
    assert captured_sampling_params.num_outputs_per_prompt == 1
    assert captured_sampling_params.num_inference_steps == 4
    assert captured_sampling_params.guidance_scale == 7.5
+    assert captured_sampling_params.generator_device == "cpu"

+    # Size exceeding max_generated_image_size (1024*1792) returns 400
    response = test_client.post(
        "/v1/images/edits",
        files=[("image", img_bytes_1)],
        data={
            "prompt": "hello world.",
-            "size": "96x96",
+            "size": "2048x2048",
        },
    )
    assert response.status_code == 400
@@ -0,0 +1,146 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
+"""Unit tests for OpenAI-compatible video API encoding helpers."""
+
+import numpy as np
+import pytest
+import torch
+
+from vllm_omni.diffusion.postprocess import rife_interpolator
+from vllm_omni.entrypoints.openai import video_api_utils
+
+pytestmark = [pytest.mark.core_model, pytest.mark.cpu]
+
+
+def _install_fake_video_mux(monkeypatch, mux_calls):
+    def _fake_mux_video_audio_bytes(frames, audio, fps, audio_sample_rate, video_codec_options=None):
+        mux_calls.append(
+            {
+                "frames": frames,
+                "audio": audio,
+                "fps": fps,
+                "audio_sample_rate": audio_sample_rate,
+                "video_codec_options": video_codec_options,
+            }
+        )
+        return b"fake-video"
+
+    monkeypatch.setattr(
+        "vllm_omni.diffusion.utils.media_utils.mux_video_audio_bytes",
+        _fake_mux_video_audio_bytes,
+    )
+
+
+def _install_fake_export_to_video(monkeypatch, export_calls):
+    def _fake_export_to_video(frames, output_path, fps):
+        export_calls.append(
+            {
+                "frames": frames,
+                "output_path": output_path,
+                "fps": fps,
+            }
+        )
+        with open(output_path, "wb") as f:
+            f.write(b"fake-video")
+
+    monkeypatch.setattr("diffusers.utils.export_to_video", _fake_export_to_video)
+
+
+def test_encode_video_bytes_exports_frames_without_interpolation(monkeypatch):
+    export_calls = []
+    _install_fake_export_to_video(monkeypatch, export_calls)
+
+    frames = [np.full((2, 2, 3), fill_value=i / 5, dtype=np.float32) for i in range(5)]
+    video_bytes = video_api_utils._encode_video_bytes(
+        frames,
+        fps=8,
+    )
+
+    assert video_bytes == b"fake-video"
+    assert len(export_calls) == 1
+    assert len(export_calls[0]["frames"]) == 5
+    assert export_calls[0]["frames"][0].shape == (2, 2, 3)
+    assert export_calls[0]["fps"] == 8
+
+
+def test_encode_video_bytes_uses_array_fast_path(monkeypatch):
+    export_calls = []
+    _install_fake_export_to_video(monkeypatch, export_calls)
+    monkeypatch.setattr(
+        video_api_utils,
+        "_normalize_frames",
+        lambda frames: (_ for _ in ()).throw(AssertionError("ndarray input should not go through frame list path")),
+    )
+
+    video = np.linspace(0.0, 1.0, num=5 * 2 * 2 * 3, dtype=np.float32).reshape(5, 2, 2, 3)
+    video_bytes = video_api_utils._encode_video_bytes(video, fps=12)
+
+    assert video_bytes == b"fake-video"
+    assert len(export_calls) == 1
+    assert len(export_calls[0]["frames"]) == 5
+    assert export_calls[0]["frames"][0].flags["C_CONTIGUOUS"]
+    assert export_calls[0]["fps"] == 12.0
+
+
+def test_encode_video_bytes_without_audio_uses_diffusers_export(monkeypatch):
+    export_calls = []
+    _install_fake_export_to_video(monkeypatch, export_calls)
+    monkeypatch.setattr(
+        "vllm_omni.diffusion.utils.media_utils.mux_video_audio_bytes",
+        lambda *args, **kwargs: (_ for _ in ()).throw(
+            AssertionError("no-audio path should not use mux_video_audio_bytes")
+        ),
+    )
+
+    video = np.linspace(0.0, 1.0, num=4 * 2 * 2 * 3, dtype=np.float32).reshape(4, 2, 2, 3)
+    video_bytes = video_api_utils._encode_video_bytes(video, fps=10)
+
+    assert video_bytes == b"fake-video"
+    assert len(export_calls) == 1
+    assert export_calls[0]["fps"] == 10
+    assert len(export_calls[0]["frames"]) == 4
+
+
+def test_rife_model_inference_runs_on_dummy_tensors():
+    model = rife_interpolator.Model().eval()
+    img0 = torch.rand(1, 3, 32, 32)
+    img1 = torch.rand(1, 3, 32, 32)
+
+    output = model.inference(img0, img1, scale=1.0)
+
+    assert output.shape == (1, 3, 32, 32)
+    assert torch.isfinite(output).all()
+
+
+def test_frame_interpolator_runs_actual_torch_tensor_path(monkeypatch):
+    model = rife_interpolator.Model().eval()
+    interpolator = rife_interpolator.FrameInterpolator()
+    monkeypatch.setattr(interpolator, "_ensure_model_loaded", lambda preferred_device=None: model)
+
+    video = torch.zeros(1, 3, 2, 32, 32)
+    output_video, multiplier = interpolator.interpolate_tensor(video, exp=1, scale=1.0)
+
+    assert multiplier == 2
+    assert output_video.shape == (1, 3, 3, 32, 32)
+    assert torch.isfinite(output_video).all()
+
+
+def test_frame_interpolator_uses_platform_device_when_tensor_is_cpu(monkeypatch):
+    chosen_devices = []
+    model = rife_interpolator.Model().eval()
+
+    def _fake_ensure_model_loaded(*, preferred_device=None):
+        chosen_devices.append(preferred_device)
+        return model
+
+    interpolator = rife_interpolator.FrameInterpolator()
+    monkeypatch.setattr(interpolator, "_ensure_model_loaded", _fake_ensure_model_loaded)
+    monkeypatch.setattr(model.flownet, "to", lambda device: model.flownet)
+    monkeypatch.setattr(rife_interpolator, "_select_torch_device", lambda: torch.device("cuda"))
+
+    video = torch.zeros(1, 3, 2, 32, 32)
+    output_video, multiplier = interpolator.interpolate_tensor(video, exp=1, scale=1.0)
+
+    assert chosen_devices == [torch.device("cuda")]
+    assert multiplier == 2
+    assert output_video.shape == (1, 3, 3, 32, 32)
@@ -14,11 +14,12 @@ import time
 from types import SimpleNamespace

 import pytest
-from fastapi import FastAPI
+from fastapi import FastAPI, HTTPException
 from fastapi.testclient import TestClient
 from PIL import Image
 from pytest_mock import MockerFixture

+from vllm_omni.diffusion.data import OmniRequestError
 from vllm_omni.entrypoints.openai import api_server
 from vllm_omni.entrypoints.openai.api_server import router
 from vllm_omni.entrypoints.openai.protocol.videos import (
@@ -29,22 +30,39 @@ from vllm_omni.entrypoints.openai.protocol.videos import (
 from vllm_omni.entrypoints.openai.serving_video import OmniOpenAIServingVideo
 from vllm_omni.entrypoints.openai.storage import LocalStorageManager
 from vllm_omni.entrypoints.openai.stores import AsyncDictStore, TaskRegistry
+from vllm_omni.inputs.data import OmniDiffusionSamplingParams

 pytestmark = [pytest.mark.core_model, pytest.mark.cpu]


 class MockVideoResult:
-    def __init__(self, videos, audios=None, sample_rate=None):
+    def __init__(
+        self,
+        videos,
+        audios=None,
+        sample_rate=None,
+        custom_output=None,
+        stage_durations=None,
+        peak_memory_mb=0.0,
+    ):
        self.multimodal_output = {"video": videos}
        if audios is not None:
            self.multimodal_output["audio"] = audios
        if sample_rate is not None:
            self.multimodal_output["audio_sample_rate"] = sample_rate
+        self._custom_output = custom_output or {}
+        self.stage_durations = stage_durations or {}
+        self.peak_memory_mb = peak_memory_mb
+
+    @property
+    def custom_output(self):
+        return self._custom_output


 class FakeAsyncOmni:
    def __init__(self):
        self.stage_configs = [SimpleNamespace(stage_type="diffusion")]
+        self.default_sampling_params_list = [OmniDiffusionSamplingParams()]
        self.captured_prompt = None
        self.captured_sampling_params_list = None

@@ -56,6 +74,18 @@ class FakeAsyncOmni:
        yield MockVideoResult(videos)


+class ErrorAsyncOmni(FakeAsyncOmni):
+    def __init__(self, exc: Exception):
+        super().__init__()
+        self._exc = exc
+
+    async def generate(self, prompt, request_id, sampling_params_list):
+        self.captured_prompt = prompt
+        self.captured_sampling_params_list = sampling_params_list
+        raise self._exc
+        yield  # pragma: no cover
+
+
 class BlockingVideoHandler:
    def __init__(self):
        self.model_name = "Wan-AI/Wan2.2-T2V-A14B-Diffusers"
@@ -67,7 +97,7 @@ class BlockingVideoHandler:
        if self.stage_configs is None:
            self.stage_configs = stage_configs

-    async def generate_videos(self, request, reference_id, *, reference_image=None):
+    async def generate_video_bytes(self, request, reference_id, *, reference_image=None):
        self.started.set()
        try:
            await asyncio.Future()
@@ -135,15 +165,81 @@ def _wait_until(predicate, timeout_s: float = 2.0, interval_s: float = 0.02):
    raise AssertionError("Timed out waiting for condition")


+def test_async_video_generation_bypasses_base64(test_client, mocker: MockerFixture):
+    """Regression test: Ensure async video generation saves raw bytes directly
+    without bouncing through base64 encoding."""
+    # We mock _encode_video_bytes (the correct path)
+    mocker.patch(
+        "vllm_omni.entrypoints.openai.serving_video._encode_video_bytes",
+        return_value=b"raw-mp4-bytes",
+    )
+
+    # We assert that encode_video_base64 is never called
+    mock_base64 = mocker.patch(
+        "vllm_omni.entrypoints.openai.serving_video.encode_video_base64",
+        side_effect=RuntimeError("Regression: async video path should not base64 encode"),
+    )
+
+    response = test_client.post(
+        "/v1/videos",
+        data={"prompt": "A base64 test."},
+    )
+    assert response.status_code == 200
+    video_id = response.json()["id"]
+
+    # Wait for completion. If it used base64, the RuntimeError would fail the task
+    _wait_for_status(test_client, video_id, VideoGenerationStatus.COMPLETED.value)
+    mock_base64.assert_not_called()
+
+
+def test_async_video_generation_with_audio_bypasses_base64(test_client, mocker: MockerFixture):
+    """Regression test: Ensure async video generation passes audio through
+    generate_video_bytes without bouncing through base64 encoding."""
+    mock_encode = mocker.patch(
+        "vllm_omni.entrypoints.openai.serving_video._encode_video_bytes",
+        return_value=b"raw-mp4-bytes",
+    )
+
+    mock_base64 = mocker.patch(
+        "vllm_omni.entrypoints.openai.serving_video.encode_video_base64",
+        side_effect=RuntimeError("Regression: async video path should not base64 encode"),
+    )
+
+    engine = test_client.app.state.openai_serving_video._engine_client
+
+    async def _generate(prompt, request_id, sampling_params_list):
+        engine.captured_prompt = prompt
+        engine.captured_sampling_params_list = sampling_params_list
+        yield MockVideoResult([object()], audios=[object()], sample_rate=48000)
+
+    engine.generate = _generate
+
+    response = test_client.post(
+        "/v1/videos",
+        data={"prompt": "A base64 test with audio."},
+    )
+    assert response.status_code == 200
+    video_id = response.json()["id"]
+
+    _wait_for_status(test_client, video_id, VideoGenerationStatus.COMPLETED.value)
+    mock_base64.assert_not_called()
+
+    mock_encode.assert_called_once()
+    kwargs = mock_encode.call_args.kwargs
+    assert "audio" in kwargs
+    assert kwargs["audio"] is not None
+    assert kwargs["audio_sample_rate"] == 48000
+
+
 def test_t2v_video_generation_form(test_client, mocker: MockerFixture):
    fps_values = []

-    def _fake_encode(video, fps):
+    def _fake_encode(video, fps, audio=None, audio_sample_rate=None, **kwargs):
        fps_values.append(fps)
-        return "Zg=="
+        return b"fake-video"

    mocker.patch(
-        "vllm_omni.entrypoints.openai.serving_video.encode_video_base64",
+        "vllm_omni.entrypoints.openai.serving_video._encode_video_bytes",
        side_effect=_fake_encode,
    )
    response = test_client.post(
@@ -175,8 +271,8 @@ def test_i2v_video_generation_form(test_client, mocker: MockerFixture):
    image_bytes = _make_test_image_bytes((48, 32))

    mocker.patch(
-        "vllm_omni.entrypoints.openai.serving_video.encode_video_base64",
-        return_value="Zg==",
+        "vllm_omni.entrypoints.openai.serving_video._encode_video_bytes",
+        return_value=b"fake-video",
    )
    response = test_client.post(
        "/v1/videos",
@@ -201,8 +297,8 @@ def test_i2v_video_generation_resizes_input_to_requested_dimensions(test_client,
    image_bytes = _make_test_image_bytes((48, 32))

    mocker.patch(
-        "vllm_omni.entrypoints.openai.serving_video.encode_video_base64",
-        return_value="Zg==",
+        "vllm_omni.entrypoints.openai.serving_video._encode_video_bytes",
+        return_value=b"fake-video",
    )
    response = test_client.post(
        "/v1/videos",
@@ -227,8 +323,8 @@ def test_i2v_video_generation_resizes_input_to_requested_dimensions(test_client,

 def test_i2v_video_generation_with_image_reference_form(test_client, mocker: MockerFixture):
    mocker.patch(
-        "vllm_omni.entrypoints.openai.serving_video.encode_video_base64",
-        return_value="Zg==",
+        "vllm_omni.entrypoints.openai.serving_video._encode_video_bytes",
+        return_value=b"fake-video",
    )
    response = test_client.post(
        "/v1/videos",
@@ -252,12 +348,12 @@ def test_i2v_video_generation_with_image_reference_form(test_client, mocker: Moc
 def test_seconds_defaults_fps_and_frames(test_client, mocker: MockerFixture):
    fps_values = []

-    def _fake_encode(video, fps):
+    def _fake_encode(video, fps, audio=None, audio_sample_rate=None, **kwargs):
        fps_values.append(fps)
-        return "Zg=="
+        return b"fake-video"

    mocker.patch(
-        "vllm_omni.entrypoints.openai.serving_video.encode_video_base64",
+        "vllm_omni.entrypoints.openai.serving_video._encode_video_bytes",
        side_effect=_fake_encode,
    )
    response = test_client.post(
@@ -281,8 +377,8 @@ def test_seconds_defaults_fps_and_frames(test_client, mocker: MockerFixture):

 def test_size_param_sets_width_height(test_client, mocker: MockerFixture):
    mocker.patch(
-        "vllm_omni.entrypoints.openai.serving_video.encode_video_base64",
-        return_value="Zg==",
+        "vllm_omni.entrypoints.openai.serving_video._encode_video_bytes",
+        return_value=b"fake-video",
    )
    response = test_client.post(
        "/v1/videos",
@@ -303,8 +399,8 @@ def test_size_param_sets_width_height(test_client, mocker: MockerFixture):

 def test_sampling_params_pass_through(test_client, mocker: MockerFixture):
    mocker.patch(
-        "vllm_omni.entrypoints.openai.serving_video.encode_video_base64",
-        return_value="Zg==",
+        "vllm_omni.entrypoints.openai.serving_video._encode_video_bytes",
+        return_value=b"fake-video",
    )
    response = test_client.post(
        "/v1/videos",
@@ -332,13 +428,155 @@ def test_sampling_params_pass_through(test_client, mocker: MockerFixture):
    assert captured.extra_args["flow_shift"] == 0.25


+def test_frame_interpolation_params_pass_to_diffusion_sampling_params(test_client, mocker: MockerFixture):
+    """Frame interpolation parameters should be forwarded to diffusion worker sampling params."""
+    mocker.patch(
+        "vllm_omni.entrypoints.openai.serving_video._encode_video_bytes",
+        return_value=b"fake-video",
+    )
+    response = test_client.post(
+        "/v1/videos",
+        data={
+            "prompt": "smooth motion",
+            "fps": "8",
+            "enable_frame_interpolation": "true",
+            "frame_interpolation_exp": "2",
+            "frame_interpolation_scale": "0.5",
+            "frame_interpolation_model_path": "local-rife",
+        },
+    )
+
+    assert response.status_code == 200
+    video_id = response.json()["id"]
+    _wait_for_status(test_client, video_id, VideoGenerationStatus.COMPLETED.value)
+
+    engine = test_client.app.state.openai_serving_video._engine_client
+    captured = engine.captured_sampling_params_list[0]
+    assert captured.enable_frame_interpolation is True
+    assert captured.frame_interpolation_exp == 2
+    assert captured.frame_interpolation_scale == 0.5
+    assert captured.frame_interpolation_model_path == "local-rife"
+
+
+def test_default_sampling_params_apply_to_video_requests(test_client, mocker: MockerFixture):
+    mocker.patch(
+        "vllm_omni.entrypoints.openai.serving_video._encode_video_bytes",
+        return_value=b"fake-video",
+    )
+    engine = test_client.app.state.openai_serving_video._engine_client
+    engine.default_sampling_params_list = [
+        OmniDiffusionSamplingParams(
+            num_inference_steps=4,
+            guidance_scale=7.5,
+            generator_device="cpu",
+            enable_frame_interpolation=True,
+            frame_interpolation_exp=2,
+            frame_interpolation_scale=0.5,
+            frame_interpolation_model_path="default-rife",
+        )
+    ]
+
+    response = test_client.post(
+        "/v1/videos",
+        data={
+            "prompt": "default param pass-through",
+        },
+    )
+
+    assert response.status_code == 200
+    video_id = response.json()["id"]
+    _wait_for_status(test_client, video_id, VideoGenerationStatus.COMPLETED.value)
+
+    captured = engine.captured_sampling_params_list[0]
+    assert captured.num_inference_steps == 4
+    assert captured.guidance_scale == 7.5
+    assert captured.generator_device == "cpu"
+    assert captured.enable_frame_interpolation is True
+    assert captured.frame_interpolation_exp == 2
+    assert captured.frame_interpolation_scale == 0.5
+    assert captured.frame_interpolation_model_path == "default-rife"
+
+
+def test_request_params_override_default_video_sampling_params(test_client, mocker: MockerFixture):
+    mocker.patch(
+        "vllm_omni.entrypoints.openai.serving_video._encode_video_bytes",
+        return_value=b"fake-video",
+    )
+    engine = test_client.app.state.openai_serving_video._engine_client
+    engine.default_sampling_params_list = [
+        OmniDiffusionSamplingParams(
+            num_inference_steps=4,
+            guidance_scale=7.5,
+            enable_frame_interpolation=True,
+            frame_interpolation_exp=2,
+            frame_interpolation_scale=0.5,
+            frame_interpolation_model_path="default-rife",
+        )
+    ]
+
+    response = test_client.post(
+        "/v1/videos",
+        data={
+            "prompt": "explicit override",
+            "num_inference_steps": "8",
+            "enable_frame_interpolation": "false",
+            "frame_interpolation_exp": "1",
+            "frame_interpolation_scale": "1.0",
+            "frame_interpolation_model_path": "custom-rife",
+        },
+    )
+
+    assert response.status_code == 200
+    video_id = response.json()["id"]
+    _wait_for_status(test_client, video_id, VideoGenerationStatus.COMPLETED.value)
+
+    captured = engine.captured_sampling_params_list[0]
+    assert captured.num_inference_steps == 8
+    assert captured.guidance_scale == 7.5
+    assert captured.enable_frame_interpolation is False
+    assert captured.frame_interpolation_exp == 1
+    assert captured.frame_interpolation_scale == 1.0
+    assert captured.frame_interpolation_model_path == "custom-rife"
+
+
+def test_worker_fps_multiplier_is_applied_to_async_encoding(test_client, mocker: MockerFixture):
+    fps_values = []
+    engine = test_client.app.state.openai_serving_video._engine_client
+
+    async def _generate(prompt, request_id, sampling_params_list):
+        engine.captured_prompt = prompt
+        engine.captured_sampling_params_list = sampling_params_list
+        import numpy as np
+
+        yield MockVideoResult([np.zeros((1, 64, 64, 3), dtype=np.uint8)], custom_output={"video_fps_multiplier": 2})
+
+    engine.generate = _generate
+
+    def _fake_encode(video, fps, **kwargs):
+        del video, kwargs
+        fps_values.append(fps)
+        return b"fake-video"
+
+    mocker.patch(
+        "vllm_omni.entrypoints.openai.serving_video._encode_video_bytes",
+        side_effect=_fake_encode,
+    )
+
+    response = test_client.post("/v1/videos", data={"prompt": "fps multiplier", "fps": "8"})
+
+    assert response.status_code == 200
+    video_id = response.json()["id"]
+    _wait_for_status(test_client, video_id, VideoGenerationStatus.COMPLETED.value)
+    assert fps_values == [16]
+
+
 def test_audio_sample_rate_comes_from_model_config(test_client, mocker: MockerFixture):
    audio_sample_rates = []

-    def _fake_encode(video, fps, audio=None, audio_sample_rate=None):
-        del video, fps, audio
+    def _fake_encode(video, fps, audio=None, audio_sample_rate=None, video_codec_options=None):
+        del video, fps, audio, video_codec_options
        audio_sample_rates.append(audio_sample_rate)
-        return "Zg=="
+        return b"fake-video"

    engine = test_client.app.state.openai_serving_video._engine_client
    engine.model_config = SimpleNamespace(
@@ -352,12 +590,14 @@ def test_audio_sample_rate_comes_from_model_config(test_client, mocker: MockerFi
    async def _generate(prompt, request_id, sampling_params_list):
        engine.captured_prompt = prompt
        engine.captured_sampling_params_list = sampling_params_list
-        yield MockVideoResult([object()], audios=[object()])
+        import numpy as np
+
+        yield MockVideoResult([np.zeros((1, 64, 64, 3), dtype=np.uint8)], audios=[object()])

    engine.generate = _generate

    mocker.patch(
-        "vllm_omni.entrypoints.openai.serving_video.encode_video_base64",
+        "vllm_omni.entrypoints.openai.serving_video._encode_video_bytes",
        side_effect=_fake_encode,
    )
    response = test_client.post(
@@ -371,6 +611,33 @@ def test_audio_sample_rate_comes_from_model_config(test_client, mocker: MockerFi
    assert audio_sample_rates == [16000]


+def test_video_job_persists_profiler_metadata(test_client, mocker: MockerFixture):
+    engine = test_client.app.state.openai_serving_video._engine_client
+
+    async def _generate(prompt, request_id, sampling_params_list):
+        engine.captured_prompt = prompt
+        engine.captured_sampling_params_list = sampling_params_list
+        yield MockVideoResult(
+            [object()],
+            stage_durations={"diffuse": 2.5, "vae.decode": 0.3},
+            peak_memory_mb=4096.5,
+        )
+
+    engine.generate = _generate
+    mocker.patch(
+        "vllm_omni.entrypoints.openai.serving_video._encode_video_bytes",
+        return_value=b"fake-video",
+    )
+
+    response = test_client.post("/v1/videos", data={"prompt": "profile me"})
+    assert response.status_code == 200
+    video_id = response.json()["id"]
+    completed = _wait_for_status(test_client, video_id, VideoGenerationStatus.COMPLETED.value)
+
+    assert completed["stage_durations"] == {"diffuse": 2.5, "vae.decode": 0.3}
+    assert completed["peak_memory_mb"] == 4096.5
+
+
 def test_missing_handler_returns_503():
    app = FastAPI()
    app.include_router(router)
@@ -428,8 +695,8 @@ def test_invalid_seconds_returns_422(test_client):

 def test_negative_prompt_and_seed_pass_through(test_client, mocker: MockerFixture):
    mocker.patch(
-        "vllm_omni.entrypoints.openai.serving_video.encode_video_base64",
-        return_value="Zg==",
+        "vllm_omni.entrypoints.openai.serving_video._encode_video_bytes",
+        return_value=b"fake-video",
    )
    response = test_client.post(
        "/v1/videos",
@@ -498,12 +765,16 @@ def test_video_request_validation():

    with pytest.raises(ValueError):
        VideoGenerationRequest(prompt="test", image_reference={"file_id": "file-1", "image_url": "https://example.com"})
+    with pytest.raises(ValueError):
+        VideoGenerationRequest(prompt="test", frame_interpolation_exp=0)
+    with pytest.raises(ValueError):
+        VideoGenerationRequest(prompt="test", frame_interpolation_scale=0)


 def test_list_videos_supports_order_after_and_limit(test_client, mocker: MockerFixture):
    mocker.patch(
-        "vllm_omni.entrypoints.openai.serving_video.encode_video_base64",
-        return_value="Zg==",
+        "vllm_omni.entrypoints.openai.serving_video._encode_video_bytes",
+        return_value=b"fake-video",
    )
    ids = []
    for i in range(3):
@@ -571,8 +842,8 @@ def test_list_videos_supports_order_after_and_limit(test_client, mocker: MockerF

 def test_delete_completed_job_removes_file_and_metadata(test_client, mocker: MockerFixture):
    mocker.patch(
-        "vllm_omni.entrypoints.openai.serving_video.encode_video_base64",
-        return_value="Zg==",
+        "vllm_omni.entrypoints.openai.serving_video._encode_video_bytes",
+        return_value=b"fake-video",
    )
    create_resp = test_client.post("/v1/videos", data={"prompt": "Delete this video"})
    assert create_resp.status_code == 200
@@ -643,8 +914,8 @@ def test_video_response_file_extension_is_robust():
 def test_extra_params_merged_into_extra_args(test_client, mocker: MockerFixture):
    """extra_params JSON object is merged into sampling_params.extra_args."""
    mocker.patch(
-        "vllm_omni.entrypoints.openai.serving_video.encode_video_base64",
-        return_value="Zg==",
+        "vllm_omni.entrypoints.openai.serving_video._encode_video_bytes",
+        return_value=b"fake-video",
    )
    extra_params = {
        "is_enable_stage2": True,
@@ -674,8 +945,8 @@ def test_extra_params_merged_into_extra_args(test_client, mocker: MockerFixture)
 def test_extra_params_none_by_default(test_client, mocker: MockerFixture):
    """When extra_params is omitted, extra_args stays empty."""
    mocker.patch(
-        "vllm_omni.entrypoints.openai.serving_video.encode_video_base64",
-        return_value="Zg==",
+        "vllm_omni.entrypoints.openai.serving_video._encode_video_bytes",
+        return_value=b"fake-video",
    )
    response = test_client.post(
        "/v1/videos",
@@ -715,8 +986,8 @@ def test_extra_params_invalid_json(test_client):
 def test_extra_params_merged_with_existing_extra_args(test_client, mocker: MockerFixture):
    """extra_params is merged on top of existing extra_args (e.g. flow_shift)."""
    mocker.patch(
-        "vllm_omni.entrypoints.openai.serving_video.encode_video_base64",
-        return_value="Zg==",
+        "vllm_omni.entrypoints.openai.serving_video._encode_video_bytes",
+        return_value=b"fake-video",
    )
    response = test_client.post(
        "/v1/videos",
@@ -735,3 +1006,59 @@ def test_extra_params_merged_with_existing_extra_args(test_client, mocker: Mocke
    assert captured.extra_args["flow_shift"] == 0.5
    assert captured.extra_args["use_zero_init"] is True
    assert captured.extra_args["zero_steps"] == 2
+
+
+def test_sample_solver_forwarded_via_extra_params(test_client, mocker: MockerFixture):
+    """sample_solver can be passed through existing extra_params for Wan2.2 online serving."""
+    mocker.patch(
+        "vllm_omni.entrypoints.openai.serving_video._encode_video_bytes",
+        return_value=b"fake-video",
+    )
+    response = test_client.post(
+        "/v1/videos",
+        data={
+            "prompt": "A fox running through snow.",
+            "extra_params": json.dumps({"sample_solver": "euler"}),
+        },
+    )
+
+    assert response.status_code == 200
+    video_id = response.json()["id"]
+    _wait_for_status(test_client, video_id, VideoGenerationStatus.COMPLETED.value)
+    engine = test_client.app.state.openai_serving_video._engine_client
+    captured = engine.captured_sampling_params_list[0]
+    assert captured.extra_args["sample_solver"] == "euler"
+
+
+@pytest.mark.asyncio
+async def test_run_generation_maps_omni_request_error_to_http_exception():
+    serving = OmniOpenAIServingVideo.for_diffusion(
+        diffusion_engine=ErrorAsyncOmni(
+            OmniRequestError(
+                "oom",
+                status_code=507,
+                request_id="req-oom",
+                stage_id=2,
+                error_type="OutOfMemoryError",
+                detail={"retryable": False},
+            )
+        ),
+        model_name="Wan-AI/Wan2.2-T2V-A14B-Diffusers",
+    )
+
+    with pytest.raises(HTTPException) as exc_info:
+        await serving._run_generation(
+            {"prompt": "test"},
+            OmniDiffusionSamplingParams(),
+            "req-oom",
+        )
+
+    exc = exc_info.value
+    assert exc.status_code == 507
+    assert exc.detail == {
+        "message": "oom",
+        "request_id": "req-oom",
+        "stage_id": 2,
+        "error_type": "OutOfMemoryError",
+        "detail": {"retryable": False},
+    }
@@ -69,6 +69,20 @@ def test_default_stage_config_propagates_ulysses_mode():
    assert parallel_config.ulysses_mode == "advanced_uaa"


+def test_default_stage_config_includes_default_sampling_params():
+    """Ensure default sampling params survive the default diffusion-stage builder."""
+    stage_cfg = AsyncOmniEngine._create_default_diffusion_stage_cfg(
+        {
+            "default_sampling_params": '{"0": {"generator_device":"cpu", "guidance_scale":7.5}}',
+        }
+    )[0]
+
+    assert stage_cfg["default_sampling_params"] == {
+        "generator_device": "cpu",
+        "guidance_scale": 7.5,
+    }
+
+
 def test_serve_cli_accepts_ulysses_mode():
    """Ensure diffusion serve CLI exposes ulysses_mode and wires it to parallel_config."""
    parser = FlexibleArgumentParser()
@@ -4,6 +4,8 @@ from unittest.mock import MagicMock, patch

 import pytest

+from vllm_omni.diffusion.data import OmniRequestError
+
 pytestmark = [pytest.mark.core_model, pytest.mark.cpu]


@@ -204,6 +206,45 @@ class TestOmniBaseProfilerSignatureConsistency:
        # stages should default to None
        assert sig.parameters["stages"].default is None

+
+class TestOmniBaseOutputHandling:
+    def test_handle_output_message_raises_structured_omni_error(self):
+        mock_engine = MagicMock()
+        mock_engine.num_stages = 1
+        mock_engine.default_sampling_params_list = [MagicMock()]
+        mock_engine.get_stage_metadata.return_value = {
+            "final_output_type": "image",
+            "final_output": True,
+        }
+
+        with (
+            patch("vllm_omni.entrypoints.omni_base.AsyncOmniEngine", return_value=mock_engine),
+            patch("vllm_omni.entrypoints.omni_base.omni_snapshot_download", side_effect=lambda x: x),
+            patch("vllm_omni.entrypoints.omni_base.weakref.finalize"),
+        ):
+            from vllm_omni.entrypoints.omni_base import OmniBase
+
+            omni_base = OmniBase(model="test-model")
+
+        with pytest.raises(OmniRequestError, match="oom") as exc_info:
+            omni_base._handle_output_message(
+                {
+                    "type": "error",
+                    "error": "oom",
+                    "status_code": 507,
+                    "request_id": "req-1",
+                    "stage_id": 2,
+                    "error_type": "OutOfMemoryError",
+                    "detail": {"retryable": False},
+                }
+            )
+
+        assert exc_info.value.status_code == 507
+        assert exc_info.value.request_id == "req-1"
+        assert exc_info.value.stage_id == 2
+        assert exc_info.value.error_type == "OutOfMemoryError"
+        assert exc_info.value.detail == {"retryable": False}
+
    def test_stop_profile_default_values(self):
        """Verify stop_profile has correct default parameter values."""
        import inspect
@@ -10,6 +10,7 @@ from vllm.entrypoints.openai.models.protocol import BaseModelPath
 from vllm.entrypoints.openai.models.serving import OpenAIServingModels
 from vllm.sampling_params import RequestOutputKind, SamplingParams

+from vllm_omni.diffusion.data import OmniRequestError
 from vllm_omni.entrypoints.async_omni import AsyncOmni
 from vllm_omni.entrypoints.omni import Omni

@@ -540,12 +541,16 @@ async def test_async_omni_propagates_engine_error(monkeypatch: pytest.MonkeyPatc

    app = AsyncOmni("dummy-model")
    try:
-        with pytest.raises(RuntimeError, match="engine boom"):
+        with pytest.raises(OmniRequestError, match="engine boom") as exc_info:
            async for _ in app.generate(prompt="hello", request_id="req-1"):
                pass
    finally:
        app.shutdown()

+    assert exc_info.value.request_id == "req-1"
+    assert exc_info.value.stage_id == 0
+    assert exc_info.value.status_code == 500
+

 def test_omni_generate_py_generator_yields_final_outputs_for_each_request(monkeypatch: pytest.MonkeyPatch):
    sampling_params = [SamplingParams(max_tokens=8) for _ in range(3)]
@@ -14,6 +14,7 @@ from vllm_omni.diffusion.data import (
    DiffusionParallelConfig,
    OmniDiffusionConfig,
 )
+from vllm_omni.diffusion.model_metadata import QWEN_IMAGE_EDIT_PLUS_MAX_INPUT_IMAGES


 def _roundtrip_diffusion_config(**kwargs) -> OmniDiffusionConfig:
@@ -106,3 +107,12 @@ class TestCreateDefaultDiffusion:
        ea = stages[0]["engine_args"]
        assert ea["enforce_eager"] is True
        assert ea["lora_path"] == "/tmp/lora"
+
+
+def test_qwen_image_edit_plus_sets_generic_multimodal_limit():
+    od_config = OmniDiffusionConfig(model="Qwen/Qwen-Image-Edit-2511", model_class_name="QwenImageEditPlusPipeline")
+
+    od_config.update_multimodal_support()
+
+    assert od_config.supports_multimodal_inputs is True
+    assert od_config.max_multimodal_image_inputs == QWEN_IMAGE_EDIT_PLUS_MAX_INPUT_IMAGES
@@ -36,6 +36,7 @@ class Attention(nn.Module):
        scatter_idx: int = 2,
        gather_idx: int = 1,
        use_sync: bool = False,
+        skip_sequence_parallel: bool = False,
    ):
        super().__init__()
        self.attn_backend = get_attn_backend(-1)
@@ -62,6 +63,7 @@ class Attention(nn.Module):
        self.gather_idx = gather_idx
        self.use_sync = use_sync
        self.causal = causal
+        self.skip_sequence_parallel = skip_sequence_parallel

        self.use_ring = False
        self.ring_pg = None
@@ -98,6 +100,8 @@ class Attention(nn.Module):
        (e.g., in noise_refiner/context_refiner before unified_prepare in Z-Image).
        This avoids unnecessary SP communication for layers not covered by _sp_plan.
        """
+        if self.skip_sequence_parallel:
+            return self._no_parallel_strategy
        if is_forward_context_available():
            ctx = get_forward_context()
            if not ctx.sp_active:
@@ -20,24 +20,6 @@ from vllm_omni.diffusion.data import DiffusionCacheConfig
 logger = init_logger(__name__)


-def enable_hunyuan_image3_teacache(pipeline: Any, config: DiffusionCacheConfig) -> None:
-    """
-    Enable TeaCache for HunyuanImage3 model.
-
-    HunyuanImage3 uses a GPT-based architecture with KV cache, which is incompatible
-    with the standard hook-based TeaCache approach. Instead, we store the TeaCacheConfig
-    on the pipeline so the denoising loop can implement caching directly.
-    """
-    teacache_config = TeaCacheConfig(
-        transformer_type="HunyuanImage3Pipeline",
-        rel_l1_thresh=config.rel_l1_thresh,
-        coefficients=config.coefficients,
-    )
-    pipeline._tea_cache_config = teacache_config
-
-    logger.info(f"TeaCache enabled for HunyuanImage3 with rel_l1_thresh={teacache_config.rel_l1_thresh}")
-
-
 def enable_bagel_teacache(pipeline: Any, config: DiffusionCacheConfig) -> None:
    """
    Enable TeaCache for Bagel model.
@@ -88,7 +70,6 @@ def enable_flux2_klein_teacache(pipeline: Any, config: DiffusionCacheConfig) ->
 CUSTOM_TEACACHE_ENABLERS = {
    "BagelPipeline": enable_bagel_teacache,
    "Flux2KleinPipeline": enable_flux2_klein_teacache,
-    "HunyuanImage3Pipeline": enable_hunyuan_image3_teacache,
 }


@@ -176,17 +157,6 @@ class TeaCacheBackend(CacheBackend):
                                Currently not used by TeaCache but accepted for interface consistency.
            verbose: Whether to log refresh operations (default: True)
        """
-        # HunyuanImage3: tea cache state is managed inside the denoising loop,
-        # so refresh is a no-op (state is re-initialized every __call__).
-        if (
-            hasattr(pipeline, "_tea_cache_config")
-            and isinstance(pipeline._tea_cache_config, TeaCacheConfig)
-            and pipeline.__class__.__name__ == "HunyuanImage3Pipeline"
-        ):
-            if verbose:
-                logger.debug(f"TeaCache state refreshed for HunyuanImage3 (num_inference_steps={num_inference_steps})")
-            return
-
        # Extract transformer from pipeline
        transformer = pipeline.transformer

@@ -54,16 +54,6 @@ _MODEL_COEFFICIENTS = {
        -12.281286412689623,
        1.0733905006198015,
    ],
-    # HunyuanImage3 pipeline coefficients
-    # Calibrated via polyfit on 3920 data points (80 prompts × 49 steps).
-    # Maps time_embed rel_l1 (range ~0.12-0.54) to output rel_l1 (range ~0.01-0.53).
-    "HunyuanImage3Pipeline": [
-        1.04117826e02,
-        -1.26848482e02,
-        5.68168652e01,
-        -1.04182570e01,
-        6.78098549e-01,
-    ],
 }


@@ -221,7 +221,8 @@ def extract_qwen_context(
    block = module.transformer_blocks[0]
    img_mod_params = block.img_mod(temb)
    img_mod1, _ = img_mod_params.chunk(2, dim=-1)
-    img_modulated, _ = block.img_norm1(hidden_states, img_mod1)
+    img_scale1, img_shift1, _ = block._modulate(img_mod1)
+    img_modulated = block.img_norm1(hidden_states, img_scale1, img_shift1)

    # ============================================================================
    # DEFINE TRANSFORMER EXECUTION (Qwen-specific)
@@ -17,6 +17,7 @@ from vllm.model_executor.layers.quantization.base_config import (
    QuantizationConfig,
 )

+from vllm_omni.diffusion.model_metadata import get_diffusion_model_metadata
 from vllm_omni.diffusion.utils.network_utils import is_port_available
 from vllm_omni.quantization import build_quant_config

@@ -468,8 +469,10 @@ class OmniDiffusionConfig:
    # Scheduler flow_shift for Wan2.2 (12.0 for 480p, 5.0 for 720p)
    flow_shift: float | None = None

-    # support multi images input
+    # Support multi-image inputs and expose any model-specific request limit
+    # through a generic config field so serving code stays model-agnostic.
    supports_multimodal_inputs: bool = False
+    max_multimodal_image_inputs: int | None = None

    log_level: str = "info"

@@ -615,7 +618,54 @@ class OmniDiffusionConfig:
            raise ValueError("max_cpu_loras must be >= 1 for diffusion LoRA")

    def update_multimodal_support(self) -> None:
-        self.supports_multimodal_inputs = self.model_class_name in {"QwenImageEditPlusPipeline"}
+        # Resolve serving-visible multimodal behavior from shared metadata
+        # instead of importing concrete pipeline modules into the config layer.
+        metadata = get_diffusion_model_metadata(self.model_class_name)
+        self.supports_multimodal_inputs = metadata.supports_multimodal_inputs
+        self.max_multimodal_image_inputs = metadata.max_multimodal_image_inputs
+
+    def enrich_config(self) -> None:
+        """Load model metadata from HuggingFace and populate config fields.
+
+        Diffusers-style models expose ``model_index.json`` with ``_class_name``.
+        Non-diffusers models (e.g. Bagel, NextStep) only have ``config.json``,
+        so we fall back to reading that and mapping model_type manually.
+        """
+        from vllm.transformers_utils.config import get_hf_file_to_dict
+
+        try:
+            config_dict = get_hf_file_to_dict("model_index.json", self.model)
+            if config_dict is not None:
+                if self.model_class_name is None:
+                    self.model_class_name = config_dict.get("_class_name", None)
+                self.update_multimodal_support()
+
+                tf_config_dict = get_hf_file_to_dict("transformer/config.json", self.model)
+                self.tf_model_config = TransformerConfig.from_dict(tf_config_dict)
+            else:
+                raise FileNotFoundError("model_index.json not found")
+        except (AttributeError, OSError, ValueError, FileNotFoundError):
+            cfg = get_hf_file_to_dict("config.json", self.model)
+            if cfg is None:
+                raise ValueError(f"Could not find config.json or model_index.json for model {self.model}")
+
+            self.tf_model_config = TransformerConfig.from_dict(cfg)
+            model_type = cfg.get("model_type")
+            architectures = cfg.get("architectures") or []
+
+            if model_type == "bagel" or "BagelForConditionalGeneration" in architectures:
+                self.model_class_name = "BagelPipeline"
+                self.tf_model_config = TransformerConfig()
+                self.update_multimodal_support()
+            elif model_type == "nextstep":
+                if self.model_class_name is None:
+                    self.model_class_name = "NextStep11Pipeline"
+                self.tf_model_config = TransformerConfig()
+                self.update_multimodal_support()
+            elif architectures and len(architectures) == 1:
+                self.model_class_name = architectures[0]
+            else:
+                raise

    @classmethod
    def from_kwargs(cls, **kwargs: Any) -> "OmniDiffusionConfig":
@@ -656,6 +706,7 @@ class DiffusionOutput:
    output: torch.Tensor | None = None
    trajectory_timesteps: list[torch.Tensor] | None = None
    trajectory_latents: torch.Tensor | None = None
+    trajectory_log_probs: torch.Tensor | None = None
    trajectory_decoded: list[torch.Tensor] | None = None
    error: str | None = None

@@ -675,6 +726,49 @@ class DiffusionOutput:
    peak_memory_mb: float = 0.0


+@dataclass
+class OmniRequestError(RuntimeError):
+    def __init__(
+        self,
+        message: str,
+        *,
+        status_code: int = 500,
+        request_id: str | None = None,
+        stage_id: int | None = None,
+        error_type: str | None = None,
+        detail: dict[str, Any] | None = None,
+    ):
+        super().__init__(message)
+        self.status_code = status_code
+        self.request_id = request_id
+        self.stage_id = stage_id
+        self.error_type = error_type or self.__class__.__name__
+        self.detail = detail or {}
+
+
+def normalize_omni_error(
+    exc: Exception,
+    *,
+    status_code: int = 500,
+    request_id: str | None = None,
+    stage_id: int | None = None,
+) -> OmniRequestError:
+    if isinstance(exc, OmniRequestError):
+        if exc.request_id is None:
+            exc.request_id = request_id
+        if exc.stage_id is None:
+            exc.stage_id = stage_id
+        return exc
+
+    return OmniRequestError(
+        str(exc),
+        status_code=status_code,
+        request_id=request_id,
+        stage_id=stage_id,
+        error_type=type(exc).__name__,
+    )
+
+
 class AttentionBackendEnum(enum.Enum):
    FA = enum.auto()
    SLIDING_TILE_ATTN = enum.auto()
Author	SHA1	Message	Date
lyj-jjj	5fd676f086	cherry-pick pr3067 (#3087 ) Signed-off-by: lyj-jjj <liuyingjun5@huawei.com>	2026-04-24 09:56:49 +08:00
WeiQing Chen	2980618f70	[release v0.18.0.post1] Fix default sampling params for /v1/videos (#3021 ) Signed-off-by: david6666666 <530634352@qq.com>	2026-04-23 11:29:54 +08:00
bjf-frz	6d76e7d343	[cherry-pick][release/v.0.18.0.post1]remove duplicate video preprocess in Wan2.2 pipeline (#3005 ) Signed-off-by: bjf-frz <frz123db@gmail.com>	2026-04-22 11:09:27 +08:00
fan2956	caa26e1d04	[Cherry-pick][v0.18.0.post1][Bugfix] Fix VAE parallelism dist.gather performance bottleneck on NPU (#2970 ) Signed-off-by: fan2956 <zhoufan53@huawei.com>	2026-04-21 20:36:08 +08:00
WeiQing Chen	89f733d941	[codex][release/v0.18.0.post1] revert Wan2.2 pipeline changes from #2878 (#2937 ) Signed-off-by: david6666666 <530634352@qq.com>	2026-04-20 19:22:11 +08:00
fan2956	8a1ff4e994	[Cherry-pick][v0.18.0.post1][Perf] Optimize Wan2.2 device free on image preprocess (#2854 ) Signed-off-by: fan2956 <zhoufan53@huawei.com>	2026-04-20 14:51:38 +08:00
WeiQing Chen	2116e88638	[cherry-pick][release/v0.18.0.post1] cherry-pick #2847 #2780 #2840 #2876 #2877 (#2878 ) Signed-off-by: david6666666 <530634352@qq.com> Signed-off-by: David Chen <530634352@qq.com> Signed-off-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>	2026-04-20 11:14:12 +08:00
fan2956	783086bcbf	[PERF] Wan2.2 support RMSNorm NPU (#2576 ) Signed-off-by: fan2956 <zhoufan53@huawei.com>	2026-04-18 15:46:37 +08:00
WeiQing ChenandSamit	fa5d358c62	[cherry-pick releasev0.18.0post1] Codex/release v0.18.0.post1 cherry-pick #2735/#2736/#2555 (#2813 ) Signed-off-by: samithuang <285365963@qq.com> Signed-off-by: David Chen <530634352@qq.com> Signed-off-by: Samit <285365963@qq.com> Signed-off-by: david6666666 <530634352@qq.com> Co-authored-by: Samit <285365963@qq.com>	2026-04-17 15:44:45 +08:00
fan2956andgcanlin	f70fd1f58b	[Cherry-pick][v0.18.0.post1][PERF] Wan2.2 support adalayernorm on NPU (#2575 ) Signed-off-by: fan2956 <zhoufan53@huawei.com> Signed-off-by: gcanlin <canlinguosdu@gmail.com> Co-authored-by: gcanlin <canlinguosdu@gmail.com>	2026-04-16 08:47:38 +08:00
fan2956	91fa5c51d7	[Bugfix] fix mindiesd laserattention unsupported error (#2674 ) Signed-off-by: fan2956 <zhoufan53@huawei.com>	2026-04-10 16:19:17 +08:00
WeiQing Chen	a109f777fd	[cherry-pick release/v0.18.0post1][Bugfix] Fix Qwen-Image min-size normalization for tiny requests (#2637 ) (#2648 ) Signed-off-by: David Chen <530634352@qq.com>	2026-04-09 21:04:39 +08:00
bjf-frz	ca7ee8d6e7	[release/v0.18.0.post1][Bugfix] fix oom no response to user (#2638 ) Signed-off-by: bjf-frz <frz123db@gmail.com>	2026-04-09 20:31:58 +08:00
WeiQing Chen	589e664947	[cherry-pick release/v0.18.0post1][Bugfix] Enforce --max-generated-image-size on /v1/images/generations… (#2625 ) Signed-off-by: David Chen <530634352@qq.com>	2026-04-09 14:42:23 +08:00
WeiQing Chen Y. Fisher Canlin Guo	73d31edf63	[cherry-pick][release/v0.18.0post1][Perf] Wan2.2 I2V optimization: convert datatype from FP32 to BF16 in… (#2618 ) Signed-off-by: KexiongYu <yukexiong1@huawei.com> Signed-off-by: Canlin Guo <961750412@qq.com> Signed-off-by: David Chen <530634352@qq.com> Co-authored-by: Y. Fisher <yukexiong1@huawei.com> Co-authored-by: Canlin Guo <961750412@qq.com>	2026-04-09 09:57:14 +08:00
WeiQing Chen Canlin Guo Hongsheng Liu	2f289c78db	[cherry-pick][Feature] Support vae tiling parallel encode (#2368 ) (#2538 ) Signed-off-by: gcanlin <canlinguosdu@gmail.com> Signed-off-by: David Chen <530634352@qq.com> Co-authored-by: Canlin Guo <canlinguosdu@gmail.com> Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>	2026-04-07 11:16:44 +08:00
WeiQing ChenandCanlin Guo	9b23589c80	[cherry-pick][Perf] Skip Wan2.2 cross attn Ulysses SP (#2459 ) (#2537 ) Signed-off-by: gcanlin <canlinguosdu@gmail.com> Signed-off-by: David Chen <530634352@qq.com> Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>	2026-04-07 10:57:02 +08:00
WeiQing ChenandCanlin Guo	5845eb743c	[cherry-pick][Perf] Optimize Wan2.2 rotary embedding (#2393 ) (#2536 ) Signed-off-by: gcanlin <canlinguosdu@gmail.com> Signed-off-by: David Chen <530634352@qq.com> Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>	2026-04-07 10:56:51 +08:00
WeiQing Chen	300411c9bd	[cherry-pick][release/v0.18.0post1] fix: return 400 for unsupported multi-image edits such as Qw… (#2352 ) Signed-off-by: David Chen <530634352@qq.com>	2026-03-31 11:19:23 +08:00
WeiQing Chen	3485569742	[cherry-pick] [release/v0.18.0post1] fix: validate layered image layers range (#2334 ) (#2349 ) Signed-off-by: David Chen <530634352@qq.com>	2026-03-31 11:18:42 +08:00
WeiQing ChenandCanlin Guo	3edc2a9379	[cherry-pick] [release/v0.18.0.post1]fix: handle Qwen-Image-Layered layered RGBA output for jpeg … (#2329 ) Signed-off-by: David Chen <530634352@qq.com> Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>	2026-03-30 20:41:03 +08:00