Enable twine upload to PyPI in release pipeline

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-17 03:54:51 -07:00
13 changed files with 76 additions and 358 deletions
@@ -1,97 +0,0 @@
 ---
 name: precheck-pr
 description: Self-check your branch before creating a PR — catch dead code, verify accuracy/perf claims, validate PR title format, and confirm merge readiness. Use when the user says "precheck", "self review", "pre-submit check", or "check my PR before I open it." Never posts to GitHub.
 ---
 # PR Pre-Check
 Self-review your branch before creating a PR against `vllm-project/vllm-omni`. Two modes: **quick** catches showstoppers, **full** does a thorough maintainer-grade review. Never posts to GitHub; the report is for the contributor's terminal only.
 ## Mode Selection
 | Mode | When | Time |
 |------|------|------|
 | **Quick** | About to push, final sanity check | ~3 min |
 | **Full** | Ready for review, want maintainer-level scan | ~10 min |
 Default to quick if unsure. Run full before marking a PR "ready for review."
 ## Workflow
 ### Step 1: Detect Base Branch
 ```bash
 BASE_SHA=$(git merge-base HEAD origin/main 2>/dev/null \
         || git merge-base HEAD main 2>/dev/null \
         || echo origin/main)
 echo "diffing against ${BASE_SHA}"
 git diff --name-only ${BASE_SHA}...HEAD
 ```
 ### Step 2: Validate PR Title
 Check the most recent commit message (or branch name if no commit yet) against the project convention. Valid prefixes:
 | Prefix | Applies to |
 |--------|-----------|
 | `[Bugfix]` | Bug fixes |
 | `[CI/Build]` | Build or CI improvements |
 | `[Doc]` | Documentation changes |
 | `[Model]` | New/improved models (include model name) |
 | `[Frontend]` | Frontend changes (API server, OmniLLM class, etc.) |
 | `[Kernel]` | CUDA/kernel changes |
 | `[Core]` | Core logic changes (OmniProcessor, OmniARScheduler, etc.) |
 | `[Hardware][Vendor]` | Hardware-specific (e.g., `[Hardware][Ascend]`) |
 | `[Misc]` | Other changes (use sparingly) |
 ✗ if: missing prefix, wrong case (`[bugfix]`), or WIP/Draft in title.
 ⚠ if: `[Model]` prefix without the model identifier (e.g., `[Model] Add new model` — should be `[Model] Add <ModelName> ...`).
 ### Step 3: Categorize the PR
 | Diff contains | PR type |
 |---------------|---------|
 | New files under `vllm_omni/model_executor/models/<name>/` | **New Model** |
 | Changes to `vllm_omni/diffusion/` | **Diffusion Model** |
 | `[Bugfix]` prefix or single-file fix | **Bug Fix** |
 | Perf/benchmark/throughput claims in commit msg or diff | **Performance** |
 | Everything else | **General** |
 If multiple rows apply (e.g., a diffusion model is also a new model), union the checklists.
 ### Step 4: Run Checklist
 Ask: "Quick mode or full mode?" Then walk the checklist for the detected PR type from [references/checklists.md](references/checklists.md). Each item produces ✓, ✗, or ⚠.
 ### Step 5: Print Report
 ```
 Pre-check report for <branch>
  Mode: quick | full
  Type: <new-model | diffusion-model | bug-fix | perf | general>
  Dimension          Result
  ─────────────────  ──────
  PR title format    ✓
  PR desc integrity  ✓
  Registry/config    ✓
  Dead code          ⚠ 2 warnings
  Accuracy           ✓
  Benchmark          ✗ missing software versions
  Verdict: 1 blocking | 2 warnings | recommend fixing ✗ before PR
 ```
 **Severity:**
 | Mark | Meaning |
 |------|---------|
 | ✗ | Blocking — fix before opening PR |
 | ⚠ | Warning — consider fixing |
 | ✓ | Pass |
 | — | Skipped (not applicable) |
 ### Stop Here
 Do not post comments, open PRs, or modify files. The report is for the contributor's terminal only.
@@ -1,135 +0,0 @@
 # Pre-Check Checklists
 Each item is a concrete check to run against the diff. ✗ = fix before PR. ⚠ = consider fixing.
 ---
 ## All PRs: Title Format (check first)
 PR title must follow the project convention documented in `docs/contributing/README.md`:
 | Prefix | Applies to |
 |--------|-----------|
 | `[Bugfix]` | Bug fixes |
 | `[CI/Build]` | Build or CI improvements |
 | `[Doc]` | Documentation changes |
 | `[Model]` | New/improved models (include model name) |
 | `[Frontend]` | Frontend changes (API server, OmniLLM class, etc.) |
 | `[Kernel]` | CUDA/kernel changes |
 | `[Core]` | Core logic changes (OmniProcessor, OmniARScheduler, etc.) |
 | `[Hardware][Vendor]` | Hardware-specific (e.g., `[Hardware][Ascend]`) |
 | `[Misc]` | Other changes (use sparingly) |
 **Examples:**
 ```
 ✓ [Model] Add MiniCPM-o 4.5 support
 ✓ [Bugfix] Fix Ovis image text encoder dtype
 ✓ [Core] Qwen3-Omni performance optimization
 ✓ [Hardware][Ascend] Fix VoxCPM2 on Ascend
 ✗ Fix Ovis image text encoder dtype              ← missing prefix
 ✗ [bugfix] Fix Ovis image text encoder           ← wrong case
 ✗ [WIP] Add model                                ← WIP in title
 ```
 - [ ] **Title has valid prefix** — ✗ if missing prefix, wrong case, or WIP/Draft in title. For `[Model]` PRs, the title MUST also include the model identifier (e.g., `[Model] Add <ModelName> ...`). ⚠ if missing the model name.
 ---
 ## Bug Fix PRs
 ### Quick
 - [ ] **PR body has 4 sections:** what broke (error message), repro steps, root cause, fix description
 - [ ] **Fix is minimal:** diff touches only files directly related to the bug. No unrelated refactors or style changes.
 - [ ] **Repro is runnable:** the PR body includes a copy-paste command that reproduces the bug
 - [ ] **Branch is rebased:** `git merge-base HEAD origin/main` is recent, no merge conflicts
 ### Full (adds)
 - [ ] **Regression test exists:** `git diff --name-only` includes at least one `tests/` file
 - [ ] **Regression test reproduces the original error:** the test would fail on main, pass on this branch
 - [ ] **Fix matches root cause exactly:** no "fix the symptom + something else" — if the root cause is dtype, the fix is dtype, not dtype + reformatting
 - [ ] **No silent failure risk:** no bare `except: pass`, no `try: ... except: return None`, no empty fallback added
 - [ ] **Upstream pattern match:** the fix follows the same pattern as existing code for similar cases (grep for analogous `from_pretrained` calls, etc.)
 - [ ] **Environment documented:** torch/transformers/vllm versions listed if the bug is version-dependent
 ---
 ## Performance PRs
 ### Quick
 - [ ] **Before/after numbers in PR body:** at minimum, one metric (RTF, RPS, latency, VRAM) with main vs PR values
 - [ ] **Hardware specified:** GPU model, count, VRAM. "L4" or "H20" is sufficient. "tested on GPU" is not.
 - [ ] **Concurrency scaling:** if the optimization claims better throughput, numbers at 2+ concurrency levels
 - [ ] **No unexplained regressions:** if any metric goes down at any concurrency level, explain why
 - [ ] **Branch is rebased:** `git merge-base HEAD origin/main` is recent, no merge conflicts
 ### Full (adds)
 - [ ] **Software versions:** torch, CUDA, vllm, vllm-omni versions in PR body
 - [ ] **Warmup stated:** "1 warmup + 3 measured" or equivalent
 - [ ] **Stddev or range:** not just a single best run. "average of 2 rounds" is minimum.
 - [ ] **VRAM measured:** peak GPU memory before and after
 - [ ] **Benchmark script checked in:** under `tests/` or `examples/`, runnable by reviewer
 - [ ] **Exact command line in PR:** not pseudocode, not "run the benchmark"
 - [ ] **Pytest summary included:** `36 passed, 3 skipped, 17 warnings in 2897.66s` — proves the whole suite passes
 - [ ] **Config changes documented:** if `max_num_seqs`, `gpu_memory_utilization`, or deploy YAML changed, old and new values stated with reasoning
 ---
 ## New Model PRs
 ### Quick
 - [ ] **PR body file list matches `git diff --name-only`:** every file path in the description exists in the diff
 - [ ] **Registry entries match claimed count:** count `_OMNI_MODELS` or `_DIFFUSION_MODELS` entries added vs claimed in body
 - [ ] **Pipeline config `custom_process_input_func` paths resolve:** each referenced module.function exists
 - [ ] **`__init__.py` exports match registry:** each registered class appears in `__init__.py` and `__all__`
 - [ ] **Shell script modes match README:** `run_curl.sh` supports every mode the README documents
 - [ ] **Branch is rebased:** `git merge-base HEAD origin/main` is recent, no merge conflicts
 ### Full (adds)
 - [ ] **Dead code scan (5 patterns):**
  1. Dead `forward()` — defined but never called through the pipeline path
  2. Dead factory/builder functions — instantiated nowhere
  3. Dead wrappers — thin wrappers with no callers
  4. Dead branch guards — `if version > X` where X is always satisfied
  5. Unused parameters — `__init__` args never accessed
 - [ ] **Copy-paste detection:** no string constant defined in 3+ files, no cross-module validation duplication, no near-identical shape coercion fns
 - [ ] **Import hygiene:** no re-export-only imports, no module-level side effects without docs, no `import os` in function body when already at top. Function-body imports are acceptable when the import is only reachable from a closure or factory return value (i.e., `import` would otherwise execute at module load even though the user may never trigger that path). Either lift to top-of-file or annotate with `# noqa: PLC0415`.
 - [ ] **Accuracy:** at least one test compares output against reference; audio = valid WAV at correct sample rate; determinism verified if seed claimed. ⚠ if missing and not explicitly deferred to a follow-up PR or RFC section; — if explicitly deferred (the deferral target SHOULD be linked in the PR body).
 - [ ] **Performance:** hardware, software versions, warmup, and metrics (RTF, VRAM, RPS, latency) stated. ⚠ if missing and not explicitly deferred; — if explicitly deferred.
 - [ ] **Benchmark settings:** model config (TP, PP, max_model_len, enforce_eager, quant), runtime config (batch, input/output spec), environment versions
 - [ ] **Benchmark script checked in + exact command line + pytest summary**
 ---
 ## Diffusion Model PRs
 Diffusion models live under `vllm_omni/diffusion/`. In addition to the New Model checklist above, verify:
 - [ ] **Transformer adapter follows contract:** inherits from `nn.Module`, implements `load_weights()`, uses `vllm_omni.diffusion.attention.layer.Attention`
 - [ ] **Pipeline has both offline and online paths:** `forward()` for batch generation, streaming path for online serving
 - [ ] **Cache-DiT integration:** at least one acceleration (TeaCache, FBCache, etc.) is wired up. ✗ for new **top-level** pipelines. For **subclass** pipelines that inherit `__call__` from a parent where Cache-DiT is already absent, downgrade to ⚠ and recommend filing a follow-up issue against the parent pipeline family.
 - [ ] **Parallelism config:** TP/SP/USP/CFG-Parallel options exposed in pipeline config
 - [ ] **Docs table updated:** model added to `docs/models/supported_models.md` with correct metadata
 - [ ] **E2E test exists:** at least one test exercises the full generate path
 ---
 ## General PRs
 ### Quick
 - [ ] **PR body matches diff:** description doesn't claim changes not in the diff
 - [ ] **Branch is rebased:** `git merge-base HEAD origin/main` is recent, no merge conflicts
 - [ ] **CI gates passing:** DCO, pre-commit, build — check `gh pr view --json statusCheckRollup` if a PR already exists
 ### Full (adds)
 - [ ] **No new dead code in changed files:** scan for unreachable paths, unused imports
 - [ ] **Test coverage:** if core code changed (`engine/`, `stages/`, `connectors/`), tests are added or existing tests cover the path
 - [ ] **Import hygiene:** no redundant imports added
 - [ ] **No unrelated changes:** diff scope matches PR description
@@ -22,12 +22,7 @@ include:
 - `add-tts-model`: covers integration of new TTS models and related serving
  workflows
 - `generate-release-note`: helps prepare release notes for repository changes
 - `precheck-pr`: self-check a branch before creating a PR — validates PR title
  format, catches dead code, verifies accuracy/perf claims, and confirms merge
  readiness
 - `review-pr`: provides a structured workflow for reviewing pull requests
 - `vllm-omni-npu-model-runner-upgrade`: upgrades NPU model runners to align with the
  latest vllm-ascend NPUModelRunner
 ## Maintenance Guidelines
@@ -1,6 +1,4 @@
 <!-- markdownlint-disable -->
 **Before submitting:** run the precheck-pr skill with code agent for a self-check against project conventions — catches dead code, missing benchmarks, and title format issues.
 PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTTOM) HAVE BEEN CONSIDERED.
 ## Purpose
@@ -20,5 +18,4 @@ PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTT
 - [ ] (Optional) Release notes update. If your change is user-facing, please update the release notes draft.
 </details>
-**BEFORE SUBMITTING,** please read <https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md>
+**BEFORE SUBMITTING, PLEASE READ <https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md>** (anything written below this line will be removed by GitHub Actions)
 (anything written below this line will be removed by GitHub Actions)
@@ -116,20 +116,6 @@ Only specific types of PRs will be reviewed. The PR title is prefixed appropriat
 !!! note
    If the PR spans more than one category, please include all relevant prefixes.
 ### Pre-Check Before Submitting
 Before submitting a PR, run the precheck-pr skill with code agent for a self-review against project conventions:
 ```
 /precheck-pr
 ```
 The skill offers two modes:
 - **Quick (~3 min):** catches showstoppers — PR title format, missing benchmark claims, rebase status
 - **Full (~10 min):** thorough maintainer-grade review — dead code scan, copy-paste detection, import hygiene
 The precheck covers five PR types: Bug Fix, Performance, New Model, Diffusion Model, and General. Each type has a tailored checklist that validates evidence quality (repro steps, A/B benchmarks, registry entries, etc.). See the skill in `.claude/skills/precheck-pr/` for the full checklist.
 ### Local Test
 Please run the L1 and L2 test cases locally first and attach the results before contacting us to add the "ready" label. Please refer to the [test instructions](./ci/test_guide.md) for running the test cases.
@@ -1,16 +1,13 @@
 # tests/entrypoints/openai_api/test_serving_audio_generate.py
 import logging
 from inspect import Signature, signature
-from types import SimpleNamespace
+from unittest.mock import MagicMock, patch
 from unittest.mock import AsyncMock, MagicMock, patch
 import pytest
 import torch
-from fastapi import FastAPI, Request
+from fastapi import FastAPI
 from fastapi.testclient import TestClient
 from vllm.v1.engine.exceptions import EngineGenerateError
 from vllm_omni.entrypoints.omni_base import OmniEngineDeadError
 from vllm_omni.entrypoints.openai import api_server as api_server_module
 from vllm_omni.entrypoints.openai.protocol.audio import (
    CreateAudio,
    OpenAICreateAudioGenerateRequest,
@@ -109,29 +106,6 @@ def client(test_app):
    return TestClient(test_app)
 def _make_api_server_test_app(handler, *, request_id: str = "audio-gen-req-1"):
    app = FastAPI()
    app.state.openai_serving_audio_generate = handler
    app.state.engine_client = SimpleNamespace(
        engine=SimpleNamespace(is_alive=lambda: False),
        errored=True,
    )
    app.state.server = SimpleNamespace()
    @app.middleware("http")
    async def add_request_metadata(request: Request, call_next):
        request.state.request_metadata = SimpleNamespace(request_id=request_id)
        return await call_next(request)
    app.add_api_route(
        "/v1/audio/generate",
        api_server_module.create_audio_generate,
        methods=["POST"],
        response_model=None,
    )
    return app
 # Request Validation (Pydantic model)
 class TestRequestValidation:
    """Validate OpenAICreateAudioGenerateRequest pydantic constraints."""
@@ -498,35 +472,6 @@ class TestErrorHandling:
        assert "Audio generation failed" in resp.error.message
    @pytest.mark.parametrize(
        ("exc", "expected_message", "expected_stage_id"),
        [
            (OmniEngineDeadError("engine dead", error_stage_id=2), "engine dead", 2),
            (EngineGenerateError("engine generate failed"), "engine generate failed", None),
        ],
    )
    def test_api_server_engine_error_response_includes_request_and_stage_id(
        self,
        exc,
        expected_message,
        expected_stage_id,
    ):
        handler = MagicMock()
        handler.create_audio_generate = AsyncMock(side_effect=exc)
        app = _make_api_server_test_app(handler)
        with patch.object(api_server_module, "terminate_if_errored") as terminate_mock:
            with TestClient(app) as client:
                response = client.post("/v1/audio/generate", json={"input": "Hello"})
        assert response.status_code == 500
        payload = response.json()
        assert payload["error"]["message"] == expected_message
        assert payload["error"]["code"] == 500
        assert payload["error"]["request_id"] == "audio-gen-req-1"
        assert payload["error"]["error_stage_id"] == expected_stage_id
        terminate_mock.assert_called_once()
 # End-to-End via TestClient
 class TestAudioGenerateAPI:
@@ -265,6 +265,44 @@ class LTX2Pipeline(nn.Module, CFGParallelMixin, ProgressBarMixin):
        self._num_timesteps = None
        self._current_timestep = None
    @staticmethod
    def _pack_text_embeds(
        text_hidden_states: torch.Tensor,
        sequence_lengths: torch.Tensor,
        device: str | torch.device,
        padding_side: str = "left",
        scale_factor: int = 8,
        eps: float = 1e-6,
    ) -> torch.Tensor:
        batch_size, seq_len, hidden_dim, num_layers = text_hidden_states.shape
        original_dtype = text_hidden_states.dtype
        token_indices = torch.arange(seq_len, device=device).unsqueeze(0)
        if padding_side == "right":
            mask = token_indices < sequence_lengths[:, None]
        elif padding_side == "left":
            start_indices = seq_len - sequence_lengths[:, None]
            mask = token_indices >= start_indices
        else:
            raise ValueError(f"padding_side must be 'left' or 'right', got {padding_side}")
        mask = mask[:, :, None, None]
        masked_text_hidden_states = text_hidden_states.masked_fill(~mask, 0.0)
        num_valid_positions = (sequence_lengths * hidden_dim).view(batch_size, 1, 1, 1)
        masked_mean = masked_text_hidden_states.sum(dim=(1, 2), keepdim=True) / (num_valid_positions + eps)
        x_min = text_hidden_states.masked_fill(~mask, float("inf")).amin(dim=(1, 2), keepdim=True)
        x_max = text_hidden_states.masked_fill(~mask, float("-inf")).amax(dim=(1, 2), keepdim=True)
        normalized_hidden_states = (text_hidden_states - masked_mean) / (x_max - x_min + eps)
        normalized_hidden_states = normalized_hidden_states * scale_factor
        normalized_hidden_states = normalized_hidden_states.flatten(2)
        mask_flat = mask.squeeze(-1).expand(-1, -1, hidden_dim * num_layers)
        normalized_hidden_states = normalized_hidden_states.masked_fill(~mask_flat, 0.0)
        normalized_hidden_states = normalized_hidden_states.to(dtype=original_dtype)
        return normalized_hidden_states
    def _get_gemma_prompt_embeds(
        self,
        prompt: str | list[str],
@@ -304,9 +342,16 @@ class LTX2Pipeline(nn.Module, CFGParallelMixin, ProgressBarMixin):
        )
        text_encoder_hidden_states = text_encoder_outputs.hidden_states
        text_encoder_hidden_states = torch.stack(text_encoder_hidden_states, dim=-1)
-        # `diffusers>=0.38` moved per_layer_masked_mean_norm inside the connector,
+        sequence_lengths = prompt_attention_mask.sum(dim=-1)
-        # so pack to 3D here and let the connector handle normalization.
+
-        prompt_embeds = text_encoder_hidden_states.flatten(2, 3).to(dtype=dtype)
+        prompt_embeds = self._pack_text_embeds(
            text_encoder_hidden_states,
            sequence_lengths,
            device=device,
            padding_side=self.tokenizer.padding_side,
            scale_factor=scale_factor,
        )
        prompt_embeds = prompt_embeds.to(dtype=dtype)
        _, seq_len, _ = prompt_embeds.shape
        prompt_embeds = prompt_embeds.repeat(1, num_videos_per_prompt, 1)
@@ -848,9 +893,9 @@ class LTX2Pipeline(nn.Module, CFGParallelMixin, ProgressBarMixin):
            device=device,
        )
        # Compute positive prompt connectors
-        tokenizer_padding_side = getattr(self.tokenizer, "padding_side", "left")
+        additive_attention_mask = (1 - prompt_attention_mask.to(prompt_embeds.dtype)) * -1000000.0
        connector_prompt_embeds, connector_audio_prompt_embeds, connector_attention_mask = self.connectors(
-            prompt_embeds, prompt_attention_mask, padding_side=tokenizer_padding_side
+            prompt_embeds, additive_attention_mask, additive_mask=True
        )
        # Compute negative prompt connectors when CFG is enabled
@@ -858,14 +903,17 @@ class LTX2Pipeline(nn.Module, CFGParallelMixin, ProgressBarMixin):
        negative_connector_audio_prompt_embeds = None
        negative_connector_attention_mask = None
        if self.do_classifier_free_guidance:
            negative_additive_attention_mask = (
                1 - negative_prompt_attention_mask.to(negative_prompt_embeds.dtype)
            ) * -1000000.0
            (
                negative_connector_prompt_embeds,
                negative_connector_audio_prompt_embeds,
                negative_connector_attention_mask,
            ) = self.connectors(
                negative_prompt_embeds,
-                negative_prompt_attention_mask,
+                negative_additive_attention_mask,
-                padding_side=tokenizer_padding_side,
+                additive_mask=True,
            )
        latent_num_frames = (num_frames - 1) // self.vae_temporal_compression_ratio + 1
@@ -5,9 +5,9 @@
 Fully independent LTX-2.3 pipeline for vLLM-Omni.
 This pipeline does NOT inherit from LTX2Pipeline because:
- LTX-2.3 connectors run per_token_rms_norm + per-modality video/audio
+- LTX-2.3 uses a different text encoding strategy (flatten ALL 49 hidden states
-  projection internally (per_modality_projections=True),
+  vs. LTX-2's _pack_text_embeds with per-layer normalization and pooling)
-  versus LTX-2's per_layer_masked_mean_norm + shared projection path
+- LTX-2.3 connectors expect the padding_side API (not additive_mask)
 - LTX-2.3 uses a BWE vocoder outputting 48kHz audio (not 16kHz)
 - LTX-2.3 transformer requires the sigma parameter for prompt modulation
 - CPU offloading is required for the 22B transformer (~44GB VRAM)
@@ -230,10 +230,10 @@ class LTX23Pipeline(nn.Module, ProgressBarMixin):
    ):
        """Encode prompts using Gemma-3-12B, returning ALL 49 hidden states flattened.
-        Stacks all 49 hidden states and flattens to [B, seq, hidden * 49]. The
+        LTX-2.3 differs from LTX-2 in text encoding:
-        connectors unflatten, apply per_token_rms_norm, and project internally
+        - LTX-2: uses _pack_text_embeds (layer selection + pooling)
-        (same shape contract as LTX-2 since the `diffusers==0.38` connector
+        - LTX-2.3: stacks ALL 49 hidden states and flattens to [B, seq, 188160]
-        migration; the two differ only in the connector's internal norm path).
+          The connectors unflatten, apply per_token_rms_norm, and project internally.
        """
        device = device or self.device
        dtype = dtype or self.text_encoder.dtype
@@ -460,9 +460,9 @@ class LTX2ImageToVideoPipeline(LTX2Pipeline):
            device=device,
        )
        # Compute positive prompt connectors
-        tokenizer_padding_side = getattr(self.tokenizer, "padding_side", "left")
+        additive_attention_mask = (1 - prompt_attention_mask.to(prompt_embeds.dtype)) * -1000000.0
        connector_prompt_embeds, connector_audio_prompt_embeds, connector_attention_mask = self.connectors(
-            prompt_embeds, prompt_attention_mask, padding_side=tokenizer_padding_side
+            prompt_embeds, additive_attention_mask, additive_mask=True
        )
        # Compute negative prompt connectors when CFG is enabled
@@ -470,14 +470,17 @@ class LTX2ImageToVideoPipeline(LTX2Pipeline):
        negative_connector_audio_prompt_embeds = None
        negative_connector_attention_mask = None
        if self.do_classifier_free_guidance:
            negative_additive_attention_mask = (
                1 - negative_prompt_attention_mask.to(negative_prompt_embeds.dtype)
            ) * -1000000.0
            (
                negative_connector_prompt_embeds,
                negative_connector_audio_prompt_embeds,
                negative_connector_attention_mask,
            ) = self.connectors(
                negative_prompt_embeds,
-                negative_prompt_attention_mask,
+                negative_additive_attention_mask,
-                padding_side=tokenizer_padding_side,
+                additive_mask=True,
            )
        latent_num_frames = (num_frames - 1) // self.vae_temporal_compression_ratio + 1
@@ -619,19 +619,6 @@ def build_vllm_config(
        )
    filtered_engine_args_dict = filter_dataclass_kwargs(OmniEngineArgs, engine_args_dict)
    # _to_dict serializes dataclass fields (e.g. StructuredOutputsConfig) into
    # plain dicts.  When OmniEngineArgs is instantiated with the dict, these
    # fields remain dicts instead of being reconstructed as dataclass objects.
    # Later, EngineArgs.create_engine_config() does
    #   self.structured_outputs_config.reasoning_parser = ...
    # which fails on a plain dict.  Reconstruct the dataclass here.
    soc = filtered_engine_args_dict.get("structured_outputs_config")
    if isinstance(soc, dict):
        from vllm.config import StructuredOutputsConfig
        filtered_engine_args_dict["structured_outputs_config"] = StructuredOutputsConfig(**soc)
    omni_engine_args = OmniEngineArgs(**filtered_engine_args_dict)
    # Multi-stage pipelines (qwen3_tts code2wav, etc.) set max_model_len
@@ -1164,8 +1164,6 @@ async def create_audio_generate(request: OpenAICreateAudioGenerateRequest, raw_r
                status_code=result.error.code if result.error else 400,
            )
        return result
    except (EngineGenerateError, EngineDeadError) as exc:
        return _create_engine_error_json_response(raw_request, exc)
    except Exception as e:
        raise HTTPException(status_code=HTTPStatus.INTERNAL_SERVER_ERROR.value, detail=str(e)) from e
@@ -2116,7 +2116,7 @@ class OmniOpenAIServingChat(OpenAIServingChat, AudioMixin):
                    index=output.index,
                    delta=DeltaMessage(role=role, content=audio_base64),
                    logprobs=None,
-                    finish_reason=output.finish_reason,
+                    finish_reason="stop",
                    stop_reason=output.stop_reason,
                    token_ids=(as_list(output.token_ids) if request.return_token_ids else None),
                )
@@ -2126,7 +2126,7 @@ class OmniOpenAIServingChat(OpenAIServingChat, AudioMixin):
                    message=ChatMessage(role=role, audio=audio_obj),
                    logprobs=None,
                    finish_reason="stop",
-                    stop_reason=output.stop_reason,
+                    stop_reason=None,
                )
            choices.append(choice_data)
        return choices
@@ -96,16 +96,7 @@ from vllm_omni.model_executor.models.hunyuan_image3.siglip2 import LightProjecto
 logger = init_logger(__name__)
-@support_torch_compile(
+@support_torch_compile
    dynamic_arg_dims={
        "input_ids": 0,
        # positions is (3, num_tokens) for mRoPE, so the token dimension is
        # the last dimension rather than dim 0.
        "positions": -1,
        "intermediate_tensors": 0,
        "inputs_embeds": 0,
    }
 )
 class HunyuanModel(HunYuanModel):
    def _split_qkv_weight(self, qkv: torch.Tensor):
        num_attention_heads = self.config.num_attention_heads