[Test] Fit Qwen2.5 online serving placement to its two-card allocation

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>

.agents/skills/3d-animation-short-generator/SKILL.md

@@ -0,0 +1,64 @@
+---
+name: 3d-animation-short-generator
+description: Develop stylized 3D narrative shorts through story, character and environment references, timed shot tables, storyboards, generation, and assembly. Use for coherent animated storytelling or its planning stages, not isolated image edits or photorealistic live action.
+metadata:
+  compatibility: File-based agent workflow with sibling h3-prompt-writing. Media generation and editing require available tools or a configured backend.
+---
+
+# 3D Animation Short Generator
+
+Read the [portable workflow](../h3-prompt-writing/references/portable-workflow.md).
+For a complete film, progress from story to references, shot table, text storyboard,
+clips, and assembly. Keep these as local artifacts, using existing user decisions
+and authorization. If only a planning stage is requested, deliver that stage.
+
+## Story and visual contract
+
+Capture premise, desired emotional outcome, duration, ratio, model, and audio mode.
+Use warm stylized 3D as a default only when style is unspecified: readable geometric
+silhouettes, designed hair clumps, tactile materials, soft skin shading where
+appropriate, expressive brows/eyes, and elastic posing. User-selected 2D or other
+styles override this preset rather than receiving accidental 3D texture.
+
+Create an active protagonist with a want, obstacle, meaningful choice, and payoff.
+Connect the ending to an earlier visual/emotional anchor. Use as many beats as the
+duration needs; do not pad a tiny story into an eight-beat formula. Dialogue should
+change a relationship or move the action, and use the user's requested language.
+
+## Reference design
+
+- Character references lock name, silhouette, proportions, face, hair, costume,
+  signature props, and color. Label identity in the production document; clean
+  rendering anchors should not carry labels that could leak into the video.
+- Scene references contain environments only unless the user requested a composite.
+  Record landmarks, lighting direction/time of day, entrances, exits, and prop state.
+- A reference board with multiple views is for design review. Prefer appropriate
+  clean views for model input so grids and pose sheets do not become final frames.
+- When an asset changes, update all affected prompts and clips, with a manifest
+  identifying the current revision.
+
+## Shot planning
+
+Read [shot-table-spec.md](references/shot-table-spec.md) for the six-column table
+and continuity checks. Then read
+[storyboard-guidelines.md](references/storyboard-guidelines.md) for the matching
+text storyboard and optional pencil previews. The text storyboard is the source
+of shot timing; preview art does not silently replace it.
+
+Vary shot size and elastic performance to serve the story. Camera tilts, close-ups,
+or exaggeration are choices for specific beats, not quotas. Track off-screen
+characters and spatial landmarks so they do not teleport back into later shots.
+
+## Generation and assembly
+
+Before rendering, read [model-selection.md](references/model-selection.md) and
+[fallback-policy.md](references/fallback-policy.md). Use the chosen model through
+available tools. Compile prompts using the H3 base or reference guide as appropriate.
+Remove storyboard-only labels, arrows, panel borders, and sketch styling from
+rendering inputs. Tie each clip to the current scene and character references.
+
+Read [qc-checklist.md](references/qc-checklist.md) for assembly and final review.
+Keep the planned shot order and audio continuity. Use a continuous score when
+requested, duck it under dialogue/SFX, and do not add subtitles unasked. Deliver
+the actual film or the requested planning package, with reproducible prompts and
+any unresolved identity, scene, or timing defects.

.agents/skills/3d-animation-short-generator/agents/openai.yaml

@@ -0,0 +1,4 @@
+interface:
+  display_name: "3D Animation Shorts"
+  short_description: "Plan stories, character anchors, shots, and edits"
+  default_prompt: "Use $3d-animation-short-generator to prepare the requested H3 prompts or video with the available tools."

.agents/skills/3d-animation-short-generator/references/fallback-policy.md

@@ -0,0 +1,21 @@
+# Targeted generation recovery
+
+Compare the failed result against the shot's actual identity/spatial anchors.
+Classify the failure: wrong input or API mapping, reference drift, overloaded
+action, timing, audio, or a render/export failure. Fix transport/configuration
+errors before spending another generation attempt.
+
+For drift, quote the relevant anchors and remove unrelated reference roles. For
+overloaded action, simplify or split the shot while preserving the requested total
+duration and handoffs. For a scene change, use the new scene reference and avoid
+conditioning on a previous scene when that would contradict the new setting.
+
+Use the retry/resource limit from the shared portable workflow. A persistent
+failure should lead to a concrete choice (simpler action, a supplied reference,
+an authorized alternate model, or an explicitly incomplete shot), not automatic
+provider switching or an endless loop. Never present a placeholder as a finished
+clip or hide an incorrect scene in final assembly.
+
+For optional pencil previews, reduce annotation or panel count. Use the text
+storyboard if visualization is not essential; do not rerender correct video
+solely because a human-review sketch changed.

.agents/skills/3d-animation-short-generator/references/model-selection.md

@@ -0,0 +1,21 @@
+# Model and format selection
+
+Honor an existing model choice. Otherwise prefer MiniMax H3 when available, and
+inspect the running backend for partitions, reference modes, audio, resolutions,
+duration limits, and long-video options. Do not assume hosted and local services
+have identical limits, quality, prices, or parameter names.
+
+If another model is requested, use its real documented input contract. A provider
+switch needs the user's authorization; never silently use a different service
+because a preset names it as a fallback. For an authorized mixed-model workflow,
+record model and settings per shot and check cross-shot style differences.
+
+Keep delivery resolution separate from native generation resolution. Record actual
+width/height, fps, frame count, and any crop, retiming, or upscaling. Do not claim
+native 2K when the video was generated at 768p and restored later.
+
+Compile shot action, camera, identity, scene, and audio into the chosen model's
+prompt format. For H3 use the bundled writing guides. Strip production labels and
+preview grids. Prompt for designed silhouettes, elastic action, material and
+lighting details when these belong to the user's 3D style; do not add text/UI just
+because a model can generate them.

.agents/skills/3d-animation-short-generator/references/qc-checklist.md

@@ -0,0 +1,19 @@
+# Assembly and review
+
+Use current clip revisions in shot-table order. Preserve useful dialogue and SFX;
+mix a continuous requested score underneath them. Inspect edits at sound bridges
+and dialogue boundaries, avoiding cut vowels and accidental audio gaps. Only add
+subtitles, music, or retiming consistent with the brief.
+
+Review identity, landmarks, screen direction, light/time of day, prop state,
+emotional payoff, intelligibility, SFX timing, and music balance. Look for leaked
+storyboard panels, sketch lines, arrows, labels, timecodes, or watermarks. Verify
+the complete file's duration, dimensions, fps, frame count, audio, and decode.
+
+Keep planning documents, references, raw clips, and final edits in distinguishable
+folders. Record the exact assets used in the final edit. After a character, scene,
+prompt, clip, or score is revised, update its downstream dependents and archive the
+previous result; do not silently combine stale and current versions.
+
+Report observed defects and what was repaired, including external editing or
+upscaling. A successful API response proves completion, not visual correctness.

.agents/skills/3d-animation-short-generator/references/shot-table-spec.md

@@ -0,0 +1,30 @@
+# Six-column shot table
+
+| Shot ID & Duration | Continuity Handoff | Reference Anchors (Spatial + Identity) | Hook Type | Shot Description (Per-Second Directives) | Audio & Dialogue Track |
+| --- | --- | --- | --- | --- | --- |
+
+Use stable IDs such as S01. Record incoming and outgoing image state, prop
+position, eyeline, motion, and emotional state in Continuity Handoff.
+
+Reference Anchors contains named landmarks and screen-relative positions; each
+character's screen position, facing and pose; recently exited characters' locations;
+lighting baseline and modifiers; and exact character/scene asset paths or IDs.
+An explicit scene/time change can replace these anchors; an accidental flip cannot.
+
+Hook Type names the shot's purpose: setup, reveal, reversal, joke, suspense,
+tenderness, chase, or callback. Do not force a joke or cut into a quiet scene.
+
+For each second or meaningful sub-second beat, cover action/pose/expression,
+camera, spatial/prop state, sound, and the handoff to the next interval. Use compact
+phrases rather than inventing movement in intentionally still moments. Include
+anticipation, squash/stretch, overshoot, and follow-through where appropriate.
+
+Keep the complete dialogue/narration script in Audio & Dialogue Track, with
+speaker, timing, delivery, and SFX. Distinguish off-screen narration from speaking
+characters; a visible listener should not lip-sync the narrator's words.
+
+Before rendering, check full time coverage, supported generation durations, a
+readable number of active characters, matching scene/identity references, and
+coherent row-to-row handoffs. Mark intentional cuts/time jumps. Fix contradictory
+states rather than using a generic continuity assurance. A visual shot may span
+multiple generation windows when the verified backend supports continuation.

.agents/skills/3d-animation-short-generator/references/storyboard-guidelines.md

@@ -0,0 +1,29 @@
+# Text storyboard and optional visualization
+
+Maintain one `storyboard.md` with a section per shot. Each section contains:
+
+1. Shot ID, title, duration, and narrative purpose.
+2. Exact scene and character reference IDs.
+3. Spatial anchors: landmarks, character positions/facing, off-screen characters,
+   key/fill/rim lighting, and important props.
+4. Incoming and outgoing continuity state.
+5. Time-ordered beats: pose/expression, camera, audio, and anchor changes.
+
+Use per-second or finer intervals only where meaningful; cover the complete shot.
+For narrated moments, specify whether a visible character speaks or only reacts.
+Optional ASCII staging diagrams can clarify screen direction but are not images
+to supply as rendering references.
+
+For heavy revision of a shot, extract its section to a named file and link it from
+the main document. Choose one authoritative version; reintegrate or update the
+link after revision so later generation cannot read stale content.
+
+Only generate pencil storyboards when requested or already authorized as previews.
+Use equal-sized time-ordered panels, with timecode, pose/expression, camera arrow,
+and audio/anchor notes. Suggested layouts: three panels in a strip, four in a 2×2
+grid, six in 2×3. More complex scenes may need fewer representative key poses.
+Keep shot/character/scene labels readable for human review, but never feed these
+annotated grids as clean final frames. The text timeline remains authoritative.
+
+If preview panels fail, simplify annotation or rely on text within the agreed
+scope. An optional visualization failure need not block a usable text storyboard.

.agents/skills/H3-SOURCES.md

@@ -0,0 +1,53 @@
+# MiniMax H3 skill provenance
+
+Upstream: https://github.com/MiniMax-AI/MiniMax-H3
+
+Source snapshot: `d21241f0a4b3acbb34c97dae47fa417b7065e438`, retrieved 2026-09-19.
+
+Source directory:
+https://github.com/MiniMax-AI/MiniMax-H3/tree/d21241f0a4b3acbb34c97dae47fa417b7065e438/skills
+
+The nine upstream skill names are retained. Each local skill corresponds to the
+same-named upstream `skills/<name>/` directory. Credit for the original creative
+workflows and templates belongs to MiniMax-AI and the upstream contributors.
+
+## Files and adaptation
+
+- `h3-prompt-writing/references/base-format.md` and `ref-format.md` are new,
+  condensed descriptions of the public prompt interface. They preserve necessary
+  field names and notation but do not copy the upstream guides or their examples.
+  The entrypoint adds a portable execution reference and clarifies clip/window
+  duration for verified long-video backends.
+- The eight style-specific `SKILL.md` files are rewritten portable adaptations of
+  the upstream workflows. Their reference documents retain the relevant creative
+  methods while removing host-specific execution assumptions.
+- The two `co-op-game-intro-generator/references/` templates retain the upstream
+  creative structure with an added portable contract and explicit user overrides.
+  Their instructions are normalized to English, and the video template expresses
+  visible UI copy as localized variables with English defaults. It also removes
+  an unrequested gender restriction and a fixed yellow/black environment
+  transformation.
+- The five 3D animation references are condensed adaptations of their upstream
+  counterparts: shot table, storyboards, model selection, fallback, and review.
+- Product, papercraft, collage, and music references extract reusable direction
+  from their corresponding upstream skill. The shared portable workflow, catalog,
+  and UI metadata are additions for this repository.
+
+No upstream `LICENSE`, `COPYING`, or `NOTICE` file was present in the Git tree at
+the pinned snapshot. Its README links to the MiniMax H3 Community License on the
+model release. Because that license has redistribution conditions, this package
+does not include the upstream prompt guides verbatim. This provenance record
+preserves attribution and does not assert that this repository's code license
+relicenses upstream material.
+
+## Runtime changes
+
+Canvas documents become local files and asset manifests; choice-card UI becomes
+the host's normal clarification mechanism when a decision is actually missing.
+Available generation/editing tools replace proprietary dispatch calls. Previous
+user choices and authorization persist across steps. Providers, prices, resolution
+support, and long-video features are checked rather than hardcoded as guarantees.
+
+User intent overrides style presets, including instrumental/no-text work, selected
+language, final morph shape, no-cut sequences, and custom player designs. Native
+generation and external processing remain separately identified in delivery.

.agents/skills/README.md

@@ -0,0 +1,96 @@
+# Portable MiniMax H3 Skills
+
+This directory contains one general prompt-writing skill and eight creative
+skills adapted from
+[MiniMax-AI/MiniMax-H3](https://github.com/MiniMax-AI/MiniMax-H3/tree/main/skills).
+See [H3-SOURCES.md](H3-SOURCES.md) for the source revision, attribution, and
+adaptation scope.
+
+## Skill catalog
+
+| Skill | Intended use |
+| --- | --- |
+| [h3-prompt-writing](h3-prompt-writing/SKILL.md) | Structured prompts for T2VA, I2VA, FL2VA, L2VA, and Ref2VA |
+| [minimalist-product-ad-generator](minimalist-product-ad-generator/SKILL.md) | Minimal product ads, isolated product references, copy, and pacing |
+| [3d-animation-short-generator](3d-animation-short-generator/SKILL.md) | 3D narrative shorts, character and environment design, shot tables, storyboards, and editing |
+| [papercraft-stop-motion-explainer](papercraft-stop-motion-explainer/SKILL.md) | Layered paper art, puppets, mechanisms, and miniature educational scenes |
+| [brand-promo-video-generator](brand-promo-video-generator/SKILL.md) | Brand promos, verified assets and claims, product interaction, and calls to action |
+| [music-video-subtitle-generator](music-video-subtitle-generator/SKILL.md) | Music videos, instrumental visual narratives, beat planning, and optional lyric typography |
+| [co-op-game-intro-generator](co-op-game-intro-generator/SKILL.md) | Two-player game menus, confirmation images, and animated world entry |
+| [paper-collage-explainer-generator](paper-collage-explainer-generator/SKILL.md) | Halftone photo collage, incremental paper assembly, and visual metaphors |
+| [handdrawn-live-video-generator](handdrawn-live-video-generator/SKILL.md) | Live action with rough luminous drawings, physical contact, continuous morphing, and delayed pursuit |
+
+## Usage
+
+Codex discovers repository skills from `.agents/skills/`. Clients that scan
+skills only when a session starts may need a new session. Claude Code reads the
+same files through the same-named relative symlinks in `.claude/skills/`. Other
+agents that support Markdown skills can load the relevant `SKILL.md` directly
+and follow its links to supporting references.
+
+Examples:
+
+```text
+Use $music-video-subtitle-generator to design a 30-second two-dimensional
+fluid-motion graphic for this instrumental track. Do not add lyrics or
+subtitles. Preserve leftward motion and initially return only a storyboard and
+three H3 prompts.
+```
+
+```text
+Use $co-op-game-intro-generator to create a two-player game intro. The players
+are Lin and Mei, the style is layered paper sculpture, and the title is
+Together. Create the confirmation image first and wait for my approval before
+generating video.
+```
+
+## MV smoke test
+
+Test `music-video-subtitle-generator` together with `h3-prompt-writing` using a
+user-owned song with timestamped lyrics and an original adult-character
+reference image:
+
+```text
+Use $music-video-subtitle-generator and $h3-prompt-writing to turn this song
+into a historical-romance music video about a woman and a man. Intercut
+narrative acting with the woman's singing performance, do not display lyric
+text, split the work according to the backend's verified clip duration, and
+preserve the complete original song in the final master.
+```
+
+Verify that the global timeline matches the source audio duration, lyrics remain
+verbatim, and each generation window records its incoming state, outgoing state,
+and local timing. Ref2VA prompts must preserve the six-section field order and
+resolve every reference label to a real input. The assembled video must decode
+completely, and its final audio must come from the supplied source rather than a
+model reconstruction.
+
+When moving the collection to another repository or a user-level skill folder,
+copy all nine skill directories and preserve their sibling layout. The eight
+creative skills share `h3-prompt-writing/references/portable-workflow.md` and the
+H3 format guides, so copying only one `SKILL.md` is insufficient. Keep each
+skill's `references/` and `agents/` directories with it. `agents/openai.yaml`
+provides optional Codex UI metadata and does not restrict the workflow to an
+OpenAI-based agent or require a dedicated plugin.
+
+## Execution boundaries
+
+Prompt writing, storyboarding, and planning require only an agent that can read
+and write files. Actual image, video, speech, music, and editing work uses tools
+or services available in the current environment. When a capability is missing,
+deliver the complete usable production package and identify the unfinished media
+step.
+
+The user's style, language, assets, audio, model selection, and existing
+authorization take priority. An MV template does not add lyrics to an
+instrumental track, and a clean motion-graphics request does not inherit film
+grain. A prompt-only request does not start generation. When the user requests
+approval of a confirmation image before rendering, preserve that checkpoint.
+
+Treat vLLM-Omni long-video extensions as capabilities of the checked-out branch
+and active backend. Do not present a development branch's continuation parameters
+as a universal H3 API. Preserve real requests and raw outputs, distinguish model
+results from post-production and upscaling, and inspect actual media properties.
+
+These skills contain no model weights and do not automatically start a service
+or invoke a paid generation endpoint.

.agents/skills/brand-promo-video-generator/SKILL.md

@@ -0,0 +1,66 @@
+---
+name: brand-promo-video-generator
+description: Plan, prompt, or produce brand, product, website, app, and service promotional shorts using traceable assets, accurate claims, timed beats, and clear calls to action. Use for launch and campaign videos, not invented product evidence or long narrative films.
+metadata:
+  compatibility: File-based workflow with sibling h3-prompt-writing and available research, media generation, inspection, and editing tools.
+---
+
+# Brand Promo Video Generator
+
+Read the [portable workflow](../h3-prompt-writing/references/portable-workflow.md).
+Use supplied brand assets, source links, and existing session decisions. No
+proprietary tool allowlist or canvas is required. A completed promo requires real
+generation/editing capability; otherwise deliver an explicitly incomplete production
+package with finished prompts and a concrete generation handoff.
+
+## Establish facts and assets
+
+Inspect product images, UI screenshots, logo exports, typography, colors, and
+campaign material. Resolve duration, ratio, audience, campaign focus, exact claims,
+CTA, copy language, narration, and required deliverable from context. Ask for
+missing identity-bearing assets when they are essential; do not fabricate logos,
+product UI, screenshots, results, or performance claims.
+
+Build a compact truth sheet from user-supplied originals and official sources.
+Record asset ID, role, local path, source URL/note, and verification status. Do not
+confuse a retrieved search thumbnail with an original brand asset. Distinguish
+verified facts from clearly labeled concept copy. Preserve the requested language;
+chat language alone does not justify translating supplied campaign text.
+
+## Design a product-specific story
+
+Choose a spine that exposes real cause and effect:
+
+- Software: user intent → actual interaction → useful output → evidence → CTA.
+- Physical product: reveal → interaction → mechanism/detail → use → benefit → CTA.
+- Service: situation → process → evidence → outcome → promise.
+- Image-led brand: authentic imagery → repeated visual motif → emotional payoff.
+
+Plan time/frame ranges, asset IDs, one main action, product evidence, exact copy,
+readable hold, color state, camera, transition, and rhythm. Use the actual output
+fps for frame timing. Five to eight major beats can fit a 15-second promo, but
+readability takes priority over a fixed beat count.
+
+Let product edges, UI movement, light, cursor paths, or matched geometry motivate
+transitions. Use a few energy peaks and deliberate settling moments. Preserve logo
+clear space and readable screenshots. Avoid invented HUDs or metrics that look like
+product proof. Summarize the truth sheet and plan before rendering; continue under
+existing authorization unless the user requested approval at that stage.
+
+## Produce and inspect
+
+Use authentic still/UI plates for exact content. Generated material can supply
+conceptual scenes, atmosphere, and transition geometry with clear provenance.
+Compile actual H3 references and shots using the appropriate bundled guide.
+Check backend resolution, duration, and audio support before calling it.
+
+Use native instrumental/SFX direction when appropriate; do not generate a second
+score automatically. Separate narration or music is useful when requested for
+control or remixing. Preserve original SFX during assembly when they matter, and
+duck music under speech. Do not add subtitles unless requested.
+
+Check claims and exact product names against sources, inspect logo geometry and
+copy, verify real UI states, and review transitions, aspect ratio, duration, audio,
+and decoding. For wrong identity-bearing content, use the correct supplied/source
+asset rather than polishing an imitation. Deliver the video, prompts/timeline,
+asset manifest, and any remaining factual or visual limitations.

.agents/skills/brand-promo-video-generator/agents/openai.yaml

@@ -0,0 +1,4 @@
+interface:
+  display_name: "Brand Promo Videos"
+  short_description: "Build brand promos from verified assets and claims"
+  default_prompt: "Use $brand-promo-video-generator to prepare the requested H3 prompts or video with the available tools."

.agents/skills/co-op-game-intro-generator/SKILL.md

@@ -0,0 +1,73 @@
+---
+name: co-op-game-intro-generator
+description: Design or generate a two-player co-op game menu and opening animation from player names, title, style, and optional identity references. Use for character-led game intros, not playable game implementation or complex multi-page interfaces.
+metadata:
+  compatibility: File-based workflow with sibling h3-prompt-writing and bundled image/video templates. Rendering uses available generation tools.
+---
+
+# Co-op Game Intro Generator
+
+Read the [portable workflow](../h3-prompt-writing/references/portable-workflow.md).
+Before writing a confirmation-image prompt, read
+[h3-confirmation-image-template.md](references/h3-confirmation-image-template.md).
+Before writing the video prompt, read
+[h3-video-prompt-template.md](references/h3-video-prompt-template.md).
+Both templates are required package resources, not remote tools.
+
+## Resolve player and style bindings
+
+Reuse or collect the two player names, game title, visual style, desired duration,
+ratio, and optional identity references. The template defaults to a 15-second,
+16:9 menu-to-world sequence; adapt it to explicit user requirements. Do not assume
+the players' gender, body shape, outfits, or equipment from a generic example.
+
+Use each portrait only for recognizable identity anchors: face silhouette, hair,
+glasses, proportions, and distinguishing traits. Reinterpret rendering, skin
+texture, lighting, and costume in the selected visual style. If a portrait or body
+comparison reference is absent, write a text design rather than inventing an asset
+label. Maintain player-to-name mapping across all references and frames.
+
+## Confirmation image
+
+Fill the template's style, palette, characters, title, and name variables. Keep
+the two characters central, player cards toward the upper left, menu on the right,
+and the continue action as the main interaction focus. The title establishes
+identity without competing with that focus. Buttons, icons, typography, and all
+player/equipment panels share one palette and style. Keep menu labels on one line.
+
+The style changes material, shapes, fonts, and decoration; it should not randomly
+rearrange the menu hierarchy. User-requested layout or ratio changes take priority
+over template defaults. Preserve the exact supplied title/names and requested copy.
+
+For image-only/prompt-only work, stop at that deliverable. For video production,
+generate and inspect the confirmation image if within scope. Honor a request to
+approve the image first; otherwise use the established end-to-end authorization
+and continue after reporting the selected image and fixing material defects.
+
+## Video event plan
+
+Fill the video template from the current image and player bindings. Its baseline
+sequence is menu → first player's light equipment → second player's heavy equipment
+→ shared confirmation → world loading → both enter the world. Equipment, world,
+body contrast, and timing are editable design choices; preserve requested identity
+and style instead of forcing a cyberpunk example onto another setting.
+
+Bind UI reference, each actual portrait, and any real body-comparison image to
+distinct roles. Compile this creative template into the H3 mode guide's fields;
+the mixed-language template is not itself an API request. Preserve literal visible
+UI text in its required language, and remove all unfilled variables before dispatch.
+Do not treat a composition reference as an exact first-frame anchor unless the
+selected backend actually supports that role.
+
+## Review and repair
+
+Check both names, exact menu strings, layout, player identity, side assignment,
+equipment distinction, and the transition into the requested world. Match UI
+hover/click, mechanical locks, loading, and footsteps to the visible events.
+For identity swaps, strengthen name/position/reference bindings; for weak style,
+revise palette, material, character, and typography together. Reduce excessive
+UI text if necessary without silently changing required player names.
+
+Deliver the requested prompts/assets or the inspected intro video, with reference
+roles and any text/timing limitations. Missing templates block template execution;
+missing generation capability still permits a complete prompt handoff.

.agents/skills/co-op-game-intro-generator/agents/openai.yaml

@@ -0,0 +1,4 @@
+interface:
+  display_name: "Co-op Game Intros"
+  short_description: "Design two-player menus and world-entry animation"
+  default_prompt: "Use $co-op-game-intro-generator to prepare the requested H3 prompts or video with the available tools."

.agents/skills/co-op-game-intro-generator/references/h3-confirmation-image-template.md

@@ -0,0 +1,397 @@
+# H3 Confirmation Image Template
+
+## Portable adaptation
+
+This is a creative template, not a tool call. User requirements override its
+default ratio, layout, poses, and wording. Fill every relevant variable from the
+brief; set `{character_count}` to two for this skill. When portraits are absent,
+replace portrait references with explicit character designs. Do not claim to
+reference an image that was not supplied. Use the image-generation capability
+available in the host environment; no specific image provider is required.
+
+## Core Rule
+Generate a polished 16:9 landscape confirmation image for a cooperative game main menu.
+The image consists of two independent systems:
+• UI Framework (fixed)
+• Visual Style (dynamic)
+The UI Framework defines the layout, composition, information hierarchy, interaction logic, reading flow, menu structure, and readability. These elements must remain unchanged.
+{visual_style} only controls the artistic appearance, including rendering medium, materials, color language, lighting mood, environment style, character design, costume language, UI materials, icon language, typography appearance, decorative motifs, and overall atmosphere.
+Changing {visual_style} should feel like applying a different art direction to the same professionally designed game menu, never redesigning the interface itself.
+If any default description conflicts with {visual_style}, prioritize {visual_style} while preserving the UI framework.
+
+---
+
+## Design Goal
+Create a premium commercial console game main menu mockup in 16:9 landscape format. The image should combine the quality of a polished game promotional artwork with the clarity of a professional game UI. Characters and interface must feel naturally integrated, with strong visual hierarchy, excellent readability, clean composition, and a clear interaction focus.
+
+---
+
+## Overall Style
+Follow the user-selected visual style: {visual_style}.
+The selected style controls the rendering medium, illustration style, materials, palette language, environment, character appearance, outfit design, UI appearance, icon language, typography treatment, decorative elements, and atmosphere.
+All visual elements must belong to one unified artistic language.
+Do not mix incompatible visual styles within the same image.
+The UI framework, layout, hierarchy, composition, and interaction logic must remain consistent regardless of style.
+
+---
+
+## Color Palette
+Derive the entire color system from {visual_style}.
+Use:
+• {main_background_color} as the primary background color.
+• {ui_body_color} as the main UI color.
+• {text_color} as the primary text color.
+• {functional_accent_color} as the interaction highlight color.
+• Red only for warning, danger, or exit actions.
+Limit the overall palette to five primary colors.
+All buttons, player cards, icons, borders, highlights, decorative elements, typography, and visual effects must follow the same color system.
+Avoid introducing unrelated colors outside the selected palette.
+
+---
+
+## Composition
+Use a fixed 16:9 landscape layout.
+Place {character_count} playable characters in the center as the visual subject.
+The UI surrounds the characters without blocking them.
+Place the player information cards in the upper-left corner.
+Place the main menu vertically on the right.
+Place a decorative horizontal strip along the bottom, adapting naturally to {visual_style} (such as ribbon, warning strip, wooden plank, stone slab, energy bar, scroll, etc.).
+Keep a clear Z-shaped reading flow:
+Player Cards → Characters → Menu → Continue Button.
+The Continue button must always remain the primary visual focus.
+Maintain generous negative space and excellent readability.
+
+---
+
+## Background
+Generate a background naturally derived from {visual_style}.
+The background supports the interface rather than competing with it.
+Keep enough clean negative space behind all important UI elements.
+Use subtle textures, environmental details, and decorative motifs only where appropriate for the selected style.
+Avoid excessive visual noise, dense decoration, heavy dirt, scratches, paint splatter, or distracting textures.
+The overall background should feel polished, premium, modern, and suitable for a commercial game menu.
+
+---
+
+## Character Style
+All characters strictly follow {visual_style}.
+The selected style determines:
+• rendering medium
+• body proportion
+• facial stylization
+• costume language
+• material treatment
+• lighting response
+• animation style
+• overall character quality
+Only the character identity anchors and pose remain fixed. Facial rendering, eye design, nose simplification, mouth shape, skin treatment, expression style, and head proportion must be fully reinterpreted in {visual_style}.
+No default character style may override {visual_style}.
+
+---
+
+## Character A / PLAYER 1
+### Identity
+Use character reference image 1 only for identity mapping.
+The reference image donates identity anchors only: recognizable face silhouette, hairstyle, glasses if present, relative facial proportions, and distinctive personal traits.
+It does not donate photographic realism, real skin texture, real-world lighting, camera quality, or the original image style.
+Do not lose the character's identity anchors, but the face must be redesigned into {visual_style}.
+All facial rendering, eye design, nose simplification, mouth shape, skin treatment, expression style, and head proportion must follow {visual_style}.
+Do not swap identity with other characters.
+
+### Appearance
+Character rendering, expression, outfit, materials, and accessories are fully derived from {visual_style} while maintaining a recognizable playable-game-character silhouette.
+
+### Action
+Character A sits cross-legged on the ground.
+Both hands support the body on the floor.
+The upper body leans slightly backward.
+Character A looks upward toward the camera.
+
+---
+
+## Character B / PLAYER 2
+### Identity
+Use character reference image 2 only for identity mapping.
+The reference image donates identity anchors only: recognizable face silhouette, hairstyle, glasses if present, relative facial proportions, and distinctive personal traits.
+It does not donate photographic realism, real skin texture, real-world lighting, camera quality, or the original image style.
+Do not lose the character's identity anchors, but the face must be redesigned into {visual_style}.
+All facial rendering, eye design, nose simplification, mouth shape, skin treatment, expression style, and head proportion must follow {visual_style}.
+Do not swap identity with other characters.
+
+### Appearance
+Character rendering, expression, outfit, materials, and accessories are fully derived from {visual_style} while maintaining a recognizable playable-game-character silhouette.
+
+### Action
+Character B sits cross-legged on the ground.
+Both hands rest naturally in front of the legs.
+The upper body leans slightly forward.
+Character B looks directly toward the camera.
+
+---
+
+## Lighting
+Lighting follows {visual_style} while preserving clear readability.
+Use a primary top light, secondary fill light, soft ambient bounce, contact shadows, gentle rim lighting, and high-quality global illumination where appropriate.
+Faces should remain clearly visible.
+Characters must integrate naturally into the environment while maintaining strong separation from the background.
+
+---
+
+## Game UI
+Create a professional commercial console game main menu.
+The menu structure, interaction hierarchy, spacing, composition, reading flow, and information architecture remain fixed.
+{visual_style} only changes the artistic appearance of the interface.
+The selected style may affect:
+• UI materials
+• surface textures
+• border treatment
+• corner shape
+• decorative motifs
+• color rendering
+• lighting treatment
+• icon language
+• typography appearance
+• interaction effects
+It must NOT change:
+• menu hierarchy
+• button order
+• button placement
+• player-card placement
+• composition
+• interaction logic
+• information architecture
+• readability
+The entire interface should feel like one professionally designed game UI system rather than separate elements assembled together.
+
+---
+
+## UI Visual Language
+Derive the complete UI appearance from {visual_style}.
+The selected style determines:
+• button materials
+• border language
+• panel materials
+• decorative details
+• edge treatment
+• visual weight
+• highlight style
+• interaction feedback
+• shadow treatment
+• texture language
+Possible materials include wood, stone, paper, leather, cloth, glass, crystal, hologram, neon, metal, ceramic, carved surfaces, painted surfaces, magical energy, pixel blocks, watercolor paper, ink brush, comic graphics or any other material naturally belonging to {visual_style}.
+Every UI element must share the same material language and artistic style.
+
+---
+
+## Button Layout Rules
+Place all menu buttons vertically on the right side.
+Maintain a consistent vertical rhythm.
+Every button uses the same design family.
+Button width automatically adapts to text length.
+Keep all menu titles on a single line.
+No automatic wrapping.
+No stacked text.
+Text remains horizontally centered.
+Keep consistent padding.
+Maintain equal spacing between buttons.
+Avoid visual clutter.
+
+---
+
+## Continue Button
+The Continue button is always the primary visual focus.
+Make it visually dominant using the emphasis method naturally belonging to {visual_style}.
+Possible emphasis methods include:
+• larger size
+• stronger contrast
+• brighter color
+• unique material
+• distinctive border
+• lighting
+• glow
+• engraving
+• magical energy
+• holographic effect
+• embossed frame
+• animated visual cue
+Do not force glow or neon if they conflict with {visual_style}.
+The Continue button should immediately attract the player's attention.
+
+---
+
+## Start New Game Button
+Place above Continue.
+Use the standard secondary button appearance.
+Keep the visual language consistent with the selected style.
+Add one small style-appropriate icon if suitable.
+Maintain single-line text.
+
+---
+
+## Settings Button
+Use the standard secondary button appearance.
+Use one settings icon naturally derived from {visual_style}.
+Maintain consistent proportions, spacing, and hierarchy.
+
+---
+
+## Exit Game Button
+Use the standard secondary button appearance.
+Use red only as the danger indicator.
+Red should remain an accent rather than the dominant color.
+Add one style-appropriate exit icon if suitable.
+
+---
+
+## Player Cards
+Create an upper-left player information system for {character_count} players.
+The appearance follows {visual_style}.
+Card materials, borders, corners, textures, shadows, decorations, and icons are automatically derived from the selected style.
+Each player card contains:
+• Player label
+• Player nickname
+• READY status
+Player names:
+PLAYER 1
+{player1_name}
+PLAYER 2
+{player2_name}
+READY always uses the functional accent color.
+The cards should remain highly readable and visually connected to the rest of the interface.
+
+---
+
+## Icon System
+Derive all icons from {visual_style}.
+Use one unified icon family throughout the entire interface.
+Possible icon languages include:
+• minimal outline
+• solid filled
+• geometric
+• comic
+• pixel
+• engraved
+• carved
+• painted
+• brush
+• rune
+• glyph
+• holographic
+• crystal
+• paper-cut
+• origami
+• embossed
+• low-poly
+• flat design
+• stylized 3D
+All icons must share:
+• the same artistic style
+• the same material language
+• the same perspective
+• the same lighting treatment
+• the same stroke language
+• the same color hierarchy
+Icons should remain simple, recognizable, and functional.
+Each interface block may contain at most one horizontal row of icons.
+No wrapping.
+No vertical stacking.
+Icons should support the interface rather than dominate it.
+
+---
+
+## Typography
+Typography hierarchy remains fixed.
+Typography appearance follows {visual_style}.
+The selected style controls:
+• font family
+• decorative treatment
+• texture
+• material
+• outline
+• lighting
+• shadow
+• wear pattern
+Maintain:
+• excellent readability
+• strong visual hierarchy
+• single-line menu buttons
+• consistent spacing
+• high contrast
+Button width should always adapt to text length.
+Never allow menu titles to wrap onto multiple lines.
+
+---
+
+## Game Title
+Display the game title:
+{game_title}
+The title should become the primary branding element of the menu.
+Its material, decorative style, typography, and rendering all follow {visual_style}.
+Avoid imitating existing commercial game logos.
+Create an original title treatment inspired by the selected style.
+Keep the title highly readable.
+Avoid random symbols or meaningless letters.
+
+---
+
+## Interaction States
+Represent the interface as an active game menu.
+Buttons may display appropriate interaction feedback naturally belonging to {visual_style}.
+Possible interaction feedback includes:
+• hover
+• pressed state
+• magical glow
+• energy pulse
+• holographic highlight
+• carved illumination
+• ink spread
+• brush bloom
+• metallic reflection
+• paper emboss
+• animated border
+• pixel flashing
+Only use interaction effects that naturally belong to the selected style.
+Avoid forcing modern UI effects into incompatible artistic styles.
+
+---
+
+## Rendering
+Generate a polished AAA-quality commercial game menu artwork.
+Characters, environment, UI, typography, icons, and decorative elements should feel like one cohesive production.
+Maintain premium rendering quality.
+Strong visual hierarchy.
+Professional commercial presentation.
+High readability.
+Balanced composition.
+Clean finish.
+
+---
+
+## Negative Constraints
+No split screen.
+No duplicated characters.
+No identity swap.
+No loss of identity anchors. No photorealistic face unless {visual_style} explicitly requires photorealism. No realistic photo face pasted onto a stylized body. No mismatched face style.
+No hairstyle changes.
+No unreadable UI.
+No distorted buttons.
+No overlapping interface.
+No random text.
+No random symbols.
+No watermark.
+No copied game logo.
+No branded interface.
+No inconsistent icon styles.
+No mixed UI materials.
+No excessive decoration.
+No overwhelming background.
+No visual clutter.
+No broken layout.
+No low readability.
+
+---
+
+## Medium Lock
+Render as a polished commercial game key art combined with a professional console game main menu mockup.
+The interface remains fixed.
+Only the artistic appearance changes according to {visual_style}.

.agents/skills/co-op-game-intro-generator/references/h3-video-prompt-template.md

@@ -0,0 +1,249 @@
+# H3 Video Prompt Template
+
+## Portable adaptation
+
+This is a creative planning template. User requirements override its default
+15-second timeline, body proportions, equipment, environment, ratio, and palette.
+Retiming must preserve readable event order. Use only supplied reference assets:
+omit `{height_ref}` or identity-image bindings when absent and replace their role
+with text design constraints. Compile the filled plan into the H3 base or Ref2VA
+format before rendering; preserve exact visible UI strings in the user's language.
+
+Use available tools and the actual backend's parameter schema. Audio directions
+describe desired content rather than asserting that a particular API supports it.
+
+## Template completion
+
+Resolve the palette variables `{background_color}`, `{ui_color}`, `{text_color}`,
+`{functional_accent_color}`, and `{power_accent_color}` from the selected style.
+Keep the five values distinct enough for readable UI contrast. Use the power
+accent for danger, exit, and warning states unless the user supplies a separate
+semantic-color requirement.
+
+Fill visible copy in the user's requested language. The following English values
+are defaults only, not mandatory output:
+
+- `{player1_label}` / `{player2_label}`: `PLAYER 1` / `PLAYER 2`
+- `{ready_label}`: `READY`
+- `{start_label}`, `{continue_label}`, `{settings_label}`, `{exit_label}`:
+  `START NEW GAME`, `CONTINUE`, `SETTINGS`, `EXIT GAME`
+- `{player1_equipment_label}`: `RIGHT ARM EQUIPMENT`
+- `{player1_option_a}` / `{player1_option_b}`: `PHANTOM GRIP` / `CHRONOS CLAW`
+- `{player2_equipment_label}`: `ARMAMENT CUSTOMIZATION`
+- `{hand_label}`, `{forearm_label}`, `{elbow_label}`, `{upper_arm_label}`:
+  `HAND`, `FOREARM`, `ELBOW`, `UPPER ARM`
+- `{confirm_label}` / `{loading_label}`: `CONFIRM CONFIG` / `LOADING`
+
+Before compiling the rendering prompt, search the filled template for every
+remaining token enclosed in braces. Replace required values. For an absent optional
+reference, remove its binding and rewrite every dependent clause as a text-only
+constraint. Do not dispatch a prompt while any template token remains.
+
+## Prompt principle
+Use the same method as the GPT confirmation-image prompt:
+
+**Fixed video event framework + dynamic user-style fill + locked character identity + palette-linked UI system.**
+
+The video prompt must not blindly preserve the source prompt's default style. It must preserve the timeline, UI events, player positions, equipment logic, and text structure, while dynamically rewriting visual treatment, palette, character rendering, lighting, UI surface, icons, and city-world style according to the user's selected style.
+
+## Priority order
+1. User-selected style: {visual_style}
+2. Confirmed image / UI reference: {ui_ref}
+3. PLAYER 1 identity reference: {player1_ref}
+4. PLAYER 2 identity reference: {player2_ref}
+5. Height/body comparison reference: {height_ref}
+6. Fixed source video event framework
+
+## Reference roles
+- {ui_ref}: confirmed first image. Use it to lock UI layout, menu hierarchy, color system, typography scale, button structure, character-game integration, and overall composition logic.
+- {player1_ref}: PLAYER 1 identity anchor. Lock exact face, hairstyle, glasses if present, facial proportions, body identity, and nickname mapping to {player1_name}.
+- {player2_ref}: PLAYER 2 identity anchor. Lock exact face, hairstyle, facial proportions, body identity, and nickname mapping to {player2_name}.
+- {height_ref}: body comparison anchor. Lock the visible contrast between the two players and prevent identical body proportions.
+
+## Global style baseline
+The overall art direction must prioritize the user's selected {visual_style}.
+Preserve these fixed qualities: a game main-menu interface, premium game-trailer
+finish, deep integration between the UI and characters, modern commercial game
+UI design, strong visual impact, clean composition, and restrained decoration.
+Derive the remaining visual treatment from {visual_style}: character rendering,
+expressions, costume language, color system, lighting temperature, UI materials,
+button icons, typography texture, and the world revealed after loading.
+
+## Palette system
+Derive the complete video palette from {visual_style}, with linked UI colors:
+
+- Use {background_color} as the primary background and world color.
+- Use {ui_color} as the main UI color.
+- Use {text_color} as the text color.
+- Use {functional_accent_color} as PLAYER 1's functional accent color.
+- Use {power_accent_color} as PLAYER 2's power accent and for danger, exit,
+  and warning states.
+- Limit the complete palette to five colors.
+- Use a vivid, modern, high-contrast color language consistent with
+  {visual_style}.
+
+Menus, equipment panels, player cards, buttons, the HUD, loading bars, icons,
+and text must share this color system. Do not introduce unrelated colors.
+
+## Character identity and style lock
+PLAYER 1 follows the facial identity in {player1_ref}. Preserve the face,
+facial proportions, hairstyle, glasses when present, personal identity, and the
+mapping to the nickname {player1_name}. Adapt expression, rendering, clothing,
+and mechanical equipment to {visual_style}. PLAYER 1 always stays on the left,
+with a taller, slender, agile build. Equipment uses the functional accent color,
+and the mechanical claw is light, narrow, and flexible.
+
+PLAYER 2 follows the facial identity in {player2_ref}. Preserve the face,
+facial proportions, hairstyle, personal identity, and the mapping to the
+nickname {player2_name}. Adapt expression, rendering, clothing, and mechanical
+equipment to {visual_style}. PLAYER 2 always stays on the right, with a shorter,
+broader, power-oriented build. Equipment uses {power_accent_color}, and the
+mechanical fist is broad, heavy, and visibly weighty.
+
+Do not swap character identities, merge faces, swap nicknames, or converge the
+two body types.
+
+## Fixed timeline framework
+
+### [0-2 seconds] - Two-player main menu
+
+Composition and camera: A high-angle extreme wide shot follows the composition
+logic of {ui_ref}. The camera tilts slightly downward and slowly pushes in.
+
+Visuals: PLAYER 1 ({player1_name}) and PLAYER 2 ({player2_name}) sit side by
+side at the center. PLAYER 1 remains on the left and PLAYER 2 on the right. They
+look up toward the camera with only natural breathing, blinking, and subtle body
+movement.
+
+UI: The upper-left player card displays `{player1_label}`, `{player1_name}`, and
+`{ready_label}` exactly as filled. The second card in the upper-center or
+upper-left two-card system displays `{player2_label}`, `{player2_name}`, and
+`{ready_label}` exactly as filled. The vertical menu on the right displays
+`{start_label}`, `{continue_label}`, `{settings_label}`, and `{exit_label}`
+exactly as filled. `{continue_label}` is the visual center and primary
+highlighted button.
+
+Dynamic style fill: Adapt the menu background, ground texture, button shapes,
+icons, typography, borders, glow, and sticker treatment to {visual_style}, while
+preserving the layout and hierarchy in {ui_ref}.
+
+Sound: Menu ambience, subtle UI-hover sounds, and a low electronic atmosphere
+before the click. Ignore this direction when generating silent video.
+
+### [2-4 seconds] - PLAYER 1 right-arm configuration
+
+Composition and camera: In a medium shot, the camera pushes smoothly from the
+main menu toward PLAYER 1's right arm. PLAYER 2 remains visible and stable in the
+background.
+
+UI: The right-side menu contracts and slides away. A panel with functional-accent
+guide lines slides in from the left and displays `{player1_label}` and
+`{player1_equipment_label}` exactly as filled. The equipment list first
+highlights `{player1_option_a}`, then moves the selection to
+`{player1_option_b}`.
+
+Action: PLAYER 1's right cuff opens automatically and a lightweight mechanism
+unfolds beneath the forearm. The fingers spread as long claw-like joints slide
+into position and lock one by one, briefly revealing fine wiring, miniature
+pistons, and metal connectors. Functional-accent LEDs illuminate sequentially
+when configuration completes.
+
+Dynamic style fill: Adapt the mechanism, panel material, icons, lines, locking
+animation, and light effects to {visual_style}. Keep the result light, precise,
+and flexible, proportioned to PLAYER 1 without changing the face, hairstyle, or
+main clothing design.
+
+Sound: Precise mechanical unfolding, light UI switching tones, and small locking
+clicks.
+
+### [4-7 seconds] - PLAYER 2 heavy-arm configuration
+
+Composition and camera: A medium shot tracks smoothly between the players and
+arcs toward PLAYER 2's left side. PLAYER 1 remains in the background, quietly
+examining the configured mechanical hand.
+
+UI: A {power_accent_color} panel slides in and displays `{player2_label}` and
+`{player2_equipment_label}` exactly as filled. Its grid contains `{hand_label}`,
+`{forearm_label}`, `{elbow_label}`, and `{upper_arm_label}`, with the selection
+moving quickly but legibly through all four.
+
+Action: PLAYER 2's left sleeve opens in sections. Heavy forearm plates spring
+outward, the previous components detach, and new armor slides into place along
+guide rails. A thick mechanical bearing replaces the elbow joint, and the broad
+mechanical hand reassembles and locks. Thick wiring, hydraulic pistons, and a
+dark metal skeleton appear briefly during the replacement. A restrained warm
+indicator illuminates as each component locks.
+
+Dynamic style fill: Adapt the heavy arm, UI panel, component icons, materials,
+and light effects to {visual_style}. Keep it broad, weighty, and powerful, in
+clear contrast with PLAYER 1's lightweight claw.
+
+Sound: Low motor movement, heavy mechanical engagement, and weighty locking
+feedback.
+
+### [7-8.5 seconds] - Shared configuration confirmation
+
+Composition and camera: Pull back smoothly to a medium two-player composition,
+with PLAYER 1 on the left and PLAYER 2 on the right.
+
+UI: The two equipment panels converge at center to form a shared button that
+displays `{confirm_label}` exactly as filled. Adapt its border, glow, icon, and
+sticker treatment to {visual_style}, while keeping the hierarchy and text legible.
+
+Action: The cursor clicks the button. A functional-accent energy pulse travels
+through PLAYER 1's mechanical claw, while a {power_accent_color} pulse travels
+through PLAYER 2's fist. All UI panels contract inward and disappear. Both
+players uncross their legs and adjust their posture: PLAYER 1 lifts one knee
+lightly and articulates the claw fingers in sequence; PLAYER 2 plants one foot
+firmly and slowly closes the heavy fist.
+
+Sound: Confirmation tone, two-color energy pulses, and UI contraction.
+
+### [8.5-10 seconds] - Shared world loading
+
+Composition and camera: A wide shot reveals a shared loading bar at the bottom.
+
+UI: The loading bar displays `{loading_label}` exactly as filled and fills
+rapidly from 0% to 100%. Its left half uses {functional_accent_color} and its
+right half uses {power_accent_color}. Adapt the HUD and loading-bar shape,
+border, texture, and typography to {visual_style}, while keeping them legible.
+
+Environment transformation: Continuously transform the confirmed
+{visual_style} menu background into a game world in the same style. Menu strips,
+color blocks, and patterns become corresponding roads, structures, and world
+elements. Preserve the confirmed palette rather than forcing a yellow-and-black
+or cyberpunk city.
+
+Key constraint: The transformation remains continuous and natural, with no hard
+cut, smoke occlusion, or identity change.
+
+Sound: Rising loading tone and an ambience transition from menu to game world.
+
+### [10-15 seconds] - Two-player world entry
+
+Composition and camera: Move from an extreme wide shot into a third-person
+tracking view. At 100% loading, both characters stand simultaneously. The camera
+descends smoothly and arcs behind them into a stable two-player cooperative view.
+
+World style: Generate the complete world from {visual_style}. Preserve the game
+opening structure: dense buildings, roads, signal lights, signs, crowds or other
+moving background elements, passing vehicles, overhead wires, industrial pipes,
+and a distant skyline. Materials, architecture, signage, lighting, and motion
+must follow {visual_style}. Do not let the source template's cyberpunk default
+override the user unless the selected style is cyberpunk.
+
+Character relationship: Clearly show the players from behind and preserve their
+body contrast. PLAYER 1 is on the left, tall and slender, with the lightweight
+claw hanging naturally. PLAYER 2 is on the right, shorter and broader, with the
+heavy fist slightly raised. PLAYER 1 steps forward first, PLAYER 2 follows, and
+they enter the street side by side.
+
+HUD: The HUD fades in. A mini-map appears in the upper right. Separate status
+bars labeled `{player1_name}` and `{player2_name}` appear in the lower left.
+The first uses {functional_accent_color}; the second uses {power_accent_color}.
+A shared objective marker appears in the street ahead.
+
+Sound: City ambience, distant vehicles, footsteps, and a HUD fade-in cue.
+
+## Negative constraints
+No extra player, duplicated character, identity or username swapping, merged bodies, unintended face/hair changes, missing player, unrequested split screen or cuts, random camera shake, floating body parts, gruesome dismemberment, unwanted weapons, style-incompatible neon, unreadable UI, random letters, misspelled usernames, extra menu options, copied game logo, or watermark. Preserve the user's chosen character designs and palette.

.agents/skills/h3-prompt-writing/SKILL.md

@@ -0,0 +1,42 @@
+---
+name: h3-prompt-writing
+description: Write MiniMax H3 video generation prompts for T2VA, I2VA, FL2VA, L2VA, and Ref2VA. Use when rewriting multimodal requests into H3 prompt structures, composing integrated_multimodal_description, overall_soundscape, and non_diegetic_music, aligning keyframes, or defining reference labels for images, videos, and audio.
+metadata:
+  compatibility: Portable to any agent that can read local files — no external API calls, MiniMax Hub tools, or proprietary runtime required. The agents/openai.yaml file only adds optional ChatGPT/Codex UI metadata; it does not restrict the skill to OpenAI agents.
+---
+
+# H3 Prompt Writing
+
+## Workflow
+
+0. Read [portable-workflow.md](references/portable-workflow.md) for scope, available tools, durable artifacts, and generation handoff. For prompt-only work, no generation tools are required.
+1. Identify the input mode: T2VA, I2VA, FL2VA, L2VA, or full-reference Ref2VA.
+2. For base text/keyframe modes, read `references/base-format.md` and follow its final prompt structure.
+3. For full-reference mode, read `references/ref-format.md` and follow its six-section rewrite format.
+4. Preserve the exact field names, section order, labels, and timing notation from the selected guide.
+
+## Base Modes
+
+- T2VA: build the full audiovisual timeline from text.
+- I2VA: start from the first frame and develop forward from it.
+- FL2VA: describe the continuous path between the first and last frames.
+- L2VA: infer a plausible opening and converge to the supplied last frame.
+
+Use `integrated_multimodal_description`, `overall_soundscape`, and `non_diegetic_music` in the order shown in `references/base-format.md`.
+
+## Full-Reference Mode
+
+Ref2VA rewrites use `subject_definitions`, `summary`, `retention_analysis`, `detailed_description`, `overall_soundscape`, and `non_diegetic_music` in that order. Reference labels stay consistent across all sections.
+
+Read `references/ref-format.md` for label rules and retention analysis.
+
+## Output Rules
+
+- Write rewrite sections in English; preserve dialogue, lyrics, and visible scene text in their original language.
+- Describe each shot by composition, subjects, environment, actions, camera, sound, and the exact point where referenced content appears.
+- Avoid plot summaries, unresolved reference labels, and timing that does not match the requested duration.
+## Tips for Better Results
+- Match the description to the effective duration of the current clip or generation window. The upstream guides describe 4–15-second clips; longer work requires a verified backend extension or an explicit assembly plan, as described in the portable workflow.
+- Keep reference labels consistent (e.g. `<Picture 1>`, `<Video 1>`, `<Audio 1>`) across every section.
+- Prefer concrete visual and audio details over abstract words like "cinematic" or "beautiful".
+- When using keyframes (I2VA / FL2VA / L2VA), clearly state how the first and/or last frame connects to the timeline.

.agents/skills/h3-prompt-writing/agents/openai.yaml

@@ -0,0 +1,4 @@
+interface:
+  display_name: "H3 Prompt Writing"
+  short_description: "Write structured H3 prompts for each input mode"
+  default_prompt: "Use $h3-prompt-writing to prepare the requested H3 prompts or video with the available tools."

.agents/skills/h3-prompt-writing/references/base-format.md

@@ -0,0 +1,86 @@
+# H3 base-mode prompt format
+
+Use this reference for T2VA, I2VA, FL2VA, and L2VA prompt writing. It is a
+compact, repository-authored description of the required output contract, not a
+copy of an upstream prompt guide.
+
+## Select the anchor mode
+
+- T2VA has no image anchor. Build the audiovisual sequence from the brief.
+- I2VA binds `<Picture 1>` to the first frame at 0.00 seconds.
+- FL2VA binds `<Picture 1>` to the first frame and `<Picture 2>` to the final
+  frame. Describe the observable path between them.
+- L2VA binds `<Picture 1>` to the final frame. Infer a compatible opening and
+  converge to the supplied image.
+
+For an image-anchored mode, put a plain-English alignment statement before the
+structured fields. State each picture label, its shot, and its exact target time.
+The ending time must match the effective clip duration, formatted to two decimal
+places. Do not invent a picture label when no image was supplied.
+
+## Emit three fields in order
+
+```text
+integrated_multimodal_description: [Shot 1] ...
+
+overall_soundscape: ...
+
+non_diegetic_music: ...
+```
+
+`integrated_multimodal_description` is the playback-ordered visual and diegetic
+audio plan. Establish the initial style, framing, subjects, environment, and
+spatial relationships, then describe actions and state changes. For keyframe
+modes, explicitly preserve or approach the relevant composition, identity,
+clothing, colors, objects, and geometry.
+
+`overall_soundscape` summarizes ambience, physical effects, and non-verbal human
+sound. Do not repeat dialogue or music already described elsewhere. Use `N/A`
+only for explicitly silent output.
+
+`non_diegetic_music` describes music heard by the audience but not the characters.
+Specify audible properties such as instrumentation, tempo, rhythm, dynamics, and
+ending behavior. Put source-visible music, singing, radio, or instrument playing
+in the main description. Use `N/A` when no score is wanted.
+
+## Timeline notation
+
+- Start the opening with `[Shot 1]` and no timestamp.
+- Mark each later cut with a sequential label and a strictly increasing time,
+  for example `[Shot 2] At 00:03.500, ...`.
+- Use a new shot for an actual edit or meaningful viewpoint/time change. Keep a
+  continuous action or morph in one shot and describe its internal timing.
+- Describe camera motion as an action: type, direction, useful amplitude, and
+  speed. Do not append a disconnected list of camera keywords.
+- Keep every cut and event within the requested clip duration.
+
+## Speech, text, and sound
+
+Give each speaking or singing source a stable ID such as `(S1)` or `(S2)` and
+reuse it across shots. Put only the language tag and literal words inside the
+dialogue wrapper:
+
+```text
+The courier (S1) says softly: <d>[English] The gate is open.</d>
+```
+
+Preserve supplied dialogue, lyrics, punctuation, and visible text verbatim. State
+when a voice is off-screen and keep the visible character's lips closed when that
+is required. If a line spans a cut, describe the audio continuity on both sides
+of the cut. Place visible signs, labels, and interface copy in double quotes.
+
+Tie diegetic sounds to the event that produces them. Avoid describing the same
+sound as both diegetic and non-diegetic.
+
+## Anchor-specific checks
+
+- I2VA: begin from the supplied frame rather than merely borrowing its style.
+- FL2VA: describe intermediate motion; do not provide two unrelated still-image
+  descriptions. Reach Picture 2 at the actual end time.
+- L2VA: keep the final picture assigned to the last shot, not automatically to
+  Shot 1, and make the preceding action physically plausible.
+- All modes: reference labels must map to supplied assets and remain stable.
+
+Before delivery, verify the field order, shot numbering, timestamps, reference
+labels, dialogue fidelity, requested language, duration, and absence of unresolved
+template variables.

.agents/skills/h3-prompt-writing/references/portable-workflow.md

@@ -0,0 +1,121 @@
+# Portable H3 production workflow
+
+This is the runtime contract for the nine H3 skills in this repository. Read it
+alongside the selected creative skill. The bundle uses local files and available
+tools; no proprietary canvas, choice-card widget, planner, or executor is required.
+
+## Scope and decisions
+
+- Resolve the requested deliverable first: prompt, storyboard, assets, or finished
+  video. A prompt-only request ends with the prompt; it does not start generation.
+- Reuse the user's duration, language, style, model, audio policy, supplied assets,
+  and existing authorization. State reasonable defaults for optional gaps. Ask
+  only about missing inputs or choices that materially affect the requested work.
+- Creative checkpoints are reviewable artifacts, not mandatory repeated approvals.
+  Honor an explicit request to approve a preview before rendering. When the user
+  has authorized the complete workflow or delegated creative choices, show the
+  plan and continue within that scope without requiring a special UI control.
+- A skill never grants permission to publish, send messages, spend outside an
+  authorized budget, or change models against the user's choice. User constraints
+  override a preset's style, language, shot count, timing, or audio defaults.
+
+## Tools and durable artifacts
+
+1. Inspect the actual available tools, existing scripts, endpoint documentation,
+   and supplied media. Do not invent a tool name, model parameter, or returned path.
+2. Write plans and prompts to the user-selected project folder. If none is given,
+   use a new named folder under `outputs/`, outside the skill package. Keep useful
+   artifacts such as `brief.md`, `storyboard.md`, `prompts/`, `references/`,
+   `clips/`, and `final/`; create only those needed for this deliverable.
+3. Record each reference's role and path: identity, environment, style, typography,
+   first/last frame, or audio. Tie generated clips to the prompt revision, input
+   assets, model, settings, and output paths. Update downstream references when an
+   asset changes; retain previous outputs instead of overwriting user files.
+4. Use available image generation/editing tools for requested raster assets. Use
+   an available video endpoint or local service for clips, and available audio
+   tooling for requested standalone speech/music. For assembly and inspection,
+   use an installed editor or tools such as FFmpeg, ffprobe, and OpenCV. Follow the
+   host's tool policies, including its preferred image generation mechanism.
+5. Independent tool jobs may run concurrently when resources permit; this skill
+   does not require separate agents or delegated generation.
+6. If a required capability is missing, finish the useful in-scope preparation:
+   complete prompts, asset-role manifest, timeline, and verified request example
+   where possible. State the missing capability and unfinished media step. Do not
+   silently substitute another model, claim a render happened, or call a written
+   plan a completed video. Resume generation when the capability becomes available.
+
+## Compile the creative plan into H3 prompts
+
+Read [base-format.md](base-format.md) for T2VA / I2VA / FL2VA / L2VA, or
+[ref-format.md](ref-format.md) for Ref2VA. These are writing modes, not guaranteed API
+task names. Check the selected backend's input mapping before sending a request.
+
+- Base prompts use `integrated_multimodal_description`, `overall_soundscape`,
+  `non_diegetic_music`, in that order, with the appropriate keyframe preamble.
+- Ref2VA uses `subject_definitions`, `summary`, `retention_analysis`,
+  `detailed_description`, `overall_soundscape`, `non_diegetic_music`, in that order.
+- Every reference label must resolve to a supplied asset, in the same order used
+  by the backend. Omit nonexistent optional references and their dependencies.
+- Keep the creative brief in the user's language. Structured H3 rendering prompts
+  use English prose and exact field labels, while dialogue, lyrics, and visible
+  text retain their requested language and wording. If the user explicitly wants
+  only a same-language creative prompt, deliver that without silently replacing it
+  with English. Compile a separate rendering prompt when generation is requested.
+- `[Shot 1]` has no initial timestamp. Later shot labels mark actual cuts. For a
+  continuous morph, describe timed actions inside one shot rather than turning
+  each form into a new shot. Express camera direction, speed, and useful amplitude.
+- Separate image style from character identity. A style reference must not inject
+  its people, scenery, layout, typography, texture, or lighting into unrelated
+  shots. For literal text fidelity, inspect the rendered result; a prompt is not
+  proof that the exact lettering or beat synchronization was achieved.
+
+## Longer videos and audio
+
+Use the actual backend's supported duration, dimensions, frame grid, reference
+count, audio modes, and model partition. Inspect this checkout's H3 implementation
+or serving documentation before building requests. Do not assume that a feature
+branch's long-video extension exists on main or on a hosted service.
+
+For a verified vLLM-Omni Ref2VA continuation backend, `long_video=true` with
+`long_video_mode=continuation` can accept one `continuation_prompts` entry per
+planned window only when the Ref2VA request is executed with its local text
+encoder. Step execution and stages that use an external text encoder reject
+`continuation_prompts`; omit the field and use the supported per-step inputs in
+those configurations. Compute the window count from the aligned total frames,
+window size, and overlap. Each prompt uses local window time; track global
+shot/music time separately. Repeated overlap is guidance, not additional output
+duration. Do not use this mode on a task/partition that does not support it.
+
+Otherwise split into supported clips and assemble them. Same-scene clips may use
+tail/head keyframe continuity when supported. Scene changes should have explicit
+new scene references; a previous scene's latent tail can preserve the wrong
+setting. Choose cuts, morphing, and transitions from the brief rather than imposing
+hard cuts whenever a work exceeds 15 seconds. Generation windows and visual shots
+are different: one window may contain cuts, and one shot may span multiple windows.
+
+For supplied music, use the selected excerpt as the master timeline unless the
+user requests replacement. For native audio, write the musical/SFX direction in
+the prompt and verify that the backend actually returns audio. Do not invent a
+generic `generate_audio` parameter. If available, locked-source audio latents guide
+generation but the audio-VAE reconstruction is not bit-identical to the source.
+Use explicit final audio stream-copy when an exact original track is required.
+Prompted BPM is a target, not measured timing. Avoid automatically adding lyrics,
+voiceover, subtitles, or multiple competing scores to an instrumental request.
+
+## Verification and bounded recovery
+
+Inspect reference images before using them. After rendering, inspect representative
+frames, transitions, opening/ending states, text, identities, and scene continuity.
+Check audio by listening when possible; report any limit on audio review. Check
+duration, dimensions, fps, frame count, audio presence, and decode validity with
+media tools. If preserving audio exactly, compare extracted audio stream hashes.
+
+Correct a demonstrated failure with a targeted change to the affected prompt,
+reference, or edit. Keep raw generation alongside edited/upscaled versions and
+label the difference. Use an agreed retry/resource budget; absent one, allow one
+targeted rerender, then report persistent defects and concrete options rather than
+entering an unbounded generation loop. Do not switch providers automatically.
+
+Deliver actual artifact paths, the prompts/settings needed to repeat the work,
+and material visual/audio limitations. Distinguish requested behavior, observed
+model output, and post-production repairs.

.agents/skills/h3-prompt-writing/references/ref-format.md

@@ -0,0 +1,105 @@
+# H3 full-reference prompt format
+
+Use this reference for Ref2VA prompt writing. It is a compact,
+repository-authored description of the output contract, not a copy of an
+upstream prompt guide.
+
+## Emit six fields in order
+
+```text
+subject_definitions: ...
+
+summary: ...
+
+retention_analysis: ...
+
+detailed_description: [Shot 1] ...
+
+overall_soundscape: ...
+
+non_diegetic_music: ...
+```
+
+Write the structural prose in English. Preserve the requested language and exact
+wording for dialogue, lyrics, and visible text.
+
+## Reference labels
+
+Assign labels only to supplied content and keep every label stable across all six
+fields:
+
+- `<Subject N>` identifies reusable visible content such as a person, object,
+  environment, costume, interface, action, or visual treatment.
+- `<Picture N>` identifies an image used as a concrete frame, composition, or
+  storyboard anchor. If an image only defines a subject, cite it in that subject's
+  definition instead of inventing a separate target-frame role.
+- `<Video N>` identifies a source video being edited or continued, or a video
+  whose temporal structure is referenced.
+- `<Audio N>` identifies a supplied audio signal that is copied or referenced.
+  Audio and video label numbers are independent even when they came from one file.
+
+In `subject_definitions`, give each tracked item a separate line. State its source,
+role, and the features that later sections must preserve or transform. Do not
+create labels for assets or roles that do not exist.
+
+## Summary and retention
+
+`summary` is one short paragraph describing the target and the principal reference
+relationships. Use only labels already defined. Distinguish these operations:
+
+- a concrete image frame anchor;
+- generation guided by identity, style, action, camera, or audio characteristics;
+- direct editing of a source video;
+- continuation from a source video;
+- copying an audio signal;
+- referencing audio properties without copying the signal.
+
+`retention_analysis` has one line per label and states where it is used and what
+changes. Use consistent machine-readable relationship values:
+
+- Visual: `fully_preserved`, `partially_preserved`, `attribute_transfer`, or
+  `weak_reference`.
+- Audio: `fully_copy`, `partially_copy`, `reference`, or `weak_reference`.
+
+Explain the concrete retained and changed traits after the relationship value.
+Do not claim full preservation when identity, geometry, timing, or signal content
+is intentionally altered. Do not count newly requested target action as an
+unintended loss of reference fidelity.
+
+## Detailed description
+
+`detailed_description` is the playback-ordered rendering plan. For each shot,
+cover the current composition, subject appearance and position, environment,
+lighting, actions and state changes, camera behavior, diegetic sound, and the
+point at which each reference becomes relevant.
+
+- Start with `[Shot 1]` and no timestamp.
+- Give later cuts sequential labels with strictly increasing times, such as
+  `[Shot 2] At 00:03.500, ...`.
+- Place relevant reference labels next to the described subject, frame, motion,
+  or sound; do not collect detached labels at the end.
+- Use stable `(S1)`, `(S2)`, and subsequent IDs for vocal sources. Put literal
+  speech or lyrics inside `<d>[Language] ...</d>`.
+- Preserve supplied visible text verbatim and place it in double quotes.
+- Describe continuous transformations inside one shot unless an actual cut is
+  intended. State camera direction, speed, and amplitude when useful.
+
+For video editing, explicitly identify what stays from the source timeline and
+what is replaced. For continuation, distinguish the inherited opening state from
+new content. For style reference, transfer visual properties without accidentally
+copying people, scenery, layout, or typography.
+
+## Audio fields
+
+`overall_soundscape` summarizes ambience, physical action sounds, and non-verbal
+human sounds. `non_diegetic_music` describes score audible only to the audience.
+Keep dialogue and diegetic music in `detailed_description`. Use `N/A` only when
+the corresponding layer is intentionally absent.
+
+If audio is copied, state whether the whole signal or only a range/layer is used.
+If it is referenced, describe the specific property—such as voice timbre, tempo,
+rhythm, or texture—without claiming signal reuse.
+
+Before delivery, verify the six-field order, one-to-one label definitions,
+retention coverage, shot timing, dialogue fidelity, requested language, and
+absence of unresolved template variables.

.agents/skills/handdrawn-live-video-generator/SKILL.md

@@ -0,0 +1,69 @@
+---
+name: handdrawn-live-video-generator
+description: Write or generate a continuous live-action scene with a rough luminous hand-drawn entity that contacts real objects, morphs, and escapes a slightly delayed handheld camera. Use for gentle surreal mixed-media shorts, not flat-vector-only animation or horror clips.
+metadata:
+  compatibility: Portable prompt writing with sibling h3-prompt-writing; actual generation requires an available video backend and any referenced media.
+---
+
+# Handdrawn Live-Action Fusion
+
+Read the [portable workflow](../h3-prompt-writing/references/portable-workflow.md).
+Keep the user's requested scope and language. A creative-prompt request delivers
+the prompt, without generating media. For a requested finished video, prepare
+the prompt and proceed under existing authorization using available tools.
+
+## Creative contract
+
+The default is a 15-second, 16:9 continuous scene in an everyday real space. A
+flat luminous drawn entity visibly contacts a real hand/object early, transforms
+as one traceable being, escapes through connected space, and the handheld camera
+reacts slightly late. Duration and ratio follow the user's explicit choices.
+
+Use rough crayon, chalk, colored pencil, pastel, or brush strokes with uneven fill,
+frayed edges, line jitter, and frame-by-frame redraw. Keep the live-action setting
+physically grounded and the drawn presence visibly planar. Uniform neon tubes,
+plush characters, polished CG, and clean vector strokes would change this style.
+
+Choose a fresh setting, entity, palette, contact mechanism, transformation chain,
+escape route, camera reaction, and emotional ending within the user's constraints.
+Do not ban an otherwise requested motif simply because an upstream example used
+it. The tone is warm, playful, and gently surprising rather than threatening.
+
+## Timed action structure
+
+For a 15-second brief, use these intervals as a starting structure:
+
+- 0–3s: establish obvious real/drawn contact, such as landing on a palm or slipping
+  around a finger. The real participant reacts.
+- 3–6s: begin a continuous morph while retaining a color smear, line, curve, or tail
+  from the previous shape. Give it a playful action.
+- 6–10s: let it escape into an adjacent part of the same space. The camera pans,
+  tilts, or moves only after the entity starts leaving its view.
+- 10–13s: add another readable interaction or surprise, keeping the same entity.
+- 13–15s: extend an earlier drawn motif across the space into a gentle larger
+  transformation, then resolve with a small emotional/comic detail.
+
+Retiming changes the intervals proportionally or reorganizes beats; do not leave
+15-second timestamps in a shorter prompt. Preserve a user's explicit final form
+instead of forcing the spatial finale when it conflicts with their ending.
+
+## Prompt delivery and rendering
+
+For a same-language creative prompt, begin with duration, ratio, real space, and
+the live-action/drawing fusion. Then describe phone-camera texture, the timed
+action intervals, drawn material, camera lag, specific exclusions, and ambient
+sound. Avoid unrelated headings or an essay when the user asked for copy-ready text.
+
+For actual H3 generation, create a separate structured rendering prompt using the
+appropriate base/Ref2VA guide: English prose with literal dialogue/visible text in
+the requested language. Preserve the creative intent and actual media bindings.
+Do not send unsupported model settings or promise exact first-frame conditioning
+merely because an image is present.
+
+## Review
+
+Check clear early physical contact, one continuous entity, connected geography,
+retained shape motifs, delayed camera reaction, rough planar drawing, and a
+non-threatening ending. Avoid jump scares, sudden blackouts, menacing anatomy, and
+unrequested extra characters. Deliver the actual prompt/video and state any
+remaining contact, continuity, texture, or camera-timing defects.

.agents/skills/handdrawn-live-video-generator/agents/openai.yaml

@@ -0,0 +1,4 @@
+interface:
+  display_name: "Handdrawn Live-Action Fusion"
+  short_description: "Direct drawn morphs and real-world interactions"
+  default_prompt: "Use $handdrawn-live-video-generator to prepare the requested H3 prompts or video with the available tools."

.agents/skills/minimalist-product-ad-generator/SKILL.md

@@ -0,0 +1,71 @@
+---
+name: minimalist-product-ad-generator
+description: Plan, prompt, or produce minimalist product advertising shorts from product images, preserving variants, material, color, concise typography, and rhythmic camera motion. Use for product launches and ecommerce films, not presenter-led ads or complex UI demonstrations.
+metadata:
+  compatibility: File-based agent workflow. Requires the sibling h3-prompt-writing package; rendering uses available image, video, audio, and editing tools.
+---
+
+# Minimalist Product Ad Generator
+
+Read the [portable workflow](../h3-prompt-writing/references/portable-workflow.md).
+Use [product direction](references/product-direction.md) when planning anchors,
+copy, and timed beats. Respect the requested output: prompt-only work needs no
+generation tool, while a finished film includes generation, inspection, and assembly.
+
+## Resolve the product brief
+
+Inspect supplied product photos, packaging, renders, and detail shots. Record the
+product category, main variant, actual body/accent colors, material finish, usable
+mechanisms, and requested copy. Preserve identity and product facts; visual polish
+does not authorize recoloring the product or inventing a feature.
+
+Reuse the specified duration, ratio, language, and model. With no constraints,
+suggest a 10-second H3 film and one of white studio, dark rim light, brand color
+field, or light lifestyle staging. Ask for a missing product reference when exact
+identity is required; distinguish a speculative concept from an existing product.
+If the user delegates direction, record the adopted choices and proceed.
+
+## Design the film
+
+1. Choose a product-specific spine: launch reveal, mechanism interaction, or color
+   family. Describe concrete actions such as a lid opening or a crown turning.
+2. Write short sensory/benefit copy grounded in supplied facts. Preserve the user's
+   exact approved wording and language. For newly requested English copy, 3–5 words
+   usually fit one readable line; do not force English on other-language campaigns.
+3. Plan three independent anchors: hero silhouette, material/mechanism detail, and
+   final product/copy composition. Generate only the assets needed or requested.
+   Do not combine them into a panel sheet that the video model might reproduce.
+4. Keep product color, light direction, ratio, and finish consistent across anchors.
+   Original photos establish identity; generated anchors must pass that identity
+   check before they can replace them as rendering references.
+5. Build the beat table from the reference. Include setup, action, impact, and a
+   stable closing composition; vary shot distance only when it reveals something.
+
+## Prompt and render
+
+Compile each beat into the appropriate H3 fields using the sibling writing guide.
+Bind every anchor to its actual role. Include exact visible copy, entry/exit times,
+colors, and same-line text motion in the rendering prompt, not only in the plan.
+Keep typography in intentional negative space without covering the product. A
+copy-reference image does not mean its line must persist throughout the whole film.
+
+For a short ad, prefer one coherent full-frame film when the backend supports it.
+Keep variants ordered: main variant first, supporting variants later, full family
+only when useful at the close. Avoid product walls, grids, or an anchor-sheet ending.
+
+Use native audio when appropriate to the brief. A restrained electronic score may
+use crisp plucks, airy pads, kick/sub accents and tactile wooden percussion. Set
+tempo and energy from the user's direction. Generate a separate score only if
+requested or needed for a disclosed audio repair, using an available tool.
+
+## Review and deliver
+
+Verify product geometry, variant colors, functional action, final copy, aspect
+ratio, and closing readability. Listen for excessive bass, dropouts, and unfinished
+endings. If editing to supplied music, select a suitable continuous excerpt and
+align visual impacts to observed beats; do not claim synchronization from text alone.
+
+For bad typography, first reduce simultaneous text or correct its reference. Exact
+lettering may need a disclosed overlay when consistent with the user's goal; keep
+the raw output. Deliver the film, prompt/beat plan, used copy and music excerpt,
+and any remaining product/text fidelity issues.

.agents/skills/minimalist-product-ad-generator/agents/openai.yaml

@@ -0,0 +1,4 @@
+interface:
+  display_name: "Minimalist Product Ads"
+  short_description: "Direct product anchors, copy, motion, and audio"
+  default_prompt: "Use $minimalist-product-ad-generator to prepare the requested H3 prompts or video with the available tools."

.agents/skills/minimalist-product-ad-generator/references/product-direction.md

@@ -0,0 +1,52 @@
+# Product anchors, typography, and beats
+
+## Narrative choices
+
+- Launch: negative space → hero product → material detail → real mechanism action
+  → supporting variants → stable product and copy.
+- Feature touch: stillness → interaction trigger → visible mechanism → detail
+  magnification → useful result → closing.
+- Color family: main variant alone → supporting colors enter → ordered relation
+  → unified material finish → full set and copy.
+
+## Independent anchor briefs
+
+| Anchor | What it locks | Check |
+| --- | --- | --- |
+| Hero | Main variant, recognizable silhouette, attractive opening view | Match supplied body color and product structure |
+| Detail | Material, reflection, texture, visible mechanism | No invented ports, controls, or functionality |
+| Closing | Product position, negative space, typography treatment | Real copy, readable contrast, no panel/grid layout |
+
+Each is a standalone full-frame image with the project ratio. Reference roles
+protect appearance and layout; they do not replace the storyboard or require three
+sequential static-image segments. Simplify the number of anchors if the backend
+cannot bind them all, and update the prompt labels accordingly.
+
+## Typography
+
+Use one readable line at a time, with a typeface appropriate to the brand. A clean
+semibold sans-serif is a useful default, not a requirement to copy a specific brand.
+For a two-part color treatment, use dark text on a light field or light text on a
+dark field, then emphasize the second part with a specific real product color.
+Check contrast rather than blindly coloring text like the product.
+
+The first phrase may enter gently; the second follows while the first shifts
+slightly within the same line. Do not accidentally turn this into stacked lines.
+Use actual copy in the prompt and spell out its timing. Avoid arrows, separators,
+or placeholder marks that the model may reproduce. For a 10-second ad, one middle
+copy moment and one stable closing line are often enough. Omit copy when requested.
+
+## Beat table
+
+| Time range | Shot purpose | Visual lead and camera | Variant state | Exact copy | Text color/motion | Handoff | Rhythm |
+| --- | --- | --- | --- | --- | --- | --- | --- |
+
+Use one main action per beat. For 5/10/15-second films, roughly 3–4/5–7/6–9 beats
+are planning suggestions, not required cuts. Record the ending product position
+and motion vector so the next beat can inherit them. Ensure the closing line has
+time to be read rather than using every second for motion.
+
+If replacing music, keep useful SFX unless replacement of the whole track was
+requested. Use a continuous excerpt with a clear entry, stable middle, and suitable
+ending. A short tail fade may resolve a click; do not hide an incomplete phrase
+behind a long fade. Confirm the entire result remains audible.

.agents/skills/music-video-subtitle-generator/SKILL.md

@@ -0,0 +1,74 @@
+---
+name: music-video-subtitle-generator
+description: Plan, write, audit, or produce music videos with coordinated music, imagery, camera motion, optional performance, and lyric typography. Use for beat-aware MVs and instrumental visual narratives, not ordinary subtitle cleanup or unrelated video edits.
+metadata:
+  compatibility: File-based workflow with sibling h3-prompt-writing; audio analysis, generation, and final assembly use available tools and backends.
+---
+
+# Music Video and Lyric Typography
+
+Read the [portable workflow](../h3-prompt-writing/references/portable-workflow.md)
+and, when building a timeline, [music timeline](references/music-timeline.md).
+Deliver the requested scope: audit, prompt, storyboard, or complete video.
+
+## Lock the musical and visual intent
+
+Identify duration, ratio, output resolution, music source/excerpt, visual style,
+character/scene references, performance mode, and whether text is wanted. Treat a
+supplied song as the master track unless the user requests replacement. Preserve
+provided lyrics exactly; do not translate or rewrite them unasked. For instrumental
+work, do not invent singing, lyrics, a performer, or typography. Generate original
+lyrics only when requested or clearly within an authorized songwriting brief.
+
+Measure supplied audio duration and inspect/listen for beat, phrase, breath, drop,
+and energy changes using available tools. Label estimated or planned timing when
+measurement is unavailable. If the requested excerpt exceeds the audio, resolve
+the discrepancy without silently stretching or looping it.
+
+## Reference roles
+
+- Character references control identity, silhouette, wardrobe, and presence.
+- Scene references control environment, time of day, spatial anchors, and lighting.
+- Typography references control letter treatment, layout, and motion, not people
+  or scenery visible in those references.
+- Music references control the requested track or sound style; distinguish copying
+  a supplied recording from generating similar instrumentation.
+
+Do not apply a preset's grain, darkness, hard cuts, or constant close-ups to an
+incompatible brief. A bright nighttime scene remains night. A continuous flat
+vector morph keeps smooth color fields and uses elastic shape motion for accents.
+Use hard cuts, slow holds, or dissolves only when appropriate to the user's intent.
+
+## Compose the timeline and H3 prompts
+
+Map visual actions, camera moves, scene changes, and optional text to the music's
+global time. Build a causal visual progression instead of defaulting to one person
+singing throughout. Allow room to read a shape, gesture, or lyric before changing it.
+
+For longer work, choose a supported continuation or clip-assembly approach through
+the portable workflow. Keep visual shot boundaries distinct from generation
+windows. Local window prompts must account for overlap without replaying the full
+previous action. A new scene may require a new reference rather than inheritance
+of the old scene. Record entry/exit states for each planned segment.
+
+Compile the timeline into H3's base or Ref2VA fields, preserving dialogue, lyrics,
+and visible text verbatim. Include concrete instrumentation and rhythmic accents,
+but treat requested BPM and beat alignment as targets until checked in the output.
+
+## Typography and finishing
+
+When requested, give each shot one main readable text event. Keep words away from
+eyes and critical lip motion. Match sung text to the active phrase, and distinguish
+spatial typography from subtitle bars. Reduce text complexity if the model cannot
+render it, or use a disclosed overlay consistent with the requested deliverable.
+
+For supplied audio, assemble clips to one global master track, maintaining phrase
+and lip continuity; avoid cuts inside a vowel. For native audio, inspect actual
+continuity across generation windows. Preserve useful SFX and avoid layered duplicate
+music. Retiming, grain, grading, interpolation, and upscaling are deliberate editing
+choices, not automatic finishing requirements.
+
+Verify transformation/story order, identities, scene changes, requested text,
+musical continuity, timing, and full media decode. Label raw and processed versions.
+Deliver the final file and current prompt/timeline files, with measured properties
+and remaining limitations rather than claims of perfect synchronization.

.agents/skills/music-video-subtitle-generator/agents/openai.yaml

@@ -0,0 +1,4 @@
+interface:
+  display_name: "Music Video Direction"
+  short_description: "Plan music-driven scenes and optional lyric text"
+  default_prompt: "Use $music-video-subtitle-generator to prepare the requested H3 prompts or video with the available tools."

.agents/skills/music-video-subtitle-generator/references/music-timeline.md

@@ -0,0 +1,40 @@
+# Global music and local video timing
+
+Use this planning skeleton, omitting vocal/text fields when not requested:
+
+```text
+Global: music/excerpt, duration, ratio, character/style/scene locks, cut policy.
+
+Segment S01 — global start/end; local generation start/end
+Incoming state:
+Music cue: measured beat/phrase timestamp, or clearly labeled planned accent
+Vocal line and speaker (if present):
+Visible typography (if requested):
+Visual action and camera:
+Outgoing state and transition:
+References with exact roles:
+```
+
+Keep one global music clock. For measured tempo B, the nominal quarter-note spacing
+is 60/B seconds, but real tracks may have pickups, swing, tempo changes, or inaccurate
+automatic detections. Validate important cut points against the waveform/listening.
+An onset detector also finds SFX and melodic attacks; it does not prove drum beats.
+
+Choose a phrase-complete excerpt when possible. Track breaths, lyric pauses, and
+vowel continuity for a performance MV. For an instrumental morph, accent the
+physical preparation, transformation impact, and release rather than introducing
+unrequested cuts. The final shape or story payoff needs a readable ending.
+
+For continuation, maintain a table of absolute output frames and local window
+times. Use the verified backend's native alignment and overlap behavior. Opening
+overlap continues the inherited state; new actions belong after it. Do not sum
+overlap twice or describe every window as a fresh opening shot.
+
+For separately generated clips, use a previous tail frame only when the next shot
+continues the same scene and the backend supports that keyframe role. For an
+intentional cut, retain appropriate identity references and select the new scene.
+Preserve screen direction or a useful match element when it helps the transition.
+
+When text is requested, track exact string, language, entry/exit time, screen
+region, and motion. Typography style cards must not donate their depicted characters
+or settings. Preserve user text; do not silently shorten lyrics to fit a template.

.agents/skills/paper-collage-explainer-generator/SKILL.md

@@ -0,0 +1,69 @@
+---
+name: paper-collage-explainer-generator
+description: Translate narration, ideas, or knowledge points into editorial halftone paper-collage metaphors, still prompts, storyboards, or stop-motion clips. Use for photographic cutout collage and tactile assembly, distinct from paper-puppet dioramas or generic cartoons.
+metadata:
+  compatibility: File-based workflow with sibling h3-prompt-writing; media creation uses available image/video tools and an editor when needed.
+---
+
+# Paper Collage Explainer Generator
+
+Read the [portable workflow](../h3-prompt-writing/references/portable-workflow.md)
+and [collage design](references/collage-design.md) when constructing scenes.
+Honor prompt-only and asset-only requests as well as finished video requests.
+
+## Meaning and media policy
+
+Extract each source line's meaning, emotion, action verb, visual metaphor, and
+three to six readable object groups. Preserve the user's topic and wording as
+context; do not put narration on screen merely because it is available.
+
+Default to tactile paper SFX, with no BGM, voiceover, or subtitles unless requested.
+An explainer does not inherently require speech. Existing user requests for those
+layers already establish the choice; do not ask again. A short 16:9 clip of roughly
+four seconds is a useful default for one assembly beat, subject to backend limits.
+
+## Plan the image and motion together
+
+Create a concise plan of metaphor, palette, object groups, assembly order, duration,
+and sound cues. For a series, keep paper texture, halftone treatment, keylines,
+shadows, and rhythmic language consistent; vary the metaphor and color field only
+where the meaning benefits. A user's no-text or no-music requirement applies to
+every downstream asset, prompt, clip, and edit.
+
+Design each still as the completed final composition. Use black-and-white
+photographic halftone cutouts, selective colored cardstock, warm cream keylines,
+subtle fiber/torn edges, and soft physical layer shadows on a bold color field.
+Keep layered depth and negative space; avoid excessive age, dirt, wrinkles, or
+automatic brown/kraft backgrounds. Exact editable layers require a compositing
+workflow, not a claim that a generated raster video is a layered project.
+
+Generate final-composition stills when requested/needed and inspect them. Follow
+any requested approval gate; otherwise continue within the authorized workflow.
+
+## Animate the assembly
+
+Start from a clean paper field matching the planned final color. Base pieces enter,
+then the main metaphor, then supporting objects. Each piece slides or pops in,
+bounces slightly, presses flat, pauses, and locks. End with a brief hold on the
+completed composition. Use stop-motion assembly rather than global fades, smooth
+digital drifting, fast spinning, or liquid morphs unless explicitly requested.
+
+Use the supplied final still as a last-frame anchor only when the backend supports
+it. Otherwise identify it honestly as composition/style guidance; do not describe
+a reference image as a guaranteed final-frame constraint. Compile the actual H3
+mode with the sibling guide and bind only real inputs.
+
+Map short paper slides, pops, taps, rustles, and snaps to motion. Preserve those
+SFX during assembly. When the user adds music or narration later, mix with the
+useful SFX unless they request full replacement.
+
+## Review and deliver
+
+Check the metaphor without relying on text, object count/readability, matching
+opening color, visible piece-by-piece construction, final-frame resemblance,
+controlled paper texture, and no unwanted UI/letters/music/voiceover. If the
+assembly is weak, simplify the groups and make their arrival order concrete.
+
+For multiple clips, preserve the planned order and SFX. Deliver prompts, current
+stills, and actual clips/assembly as requested. Explain any missing generation
+capability or remaining drift; keep raw output separate from edited fixes.

.agents/skills/paper-collage-explainer-generator/agents/openai.yaml

@@ -0,0 +1,4 @@
+interface:
+  display_name: "Paper Collage Explainers"
+  short_description: "Animate editorial paper metaphors with tactile SFX"
+  default_prompt: "Use $paper-collage-explainer-generator to prepare the requested H3 prompts or video with the available tools."

.agents/skills/paper-collage-explainer-generator/references/collage-design.md

@@ -0,0 +1,33 @@
+# Editorial collage design
+
+For each source idea record meaning, emotion, one active verb, a concrete visual
+metaphor, three to six object groups, and a tactile sound cue. For example, a
+collaboration concept can become separate cutout hands fitting paper routes
+together; it need not become a screen full of the word "collaboration".
+
+Style signature: flat bold color field, black-and-white halftone photographic
+cutouts, selective colored cardstock accents, warm cream keylines, soft physical
+paper shadows, fine uncoated fiber, refined irregular torn edges, layered seams.
+Control texture so the video feels handmade without becoming dirty or distressed.
+
+Palette choices may support meaning: orange/red for urgency, mustard for tools or
+warning, green for reflection, purple for memory, teal for collaboration, rose for
+theatrical contrast. These are suggestions, not universal semantic rules. Use the
+user's palette and avoid an unrelated kraft-paper opening.
+
+Still specification: source meaning → metaphor → ratio → background/accent colors
+→ object roles → foreground/midground/background → final composition → exclusions.
+
+Motion specification: matching empty color field → base paper structures → main
+subject → secondary groups → light bounce/press-flat/pause → completed composition.
+Explicitly describe material boundaries and which piece moves at each beat. A
+static camera is usually enough to reveal assembly; add movement only when useful.
+
+Sound specification: synchronize restrained slides, pops, taps, rustles, and snaps.
+Set `non_diegetic_music: N/A` for the default SFX-only H3 prompt. Do not use silence
+as a synonym for no background music. Dialogue and subtitles are optional layers.
+
+Review one established clip as a style/cadence reference for subsequent clips.
+Change the specific failure: fewer groups for clutter, more physical arrival
+steps for digital drifting, less aging for dirty paper, stronger final state for
+composition drift. Do not apply an unrelated aesthetic preset to fix one defect.

.agents/skills/papercraft-stop-motion-explainer/SKILL.md

@@ -0,0 +1,68 @@
+---
+name: papercraft-stop-motion-explainer
+description: Turn knowledge topics into layered papercraft stop-motion explainers, including visual metaphors, miniature sets, prompts, storyboards, motion and sound plans, or rendered video. Use for tactile paper dioramas and mechanisms, not ordinary vector cartoons or photographic collage.
+metadata:
+  compatibility: Portable file-based workflow with sibling h3-prompt-writing; actual media generation uses tools available in the host environment.
+---
+
+# Papercraft Stop-Motion Explainer
+
+Read the [portable workflow](../h3-prompt-writing/references/portable-workflow.md).
+For scene/prompt planning read [paper staging](references/paper-staging.md).
+Route a single image prompt, image series, short animation prompt, or storyboard
+request directly to that artifact; do not force a full film workflow.
+
+## Knowledge and direction
+
+Identify the exact topic, source facts, audience, desired learning outcome, and
+output type. Preserve technical terms and causal meaning. A visual metaphor helps
+explain a mechanism; it must not turn into a false scientific claim.
+
+Reuse requested duration and ratio. If unspecified, a 30-second, 16:9 general-audience
+concept is a reasonable starting point. Offer distinct directions only when useful:
+pop-up-book journey, paper laboratory, layered cross-section, miniature nature
+theater, or mechanism board. Each should identify one learning outcome, a paper
+mechanism that conveys it, and the signature visual moment.
+
+## Build the paper world
+
+Design characters only if they help the explanation. Specify paper construction,
+silhouette, joint/hinge design, replaceable facial parts, gestures, and repeatable
+identity cues. Keep the central knowledge object readable.
+
+Plan foreground occluders, midground mechanism, background scenery, and a distant
+paper plane when the shot benefits from depth. Record each layer's material,
+thickness, cut/fold edges, shadow, movement, and educational role. Use real paper
+mechanics such as pull-tabs, sliders, rotating discs, page folds, and pinned joints.
+Motion is stepped with small pauses and rebounds, not a smooth CG melt.
+
+Create an asset list by function: characters, scientific objects, information cards,
+environment, and moving mechanisms. For a complete production, prepare one to three
+preview briefs or images when useful and authorized. Respect any preview approval
+requested by the user; otherwise continue under the existing end-to-end scope.
+
+## Prompts and timeline
+
+Write a compact storyboard with columns: time, knowledge beat, visual action,
+paper movement, camera/transition, sound cue. Each shot explains one knowledge
+step, with a readable pause for a necessary label. Keep ratio, material, light
+direction, identity, and domain vocabulary consistent across image-series prompts.
+
+For a short image-to-video prompt, begin from the actual image and animate a few
+paper mechanisms with restrained push-in or lateral parallax. Preserve thickness,
+cut edges, texture, and layer shadows. Compile final rendering prompts using the
+bundled H3 mode guide, with truthful references and local clip timing.
+
+## Sound, generation, and review
+
+Map paper flips, cardboard slides, joint taps, soft pops, and rustles to actual
+motion. Match any requested music to the topic and cultural context rather than
+automatically using the same whimsical score. Keep speech intelligible. If narration
+is requested, preserve its approved wording, measure its duration, and adjust the
+timeline or script within the user's scope; do not silently speed up the voice.
+
+Use available generation/editing tools, then inspect both educational accuracy
+and paper construction: readable mechanism, separate layers/shadows, hinged or
+stepped motion, plausible camera movement, and a clear ending takeaway. Check
+narration tails, clicks, audio gaps, and music balance. Deliver requested artifacts
+and state missing rendering capabilities or unresolved scientific/visual issues.

.agents/skills/papercraft-stop-motion-explainer/agents/openai.yaml

@@ -0,0 +1,4 @@
+interface:
+  display_name: "Papercraft Explainers"
+  short_description: "Explain ideas with layered paper mechanisms"
+  default_prompt: "Use $papercraft-stop-motion-explainer to prepare the requested H3 prompts or video with the available tools."

.agents/skills/papercraft-stop-motion-explainer/references/paper-staging.md

@@ -0,0 +1,48 @@
+# Paper staging and prompt building blocks
+
+## Materials and depth
+
+Use layered matte cardboard, visible thickness, fibers, folds, seams, tabs, brads,
+cut edges, and physical shadows. Separate foreground, mechanism, background, and
+far background with spacing. Roughly 4–7 layers often suffice; remove layers that
+hide the explanation. Place reading-heavy labels on stable planes. Use miniature
+macro photography and moderate depth of field without blurring necessary facts.
+
+| Plane | Useful content | Motion | Purpose |
+| --- | --- | --- | --- |
+| Foreground | Paper leaves, frame, curtain, props | Small parallax or hinged reveal | Depth without concealing the mechanism |
+| Midground | Knowledge model, host, arrow/cards | Pull-tab, slider, joint, rotating disc | Main cause and effect |
+| Background | Diorama set or schematic layers | Slow paper rail shift | Context and hierarchy |
+| Far plane | Paper sky/board | Restrained change | Stable visual support |
+
+## Prompt ingredients
+
+Image: topic and core object → miniature diorama → layered paper construction →
+specific material edges → physical light/shadows → clear depth → necessary labels.
+One frame should communicate one main idea, not the entire lesson at once.
+
+Image series: repeat the style/identity anchors, then state what changes and why
+that state explains the next knowledge step. Keep the same ratio and light direction.
+
+Short animation: preserve the supplied image, make a small paper mechanism move,
+include tiny pauses/rebounds, and select one restrained camera move. Small puppet
+gestures and background rails can provide life without overwhelming the model.
+
+## Camera and transitions
+
+Use slow push-in, lateral parallax, static demonstration, macro detail, or a slight
+top-down cross-section view. A full orbit requires a set built for that view.
+Transitions may use a page flip, pop-up unfold, pull-tab, paper cloud, circular
+mask, cut-paper door, layer separation, confetti, or sticker reveal. Avoid liquid
+morphing, neon glitches, and glass shatter unless explicitly requested as contrast.
+
+## Sound and exclusions
+
+Use a few motion-specific accents: paper flip, scissors, cardboard slide, joint
+tap, tape peel, or box opening. A score, when appropriate, can use light percussion,
+plucked instruments, or topic-specific instrumentation. Do not infer narration
+merely because the video teaches something.
+
+Keep constraints concise and relevant: no glossy plastic, smooth CG morphs, metal
+surfaces replacing paper, missing thickness/shadows, or frantic camera movement.
+Include them in supported prompt fields; do not invent a negative-prompt API field.

.buildkite/amd/test-amd-merge.yml

@@ -41,7 +41,33 @@ steps:
         - export VLLM_ROCM_USE_AITER=0
         # ignore test_teacache_extractors.py because it use rocm gemm kernel from vLLM
         # that is not supported on CPU
-        - "pytest -sv tests/diffusion -m 'core_model and cpu' --ignore=tests/diffusion/cache/test_teacache_extractors.py --num-shards=$$BUILDKITE_PARALLEL_JOB_COUNT --shard-id=$$BUILDKITE_PARALLEL_JOB"
+        # Keep multi-GPU tests out of this one-GPU pool even when their module
+        # also carries the cpu marker for unit tests.
+        - "pytest -sv tests/diffusion -m 'core_model and cpu and not (cards_2 or cards_3 or cards_4 or cards_5 or cards_6 or cards_7 or cards_8)' --ignore=tests/diffusion/cache/test_teacache_extractors.py --ignore=tests/diffusion/models/pi05/test_pi05_units.py --num-shards=$$BUILDKITE_PARALLEL_JOB_COUNT --shard-id=$$BUILDKITE_PARALLEL_JOB"
+    # This module takes about 22 minutes on MI300 and was consistently assigned to shard 4,
+    # pushing that shard to the 45-minute timeout. Keep its module-scoped fixture in one lane.
+    - label: "Simple · Pi0.5 CPU Test"
+      agent_pool: mi300_1
+      depends_on: amd-build
+      mirror_hardwares: [amdproduction]
+      grade: Blocking
+      timeout_in_minutes: 45
+      commands:
+        - export VLLM_ROCM_USE_AITER=0
+        - "pytest -sv tests/diffusion/models/pi05/test_pi05_units.py -m 'core_model and cpu'"
+
+    - label: "Diffusion · AuK VAE GPU Test"
+      agent_pool: mi300_1
+      depends_on: amd-build
+      mirror_hardwares: [amdproduction]
+      grade: Blocking
+      timeout_in_minutes: 40
+      commands:
+        - export VLLM_ROCM_USE_AITER=0
+        - >-
+          timeout --signal=TERM --kill-after=2m 35m
+          pytest -sv tests/diffusion/models/auk/test_auk_vae_fastpath.py
+          -m 'core_model and rocm and cards_1' --run-level core_model
 
     - label: "Simple · Engine&Entrypoints Test"
       agent_pool: mi300_1
@@ -131,6 +157,11 @@ steps:
       grade: Blocking
       commands:
         - pytest -s -v tests/diffusion/distributed/test_tensor_parallel.py
+        - >-
+          timeout 20m pytest -s -v
+          tests/diffusion/attention/test_ulysses_uaa.py::test_ulysses_uaa_2d_mask_layout_matches_baseline
+          -m 'core_model and cards_2'
+          --run-level "core_model"
 
     - label: "Diffusion Model Test"
       agent_pool: mi300_1
@@ -138,7 +169,9 @@ steps:
       mirror_hardwares: [amdproduction]
       grade: Blocking
       commands:
-        - timeout 30m pytest -s -v tests/e2e/offline_inference/test_z_image.py
+        # A cold Z-Image run spends about six minutes loading weights before
+        # the first-request AITER compilation, which can itself exceed 23m.
+        - timeout 55m pytest -s -v tests/e2e/offline_inference/test_z_image.py
 
     - label: "Diffusion GPU Worker Test"
       agent_pool: mi300_2

.buildkite/amd/test-amd-ready.yml

@@ -41,7 +41,33 @@ steps:
         - export VLLM_ROCM_USE_AITER=0
         # ignore test_teacache_extractors.py because it use rocm gemm kernel from vLLM
         # that is not supported on CPU
-        - "pytest -sv tests/diffusion -m 'core_model and cpu' --ignore=tests/diffusion/cache/test_teacache_extractors.py --num-shards=$$BUILDKITE_PARALLEL_JOB_COUNT --shard-id=$$BUILDKITE_PARALLEL_JOB"
+        # Keep multi-GPU tests out of this one-GPU pool even when their module
+        # also carries the cpu marker for unit tests.
+        - "pytest -sv tests/diffusion -m 'core_model and cpu and not (cards_2 or cards_3 or cards_4 or cards_5 or cards_6 or cards_7 or cards_8)' --ignore=tests/diffusion/cache/test_teacache_extractors.py --ignore=tests/diffusion/models/pi05/test_pi05_units.py --num-shards=$$BUILDKITE_PARALLEL_JOB_COUNT --shard-id=$$BUILDKITE_PARALLEL_JOB"
+    # This module takes about 22 minutes on MI300 and was consistently assigned to shard 4,
+    # pushing that shard to the 45-minute timeout. Keep its module-scoped fixture in one lane.
+    - label: "Simple · Pi0.5 CPU Test"
+      agent_pool: mi300_1
+      depends_on: amd-build
+      mirror_hardwares: [amdproduction]
+      grade: Blocking
+      timeout_in_minutes: 45
+      commands:
+        - export VLLM_ROCM_USE_AITER=0
+        - "pytest -sv tests/diffusion/models/pi05/test_pi05_units.py -m 'core_model and cpu'"
+
+    - label: "Diffusion · AuK VAE GPU Test"
+      agent_pool: mi300_1
+      depends_on: amd-build
+      mirror_hardwares: [amdproduction]
+      grade: Blocking
+      timeout_in_minutes: 40
+      commands:
+        - export VLLM_ROCM_USE_AITER=0
+        - >-
+          timeout --signal=TERM --kill-after=2m 35m
+          pytest -sv tests/diffusion/models/auk/test_auk_vae_fastpath.py
+          -m 'core_model and rocm and cards_1' --run-level core_model
 
     - label: "Simple · Engine&Entrypoints Test"
       agent_pool: mi300_1
@@ -141,6 +167,11 @@ steps:
         - export VLLM_LOGGING_LEVEL=DEBUG
         - export VLLM_WORKER_MULTIPROC_METHOD=spawn
         - timeout 20m pytest -s -v tests/diffusion/distributed/test_sequence_parallel.py -m core_model
+        - >-
+          timeout 20m pytest -s -v
+          tests/diffusion/attention/test_ulysses_uaa.py::test_ulysses_uaa_2d_mask_layout_matches_baseline
+          -m 'core_model and cards_2'
+          --run-level "core_model"
         - >-
           timeout 20m pytest -s -v
           tests/diffusion/models/ltx2/test_ltx2_transformer_ulysses.py
@@ -165,6 +196,7 @@ steps:
         # registered multi-GPU marker from this one-GPU job.
         - >-
           timeout 15m pytest -sv tests/diffusion/models/
+          --ignore=tests/diffusion/models/auk/test_auk_vae_fastpath.py
           -m 'core_model and cuda and not
           (cards_2 or cards_3 or cards_4 or cards_5 or cards_6 or cards_7 or cards_8)'
           --run-level "core_model"

.buildkite/common/ci_source_file_dependencies.yml

@@ -16,10 +16,14 @@ x-qwen3-tts: &qwen3_tts
   - vllm_omni/worker/gpu_model_runner.py
   - vllm_omni/model_executor/models/qwen3_tts/
   - vllm_omni/model_executor/models/common/qwen3_code_predictor.py
+  - vllm_omni/model_executor/models/common/fused_sampling.py
+  - vllm_omni/model_executor/models/common/short_kv_attention.py
+  - vllm_omni/model_executor/models/common/talker_first_audio.py
   - vllm_omni/model_executor/models/common/snake_activation.py
   - vllm_omni/model_executor/models/common/alias_free_activation.py
   - vllm_omni/model_executor/stage_input_processors/qwen3_tts.py
   - vllm_omni/deploy/qwen3_tts.yaml
+  - vllm_omni/deploy/qwen3_tts_high_concurrency_mrv2_single_gpu.yaml
   - vllm_omni/entrypoints/openai/serving_speech.py
   - vllm_omni/entrypoints/openai/tts_adapters/
   - vllm_omni/engine/stage_pool.py
@@ -30,6 +34,9 @@ x-qwen3-tts: &qwen3_tts
   - vllm_omni/config/omni_config.py
   - vllm_omni/engine/orchestrator.py
   - vllm_omni/engine/stage_engine_core_proc.py
+  - vllm_omni/engine/cuda_mps.py
+  - vllm_omni/engine/stage_runtime.py
+  - vllm_omni/engine/stage_init_utils.py
   - vllm_omni/deploy/qwen3_tts_mrv2.yaml
   - vllm_omni/deploy/qwen3_tts_high_concurrency_mrv2.yaml
   - vllm_omni/deploy/qwen3_tts_high_concurrency_mrv2_b4.yaml
@@ -81,11 +88,16 @@ x-qwen3-omni: &qwen3_omni
   - vllm_omni/entrypoints/openai/serving_chat.py
   - vllm_omni/model_executor/models/qwen3_omni/
   - vllm_omni/model_executor/models/common/qwen3_code_predictor.py
+  - vllm_omni/model_executor/models/common/fused_sampling.py
+  - vllm_omni/model_executor/models/common/short_kv_attention.py
+  - vllm_omni/model_executor/models/common/talker_first_audio.py
   - vllm_omni/model_executor/models/common/snake_activation.py
   - vllm_omni/model_executor/models/qwen2_5_omni/qwen2_5_omni_thinker.py
   - vllm_omni/model_executor/models/qwen3_tts/cuda_graph_decoder_wrapper.py
   - vllm_omni/model_executor/stage_input_processors/qwen3_omni.py
   - vllm_omni/deploy/qwen3_omni_moe.yaml
+  - vllm_omni/deploy/qwen3_omni_moe_mrv2.yaml
+  - vllm_omni/deploy/qwen3_omni_moe_mrv2_h200.yaml
   - vllm_omni/deploy/qwen3_omni_moe_thinking.yaml
 
 x-wan22: &wan22
@@ -188,6 +200,10 @@ source_file_dependencies:
 
   # --- models ---
   # --- omni / tts jobs ---
+  entrypoints_function:
+    - tests/entrypoints/
+    - vllm_omni/entrypoints/
+
   omni_qwen3_omni_function:
     - *qwen3_omni
     - tests/e2e/offline_inference/test_qwen3_omni.py
@@ -236,6 +252,14 @@ source_file_dependencies:
     - *minicpmo_4_5
     - tests/e2e/accuracy/minicpmo_4_5/
 
+  omni_duplex_eval_ci:
+    - *minicpmo_4_5
+    - *minicpmo_4_5_duplex
+    - tests/e2e/accuracy/omni_duplex_eval/**
+    - tests/e2e/online_serving/helpers/minicpmo_4_5_duplex.py
+    - vllm_omni/benchmarks/duplex/**
+    - vllm_omni/entrypoints/cli/benchmark/omni_duplex_eval.py
+
   omni_minicpmo_4_5_perf:
     - *minicpmo_4_5
     - tests/dfx/perf/scripts/run_benchmark.py
@@ -319,7 +343,7 @@ source_file_dependencies:
 
   diffusion_wan22_perf:
     - *wan22
-    - tests/dfx/perf/scripts/run_diffusion_benchmark.py
+    - tests/dfx/perf/scripts/run_benchmark.py
     - tests/dfx/perf/tests/test_wan22_i2v_vllm_omni.json
 
   diffusion_wan22_reliability:
@@ -345,7 +369,7 @@ source_file_dependencies:
 
   diffusion_cosmos3_perf:
     - *cosmos3
-    - tests/dfx/perf/scripts/run_diffusion_benchmark.py
+    - tests/dfx/perf/scripts/run_benchmark.py
     - tests/dfx/perf/tests/test_cosmos3_vllm_omni.json
 
   diffusion_qwen_image_function:
@@ -390,6 +414,8 @@ source_file_dependencies:
   diffusion_hunyuan_image3_accuracy:
     - *hunyuan_image3
     - tests/e2e/accuracy/test_hunyuan_image3_pixel_accuracy.py
+    - tests/e2e/accuracy/test_hunyuan_image3_lora.py
+    - vllm_omni/diffusion/lora/
 
   diffusion_hunyuan_image3_perf:
     - *hunyuan_image3
@@ -404,6 +430,7 @@ source_file_dependencies:
 
   diffusion_minimax_h3_function:
     - *minimax_h3
+    - tests/helpers/assertions.py
     - tests/e2e/online_serving/test_minimax_h3_dlo_dp2_t2va.py
     - tests/e2e/online_serving/minimax_h3/
     - tests/diffusion/models/minimax_h3/
@@ -418,7 +445,7 @@ source_file_dependencies:
 
   diffusion_minimax_h3_perf:
     - *minimax_h3
-    - tests/dfx/perf/scripts/run_diffusion_benchmark.py
+    - tests/dfx/perf/scripts/run_benchmark.py
     - tests/dfx/perf/tests/test_minimax_h3_vllm_omni.json
 
   diffusion_hunyuan_video_function:
@@ -432,7 +459,7 @@ source_file_dependencies:
 
   diffusion_hunyuan_video_perf:
     - *hunyuan_video
-    - tests/dfx/perf/scripts/run_diffusion_benchmark.py
+    - tests/dfx/perf/scripts/run_benchmark.py
     - tests/dfx/perf/tests/test_hunyuanvideo15_t2v_vllm_omni.json
     - tests/dfx/perf/tests/test_hunyuanvideo15_i2v_vllm_omni.json
 
@@ -459,6 +486,11 @@ source_file_dependencies:
     - tests/e2e/online_serving/test_lingbot_video_moe.py
     - tests/e2e/offline_inference/test_lingbot_world_v2.py
 
+  diffusion_lingbot_perf:
+    - *lingbot_video
+    - tests/dfx/perf/scripts/run_benchmark.py
+    - tests/dfx/perf/tests/test_lingbot_video_vllm_omni.json
+
 
   # Invalid-param jobs: model business code only, plus the reliability scripts.
   reliability_invalid_param_h100:

.buildkite/cuda/test-merge.yml

@@ -18,7 +18,7 @@ steps:
         mirror_hardwares: l4_1
 
       - label: "Simple · Diffusion Test"
-        timeout_in_minutes: 20
+        timeout_in_minutes: 30
         commands:
           - "pytest -sv tests/diffusion -m 'core_model and cpu'"
         mirror_hardwares: l4_1
@@ -60,11 +60,7 @@ steps:
     depends_on: upload-merge-pipeline
     steps:
       - label: "Entrypoint Test with H100"
-        # TEMPORARY 60 -> 90: engine startup/sleep-wake is ~13-15 min slower per
-        # run on the vLLM 0.26 image (model loading ~2x, init ~2x; see builds
-        # 2915 vs 2916), which pushed this suite past 60 min. Revert once the
-        # 0.26 startup regression is understood and fixed.
-        timeout_in_minutes: 90
+        timeout_in_minutes: 60
         commands:
           # Turn-based server VAD over /v1/realtime is deferred to RFC #7181 PR 5
           # (see the P0 thread on #7413); the test stays in the tree and runs again there.
@@ -73,10 +69,25 @@ steps:
 
       - label: "Entrypoint Test with L4"
         timeout_in_minutes: 60
+        # Workaround https://github.com/vllm-project/vllm-omni/issues/8016: shutdown()
+        # leaves inline UniProc DiffusionWorker VRAM mapped in the pytest parent.
+        # This job does not collect the ready leftover
+        # (test_diffusion_generate_request_id, core_model); isolate the
+        # advanced_model DiT handshake instead. Revert to a single sweep after
+        # the product shutdown fix.
         commands:
-          - pytest -s -v tests/entrypoints/ -m "advanced_model and cuda and L4" --run-level "advanced_model"
+          - |
+            set -eu
+            pytest -s -v tests/entrypoints/test_omni_sleep_mode.py::TestOmniDiffusionSleepMode::test_diffusion_sleep_handshake -m "advanced_model and cuda and L4 and cards_1" --run-level "advanced_model"
+            pytest -s -v tests/entrypoints/ -m "advanced_model and cuda and L4 and cards_1" --run-level "advanced_model" --deselect tests/entrypoints/test_omni_sleep_mode.py::TestOmniDiffusionSleepMode::test_diffusion_sleep_handshake
         mirror_hardwares: l4_1
 
+      - label: "Entrypoint Test with L4 · 2-GPU"
+        timeout_in_minutes: 60
+        commands:
+          - pytest -s -v tests/entrypoints/ -m "advanced_model and cuda and L4 and cards_2" --run-level "advanced_model"
+        mirror_hardwares: l4_2
+
   - group: ":card_index_dividers: E2E Test"
     depends_on: upload-merge-pipeline
     steps:
@@ -113,6 +124,30 @@ steps:
           - pytest -s -v tests/e2e/offline_inference/test_qwen3_omni.py tests/e2e/online_serving/test_qwen3_omni.py tests/e2e/offline_inference/test_qwen3_omni_colocate_async.py -m 'advanced_model and cuda' --run-level 'advanced_model'
         mirror_hardwares: h100_2
 
+      - label: "Omni · Qwen3-Omni Async-Chunk SHM Leak"
+        source_file_dependencies:
+          - tests/e2e/online_serving/test_qwen3_omni_shm_leak.py
+          - tests/helpers/runtime.py
+          - tests/helpers/stage_config.py
+          - vllm_omni/core/sched/omni_ar_scheduler.py
+          - vllm_omni/distributed/omni_connectors/connectors/shm_connector.py
+          - vllm_omni/distributed/omni_connectors/transfer_adapter/chunk_transfer_adapter.py
+          - vllm_omni/engine/orchestrator.py
+          - vllm_omni/engine/stage_engine_core_proc.py
+          - vllm_omni/engine/stage_pool.py
+          - vllm_omni/entrypoints/openai/serving_chat.py
+          - vllm_omni/model_executor/models/qwen3_omni/
+          - vllm_omni/model_executor/models/common/qwen3_code_predictor.py
+          - vllm_omni/model_executor/models/common/snake_activation.py
+          - vllm_omni/model_executor/models/qwen2_5_omni/qwen2_5_omni_thinker.py
+          - vllm_omni/model_executor/models/qwen3_tts/cuda_graph_decoder_wrapper.py
+          - vllm_omni/model_executor/stage_input_processors/qwen3_omni.py
+          - vllm_omni/deploy/qwen3_omni_moe.yaml
+        timeout_in_minutes: 40
+        commands:
+          - pytest -s -v tests/e2e/online_serving/test_qwen3_omni_shm_leak.py -m 'advanced_model and cuda' --run-level 'advanced_model'
+        mirror_hardwares: h100_2
+
       - label: "Omni · MiniCPM-o 4.5 Test"
         source_file_dependencies: omni_minicpmo_4_5_function
         timeout_in_minutes: 50
@@ -197,3 +232,14 @@ steps:
             timeout 35m pytest -s -v tests/e2e/online_serving/minimax_h3/test_minimax_h3_turbo_lora.py -m 'advanced_model and cuda and diffusion' --run-level 'advanced_model'
             timeout 35m pytest -s -v tests/e2e/online_serving/minimax_h3/test_minimax_h3_fasth3_lora.py -m 'advanced_model and cuda and diffusion' --run-level 'advanced_model'
         mirror_hardwares: h100_2
+
+      - label: "Diffusion · MiniMax-H3 Latent Upscale and Refine"
+        source_file_dependencies: diffusion_minimax_h3_function
+        timeout_in_minutes: 40
+        commands:
+          - |
+            set -eu
+            VLLM_TEST_MINIMAX_H3_LATENT_UPSCALER=$(python3 -c 'from huggingface_hub import hf_hub_download; print(hf_hub_download(repo_id="LBH-123-AI/Minimax_h3_latent_Upscaler", filename="minimax_h3_latent_upscaler_3d_conv_v1/minimax_h3_latent_upscaler_3d_conv_v1_bf16.safetensors"))')
+            export VLLM_TEST_MINIMAX_H3_LATENT_UPSCALER
+            timeout 35m pytest -s -v tests/e2e/online_serving/minimax_h3/test_minimax_h3.py -m 'advanced_model and cuda and diffusion' --run-level 'advanced_model'
+        mirror_hardwares: h100_2

.buildkite/cuda/test-nightly.yml

@@ -21,7 +21,7 @@ steps:
           - omni_minicpmo_4_5_function
         timeout_in_minutes: 120
         commands:
-          - pytest -sv tests/e2e --run-level "full_model" -m "full_model and H100 and B200 and omni and cards_1" --ignore=tests/e2e/accuracy --ignore=tests/e2e/online_serving/test_minicpmo_4_5_duplex_expansion.py
+          - pytest -sv tests/e2e --run-level "full_model" -m "full_model and H100 and B200 and omni and cards_1" --ignore=tests/e2e/accuracy --ignore=tests/e2e/online_serving/test_minicpmo_4_5_duplex_expansion.py --ignore=tests/e2e/online_serving/test_minicpmo_4_5_window.py
 
       - label: ":full_moon: Omni · Function Test with H100 · 2-GPU"
         source_file_dependencies:
@@ -33,9 +33,10 @@ steps:
 
       - label: ":full_moon: Omni · MiniCPM-o 4.5 Duplex Test"
         source_file_dependencies: omni_minicpmo_4_5_duplex_function
-        timeout_in_minutes: 50
+        timeout_in_minutes: 120
         commands:
           - pytest -s -v tests/e2e/online_serving/test_minicpmo_4_5_duplex_expansion.py -m "full_model and cuda and H100 and B200 and omni and cards_1" --run-level "full_model"
+          - pytest -s -v tests/e2e/online_serving/test_minicpmo_4_5_window.py -m "full_model and cuda and H100 and B200 and omni and cards_1" --run-level "full_model"
 
       - label: ":full_moon: Omni · Doc Test with L4 · 4-GPU"
         source_file_dependencies: omni_qwen2_5_omni_doc
@@ -53,80 +54,78 @@ steps:
       - label: ":full_moon: Omni · Accuracy Test"
         source_file_dependencies: omni_qwen3_omni_accuracy
         timeout_in_minutes: 180
+        artifact_paths:
+          - tests/e2e/accuracy/qwen3_omni/results/qwen_omni_acc/*.json
         commands:
           - export SEED_TTS_WER_EVAL=1
           - export SEED_TTS_EVAL_DEVICE=cuda:1
-          - |
-            set +e
-            pytest -s -v tests/e2e/accuracy/qwen3_omni/test_qwen3_omni.py -m "full_model and H100 and B200 and cards_2" --run-level full_model
-            EXIT=$$?
-            buildkite-agent artifact upload "tests/e2e/accuracy/qwen3_omni/results/qwen_omni_acc/*.json"
-            exit $$EXIT
+          - pytest -s -v tests/e2e/accuracy/qwen3_omni/test_qwen3_omni.py -m "full_model and H100 and B200 and cards_2" --run-level full_model
 
       - label: ":full_moon: Omni · MiniCPM-o 4.5 · Accuracy Test"
         source_file_dependencies: omni_minicpmo_4_5_accuracy
         timeout_in_minutes: 180
+        artifact_paths:
+          - tests/e2e/accuracy/minicpmo_4_5/results/*.json
         commands:
           - export SEED_TTS_WER_EVAL=1
           - export SEED_TTS_EVAL_DEVICE=cuda:1
+          - pytest -s -v tests/e2e/accuracy/minicpmo_4_5/test_minicpmo_4_5.py -m "full_model and H100 and B200 and cuda and (cards_1 or cards_2)" --run-level full_model
+
+      - label: ":full_moon: Omni · Omni-DuplexEval CI Guard"
+        source_file_dependencies: omni_duplex_eval_ci
+        timeout_in_minutes: 240
+        env:
+          VLLM_DUPLEX_EVAL_JUDGE_URL: "http://127.0.0.1:8001"
+        commands:
           - |
             set +e
-            pytest -s -v tests/e2e/accuracy/minicpmo_4_5/test_minicpmo_4_5.py -m "full_model and H100 and B200 and cuda and (cards_1 or cards_2)" --run-level full_model
+            pytest -s -v tests/e2e/accuracy/omni_duplex_eval/test_omni_duplex_eval_ci.py \
+              -m "full_model and H100 and B200 and cuda and cards_2" \
+              --run-level full_model \
+              --basetemp /tmp/omni_duplex_ci
             EXIT=$$?
-            buildkite-agent artifact upload "tests/e2e/accuracy/minicpmo_4_5/results/*.json"
+            buildkite-agent artifact upload "tests/e2e/accuracy/omni_duplex_eval/results/*.json"
             exit $$EXIT
 
       - label: ":full_moon: Omni · Perf Test · No Async Chunk"
         source_file_dependencies: omni_qwen3_omni_perf
         key: nightly-omni-performance-no-async-chunk
         timeout_in_minutes: 300
+        artifact_paths:
+          - tests/dfx/perf/results/*.json
         commands:
           - export BENCHMARK_DIR=tests/dfx/perf/results
-          - |
-            set +e
-            pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_qwen3_omni_no_async_chunk.json -m "H100 and B200 and cards_2"
-            EXIT=$$?
-            buildkite-agent artifact upload "tests/dfx/perf/results/*.json"
-            exit $$EXIT
+          - pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_qwen3_omni_no_async_chunk.json -m "H100 and B200 and cards_2"
 
       - label: ":full_moon: Omni · Perf Test · Async Chunk"
         source_file_dependencies: omni_qwen3_omni_perf
         key: nightly-omni-performance-async-chunk
         timeout_in_minutes: 180
+        artifact_paths:
+          - tests/dfx/perf/results/*.json
         commands:
           - export BENCHMARK_DIR=tests/dfx/perf/results
-          - |
-            set +e
-            pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_qwen3_omni_async_chunk.json -m "H100 and B200 and full_model and cards_2"
-            EXIT=$$?
-            buildkite-agent artifact upload "tests/dfx/perf/results/*.json"
-            exit $$EXIT
+          - pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_qwen3_omni_async_chunk.json -m "H100 and B200 and full_model and cards_2"
 
       - label: ":full_moon: Omni · MiniCPM-o 4.5 · Perf Test"
         source_file_dependencies: omni_minicpmo_4_5_perf
         key: nightly-omni-performance-minicpmo-4-5
         timeout_in_minutes: 180
+        artifact_paths:
+          - tests/dfx/perf/results/*.json
         commands:
           - export BENCHMARK_DIR=tests/dfx/perf/results
-          - |
-            set +e
-            pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_minicpmo_4_5.json -m "H100 and B200 and cards_1"
-            EXIT=$$?
-            buildkite-agent artifact upload "tests/dfx/perf/results/*.json"
-            exit $$EXIT
+          - pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_minicpmo_4_5.json -m "H100 and B200 and cards_1"
 
       - label: ":full_moon: Omni · MiniCPM-o 4.5 · Duplex Seed-TTS Perf Test"
         source_file_dependencies: omni_minicpmo_4_5_duplex_perf
         key: nightly-omni-performance-minicpmo-4-5-duplex-seed-tts
         timeout_in_minutes: 180
+        artifact_paths:
+          - tests/dfx/perf/results/*.json
         commands:
           - export BENCHMARK_DIR=tests/dfx/perf/results
-          - |
-            set +e
-            pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_minicpmo_4_5_duplex_seed_tts.json -m "H100 and B200 and cards_1"
-            EXIT=$$?
-            buildkite-agent artifact upload "tests/dfx/perf/results/*.json"
-            exit $$EXIT
+          - pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_minicpmo_4_5_duplex_seed_tts.json -m "H100 and B200 and cards_1"
 
       - label: ":full_moon: Omni · Multi-Replica Startup Test with 4x H100"
         source_file_dependencies: omni_qwen3_omni_function
@@ -134,6 +133,20 @@ steps:
         commands:
           - pytest -s -v tests/e2e/online_serving/test_qwen3_omni_multi_replicas.py -m "full_model and H100 and B200 and cards_4" --run-level "core_model"
 
+  - group: ":card_index_dividers: Entrypoints Test"
+    key: nightly-entrypoints-test-group
+    depends_on: upload-nightly-pipeline
+    if: build.env("NIGHTLY") == "1" || build.pull_request.labels includes "nightly-test"
+    steps:
+
+      - label: ":full_moon: Entrypoint Test with H100"
+        source_file_dependencies: entrypoints_function
+        # Merge collects advanced_model (5 duplex interruptions). Nightly
+        # collects full_model (20 interruptions + BAGEL sleep).
+        timeout_in_minutes: 180
+        commands:
+          - pytest -s -v tests/entrypoints/ -m "full_model and cuda and H100 and B200 and cards_2" --run-level "full_model"
+
   - group: ":card_index_dividers: TTS Model Test"
     key: nightly-tts-test-group
     depends_on: upload-nightly-pipeline
@@ -153,29 +166,23 @@ steps:
         source_file_dependencies: tts_qwen3_tts_perf
         key: nightly-tts-performance-single
         timeout_in_minutes: 180
+        artifact_paths:
+          - tests/dfx/perf/results/*.json
         commands:
           - export BENCHMARK_DIR=tests/dfx/perf/results
           - export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
-          - |
-            set +e
-            pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_tts.json -m "H100 and B200 and cards_1"
-            EXIT=$$?
-            buildkite-agent artifact upload "tests/dfx/perf/results/*.json"
-            exit $$EXIT
+          - pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_tts.json -m "H100 and B200 and cards_1"
 
       - label: ":full_moon: TTS · Perf Test · 2-GPU"
         source_file_dependencies: tts_qwen3_tts_perf
         key: nightly-tts-performance-2gpu
         timeout_in_minutes: 180
+        artifact_paths:
+          - tests/dfx/perf/results/*.json
         commands:
           - export BENCHMARK_DIR=tests/dfx/perf/results
           - export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
-          - |
-            set +e
-            pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_tts.json -m "H100 and B200 and cards_2"
-            EXIT=$$?
-            buildkite-agent artifact upload "tests/dfx/perf/results/*.json"
-            exit $$EXIT
+          - pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_tts.json -m "H100 and B200 and cards_2"
 
   - group: ":card_index_dividers: Tiny Model Tests [Multi-GPU]"
     key: nightly-tiny-model-test-group
@@ -223,17 +230,6 @@ steps:
             -m "full_model and diffusion and H100 and B200 and cards_1"
             --run-level "full_model"
 
-      - label: ":robot_face: Robot Policy OpenPI · π0.5 · H100 · Single-GPU"
-        timeout_in_minutes: 120
-        commands:
-          - pip install --no-deps "openpi-client @ git+https://github.com/Physical-Intelligence/openpi.git@215abfb217dbac7d5f1273282331b9b1866c0479#subdirectory=packages/openpi-client"
-          - >-
-            pytest -sv
-            tests/e2e/online_serving/test_pi05_expansion.py
-            -m "full_model and diffusion and H100 and cards_1"
-            --run-level "full_model"
-        mirror_hardwares: h100_1
-
       - label: ":full_moon: Diffusion X2I(&A&T) · Function Test with H100 · 2-GPU"
         source_file_dependencies: diffusion_qwen_image_function
         timeout_in_minutes: 120
@@ -271,39 +267,33 @@ steps:
         commands:
           - export HUNYUAN_IMAGE3_MODEL="tencent/HunyuanImage-3.0-Instruct"
           - export HUNYUAN_IMAGE3_DEVICES="0,1,2,3"
-          - pytest -s -v tests/e2e/accuracy/test_hunyuan_image3_pixel_accuracy.py -m "full_model and H100 and B200 and cards_4" --run-level full_model
+          - pytest -s -v tests/e2e/accuracy/test_hunyuan_image3_pixel_accuracy.py tests/e2e/accuracy/test_hunyuan_image3_lora.py -m "full_model and H100 and B200 and cards_4" --run-level full_model
 
       - label: ":full_moon: Diffusion X2I(&A&T) · Perf Test · Qwen-Image · Single-GPU"
         source_file_dependencies: diffusion_qwen_image_perf
         key: nightly-diffusion-x2iat-performance-qwen-image-single
         timeout_in_minutes: 180
+        artifact_paths:
+          - tests/dfx/perf/results/diffusion_result_*.json
+          - tests/dfx/perf/results/logs/*.log
         commands:
           - export DIFFUSION_BENCHMARK_DIR=tests/dfx/perf/results
           - export CACHE_DIT_VERSION=1.5.0
-          - |
-            set +e
-            pytest -s -v tests/dfx/perf/scripts/run_diffusion_benchmark.py --test-config-file tests/dfx/perf/tests/test_qwen_image_vllm_omni.json -m "H100 and B200 and cards_1"
-            EXIT=$$?
-            buildkite-agent artifact upload "tests/dfx/perf/results/diffusion_result_*.json"
-            buildkite-agent artifact upload "tests/dfx/perf/results/logs/*.log"
-            exit $$EXIT
+          - pytest -s -v tests/dfx/perf/scripts/run_diffusion_benchmark.py --test-config-file tests/dfx/perf/tests/test_qwen_image_vllm_omni.json -m "H100 and B200 and cards_1"
 
       - label: ":full_moon: Diffusion X2I(&A&T) · Perf Test · Qwen-Image · 4-GPU"
         source_file_dependencies: diffusion_qwen_image_perf
         key: nightly-diffusion-x2iat-performance-qwen-image-4gpu
         timeout_in_minutes: 180
+        artifact_paths:
+          - tests/dfx/perf/results/diffusion_result_*.json
+          - tests/dfx/perf/results/logs/*.log
         commands:
           - export DIFFUSION_BENCHMARK_DIR=tests/dfx/perf/results
           - export CACHE_DIT_VERSION=1.5.0
           # Do not pin DIFFUSION_ATTENTION_BACKEND: H100 auto-selects FLASH_ATTN;
           # B200 rejects explicit FLASH_ATTN without FA4 and uses CUDNN/TRTLLM.
-          - |
-            set +e
-            pytest -s -v tests/dfx/perf/scripts/run_diffusion_benchmark.py --test-config-file tests/dfx/perf/tests/test_qwen_image_vllm_omni.json -m "H100 and B200 and cards_4"
-            EXIT=$$?
-            buildkite-agent artifact upload "tests/dfx/perf/results/diffusion_result_*.json"
-            buildkite-agent artifact upload "tests/dfx/perf/results/logs/*.log"
-            exit $$EXIT
+          - pytest -s -v tests/dfx/perf/scripts/run_diffusion_benchmark.py --test-config-file tests/dfx/perf/tests/test_qwen_image_vllm_omni.json -m "H100 and B200 and cards_4"
 
   - group: ":card_index_dividers: Diffusion X2V Model Test"
     key: nightly-diffusion-x2v-group
@@ -399,15 +389,11 @@ steps:
         source_file_dependencies: diffusion_wan22_perf
         key: nightly-diffusion-x2v-performance-single
         timeout_in_minutes: 180
+        artifact_paths:
+          - tests/dfx/perf/results/*.json
         commands:
-          - export DIFFUSION_BENCHMARK_DIR=tests/dfx/perf/results
-          - |
-            set +e
-            pytest -s -v tests/dfx/perf/scripts/run_diffusion_benchmark.py --test-config-file tests/dfx/perf/tests/test_wan22_i2v_vllm_omni.json -m "H100 and B200 and cards_1"
-            EXIT=$$?
-            buildkite-agent artifact upload "tests/dfx/perf/results/diffusion_result_*.json"
-            buildkite-agent artifact upload "tests/dfx/perf/results/logs/*.log"
-            exit $$EXIT
+          - export BENCHMARK_DIR=tests/dfx/perf/results
+          - pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_wan22_i2v_vllm_omni.json -m "H100 and B200 and cards_1"
 
       - label: ":full_moon: Diffusion X2V · Perf Test · 2-GPU"
         source_file_dependencies:
@@ -415,16 +401,16 @@ steps:
           - diffusion_cosmos3_perf
         key: nightly-diffusion-x2v-performance-2gpu
         timeout_in_minutes: 180
+        artifact_paths:
+          - tests/dfx/perf/results/*.json
         commands:
-          - export DIFFUSION_BENCHMARK_DIR=tests/dfx/perf/results
+          - export BENCHMARK_DIR=tests/dfx/perf/results
           - |
             set +e
-            pytest -s -v tests/dfx/perf/scripts/run_diffusion_benchmark.py --test-config-file tests/dfx/perf/tests/test_wan22_i2v_vllm_omni.json -m "H100 and B200 and cards_2"
+            pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_wan22_i2v_vllm_omni.json -m "H100 and B200 and cards_2"
             EXIT1=$$?
-            pytest -s -v tests/dfx/perf/scripts/run_diffusion_benchmark.py --test-config-file tests/dfx/perf/tests/test_cosmos3_vllm_omni.json -m "H100 and B200 and cards_2"
+            pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_cosmos3_vllm_omni.json -m "H100 and B200 and cards_2"
             EXIT2=$$?
-            buildkite-agent artifact upload "tests/dfx/perf/results/diffusion_result_*.json"
-            buildkite-agent artifact upload "tests/dfx/perf/results/logs/*.log"
             if [ $$EXIT1 -ne 0 ]; then exit $$EXIT1; fi
             exit $$EXIT2
 
@@ -432,16 +418,16 @@ steps:
         source_file_dependencies: diffusion_hunyuan_video_perf
         key: nightly-diffusion-x2v-performance-hunyuanvideo15
         timeout_in_minutes: 180
+        artifact_paths:
+          - tests/dfx/perf/results/*.json
         commands:
-          - export DIFFUSION_BENCHMARK_DIR=tests/dfx/perf/results
+          - export BENCHMARK_DIR=tests/dfx/perf/results
           - |
             set +e
-            pytest -s -v tests/dfx/perf/scripts/run_diffusion_benchmark.py --test-config-file tests/dfx/perf/tests/test_hunyuanvideo15_t2v_vllm_omni.json -m "H100 and B200 and (cards_1 or cards_2)"
+            pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_hunyuanvideo15_t2v_vllm_omni.json -m "H100 and B200 and (cards_1 or cards_2)"
             EXIT1=$$?
-            pytest -s -v tests/dfx/perf/scripts/run_diffusion_benchmark.py --test-config-file tests/dfx/perf/tests/test_hunyuanvideo15_i2v_vllm_omni.json -m "H100 and B200 and (cards_1 or cards_2)"
+            pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_hunyuanvideo15_i2v_vllm_omni.json -m "H100 and B200 and (cards_1 or cards_2)"
             EXIT2=$$?
-            buildkite-agent artifact upload "tests/dfx/perf/results/diffusion_result_*.json"
-            buildkite-agent artifact upload "tests/dfx/perf/results/logs/*.log"
             if [ $$EXIT1 -ne 0 ]; then exit $$EXIT1; fi
             exit $$EXIT2
 
@@ -449,15 +435,21 @@ steps:
         source_file_dependencies: diffusion_minimax_h3_perf
         key: nightly-diffusion-x2v-performance-minimax-h3
         timeout_in_minutes: 180
+        artifact_paths:
+          - tests/dfx/perf/results/*.json
         commands:
-          - export DIFFUSION_BENCHMARK_DIR=tests/dfx/perf/results
-          - |
-            set +e
-            pytest -s -v tests/dfx/perf/scripts/run_diffusion_benchmark.py --test-config-file tests/dfx/perf/tests/test_minimax_h3_vllm_omni.json -m "H100 and B200 and cards_4"
-            EXIT=$$?
-            buildkite-agent artifact upload "tests/dfx/perf/results/diffusion_result_*.json"
-            buildkite-agent artifact upload "tests/dfx/perf/results/logs/*.log"
-            exit $$EXIT
+          - export BENCHMARK_DIR=tests/dfx/perf/results
+          - pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_minimax_h3_vllm_omni.json -m "H100 and B200 and cards_4"
+
+      - label: ":full_moon: Diffusion X2V · Perf Test · LingBot"
+        source_file_dependencies: diffusion_lingbot_perf
+        key: nightly-diffusion-x2v-performance-lingbot
+        timeout_in_minutes: 180
+        artifact_paths:
+          - tests/dfx/perf/results/*.json
+        commands:
+          - export BENCHMARK_DIR=tests/dfx/perf/results
+          - pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_lingbot_video_vllm_omni.json -m "H100 and B200 and cards_1"
 
   - group: ":card_index_dividers: Diffusion Test"
     key: nightly-diffusion-test-group
@@ -533,6 +525,9 @@ steps:
       - nightly-diffusion-x2v-performance-single
       - nightly-diffusion-x2v-performance-2gpu
     if: build.env("NIGHTLY") == "1" && build.env("EMAIL_DISTRIBUTION") == "1"
+    artifact_paths:
+      - tests/dfx/perf/results/*.xlsx
+      - tests/dfx/perf/results/*.html
     commands:
       - pip install openpyxl
       - export DEFAULT_INPUT_DIR=tests/dfx/perf/results
@@ -548,7 +543,5 @@ steps:
       - python tools/nightly/generate_nightly_perf_excel.py
       - python tools/nightly/generate_nightly_perf_html.py
       - python tools/nightly/send_nightly_email.py --report-file "tests/dfx/perf/results/*.xlsx, tests/dfx/perf/results/*.html"
-      - buildkite-agent artifact upload "tests/dfx/perf/results/*.xlsx"
-      - buildkite-agent artifact upload "tests/dfx/perf/results/*.html"
     agents:
       queue: "cpu_queue_premerge"

.buildkite/cuda/test-ready.yml

@@ -16,8 +16,27 @@ steps:
           - "pytest -sv tests/model_executor -m 'core_model and cpu'"
         mirror_hardwares: l4_1
 
+      - label: "Simple · CosyVoice3 Packed Flow Test"
+        timeout_in_minutes: 15
+        source_file_dependencies:
+          - tests/model_executor/models/cosyvoice3/test_cosyvoice3_full_batch.py
+          - vllm_omni/model_executor/models/cosyvoice3/
+        commands:
+          - "pytest -sv tests/model_executor/models/cosyvoice3/test_cosyvoice3_full_batch.py -k packed -m 'core_model and cuda' --run-level=core_model"
+        mirror_hardwares: h100_1
+
+      - label: "Simple · MRV2 FA3 Graph Bounds Test"
+        timeout_in_minutes: 15
+        source_file_dependencies:
+          - tests/worker_v2/test_capture_query_bound_gpu.py
+          - tests/worker_v2/test_decode_query_bound_gpu.py
+          - vllm_omni/worker_v2/model_states/omni_model_state.py
+        commands:
+          - "pytest -sv tests/worker_v2/test_capture_query_bound_gpu.py tests/worker_v2/test_decode_query_bound_gpu.py -m 'core_model and cuda and H100' --run-level=core_model"
+        mirror_hardwares: h100_1
+
       - label: "Simple · Diffusion Test"
-        timeout_in_minutes: 20
+        timeout_in_minutes: 30
         commands:
           - "pytest -sv tests/diffusion -m 'core_model and cpu'"
         mirror_hardwares: l4_1
@@ -69,6 +88,7 @@ steps:
         source_file_dependencies: diffusion_distributed_attention
         commands:
           - timeout 20m pytest -sv tests/diffusion/distributed -m 'core_model and cuda and L4 and cards_2' --run-level "core_model"
+          - timeout 10m pytest -sv tests/diffusion/attention/test_ulysses_uaa.py -m 'core_model and cuda and L4 and cards_2' --run-level "core_model"
         mirror_hardwares: l4_2
 
       - label: "Diffusion · Distributed & Attention Test · 4-GPU"
@@ -117,11 +137,18 @@ steps:
 
   - label: "Entrypoints Test"
     depends_on: upload-ready-pipeline
+    # Workaround https://github.com/vllm-project/vllm-omni/issues/8016: shutdown()
+    # leaves inline UniProc DiffusionWorker VRAM mapped in the pytest parent.
+    # Isolate that leftover case in its own process so the driver drops the
+    # mapping; the rest stays a directory sweep with --deselect. Revert to a
+    # single sweep after the product shutdown fix.
     commands:
       - |
         timeout 30m bash -c "
+          set -eu
           export VLLM_WORKER_MULTIPROC_METHOD=spawn
-          pytest -sv tests/entrypoints/ -m 'core_model and cuda' --run-level 'core_model'
+          pytest -sv tests/entrypoints/test_async_omni.py::test_diffusion_generate_request_id -m 'core_model and cuda' --run-level 'core_model'
+          pytest -sv tests/entrypoints/ -m 'core_model and cuda' --run-level 'core_model' --deselect tests/entrypoints/test_async_omni.py::test_diffusion_generate_request_id
         "
     mirror_hardwares: l4_1
 
@@ -188,6 +215,16 @@ steps:
           - pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_minicpmo_4_5_ready.json -m 'core_model and cuda'
         mirror_hardwares: h100_1
 
+      - label: "TTS · Higgs v3 Full Response Test"
+        source_file_dependencies:
+          - tests/e2e/online_serving/test_higgs_audio_v3.py
+          - vllm_omni/model_executor/models/higgs_audio_v3/
+          - vllm_omni/model_executor/stage_input_processors/higgs_audio_v3.py
+          - vllm_omni/deploy/higgs_multimodal_qwen3.yaml
+        commands:
+          - timeout 20m pytest -s -v tests/e2e/online_serving/test_higgs_audio_v3.py::test_full_response_v1_preserves_audio -m 'core_model and cuda' --run-level 'core_model'
+        mirror_hardwares: h100_1
+
       - label: "TTS · Qwen3-TTS CustomVoice Test"
         source_file_dependencies: tts_qwen3_tts_function
         commands:
@@ -204,7 +241,7 @@ steps:
           - |
             timeout 40m bash -c "
               export VLLM_IMAGE_FETCH_TIMEOUT=60
-              pytest -s -v tests/e2e/online_serving/test_wan22_t2v.py -m 'core_model' --run-level 'core_model'
+              pytest -s -v tests/e2e/online_serving/test_wan22_t2v.py -m 'core_model and cuda' --run-level 'core_model'
             "
         mirror_hardwares: h100_1
 

.buildkite/cuda/test-weekly.yml

@@ -12,6 +12,8 @@ steps:
     depends_on: upload-weekly-pipeline
     if: build.env("WEEKLY") == "1"
     timeout_in_minutes: 60
+    artifact_paths:
+      - coverage-core-model-cpu-*.xml.gz
     commands:
       - |
         set +e
@@ -19,7 +21,6 @@ steps:
         pytest -sv tests/ -m 'core_model and cpu' --cov=vllm_omni --cov-report=term-missing:skip-covered --cov-report=xml:$${REPORT}
         EXIT=$$?
         gzip -9 -f "$${REPORT}"
-        buildkite-agent artifact upload "$${REPORT}.gz"
         exit $$EXIT
     mirror_hardwares: l4_1
 
@@ -79,41 +80,32 @@ steps:
         source_file_dependencies: omni_qwen3_omni_perf
         key: weekly-omni-performance-vllm-text
         timeout_in_minutes: 180
+        artifact_paths:
+          - tests/dfx/perf/results/*.json
         commands:
           - export BENCHMARK_DIR=tests/dfx/perf/results
-          - |
-            set +e
-            pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_qwen3_omni_vllm_text.json
-            EXIT=$$?
-            buildkite-agent artifact upload "tests/dfx/perf/results/*.json"
-            exit $$EXIT
+          - pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_qwen3_omni_vllm_text.json
         mirror_hardwares: h100_2
 
       - label: ":full_moon: Omni · Perf Test · Async Chunk · Random"
         key: weekly-omni-performance-async-chunk-random
         timeout_in_minutes: 180
+        artifact_paths:
+          - tests/dfx/perf/results/*.json
         commands:
           - export BENCHMARK_DIR=tests/dfx/perf/results
-          - |
-            set +e
-            pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_qwen3_omni_async_chunk.json -m "H100 and slow"
-            EXIT=$$?
-            buildkite-agent artifact upload "tests/dfx/perf/results/*.json"
-            exit $$EXIT
+          - pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_qwen3_omni_async_chunk.json -m "H100 and slow"
         mirror_hardwares: h100_2
 
       - label: ":full_moon: Omni · Perf Test · Multi-Replica"
         source_file_dependencies: omni_qwen3_omni_perf
         key: weekly-omni-performance-multi-replicas
         timeout_in_minutes: 180
+        artifact_paths:
+          - tests/dfx/perf/results/*.json
         commands:
           - export BENCHMARK_DIR=tests/dfx/perf/results
-          - |
-            set +e
-            pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_qwen3_omni_multi_replicas.json
-            EXIT=$$?
-            buildkite-agent artifact upload "tests/dfx/perf/results/*.json"
-            exit $$EXIT
+          - pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_qwen3_omni_multi_replicas.json
         mirror_hardwares: h100_3
 
   - group: ":card_index_dividers: E2E Tests"
@@ -160,15 +152,122 @@ steps:
             --run-level "full_model"
         mirror_hardwares: h100_4
 
-      - label: "TTS · L4"
-        timeout_in_minutes: 120
+      - label: "TTS · Moss · L4"
+        timeout_in_minutes: 90
         commands:
           - export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
           - export VLLM_USE_DEEP_GEMM="0"
           - export VLLM_MOE_USE_DEEP_GEMM="0"
           - |
             failed=0
-            for f in $(pytest --collect-only -q tests/e2e/ -m "slow and L4 and tts" --run-level "full_model" | grep '\.py::' | sed -E 's|::.*||' | sort -u); do
+            for f in \
+              tests/e2e/offline_inference/test_moss_tts_expansion.py \
+              tests/e2e/offline_inference/test_moss_tts_nano_expansion.py \
+              tests/e2e/offline_inference/test_moss_tts_realtime.py \
+              tests/e2e/offline_inference/test_moss_tts_realtime_expansion.py \
+              tests/e2e/online_serving/test_moss_tts_expansion.py \
+              tests/e2e/online_serving/test_moss_tts_nano_expansion.py
+            do
+              pytest -s -v "$$f" -m "slow and L4 and tts" --run-level "full_model" || failed=1
+            done
+            exit $$failed
+        mirror_hardwares: l4_1
+
+      - label: "TTS · VoxCPM2 · Voxtral · L4"
+        timeout_in_minutes: 90
+        commands:
+          - export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
+          - export VLLM_USE_DEEP_GEMM="0"
+          - export VLLM_MOE_USE_DEEP_GEMM="0"
+          - |
+            failed=0
+            for f in \
+              tests/e2e/offline_inference/test_voxcpm2_tts.py \
+              tests/e2e/online_serving/test_voxcpm2_tts.py \
+              tests/e2e/online_serving/test_voxcpm2_tts_expansion.py \
+              tests/e2e/online_serving/test_voxcpm2_tts_lora.py \
+              tests/e2e/offline_inference/test_voxtral_tts_expansion.py \
+              tests/e2e/online_serving/test_voxtral_tts_expansion.py
+            do
+              pytest -s -v "$$f" -m "slow and L4 and tts" --run-level "full_model" || failed=1
+            done
+            exit $$failed
+        mirror_hardwares: l4_1
+
+      - label: "TTS · Higgs · L4"
+        timeout_in_minutes: 90
+        commands:
+          - export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
+          - export VLLM_USE_DEEP_GEMM="0"
+          - export VLLM_MOE_USE_DEEP_GEMM="0"
+          - |
+            failed=0
+            for f in \
+              tests/e2e/offline_inference/test_higgs_audio_v2_expansion.py \
+              tests/e2e/online_serving/test_higgs_audio_v2_expansion.py \
+              tests/e2e/online_serving/test_higgs_audio_v3.py
+            do
+              pytest -s -v "$$f" -m "slow and L4 and tts" --run-level "full_model" || failed=1
+            done
+            exit $$failed
+        mirror_hardwares: l4_1
+
+      - label: "TTS · OmniVoice · Step-Audio · L4"
+        timeout_in_minutes: 90
+        commands:
+          - export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
+          - export VLLM_USE_DEEP_GEMM="0"
+          - export VLLM_MOE_USE_DEEP_GEMM="0"
+          - |
+            failed=0
+            for f in \
+              tests/e2e/offline_inference/test_omnivoice_expansion.py \
+              tests/e2e/online_serving/test_omnivoice_expansion.py \
+              tests/e2e/online_serving/test_omnivoice_parity.py \
+              tests/e2e/offline_inference/test_step_audio2_expansion.py
+            do
+              pytest -s -v "$$f" -m "slow and L4 and tts" --run-level "full_model" || failed=1
+            done
+            exit $$failed
+        mirror_hardwares: l4_1
+
+      - label: "TTS · Ming · GLM · IndexTTS · Breeze · L4"
+        timeout_in_minutes: 90
+        commands:
+          - export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
+          - export VLLM_USE_DEEP_GEMM="0"
+          - export VLLM_MOE_USE_DEEP_GEMM="0"
+          - |
+            failed=0
+            for f in \
+              tests/e2e/offline_inference/test_ming_tts_expansion.py \
+              tests/e2e/online_serving/test_ming_tts_expansion.py \
+              tests/e2e/offline_inference/test_glm_tts_expansion.py \
+              tests/e2e/online_serving/test_glm_tts_expansion.py \
+              tests/e2e/offline_inference/test_indextts2_expansion.py \
+              tests/e2e/online_serving/test_indextts2_expansion.py \
+              tests/e2e/online_serving/test_breeze_tts_2.py
+            do
+              pytest -s -v "$$f" -m "slow and L4 and tts" --run-level "full_model" || failed=1
+            done
+            exit $$failed
+        mirror_hardwares: l4_1
+
+      - label: "TTS · Audex · CosyVoice · Qwen3-TTS · L4"
+        timeout_in_minutes: 90
+        commands:
+          - export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
+          - export VLLM_USE_DEEP_GEMM="0"
+          - export VLLM_MOE_USE_DEEP_GEMM="0"
+          - |
+            failed=0
+            for f in \
+              tests/e2e/offline_inference/test_audex_tts.py \
+              tests/e2e/offline_inference/test_audex_s2t.py \
+              tests/e2e/offline_inference/test_audex_tta.py \
+              tests/e2e/offline_inference/test_cosyvoice3_expansion.py \
+              tests/e2e/online_serving/test_qwen3_tts_word_timestamps.py
+            do
               pytest -s -v "$$f" -m "slow and L4 and tts" --run-level "full_model" || failed=1
             done
             exit $$failed
@@ -232,6 +331,17 @@ steps:
 
       # Split the old shared H100 Single-GPU sweep: one pytest process collected
       # every slow+cards_1 diffusion file and exceeded 150 minutes.
+      - label: ":robot_face: Robot Policy OpenPI · π0.5 · H100 · Single-GPU"
+        timeout_in_minutes: 120
+        commands:
+          - pip install --no-deps "openpi-client @ git+https://github.com/Physical-Intelligence/openpi.git@215abfb217dbac7d5f1273282331b9b1866c0479#subdirectory=packages/openpi-client"
+          - >-
+            pytest -sv
+            tests/e2e/online_serving/test_pi05_expansion.py
+            -m "slow and diffusion and H100 and cards_1"
+            --run-level "full_model"
+        mirror_hardwares: h100_1
+
       - label: "Diffusion · Qwen-Image · H100 · Single-GPU"
         timeout_in_minutes: 120
         commands:
@@ -424,6 +534,7 @@ steps:
           - >-
             pytest -sv
             tests/e2e/online_serving/test_bagel_expansion.py
+            tests/e2e/offline_inference/test_bagel_expansion.py
             -m "slow and diffusion and H100 and cards_2"
             --run-level "full_model"
         mirror_hardwares: h100_2

.buildkite/intel/pipeline-intel.yml

@@ -10,7 +10,7 @@ steps:
           DOCKER_BUILDKIT: "1"
           # Buildkite will automatically replace this with the actual commit hash
           VLLM_IMAGE_TAG: "${BUILDKITE_COMMIT}"
-          VLLM_VERSION: "v0.29.0"
+          VLLM_VERSION: "v0.30.0"
         priority: 100
         timeout_in_minutes: 60
         soft_fail: false

.buildkite/npu/test-npu-nightly.yml

@@ -64,6 +64,11 @@ steps:
             tests/e2e/online_serving/test_minicpmo_4_5_duplex_expansion.py
             -m "npu and A3 and omni"
             --run-level full_model
+          - >-
+            pytest -s -v
+            tests/e2e/online_serving/test_minicpmo_4_5_window.py
+            -m "full_model and npu and A3 and omni"
+            --run-level full_model
       - label: ":full_moon: Omni · MiniCPM-o 4.5 · Accuracy Test"
         source_file_dependencies: omni_minicpmo_4_5_accuracy
         timeout_in_minutes: 180
@@ -85,6 +90,25 @@ steps:
             EXIT=$$?
             buildkite-agent artifact upload "tests/e2e/accuracy/minicpmo_4_5/results/*.json"
             exit $$EXIT
+      - label: ":full_moon: Omni · Omni-DuplexEval CI Guard"
+        source_file_dependencies: omni_duplex_eval_ci
+        timeout_in_minutes: 240
+        mirror_hardwares: a3_npu_2
+        env:
+          VLLM_WORKER_MULTIPROC_METHOD: spawn
+          VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
+          HF_TOKEN: "${HF_TOKEN}"
+          VLLM_DUPLEX_EVAL_JUDGE_URL: "http://127.0.0.1:8001"
+        commands:
+          - |
+            set +e
+            pytest -s -v tests/e2e/accuracy/omni_duplex_eval/test_omni_duplex_eval_ci.py \
+              -m "full_model and npu and A3 and cards_2" \
+              --run-level full_model \
+              --basetemp /tmp/omni_duplex_ci
+            EXIT=$$?
+            buildkite-agent artifact upload "tests/e2e/accuracy/omni_duplex_eval/results/*.json"
+            exit $$EXIT
       - label: ":full_moon: Omni · MiniCPM-o 4.5 · Perf Test"
         source_file_dependencies: omni_minicpmo_4_5_perf
         key: nightly-omni-performance-minicpmo-4-5-npu
@@ -228,6 +252,42 @@ steps:
             buildkite-agent artifact upload "tests/dfx/perf/results/diffusion_result_*.json"
             buildkite-agent artifact upload "tests/dfx/perf/results/logs/*.log"
             exit $$EXIT
+      - label: ":full_moon: Diffusion X2I(&A&T) · Perf Test · Qwen-Image · Single-NPU"
+        source_file_dependencies: diffusion_qwen_image_perf
+        key: nightly-diffusion-x2iat-performance-qwen-image-single-npu
+        timeout_in_minutes: 180
+        mirror_hardwares: a3_npu_2
+        env:
+          VLLM_WORKER_MULTIPROC_METHOD: spawn
+          HF_TOKEN: "${HF_TOKEN}"
+          DIFFUSION_ATTENTION_BACKEND: TORCH_SDPA
+        commands:
+          - export DIFFUSION_BENCHMARK_DIR=tests/dfx/perf/results
+          - |
+            set +e
+            pytest -s -v tests/dfx/perf/scripts/run_diffusion_benchmark.py -m "npu and A3 and cards_1" --test-config-file tests/dfx/perf/tests/test_qwen_image_vllm_omni.json
+            EXIT=$$?
+            buildkite-agent artifact upload "tests/dfx/perf/results/diffusion_result_*.json"
+            buildkite-agent artifact upload "tests/dfx/perf/results/logs/*.log"
+            exit $$EXIT
+      - label: ":full_moon: Diffusion X2I(&A&T) · Perf Test · Qwen-Image · 4-NPU"
+        source_file_dependencies: diffusion_qwen_image_perf
+        key: nightly-diffusion-x2iat-performance-qwen-image-4npu
+        timeout_in_minutes: 180
+        mirror_hardwares: a3_npu_4
+        env:
+          VLLM_WORKER_MULTIPROC_METHOD: spawn
+          HF_TOKEN: "${HF_TOKEN}"
+          DIFFUSION_ATTENTION_BACKEND: TORCH_SDPA
+        commands:
+          - export DIFFUSION_BENCHMARK_DIR=tests/dfx/perf/results
+          - |
+            set +e
+            pytest -s -v tests/dfx/perf/scripts/run_diffusion_benchmark.py -m "npu and A3 and cards_4" --test-config-file tests/dfx/perf/tests/test_qwen_image_vllm_omni.json
+            EXIT=$$?
+            buildkite-agent artifact upload "tests/dfx/perf/results/diffusion_result_*.json"
+            buildkite-agent artifact upload "tests/dfx/perf/results/logs/*.log"
+            exit $$EXIT
   - group: ":card_index_dividers: Diffusion X2V Model Test"
     key: nightly-diffusion-x2v-group
     depends_on: upload-nightly-pipeline
@@ -258,13 +318,12 @@ steps:
           HF_HUB_OFFLINE: "1"
           DIFFUSION_ATTENTION_BACKEND: TORCH_SDPA
         commands:
-          - export DIFFUSION_BENCHMARK_DIR=tests/dfx/perf/results
+          - export BENCHMARK_DIR=tests/dfx/perf/results
           - |
             set +e
-            pytest -s -v tests/dfx/perf/scripts/run_diffusion_benchmark.py -m "npu and A3" --test-config-file tests/dfx/perf/tests/test_hunyuanvideo15_t2v_vllm_omni.json
+            pytest -s -v tests/dfx/perf/scripts/run_benchmark.py -m "npu and A3" --test-config-file tests/dfx/perf/tests/test_hunyuanvideo15_t2v_vllm_omni.json
             EXIT=$$?
-            buildkite-agent artifact upload "tests/dfx/perf/results/diffusion_result_*.json"
-            buildkite-agent artifact upload "tests/dfx/perf/results/logs/*.log"
+            buildkite-agent artifact upload "tests/dfx/perf/results/*.json"
             exit $$EXIT
       - label: ":full_moon: Diffusion X2V · Wan 2.2 Function Test"
         source_file_dependencies: diffusion_wan22_function
@@ -286,11 +345,10 @@ steps:
           HF_TOKEN: "${HF_TOKEN}"
           DIFFUSION_ATTENTION_BACKEND: TORCH_SDPA
         commands:
-          - export DIFFUSION_BENCHMARK_DIR=tests/dfx/perf/results
+          - export BENCHMARK_DIR=tests/dfx/perf/results
           - |
             set +e
-            pytest -s -v tests/dfx/perf/scripts/run_diffusion_benchmark.py -m npu --test-config-file tests/dfx/perf/tests/test_wan22_i2v_vllm_omni.json
+            pytest -s -v tests/dfx/perf/scripts/run_benchmark.py -m npu --test-config-file tests/dfx/perf/tests/test_wan22_i2v_vllm_omni.json
             EXIT=$$?
-            buildkite-agent artifact upload "tests/dfx/perf/results/diffusion_result_*.json"
-            buildkite-agent artifact upload "tests/dfx/perf/results/logs/*.log"
+            buildkite-agent artifact upload "tests/dfx/perf/results/*.json"
             exit $$EXIT

.buildkite/npu/test-npu-ready.yml

@@ -55,8 +55,8 @@ steps:
             -m 'core_model and npu and 310P'
             --run-level 'core_model'
 
-  - group: ":card_index_dividers: Omni Model Test"
-    key: ready-omni-group
+  - group: ":card_index_dividers: E2E Test"
+    key: ready-e2e-group
     depends_on: upload-ready-pipeline
     steps:
       - label: "Omni · MiniCPM-o 4.5 Test"
@@ -66,6 +66,7 @@ steps:
           VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
         commands:
           - pytest -s -v tests/e2e/offline_inference/test_minicpmo_4_5.py tests/e2e/online_serving/test_minicpmo_4_5.py -m 'core_model and npu and A3 and omni' --run-level 'core_model'
+
       - label: "Omni · MiniCPM-o 4.5 Duplex Test"
         mirror_hardwares: a3_npu_2
         env:
@@ -73,6 +74,7 @@ steps:
           VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
         commands:
           - pytest -s -v tests/e2e/online_serving/test_minicpmo_4_5_duplex.py -m 'core_model and npu and A3 and omni' --run-level 'core_model'
+
       - label: "Omni · MiniCPM-o 4.5 Perf Test"
         timeout_in_minutes: 60
         mirror_hardwares: a3_npu_2
@@ -83,10 +85,15 @@ steps:
           - export BENCHMARK_DIR=tests/dfx/perf/results
           - pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_minicpmo_4_5_ready.json -m 'core_model and npu and A3 and omni'
 
-  - group: ":card_index_dividers: TTS Model Test"
-    key: ready-tts-group
-    depends_on: upload-ready-pipeline
-    steps:
+      - label: "Omni · Qwen3-Omni Test"
+        mirror_hardwares: a3_npu_4
+        env:
+          VLLM_WORKER_MULTIPROC_METHOD: spawn
+          VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
+        commands:
+          - pytest -s -v tests/e2e/online_serving/test_qwen3_omni.py -m 'core_model and npu and A3 and omni' --run-level 'core_model'
+        timeout_in_minutes: 60
+
       - label: ":sound: TTS · Qwen3-TTS CustomVoice Test"
         mirror_hardwares: a3_npu_2
         env:
@@ -95,3 +102,55 @@ steps:
           HF_TOKEN: "${HF_TOKEN}"
         commands:
           - pytest -s -v tests/e2e/online_serving/test_qwen3_tts_customvoice.py -m 'core_model and npu' --run-level 'core_model'
+
+      - label: "Diffusion · HunyuanVideo-1.5 Test"
+        mirror_hardwares: a3_npu_2
+        env:
+            VLLM_WORKER_MULTIPROC_METHOD: spawn
+            VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
+        commands:
+            - |
+              timeout 40m bash -c "
+                export VLLM_IMAGE_FETCH_TIMEOUT=60
+                pytest -s -v tests/e2e/online_serving/test_hunyuan_video_15_expansion.py -m 'core_model and diffusion and npu' --run-level 'core_model'
+                "
+        timeout_in_minutes: 60
+
+      - label: "Diffusion · Qwen Image Test"
+        mirror_hardwares: a3_npu_2
+        env:
+            VLLM_WORKER_MULTIPROC_METHOD: spawn
+            VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
+        commands:
+            - |
+                timeout 40m bash -c "
+                export VLLM_IMAGE_FETCH_TIMEOUT=60
+                pytest -s -v tests/e2e/online_serving/test_qwen_image.py -m 'core_model and npu' --run-level 'core_model'
+                "
+        timeout_in_minutes: 60
+
+      - label: "Diffusion · Wan22 Test"
+        mirror_hardwares: a3_npu_2
+        env:
+            VLLM_WORKER_MULTIPROC_METHOD: spawn
+            VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
+        commands:
+            - |
+                timeout 40m bash -c "
+                export VLLM_IMAGE_FETCH_TIMEOUT=60
+                pytest -s -v tests/e2e/online_serving/test_wan22_t2v.py -m 'core_model and npu' --run-level 'core_model'
+                "
+        timeout_in_minutes: 60
+
+      - label: "Diffusion · Hunyuan Image Test"
+        mirror_hardwares: a3_npu_4
+        env:
+            VLLM_WORKER_MULTIPROC_METHOD: spawn
+            VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
+        commands:
+            - |
+                timeout 40m bash -c "
+                export VLLM_IMAGE_FETCH_TIMEOUT=60
+                pytest -s -v tests/e2e/online_serving/test_hunyuan_image.py -m 'core_model and diffusion and npu' --run-level 'core_model'
+                "
+        timeout_in_minutes: 60

.claude/skills/3d-animation-short-generator

@@ -0,0 +1 @@
+../../.agents/skills/3d-animation-short-generator

.claude/skills/brand-promo-video-generator

@@ -0,0 +1 @@
+../../.agents/skills/brand-promo-video-generator

.claude/skills/co-op-game-intro-generator

@@ -0,0 +1 @@
+../../.agents/skills/co-op-game-intro-generator

.claude/skills/h3-prompt-writing

@@ -0,0 +1 @@
+../../.agents/skills/h3-prompt-writing

.claude/skills/handdrawn-live-video-generator

@@ -0,0 +1 @@
+../../.agents/skills/handdrawn-live-video-generator

.claude/skills/minimalist-product-ad-generator

@@ -0,0 +1 @@
+../../.agents/skills/minimalist-product-ad-generator

.claude/skills/music-video-subtitle-generator

@@ -0,0 +1 @@
+../../.agents/skills/music-video-subtitle-generator

.claude/skills/paper-collage-explainer-generator

@@ -0,0 +1 @@
+../../.agents/skills/paper-collage-explainer-generator

.claude/skills/papercraft-stop-motion-explainer

@@ -0,0 +1 @@
+../../.agents/skills/papercraft-stop-motion-explainer

.claude/skills/readme.md

@@ -60,6 +60,14 @@ self-check.
   `test-weekly.yml`), and copy-paste local plus CI-like `pytest` commands; see
   `references/test-routing.md` for level-to-command mapping
 
+## MiniMax H3 Creative Skills
+
+The [portable H3 collection](../../.agents/skills/README.md) contains the prompt
+writing skill and eight creative workflows for product ads, 3D shorts, paper art,
+brand promos, music videos, co-op intros, and hand-drawn/live-action fusion.
+Same-named directories here link to the canonical `.agents/skills/` copies, so
+Claude Code and Codex use the same instructions and templates.
+
 ## Maintenance Guidelines
 
 - Keep skill names short and task-oriented.

.github/PULL_REQUEST_TEMPLATE.md

@@ -12,4 +12,7 @@ PLEASE FILL IN THE PR DESCRIPTION HERE.
 ## Test Result
 
 **BEFORE SUBMITTING:** read [CONTRIBUTING.md](https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md) and run the [precheck-pr skill](https://github.com/vllm-project/vllm-omni/blob/main/.claude/skills/precheck-pr/SKILL.md) with the code agent for a self-check against project conventions.
+
+> [!NOTE]
+> To avoid agent spam, vLLM-Omni limits contributors without write access to **5 open PRs**. If this blocks well-intentioned critical work, please contact a committer to request bypass list access.
 (anything written below this line will be removed by GitHub Actions)

.github/workflows/build_wheel.yml

@@ -56,6 +56,35 @@ jobs:
         run: |
           bash scripts/build_wheel.sh --python python
 
+      - name: Validate runtime package data from source archive
+        run: |
+          set -euo pipefail
+          archive_dir="$(mktemp -d)"
+          install_dir="$(mktemp -d)"
+          vcs_wheels=(dist/*.whl)
+          if [[ ${#vcs_wheels[@]} -ne 1 || ! -f "${vcs_wheels[0]}" ]]; then
+            echo "Expected exactly one VCS-built wheel" >&2
+            exit 1
+          fi
+          git archive HEAD | tar -x -C "${archive_dir}"
+          (
+            cd "${archive_dir}"
+            VLLM_OMNI_TARGET_DEVICE=cuda \
+              VLLM_OMNI_VERSION_OVERRIDE=0.0.0 \
+              python -m build --wheel --outdir archive-dist
+          )
+          wheels=("${archive_dir}"/archive-dist/*.whl)
+          if [[ ${#wheels[@]} -ne 1 || ! -f "${wheels[0]}" ]]; then
+            echo "Expected exactly one archive-built wheel" >&2
+            exit 1
+          fi
+          python -m pip install --no-deps --target "${install_dir}" "${wheels[0]}"
+          python tools/check_wheel_package_data.py \
+            --source-root "${archive_dir}" \
+            --wheel "${wheels[0]}" \
+            --vcs-wheel "${vcs_wheels[0]}" \
+            --installed-root "${install_dir}"
+
       - name: Upload Wheel Artifact
         uses: actions/upload-artifact@v7
         with:

.gitignore

@@ -243,9 +243,15 @@ datasets/
 *.csv
 *.json
 !recipes/cosmos3/negative_prompt.json
+!benchmarks/mammoth_moda2/reference_results.json
 !tests/dfx/perf/tests/*.json
 !apps/ComfyUI-vLLM-Omni/example_workflows/*.json
+# Omni-DuplexEval CI config JSON (e.g. accuracy/omni_duplex_eval)
+!tests/e2e/accuracy/omni_duplex_eval/*.json
 *.jsonl
+# Shared Hunyuan IT2I input for accuracy tests and vllm bench.
+!tests/assets/hunyuan_image3/it2i.jsonl
+!tests/assets/livekit/client_vad_replay.jsonl
 *.parquet
 
 # Output files

CONTRIBUTING.md

@@ -11,3 +11,5 @@ how to extend allowlists) are documented in the
 section of that guide.
 
 Before submitting a PR, run the [precheck-pr skill](https://github.com/vllm-project/vllm-omni/blob/main/.claude/skills/precheck-pr/SKILL.md) with the code agent for a self-check against project conventions.
+
+vLLM-Omni limits contributors without write access to **5 open PRs**; see the [Pull Request Limit](docs/contributing/README.md#pull-request-limit) section of the contributing guide.

README.md

@@ -16,6 +16,7 @@ Easy, fast, and cheap omni-modality model serving for everyone
 
 *Latest News* 🔥
 
+- [2026/09] We released [0.30.0](https://github.com/vllm-project/vllm-omni/releases/tag/v0.30.0), rebased onto vLLM 0.30.0, featuring a unified full-duplex serving framework around engine-owned sessions for [MiniCPM-o 4.5](recipes/OpenBMB/MiniCPM-o-4_5.md) and AURA, native cross-stage KV and multimodal payload transfer (Mooncake AR-to-DiT handoff and NIXL connectors), interactive world-model serving with [LingBot World](recipes/Robbyant/LingBot-World-2.0.md), and realtime [MiniMax H3](recipes/MiniMaxAI/MiniMax-H3.md) Turbo inference on NVIDIA Blackwell and Ascend 950.
 - [2026/08] We released [0.28.0](https://github.com/vllm-project/vllm-omni/releases/tag/v0.28.0), featuring production-ready [MiniMax H3](recipes/MiniMaxAI/MiniMax-H3.md) serving on GPU and NPU, a unified AR/DiT paged KV cache runtime, and enhanced realtime full-duplex serving for the [MiniCPM-o series](recipes/OpenBMB/MiniCPM-o-4_5.md).
 - [2026/08] [VeRL-Omni](https://github.com/verl-project/verl-omni) `v0.2.0` is released: faster diffusion RL powered by vLLM-Omni (request-level/step-wise batching with FA3), rebuilt Qwen3-Omni multimodal training (DPO & GSPO), plus LTX-2.3, Qwen-Image-Edit support and more. See the [release notes](https://github.com/verl-project/verl-omni/releases/tag/v0.2.0).
 - [2026/08] We released [0.26.0](https://github.com/vllm-project/vllm-omni/releases/tag/v0.26.0) - aligned with the vLLM 0.26 release line, featuring [MiniMax H3](recipes/MiniMaxAI/MiniMax-H3.md) joint video/audio generation, an experimental full-duplex realtime runtime for [MiniCPM-o 4.5](recipes/OpenBMB/MiniCPM-o-4_5.md), distributed layerwise diffusion offload, and broader model, hardware, streaming, TTS, and quantization support.
@@ -58,9 +59,9 @@ vLLM-Omni is flexible and easy to use with:
 vLLM-Omni seamlessly supports most popular open-source models on HuggingFace, including:
 
 - **Omni-modality models** (e.g. Qwen3-Omni, MiniCPM-o 4.5, Cosmos3, HunyuanImage, BAGEL)
-- **TTS models** (e.g. Qwen3-TTS, IndexTTS 2.5, CosyVoice3)
-- **Diffusion models** — image, video, and audio generation (e.g. MiniMax H3, LTX-2.5, SANA-Video, Wan2.2)
-- **Robot-policy and action models** (e.g. π0, GR00T-N1.7, DreamZero-DROID, InternVLA-A1)
+- **TTS models** (e.g. Qwen3-TTS, Tencent AuK, Breeze-TTS-2, CosyVoice3)
+- **Diffusion models** — image, video, and audio generation (e.g. MiniMax H3, LingBot World, MAGI-2, LTX-2.5, Wan2.2)
+- **Robot-policy and action models** (e.g. π0.5, GR00T-N1.7, DreamZero-DROID, InternVLA-A1)
 
 ## Getting Started
 

benchmarks/README.md

@@ -66,6 +66,13 @@ Accuracy benchmarks for image generation/editing models, adapting external suite
 - **Layout**: `accuracy/text_to_image/` (GEBench), `accuracy/image_to_image/` (GEdit-Bench)
 - **Method**: generation and judge scoring both run through local `vllm-omni serve` endpoints
 
+### [MammothModa2](mammoth_moda2/README.md) — Startup and Weight Loading
+
+Startup / model-loading benchmark for the two-stage MammothModa2 (AR → DiT) deployment, plus a raw safetensors loading micro-benchmark.
+
+- **Layout**: `mammoth_moda2/bench_startup.py` (single-process startup and first/steady request timing), `mammoth_moda2/parse_startup_log.py` (per-stage breakdown from engine logs), `mammoth_moda2/bench_storage_scenarios.sh` (cold/warm page cache on local disk vs. network storage), `mammoth_moda2/raw_load_bench.py` (safetensors → GPU without vLLM)
+- **Key metrics**: time to engine ready, per-stage spawn/init/weight-load/profile time, first vs. steady-state request latency
+
 ### Common serving metrics framework
 
 `vllm_omni/benchmarks/` extends `vllm bench serve --omni` with Omni-specific datasets, backends, and multimodal metrics. Key metrics include:

benchmarks/accuracy/common.py

@@ -1,3 +1,6 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
+
 from __future__ import annotations
 
 import base64
@@ -200,7 +203,16 @@ class VllmOmniImageClient:
             )
             last_response = response
             if response.status_code != 404:
-                response.raise_for_status()
+                try:
+                    response.raise_for_status()
+                except requests.HTTPError as error:
+                    # Keep the server's validation detail in the exception;
+                    # otherwise multipart image tests only report a bare
+                    # ``400 Bad Request`` and hide the actionable cause.
+                    raise requests.HTTPError(
+                        f"{error}: {response.text}",
+                        response=response,
+                    ) from error
                 return decode_base64_image(response.json()["data"][0]["b64_json"])
 
         assert last_response is not None

benchmarks/accuracy/image_to_image/README.md

@@ -16,9 +16,9 @@ What changed:
   `/v1/images/edits` endpoint to produce benchmark outputs in the expected
   directory structure.
 - The evaluator keeps the same VIEScore-style decomposition:
-  - `semantics_score`
-  - `quality_score`
-  - `overall_score = sqrt(semantics_score * quality_score)`
+    - `semantics_score`
+    - `quality_score`
+    - `overall_score = sqrt(semantics_score * quality_score)`
 - Judge calls are routed to a local OpenAI-compatible model served by
   `vllm-omni`, not a remote provider.
 

benchmarks/accuracy/text_to_image/README.md

@@ -12,14 +12,14 @@ Upstream mapping:
 What changed:
 
 - Generation calls the local OpenAI-compatible `vllm-omni` endpoints:
-  - `/v1/images/generations` for text-only frame generation
-  - `/v1/images/edits` for image-conditioned GUI transition generation
+    - `/v1/images/generations` for text-only frame generation
+    - `/v1/images/edits` for image-conditioned GUI transition generation
 - Evaluation still keeps the GEBench scoring dimensions:
-  - `goal`
-  - `logic`
-  - `cons`
-  - `ui`
-  - `qual`
+    - `goal`
+    - `logic`
+    - `cons`
+    - `ui`
+    - `qual`
 - Judge calls are also routed to a local OpenAI-compatible model served by
   `vllm-omni` instead of a remote service.
 

benchmarks/diffusion/README.md

@@ -164,3 +164,22 @@ batch may still pay compile or CUDA-graph capture cost.
 
 For a Qwen-Image continuous-batching replay example, see
 [`performance_dashboard/qwen_image_serving_performance.md`](./performance_dashboard/qwen_image_serving_performance.md).
+
+## HunyuanImage3 reference-prefix reuse
+
+The prefix-cache benchmark uses the unified `vllm bench serve --omni` runner,
+not this legacy diffusion benchmark. It reuses the two-image IT2I input on a
+single DiT stage, comparing dense, paged without caching, and paged with caching.
+See [Shared-reference benchmark](../../docs/design/feature/prefix_caching.md#shared-reference-benchmark)
+for the configuration, commands, and separate partial-hit accuracy coverage.
+
+The accuracy regression compares paged-no-cache, partial-hit and exact-repeat
+outputs against the checked-in official-repository reference-image goldens at
+**50 steps, CFG=2.5, seeds 43/45** (both CFG branches remain on CFGP1).
+It uses the AR-to-DiT image criteria: CLIP ≥ 90, SSIM ≥ 0.26, PSNR ≥ 12.5 dB.
+`HUNYUAN_IMAGE3_INCLUDE_DENSE=1` adds dense as another comparison to the goldens.
+It also requires actual reference-image hits and model-side query slicing,
+and checks bitwise repeatability for repeated requests at the same hit boundary.
+Images, logs, deployment YAMLs, hit traces and quality metrics are saved under
+pytest's temporary output directory. `HUNYUAN_IMAGE3_MODEL` selects a local model;
+DFX uses the repository's normal model/cache resolution.

benchmarks/diffusion/backends.py

@@ -107,6 +107,8 @@ async def async_request_image_edits(
 
     if extra_body.get("guidance_scale") is not None:
         form.add_field("guidance_scale", str(extra_body["guidance_scale"]))
+    if extra_body.get("guidance_scale_2") is not None:
+        form.add_field("guidance_scale_2", str(extra_body["guidance_scale_2"]))
     if extra_body.get("negative_prompt") is not None:
         form.add_field("negative_prompt", str(extra_body["negative_prompt"]))
     if extra_body.get("true_cfg_scale") is not None:

benchmarks/diffusion/bench_wan_vae_decode.py

@@ -0,0 +1,324 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
+"""Benchmark and exactness check for the Wan VAE decoder fast path.
+
+Loads a diffusers Wan VAE (by default the Cosmos3 one), decodes seeded latents
+for the requested video size at each ``--fast-path`` level and reports wall
+time, peak memory and, against the ``off`` level, bitwise equality, max abs
+difference and PSNR. ``--profile`` adds a torch.profiler CUDA kernel table with
+the convolution / layout-transpose share.
+
+Single GPU::
+
+    python benchmarks/diffusion/bench_wan_vae_decode.py --model nvidia/Cosmos3-Nano
+    python benchmarks/diffusion/bench_wan_vae_decode.py --model nvidia/Cosmos3-Nano \
+        --fast-path off,lossless --frames 33 --profile
+    python benchmarks/diffusion/bench_wan_vae_decode.py --tiny --frames 9 --size 256x256
+
+Multi-GPU VAE parallel decode (launch one process per GPU with ``torchrun``; the
+decode is timed across all ranks and only rank 0 prints)::
+
+    torchrun --nproc-per-node 2 benchmarks/diffusion/bench_wan_vae_decode.py --model nvidia/Cosmos3-Nano \
+        --vae-patch-parallel-size 2 --vae-parallel-mode tile
+
+``tile`` distributes the VAE's own spatial tiles over the ranks (it needs the
+video to exceed the tile size, 256x256 pixels by default) and ``spatial_shard_*``
+shards every decoder feature map along height or width with halo exchange. Both
+return the full video on rank 0 only, so equality and PSNR are computed there.
+"""
+
+from __future__ import annotations
+
+import argparse
+import math
+import os
+import time
+from collections.abc import Iterable
+
+import torch
+import torch.distributed as dist
+
+from vllm_omni.diffusion.distributed.autoencoders.autoencoder_kl_wan import (
+    DistributedAutoencoderKLWan,
+    OmniAutoencoderKLWan,
+)
+from vllm_omni.diffusion.distributed.autoencoders.wan_vae_fastpath import (
+    VAE_FAST_PATH_LEVELS,
+    install_wan_vae_fastpath,
+)
+
+TINY_CONFIG = dict(
+    base_dim=32,
+    decoder_base_dim=64,
+    z_dim=48,
+    dim_mult=[1, 2, 4, 4],
+    num_res_blocks=2,
+    temperal_downsample=[False, True, True],
+    is_residual=True,
+    patch_size=2,
+    in_channels=12,
+    out_channels=12,
+    scale_factor_temporal=4,
+    scale_factor_spatial=16,
+)
+
+DTYPES = {"bf16": torch.bfloat16, "fp16": torch.float16, "fp32": torch.float32}
+VAE_PARALLEL_MODES = ("tile", "spatial_shard_height", "spatial_shard_width")
+_CONV_TAGS = ("conv", "cudnn", "gemm", "cutlass", "xmma")
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    parser.add_argument("--model", default="nvidia/Cosmos3-Nano", help="HF id or local path of the pipeline")
+    parser.add_argument("--subfolder", default="vae")
+    parser.add_argument("--tiny", action="store_true", help="Use a small random VAE instead of loading weights")
+    parser.add_argument("--size", default="1280x720", help="Output video WxH")
+    parser.add_argument("--frames", type=int, default=189, help="Output frame count (1 + 4k)")
+    parser.add_argument("--dtype", choices=sorted(DTYPES), default="bf16")
+    parser.add_argument("--fast-path", default=",".join(VAE_FAST_PATH_LEVELS), help="Comma-separated levels")
+    parser.add_argument("--warmup", type=int, default=1)
+    parser.add_argument("--iters", type=int, default=2)
+    parser.add_argument("--seed", type=int, default=0)
+    parser.add_argument("--profile", action="store_true", help="Print a torch.profiler CUDA kernel table")
+    parser.add_argument("--profile-rows", type=int, default=25)
+    parser.add_argument("--save-output", default=None, help="Save the `off` decode output to this .pt path")
+    parser.add_argument(
+        "--vae-patch-parallel-size",
+        type=int,
+        default=1,
+        help="VAE parallel decode over this many ranks (run under torchrun with as many processes)",
+    )
+    parser.add_argument(
+        "--vae-parallel-mode",
+        choices=VAE_PARALLEL_MODES,
+        default="tile",
+        help="tile: distribute the VAE's spatial tiles; spatial_shard_*: shard feature maps with halo exchange",
+    )
+    return parser.parse_args()
+
+
+def init_distributed() -> tuple[int, int]:
+    """Join the torchrun process group (if any) and return ``(rank, world_size)``."""
+    world_size = int(os.environ.get("WORLD_SIZE", "1"))
+    if world_size <= 1:
+        return 0, 1
+    from vllm_omni.diffusion.distributed.parallel_state import (
+        init_distributed_environment,
+        initialize_model_parallel,
+    )
+
+    rank = int(os.environ["RANK"])
+    local_rank = int(os.environ.get("LOCAL_RANK", rank))
+    init_distributed_environment(world_size=world_size, rank=rank, local_rank=local_rank)
+    initialize_model_parallel(sequence_parallel_size=world_size, ulysses_degree=world_size)
+    return rank, world_size
+
+
+def load_vae(
+    args: argparse.Namespace, dtype: torch.dtype, device: torch.device, parallel: bool
+) -> OmniAutoencoderKLWan:
+    cls = DistributedAutoencoderKLWan if parallel else OmniAutoencoderKLWan
+    if args.tiny:
+        torch.manual_seed(args.seed)
+        vae = cls(**TINY_CONFIG).to(device=device, dtype=dtype).eval()
+        if parallel:
+            vae.init_distributed()
+    else:
+        vae = cls.from_pretrained(args.model, subfolder=args.subfolder, torch_dtype=dtype).to(device).eval()
+    if parallel:
+        # Same setup as the registry: parallel decode needs tiling on.
+        vae.use_tiling = True
+        vae.set_parallel_size(args.vae_patch_parallel_size, mode=args.vae_parallel_mode)
+    return vae
+
+
+def make_latents(vae: OmniAutoencoderKLWan, args: argparse.Namespace, dtype: torch.dtype, device: torch.device):
+    width, height = (int(v) for v in args.size.lower().split("x"))
+    spatial = int(getattr(vae.config, "scale_factor_spatial", 16))
+    temporal = int(getattr(vae.config, "scale_factor_temporal", 4))
+    if width % spatial or height % spatial or (args.frames - 1) % temporal:
+        raise SystemExit(f"size must be a multiple of {spatial} and frames must be 1 + {temporal}k")
+    shape = (1, vae.config.z_dim, (args.frames - 1) // temporal + 1, height // spatial, width // spatial)
+    generator = torch.Generator(device="cpu").manual_seed(args.seed)
+    latents = torch.randn(shape, generator=generator).to(device=device, dtype=dtype)
+    mean = getattr(vae.config, "latents_mean", None)
+    std = getattr(vae.config, "latents_std", None)
+    if mean is not None and std is not None:
+        mean = torch.as_tensor(mean, device=device, dtype=dtype).view(1, -1, 1, 1, 1)
+        std = torch.as_tensor(std, device=device, dtype=dtype).view(1, -1, 1, 1, 1)
+        latents = latents * std + mean
+    return latents
+
+
+@torch.inference_mode()
+def decode(vae: OmniAutoencoderKLWan, latents: torch.Tensor) -> torch.Tensor:
+    return vae.decode(latents, return_dict=False)[0]
+
+
+def sync() -> None:
+    """Wait for this rank's GPU and, under torchrun, for every other rank."""
+    torch.accelerator.synchronize()
+    if dist.is_initialized():
+        dist.barrier()
+
+
+def psnr(a: torch.Tensor, b: torch.Tensor) -> float:
+    mse = torch.mean((a.float() - b.float()) ** 2).item()
+    return math.inf if mse == 0 else 10 * math.log10(4.0 / mse)  # outputs live in [-1, 1]
+
+
+def bits_equal(a: torch.Tensor, b: torch.Tensor) -> bool:
+    if a.shape != b.shape or a.dtype != b.dtype:
+        return False
+    int_dtype = torch.int16 if a.element_size() == 2 else torch.int32
+    return bool((a.contiguous().view(int_dtype) == b.contiguous().view(int_dtype)).all())
+
+
+def profile_decode(vae: OmniAutoencoderKLWan, latents: torch.Tensor, rows: int, verbose: bool) -> None:
+    from torch.profiler import ProfilerActivity, profile
+
+    with profile(activities=[ProfilerActivity.CPU, ProfilerActivity.CUDA], record_shapes=False) as prof:
+        decode(vae, latents)
+        sync()
+    if not verbose:
+        return
+    events = [e for e in prof.key_averages() if e.device_time_total > 0]
+    events.sort(key=lambda e: e.device_time_total, reverse=True)
+    total = sum(e.device_time_total for e in events if e.device_type.name == "CUDA")
+    if total == 0:
+        total = sum(e.device_time_total for e in events)
+    conv = sum(
+        e.device_time_total
+        for e in events
+        if e.device_type.name == "CUDA" and any(tag in e.key.lower() for tag in _CONV_TAGS)
+    )
+    transpose = sum(e.device_time_total for e in events if e.device_type.name == "CUDA" and "nhwc" in e.key.lower())
+    print(
+        f"  GPU kernel time: {total / 1e3:.1f} ms  conv-like: {100 * conv / total:.1f}%  "
+        f"nchw<->nhwc transposes: {100 * transpose / total:.1f}%"
+    )
+    print(f"  {'kernel':90s} {'ms':>9s} {'%':>6s} {'calls':>7s}")
+    shown = 0
+    for e in events:
+        if e.device_type.name != "CUDA":
+            continue
+        share = 100 * e.device_time_total / total
+        print(f"  {e.key[:90]:90s} {e.device_time_total / 1e3:9.2f} {share:6.1f} {e.count:7d}")
+        shown += 1
+        if shown >= rows:
+            break
+
+
+def run_level(
+    args: argparse.Namespace,
+    level: str,
+    dtype: torch.dtype,
+    device: torch.device,
+    *,
+    rank: int,
+    parallel: bool,
+) -> tuple[torch.Tensor, dict]:
+    vae = load_vae(args, dtype, device, parallel)
+    report = install_wan_vae_fastpath(vae, level=level)
+    latents = make_latents(vae, args, dtype, device)
+    for _ in range(args.warmup):
+        decode(vae, latents)
+    sync()
+    torch.accelerator.reset_peak_memory_stats()
+    timings = []
+    output = None
+    for _ in range(args.iters):
+        sync()
+        start = time.perf_counter()
+        output = decode(vae, latents)
+        sync()
+        timings.append(time.perf_counter() - start)
+    peak = torch.tensor(torch.accelerator.max_memory_allocated() / 2**30, device=device)
+    if dist.is_initialized():
+        dist.all_reduce(peak, op=dist.ReduceOp.MAX)  # the largest rank decides whether the config fits
+    stats = {
+        "installed": report.installed,
+        "reason": report.reason,
+        "fused_silu": report.fused_silu_dtypes,
+        "patched": dict(report.patched),
+        "time_s": min(timings),
+        "peak_gib": peak.item(),
+    }
+    if args.profile:
+        if rank == 0:
+            print(f"[{level}] profile")
+        profile_decode(vae, latents, args.profile_rows, verbose=rank == 0)
+    assert output is not None
+    del vae
+    torch.accelerator.empty_cache()
+    return output.detach(), stats
+
+
+def main() -> None:
+    args = parse_args()
+    if not torch.cuda.is_available():
+        raise SystemExit("CUDA device required")
+    rank, world_size = init_distributed()
+    parallel = args.vae_patch_parallel_size > 1
+    if parallel and world_size < args.vae_patch_parallel_size:
+        raise SystemExit(
+            f"--vae-patch-parallel-size {args.vae_patch_parallel_size} needs torchrun with at least that many "
+            f"processes (WORLD_SIZE={world_size})"
+        )
+    device = torch.device("cuda", torch.accelerator.current_device_index()) if world_size > 1 else torch.device("cuda")
+    dtype = DTYPES[args.dtype]
+    levels: Iterable[str] = [level.strip() for level in args.fast_path.split(",") if level.strip()]
+    for level in levels:
+        if level not in VAE_FAST_PATH_LEVELS:
+            raise SystemExit(f"unknown fast-path level {level!r}; choose from {VAE_FAST_PATH_LEVELS}")
+    if "off" not in levels:
+        levels = ["off", *levels]
+
+    if rank == 0:
+        parallel_desc = f" vae_parallel={args.vae_parallel_mode}x{args.vae_patch_parallel_size}" if parallel else ""
+        print(
+            f"model={'tiny' if args.tiny else args.model} size={args.size} frames={args.frames} "
+            f"dtype={args.dtype} world_size={world_size}{parallel_desc}"
+        )
+    golden = None
+    baseline_time = None
+    try:
+        for level in levels:
+            output, stats = run_level(args, level, dtype, device, rank=rank, parallel=parallel)
+            if level == "off":
+                golden = output
+                baseline_time = stats["time_s"]
+                if args.save_output and rank == 0:
+                    torch.save(output.cpu(), args.save_output)
+            assert golden is not None
+            if rank != 0:
+                # Parallel decode assembles the video on rank 0 only; the other
+                # ranks just keep the collectives in lockstep.
+                del output
+                continue
+            speedup = baseline_time / stats["time_s"] if baseline_time else float("nan")
+            line = (
+                f"[{level:13s}] decode {stats['time_s'] * 1e3:9.1f} ms  speedup {speedup:5.2f}x  "
+                f"peak {stats['peak_gib']:5.2f} GiB  installed={stats['installed']}"
+            )
+            if level != "off":
+                equal = torch.equal(output, golden)
+                line += (
+                    f"  torch_equal={equal} bits_equal={bits_equal(output, golden)} "
+                    f"max_abs_diff={(output.float() - golden.float()).abs().max().item():.3e} "
+                    f"psnr={psnr(output, golden):.2f} dB"
+                )
+                if stats["installed"]:
+                    line += f"  fused_silu={stats['fused_silu'] or 'off'}"
+                else:
+                    line += f"  reason={stats['reason']}"
+            print(line)
+            del output
+            torch.accelerator.empty_cache()
+    finally:
+        if dist.is_initialized():
+            dist.destroy_process_group()
+
+
+if __name__ == "__main__":
+    main()

benchmarks/diffusion/diffusion_benchmark_serving.py

@@ -1107,6 +1107,11 @@ def calculate_metrics(
     error_outputs = [o for o in outputs if not o.success]
 
     num_success = len(success_outputs)
+    error_log_path = os.environ.get("BENCH_ERROR_LOG")
+    if error_outputs and error_log_path:
+        with open(error_log_path, "a") as f:
+            for output in error_outputs:
+                f.write(f"ERROR latency={output.latency:.3f} detail={output.error!r}\n")
     latencies = [o.latency for o in success_outputs]
     peak_memories = [o.peak_memory_mb for o in success_outputs if o.peak_memory_mb > 0]
 

benchmarks/distributed/omni_connectors/README.md

@@ -31,6 +31,7 @@ docker run -it \
 ```
 
 Parameter explanation:
+
 - `--cap-add=SYS_PTRACE`: Allow reading system topology information
 - `--cap-add=IPC_LOCK`: Allow memory locking (required for RDMA memory registration)
 - `--security-opt seccomp=unconfined`: Disable seccomp restrictions
@@ -60,6 +61,7 @@ When running single-node tests (producer and consumer on the same machine), ensu
 InfiniBand devices use LID (Local Identifier) for routing. Different devices have different LIDs and cannot communicate directly. If no device is specified, Mooncake may assign different devices to connectors, causing handshake failures.
 
 Common error:
+
 ```
 [Handshake] Failed to modify QP to RTR, check mtu, gid, peer lid, peer qp num: Invalid argument [22]
 ```
@@ -249,7 +251,7 @@ python cross_node_mooncake_transfer_engine.py \
 ### Transfer Modes
 
 | Mode | Description | Example |
-|------|-------------|---------|
+| ------ | ------------- | --------- |
 | `copy` | Normal path — tensor copied to RDMA pool (default) | `--mode copy` |
 | `zerocopy` | Zero-copy path — data created directly in RDMA pool | `--mode zerocopy` |
 | `gpu` | GPU transfer — RDMA pool on GPU, uses GPUDirect | `--mode gpu --gpu-id 0` |
@@ -281,7 +283,7 @@ python cross_node_mooncake_transfer_engine.py \
 ### Cross-Node Test Options
 
 | Option | Description | Default |
-|--------|-------------|---------|
+| -------- | ------------- | --------- |
 | `--role` | `producer` or `consumer` | Required |
 | `--local-host` | Local RDMA IP address | Required |
 | `--remote-host` | Remote RDMA IP address | Required |
@@ -303,6 +305,7 @@ python cross_node_mooncake_transfer_engine.py \
 **Cause**: QP handshake failed, usually due to device configuration mismatch.
 
 **Solution**:
+
 ```bash
 # Force using the same device
 export RDMA_DEVICE_NAME='mlx5_0'
@@ -313,6 +316,7 @@ export RDMA_DEVICE_NAME='mlx5_0'
 **Cause**: Mooncake not installed or import failed.
 
 **Solution**:
+
 ```bash
 # Check Mooncake installation
 python -c "from mooncake.engine import TransferEngine; print('OK')"
@@ -329,6 +333,7 @@ uv pip install mooncake-transfer-engine
 **Cause**: Container lacks IB device access permissions.
 
 **Solution**:
+
 ```bash
 docker run --device=/dev/infiniband --cap-add=IPC_LOCK ...
 ```
@@ -338,6 +343,7 @@ docker run --device=/dev/infiniband --cap-add=IPC_LOCK ...
 **Cause**: RDMA connection establishment failed or network latency.
 
 **Solution**:
+
 ```bash
 # Check network status
 ibstat
@@ -349,6 +355,7 @@ ibstatus
 **Cause**: CUDA environment not configured or GPU unavailable.
 
 **Solution**:
+
 ```bash
 # Check CUDA
 python -c "import torch; print(torch.cuda.is_available())"
@@ -362,7 +369,7 @@ docker run --gpus all ...
 ## Environment Variables Reference
 
 | Variable | Description | Example |
-|----------|-------------|---------|
+| ---------- | ------------- | --------- |
 | `RDMA_DEVICE_NAME` | Specify RDMA device name | `mlx5_0` |
 | `RDMA_TEST_HOST` | Specify test host IP | `10.0.0.1` |
 | `MC_TE_METRIC` | Enable Mooncake metrics | `1` |
@@ -373,7 +380,7 @@ docker run --gpus all ...
 ## Test Files Overview
 
 | File | Description | Auto-discovered by pytest |
-|------|-------------|--------------------------|
+| ------ | ------------- | -------------------------- |
 | `test_mooncake_transfer_engine_rdma.py` | Integration tests for MooncakeTransferEngineConnector (basic, E2E, lifecycle, GPU) | Yes |
 | `test_mooncake_transfer_engine_buffer.py` | Memory pool and buffer management unit tests | Yes |
 | `cross_node_mooncake_transfer_engine.py` | Cross-node (multi-machine) testing script — run manually | No (filename does not start with `test_`) |
@@ -381,7 +388,7 @@ docker run --gpus all ...
 ### test_mooncake_transfer_engine_rdma.py — Test Classes
 
 | Test Class | Memory Pool | Marker | Description |
-|------------|-------------|--------|-------------|
+| ------------ | ------------- | -------- | ------------- |
 | `TestBasicConnector` | CPU | — | Initialization, put tensor/bytes/object, cleanup, pool exhaustion |
 | `TestEndToEnd` | CPU | — | E2E RDMA transfer: tensor, bytes, object, zero-copy, large payload (100MB), mixed types, concurrency |
 | `TestLifecycle` | CPU | — | Close, context manager, double-close safety |
@@ -391,7 +398,7 @@ docker run --gpus all ...
 ### test_mooncake_transfer_engine_buffer.py — Test Classes
 
 | Test Class | Marker | Description |
-|------------|--------|-------------|
+| ------------ | -------- | ------------- |
 | `TestBufferAllocator` | — | Basic alloc/free, alignment, exhaustion/recovery, thread safety |
 | `TestAllocatorInvariants` | `slow` | Double-free safety, overlap corruption detection, adjacent-block merging, fragmentation/defrag |
 | `TestManagedBuffer` | — | Tensor views, context manager |

benchmarks/glm_image/README.md

@@ -5,7 +5,7 @@ Benchmark GLM-Image T2I (text-to-image) and I2I (image-to-image) performance acr
 ## Benchmarks
 
 | Benchmark | Script | Description |
-|-----------|--------|-------------|
+| ----------- | -------- | ------------- |
 | HuggingFace Baseline | `huggingface/inference.py` | Single-GPU transformers + diffusers pipeline |
 | vLLM-Omni Offline | `vllm-omni/inference.py` | Offline inference with continuous batching |
 | vLLM-Omni Online | `benchmark_glm_image.py` | Online serving via `/v1/chat/completions` |
@@ -27,7 +27,7 @@ CUDA_VISIBLE_DEVICES=0 python benchmarks/glm_image/huggingface/inference.py \
 ### Options
 
 | Flag | Default | Description |
-|------|---------|-------------|
+| ------ | --------- | ------------- |
 | `--model-path` | `zai-org/GLM-Image` | Model path |
 | `--mode` | `t2i` | `t2i` or `i2i` |
 | `--dataset-path` | `prompt/prompt.json` | Path to prompt.json |
@@ -54,7 +54,7 @@ CUDA_VISIBLE_DEVICES=0,1 python benchmarks/glm_image/vllm-omni/inference.py \
 ### Options
 
 | Flag | Default | Description |
-|------|---------|-------------|
+| ------ | --------- | ------------- |
 | `--model-path` | `zai-org/GLM-Image` | Model path |
 | `--deploy-config` | - | Deploy config YAML |
 | `--mode` | `t2i` | `t2i` or `i2i` |
@@ -100,7 +100,7 @@ python benchmarks/glm_image/benchmark_glm_image.py \
 ### Options
 
 | Flag | Default | Description |
-|------|---------|-------------|
+| ------ | --------- | ------------- |
 | `--mode` | `t2i` | `t2i` or `i2i` |
 | `--dataset` | `prompt` | `prompt`, `random`, or `custom` |
 | `--dataset-path` | - | JSON file path (required for `custom`) |
@@ -134,7 +134,7 @@ Custom datasets use the same JSON format and can be provided via `--dataset-path
 All three benchmarks report per-stage pipeline timings (in milliseconds):
 
 | Key | Description |
-|-----|-------------|
+| ----- | ------------- |
 | `preprocess_ms` | Input preprocessing (tokenization, multimodal encoding) |
 | `stage_0_gen_ms` | AR (autoregressive) model generation time |
 | `ar2diffusion_ms` | AR output to diffusion input conversion |
@@ -148,7 +148,7 @@ The stages are ordered by execution: `preprocess → stage_0 (AR) → ar2diffusi
 Tested on 2x GPU with 10 prompts, 1024x1024, 50 denoising steps:
 
 | Backend | Mode | Latency Mean (s) | Throughput (img/s) |
-|---------|------|-------------------|--------------------|
+| --------- | ------ | ------------------- | -------------------- |
 | HuggingFace | T2I | 72.6 | 0.014 |
 | HuggingFace | I2I | 70.9 | 0.014 |
 | vLLM-Omni Offline | T2I | 35.0 | 0.044 |

benchmarks/kernels/_magi2_gpu_bench.py

@@ -0,0 +1,194 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
+"""Cold-cache torch.cuda timing and correctness gates for MAGI-2 microbenchmarks."""
+
+import argparse
+import importlib
+import importlib.metadata
+import json
+import logging
+import math
+import os
+import subprocess
+import sys
+from collections.abc import Callable
+from pathlib import Path
+from types import ModuleType
+from typing import Any
+
+LOGGER = logging.getLogger(__name__)
+
+
+def emit(**fields: Any) -> None:
+    LOGGER.info(json.dumps(fields, sort_keys=True))
+
+
+def add_arguments(parser: argparse.ArgumentParser) -> None:
+    parser.add_argument("--tokens", type=int, nargs="+", default=[1, 4, 16, 64, 129, 4096])
+    parser.add_argument("--dtype", choices=["bfloat16", "float16", "float32"], default="bfloat16")
+    parser.add_argument("--num-tests", type=int, default=30, help="Measured samples per implementation.")
+    parser.add_argument("--warmup", type=int, default=10, help="Interleaved warmup calls before sampling.")
+    parser.add_argument("--seed", type=int, default=42)
+
+
+def load_musa(args: argparse.Namespace, parser: argparse.ArgumentParser) -> Any:
+    if min(*args.tokens, args.num_tests) <= 0 or args.warmup < 0:
+        parser.error("tokens and num-tests must be positive; warmup must be nonnegative")
+    logging.basicConfig(level=logging.INFO, format="%(message)s", stream=sys.stdout)
+    torch = importlib.import_module("torch")
+    cuda = getattr(torch, "cuda")
+    # torchada is needed only on MUSA; after it is loaded, use torch.cuda for
+    # both CUDA and MUSA as documented by torchada.
+    if hasattr(torch.version, "musa") and torch.version.musa is not None:
+        try:
+            importlib.import_module("torchada")
+        except ModuleNotFoundError as error:
+            if error.name != "torchada":
+                raise
+    has_gpu = cuda.is_available() or (
+        hasattr(torch.version, "musa") and torch.version.musa is not None and torch.musa.is_available()
+    )
+    if not has_gpu:
+        emit(status="skip", reason="CUDA/MUSA device is unavailable", torch=torch.__version__)
+        return None
+    torch.manual_seed(args.seed)
+    cuda.manual_seed_all(args.seed)
+    return torch
+
+
+def report_environment(torch: Any, args: argparse.Namespace, module: ModuleType) -> None:
+    root = Path(__file__).resolve().parents[2]
+    packages = {}
+    for name in ("torch", "torch_musa", "torchada", "triton", "vllm", "vllm-omni"):
+        try:
+            packages[name] = importlib.metadata.version(name)
+        except importlib.metadata.PackageNotFoundError:
+            packages[name] = None
+    cuda = getattr(torch, "cuda")
+    properties = cuda.get_device_properties(cuda.current_device())
+    try:
+        source_sha = subprocess.check_output(["git", "-C", str(root), "rev-parse", "HEAD"], text=True).strip()
+        source_dirty = bool(subprocess.check_output(["git", "-C", str(root), "status", "--porcelain"], text=True))
+    except (subprocess.CalledProcessError, FileNotFoundError):
+        source_sha = os.environ.get("MAGI2_BENCH_SOURCE_SHA", "unknown")
+        source_dirty = None
+    emit(
+        status="environment",
+        command=sys.argv,
+        source_root=str(root),
+        source_sha=source_sha,
+        source_dirty=source_dirty,
+        kernel_source=module.__file__,
+        device=cuda.get_device_name(cuda.current_device()),
+        sm_count=getattr(properties, "multi_processor_count", None),
+        packages=packages,
+        env={name: os.environ.get(name) for name in ("MUSA_VISIBLE_DEVICES", "TORCHDYNAMO_DISABLE")},
+        config=vars(args),
+        mode="eager operator diagnostic; no compile or graph capture",
+        comparison="forward_native versus the default CustomOp dispatch",
+        timing="torch.cuda.Event; alternating native/fused calls; vanilla 512 MiB flush outside timed regions",
+        timing_scope="whole callable, including device launch gaps; not isolated kernel duration",
+        l2_flush=True,
+        l2_flush_bytes=512 * 1024 * 1024,
+    )
+
+
+class _LaunchProbe:
+    def __init__(self, kernel: Any) -> None:
+        self.kernel = kernel
+        self.count = 0
+
+    def __getitem__(self, grid: Any) -> Any:
+        self.count += 1
+        return self.kernel[grid]
+
+
+def check_pair(
+    torch: Any,
+    reference: Callable,
+    fused: Callable,
+    module: ModuleType,
+    kernels: tuple[str, ...],
+    *,
+    rtol: float,
+    atol: float,
+) -> dict:
+    cuda = getattr(torch, "cuda")
+    expected = reference()
+    probes = {name: _LaunchProbe(getattr(module, name)) for name in kernels}
+    try:
+        for name, probe in probes.items():
+            setattr(module, name, probe)
+        actual = fused()
+        cuda.synchronize()
+    finally:
+        for name, probe in probes.items():
+            setattr(module, name, probe.kernel)
+    if any(probe.count != 1 for probe in probes.values()):
+        raise RuntimeError(f"Expected one launch per fused kernel; got {[(k, p.count) for k, p in probes.items()]}")
+    expected = expected if isinstance(expected, tuple) else (expected,)
+    actual = actual if isinstance(actual, tuple) else (actual,)
+    errors = []
+    for result, baseline in zip(actual, expected, strict=True):
+        for value in (baseline, result):
+            if not torch.isfinite(value).all().item():
+                raise AssertionError("Nonfinite benchmark output")
+            if value.numel() > 1 and (value == value.flatten()[0]).all().item():
+                raise AssertionError("Constant benchmark output")
+        torch.testing.assert_close(result, baseline, rtol=rtol, atol=atol)
+        difference = (result.float() - baseline.float()).abs()
+        errors.append(
+            {
+                "max_abs": difference.max().item(),
+                "max_rel": (difference / baseline.float().abs().clamp_min(1e-6)).max().item(),
+            }
+        )
+    return {"parity": "pass", "errors": errors, "rtol": rtol, "atol": atol, "kernels": list(kernels)}
+
+
+def _percentile(samples: list[float], quantile: float) -> float:
+    ordered = sorted(samples)
+    position = (len(ordered) - 1) * quantile
+    lower, upper = math.floor(position), math.ceil(position)
+    return ordered[lower] + (ordered[upper] - ordered[lower]) * (position - lower)
+
+
+def measure_pair(torch: Any, reference: Callable, fused: Callable, args: argparse.Namespace) -> dict:
+    cuda = getattr(torch, "cuda")
+    functions = (reference, fused)
+    order: list[int] = []
+
+    def alternate() -> Any:
+        mode = len(order) % 2
+        order.append(mode)
+        return functions[mode]()
+
+    times_ms = []
+    for _ in range(args.warmup):
+        alternate()
+    cuda.synchronize()
+    order.clear()
+    flush = torch.empty(512 * 1024 * 1024, dtype=torch.uint8, device="cuda")
+    for _ in range(2 * args.num_tests):
+        flush.zero_()
+        cuda.synchronize()
+        start = cuda.Event(enable_timing=True)
+        end = cuda.Event(enable_timing=True)
+        start.record()
+        alternate()
+        end.record()
+        end.synchronize()
+        times_ms.append(start.elapsed_time(end))
+    samples: tuple[list[float], list[float]] = ([], [])
+    for mode, milliseconds in zip(order[-len(times_ms) :], times_ms, strict=True):
+        if not math.isfinite(milliseconds) or milliseconds <= 0:
+            raise RuntimeError(f"Invalid GPU event sample: {milliseconds}")
+        samples[mode].append(milliseconds * 1000)
+    stats = {}
+    for label, values in zip(("native", "fused"), samples, strict=True):
+        if len(values) != args.num_tests:
+            raise RuntimeError("GPU timing harness returned an unexpected sample count")
+        p50 = _percentile(values, 0.5)
+        stats[label] = {"median_us": p50, "p50_us": p50, "p90_us": _percentile(values, 0.9), "samples": len(values)}
+    stats["speedup_p50"] = stats["native"]["p50_us"] / stats["fused"]["p50_us"]
+    return stats

benchmarks/kernels/_magi2_musa_bench.py

@@ -0,0 +1,194 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
+"""Cold-cache torch.cuda timing and correctness gates for MAGI-2 microbenchmarks."""
+
+import argparse
+import importlib
+import importlib.metadata
+import json
+import logging
+import math
+import os
+import subprocess
+import sys
+from collections.abc import Callable
+from pathlib import Path
+from types import ModuleType
+from typing import Any
+
+LOGGER = logging.getLogger(__name__)
+
+
+def emit(**fields: Any) -> None:
+    LOGGER.info(json.dumps(fields, sort_keys=True))
+
+
+def add_arguments(parser: argparse.ArgumentParser) -> None:
+    parser.add_argument("--tokens", type=int, nargs="+", default=[1, 4, 16, 64, 129, 4096])
+    parser.add_argument("--dtype", choices=["bfloat16", "float16", "float32"], default="bfloat16")
+    parser.add_argument("--num-tests", type=int, default=30, help="Measured samples per implementation.")
+    parser.add_argument("--warmup", type=int, default=10, help="Interleaved warmup calls before sampling.")
+    parser.add_argument("--seed", type=int, default=42)
+
+
+def load_musa(args: argparse.Namespace, parser: argparse.ArgumentParser) -> Any:
+    if min(*args.tokens, args.num_tests) <= 0 or args.warmup < 0:
+        parser.error("tokens and num-tests must be positive; warmup must be nonnegative")
+    logging.basicConfig(level=logging.INFO, format="%(message)s", stream=sys.stdout)
+    torch = importlib.import_module("torch")
+    cuda = getattr(torch, "cuda")
+    # torchada is needed only on MUSA; after it is loaded, use torch.cuda for
+    # both CUDA and MUSA as documented by torchada.
+    if hasattr(torch.version, "musa") and torch.version.musa is not None:
+        try:
+            importlib.import_module("torchada")
+        except ModuleNotFoundError as error:
+            if error.name != "torchada":
+                raise
+    has_gpu = cuda.is_available() or (
+        hasattr(torch.version, "musa") and torch.version.musa is not None and torch.musa.is_available()
+    )
+    if not has_gpu:
+        emit(status="skip", reason="CUDA/MUSA device is unavailable", torch=torch.__version__)
+        return None
+    torch.manual_seed(args.seed)
+    cuda.manual_seed_all(args.seed)
+    return torch
+
+
+def report_environment(torch: Any, args: argparse.Namespace, module: ModuleType) -> None:
+    root = Path(__file__).resolve().parents[2]
+    packages = {}
+    for name in ("torch", "torch_musa", "torchada", "triton", "vllm", "vllm-omni"):
+        try:
+            packages[name] = importlib.metadata.version(name)
+        except importlib.metadata.PackageNotFoundError:
+            packages[name] = None
+    cuda = getattr(torch, "cuda")
+    properties = cuda.get_device_properties(cuda.current_device())
+    try:
+        source_sha = subprocess.check_output(["git", "-C", str(root), "rev-parse", "HEAD"], text=True).strip()
+        source_dirty = bool(subprocess.check_output(["git", "-C", str(root), "status", "--porcelain"], text=True))
+    except (subprocess.CalledProcessError, FileNotFoundError):
+        source_sha = os.environ.get("MAGI2_BENCH_SOURCE_SHA", "unknown")
+        source_dirty = None
+    emit(
+        status="environment",
+        command=sys.argv,
+        source_root=str(root),
+        source_sha=source_sha,
+        source_dirty=source_dirty,
+        kernel_source=module.__file__,
+        device=cuda.get_device_name(cuda.current_device()),
+        sm_count=getattr(properties, "multi_processor_count", None),
+        packages=packages,
+        env={name: os.environ.get(name) for name in ("MUSA_VISIBLE_DEVICES", "TORCHDYNAMO_DISABLE")},
+        config=vars(args),
+        mode="eager operator diagnostic; no compile or graph capture",
+        comparison="forward_native versus the default CustomOp dispatch",
+        timing="torch.cuda.Event; alternating native/fused calls; vanilla 512 MiB flush outside timed regions",
+        timing_scope="whole callable, including device launch gaps; not isolated kernel duration",
+        l2_flush=True,
+        l2_flush_bytes=512 * 1024 * 1024,
+    )
+
+
+class _LaunchProbe:
+    def __init__(self, kernel: Any) -> None:
+        self.kernel = kernel
+        self.count = 0
+
+    def __getitem__(self, grid: Any) -> Any:
+        self.count += 1
+        return self.kernel[grid]
+
+
+def check_pair(
+    torch: Any,
+    reference: Callable,
+    fused: Callable,
+    module: ModuleType,
+    kernels: tuple[str, ...],
+    *,
+    rtol: float,
+    atol: float,
+) -> dict:
+    cuda = getattr(torch, "cuda")
+    expected = reference()
+    probes = {name: _LaunchProbe(getattr(module, name)) for name in kernels}
+    try:
+        for name, probe in probes.items():
+            setattr(module, name, probe)
+        actual = fused()
+        cuda.synchronize()
+    finally:
+        for name, probe in probes.items():
+            setattr(module, name, probe.kernel)
+    if any(probe.count != 1 for probe in probes.values()):
+        raise RuntimeError(f"Expected one launch per fused kernel; got {[(k, p.count) for k, p in probes.items()]}")
+    expected = expected if isinstance(expected, tuple) else (expected,)
+    actual = actual if isinstance(actual, tuple) else (actual,)
+    errors = []
+    for result, baseline in zip(actual, expected, strict=True):
+        for value in (baseline, result):
+            if not torch.isfinite(value).all().item():
+                raise AssertionError("Nonfinite benchmark output")
+            if value.numel() > 1 and (value == value.flatten()[0]).all().item():
+                raise AssertionError("Constant benchmark output")
+        torch.testing.assert_close(result, baseline, rtol=rtol, atol=atol)
+        difference = (result.float() - baseline.float()).abs()
+        errors.append(
+            {
+                "max_abs": difference.max().item(),
+                "max_rel": (difference / baseline.float().abs().clamp_min(1e-6)).max().item(),
+            }
+        )
+    return {"parity": "pass", "errors": errors, "rtol": rtol, "atol": atol, "kernels": list(kernels)}
+
+
+def _percentile(samples: list[float], quantile: float) -> float:
+    ordered = sorted(samples)
+    position = (len(ordered) - 1) * quantile
+    lower, upper = math.floor(position), math.ceil(position)
+    return ordered[lower] + (ordered[upper] - ordered[lower]) * (position - lower)
+
+
+def measure_pair(torch: Any, reference: Callable, fused: Callable, args: argparse.Namespace) -> dict:
+    cuda = getattr(torch, "cuda")
+    functions = (reference, fused)
+    order: list[int] = []
+
+    def alternate() -> Any:
+        mode = len(order) % 2
+        order.append(mode)
+        return functions[mode]()
+
+    times_ms = []
+    for _ in range(args.warmup):
+        alternate()
+    cuda.synchronize()
+    order.clear()
+    flush = torch.empty(512 * 1024 * 1024, dtype=torch.uint8, device="cuda")
+    for _ in range(2 * args.num_tests):
+        flush.zero_()
+        cuda.synchronize()
+        start = cuda.Event(enable_timing=True)
+        end = cuda.Event(enable_timing=True)
+        start.record()
+        alternate()
+        end.record()
+        end.synchronize()
+        times_ms.append(start.elapsed_time(end))
+    samples: tuple[list[float], list[float]] = ([], [])
+    for mode, milliseconds in zip(order[-len(times_ms) :], times_ms, strict=True):
+        if not math.isfinite(milliseconds) or milliseconds <= 0:
+            raise RuntimeError(f"Invalid GPU event sample: {milliseconds}")
+        samples[mode].append(milliseconds * 1000)
+    stats = {}
+    for label, values in zip(("native", "fused"), samples, strict=True):
+        if len(values) != args.num_tests:
+            raise RuntimeError("GPU timing harness returned an unexpected sample count")
+        p50 = _percentile(values, 0.5)
+        stats[label] = {"median_us": p50, "p50_us": p50, "p90_us": _percentile(values, 0.9), "samples": len(values)}
+    stats["speedup_p50"] = stats["native"]["p50_us"] / stats["fused"]["p50_us"]
+    return stats

benchmarks/kernels/benchmark_magi2_bf16_moe.py

@@ -0,0 +1,310 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
+"""Compare MAGI-2 routed MoE with equal preprocessing boundaries.
+
+python -m benchmarks.kernels.benchmark_magi2_bf16_moe --tokens 2 4096 10000
+
+Default (`--mode dispatch`): the same MAGI-2 layer is timed with its native
+reference and default BF16 fast path, including routing and all preprocessing.
+With `--mode routed`, routing probabilities/IDs are inputs to BOTH timed paths. Sorting,
+padding, packing, allocations and reduction remain inside their respective
+calls. The reference uses deterministic routed storage in diagnostic mode.
+Use --mode dispatch to measure the
+actual layer with the native and default fast paths, including the router. Unsupported
+dispatch fails explicitly; it is never silently replaced by another baseline.
+
+Stage timings are standalone diagnostics, not additive or env speedup claims.
+These synthetic local shapes are not full-model performance measurements.
+"""
+
+from __future__ import annotations
+
+import argparse
+import importlib
+import json
+import math
+import os
+import statistics
+import subprocess
+import time
+from contextlib import contextmanager
+from pathlib import Path
+from unittest.mock import patch
+
+
+def parse_args(argv=None):
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--tokens", type=int, nargs="+", default=[2, 4096])
+    parser.add_argument("--heads", type=int, default=3, help="EP4 local head count for MAGI-2 Preview.")
+    parser.add_argument("--experts", type=int, default=256, help="Released experts per head.")
+    parser.add_argument("--top-k", type=int, default=6)
+    parser.add_argument("--warmup", type=int, default=10)
+    parser.add_argument("--iters", type=int, default=50)
+    parser.add_argument("--seed", type=int, default=419)
+    parser.add_argument("--mode", choices=["routed", "dispatch"], default="dispatch")
+    parser.add_argument(
+        "--native-deterministic",
+        action="store_true",
+        help="Use the portable deterministic native reference (needed by Triton-MUSA 3.2).",
+    )
+    parser.add_argument("--stages", action="store_true")
+    parser.add_argument("--output", type=Path)
+    args = parser.parse_args(argv)
+    if min(*args.tokens, args.heads, args.experts, args.top_k, args.iters) < 1 or args.warmup < 0:
+        parser.error("Shapes/iters must be positive; warmup must be nonnegative")
+    if args.top_k > args.experts:
+        parser.error("top-k must not exceed experts per head")
+    return args
+
+
+@contextmanager
+def env_switch(enabled):
+    # Keep the legacy benchmark context API; dispatch is no longer opt-in.
+    with patch.dict(os.environ, MAGI2_DETERMINISTIC="0"):
+        yield
+
+
+def stats(values):
+    if not values or any(not math.isfinite(x) or x <= 0 for x in values):
+        raise ValueError("Missing/non-positive/non-finite timing sample")
+    ordered = sorted(values)
+    return {
+        "mean_us": statistics.mean(values),
+        "p50_us": statistics.median(values),
+        "p90_us": ordered[math.ceil(0.9 * len(ordered)) - 1],
+        "samples_us": values,
+    }
+
+
+def measure(torch, calls, args):
+    """Alternating AB/BA; cold-cache flush is outside each timed region."""
+    cuda = getattr(torch, "cuda")
+    names = list(calls)
+    sequence = names + names[::-1]
+    wall_samples = []
+    labels = []
+    counter = 0
+
+    def invoke():
+        nonlocal counter
+        name = sequence[counter % len(sequence)]
+        counter += 1
+        cuda.synchronize()
+        start = time.perf_counter_ns()
+        output = calls[name]()
+        cuda.synchronize()
+        elapsed = (time.perf_counter_ns() - start) / 1000
+        wall_samples.append(elapsed)
+        labels.append(name)
+        return output
+
+    count = len(names) * args.iters
+    cache = torch.empty(512 * 1024 * 1024, device="cuda", dtype=torch.uint8)
+    gpu_ms = []
+    for index in range(len(names) * args.warmup + count):
+        cache.zero_()
+        cuda.synchronize()
+        begin, end = cuda.Event(enable_timing=True), cuda.Event(enable_timing=True)
+        begin.record()
+        invoke()
+        end.record()
+        end.synchronize()
+        if index >= len(names) * args.warmup:
+            gpu_ms.append(begin.elapsed_time(end))
+    labels, wall_samples = labels[-count:], wall_samples[-count:]
+    result = {}
+    for name in names:
+        wall = [x for label, x in zip(labels, wall_samples) if label == name]
+        event = [float(x) * 1000 for label, x in zip(labels, gpu_ms) if label == name]
+        if len(wall) != args.iters or len(event) != args.iters:
+            raise RuntimeError("Timing harness did not return the requested sample counts")
+        result[name] = {"synchronized_wall": stats(wall), "gpu_event": stats(event)}
+    return result
+
+
+def make_inputs(torch, tokens, args):
+    # Fan-in-scaled synthetic weights. Unlike the old script, W_up and W_gate
+    # have the same distribution and top-k IDs are unique within each token.
+    torch.manual_seed(args.seed)
+    x = torch.randn(tokens, args.heads, 256, device="cuda").to(torch.bfloat16)
+    shape = (args.heads * args.experts, 256, 1280)
+    gate, up = [(torch.randn(shape, device="cuda") / 16).to(torch.bfloat16) for _ in range(2)]
+    down = (torch.randn(shape[0], 1280, 256, device="cuda") / math.sqrt(1280)).to(torch.bfloat16)
+    logits = torch.randn(args.heads, tokens, args.experts, device="cuda")
+    selected = logits.topk(args.top_k, dim=-1)
+    probabilities = selected.values.softmax(-1)
+    return x, probabilities, selected.indices, gate, up, down
+
+
+def check_and_capture(torch, moe, kernels, data, args):
+    x, probabilities, indices, gate, up, down = data
+    gather, weights, offsets = moe.global_sort_routes(probabilities, indices, args.experts)
+    sorted_routes, launches = [], []
+    scatter, invoke = kernels._deterministic_scatter, moe.invoke_fused_moe_bf16
+
+    def capture_scatter(routes, *params):
+        sorted_routes.append(routes.detach().clone())
+        return scatter(routes, *params)
+
+    def capture_launch(*params, **kwargs):
+        launches.append((params, kwargs))
+        return invoke(*params, **kwargs)
+
+    with patch.object(kernels, "_deterministic_scatter", capture_scatter):
+        moe.triton_mh_moe_forward(x, gather, weights, offsets, gate, up, down, deterministic=True)
+    packed_w13 = moe._pack_bf16_w13(gate, up)
+    route_buffers = moe._allocate_bf16_route_buffers(
+        x.shape[0] * args.heads * args.top_k,
+        args.heads * args.experts,
+        128,
+        x.device,
+    )
+    with patch.object(moe, "invoke_fused_moe_bf16", capture_launch):
+        actual = moe._bf16_fused_moe_forward(x, probabilities, indices, packed_w13, down, route_buffers)
+    if len(sorted_routes) != 1 or len(launches) != 2:
+        raise RuntimeError("Unexpected kernel dispatch: benchmark contract needs updating")
+    flat_experts = (indices + torch.arange(args.heads, device=x.device)[:, None, None] * args.experts).flatten()
+    order = flat_experts.argsort(stable=True)
+    new_routes = launches[1][0][2].reshape(-1, 256).index_select(0, order)
+    scatter_ids = gather.long() * args.heads + flat_experts[order] // args.experts
+    expected = torch.zeros(x.shape[0] * args.heads, 256, device=x.device, dtype=torch.float32)
+    expected.index_add_(0, scatter_ids, sorted_routes[0].float())
+    expected = expected.to(x.dtype).view_as(x)
+    errors = []
+    # Same expert/FP32 reduction contract as test_bf16_moe_wiring.py, rather
+    # than relaxing tolerances on the inherited BF16 scatter reduction.
+    for result, reference in ((new_routes, sorted_routes[0]), (actual, expected)):
+        if not (torch.isfinite(result).all() and torch.isfinite(reference).all()):
+            raise RuntimeError("Non-finite MoE output")
+        torch.testing.assert_close(result, reference, rtol=2e-2, atol=0.5)
+        relative_l2 = (result.float() - reference.float()).norm() / reference.float().norm().clamp_min(1e-6)
+        if relative_l2.item() >= 1e-3:
+            raise AssertionError(f"MoE relative L2 {relative_l2.item()} >= 1e-3")
+        errors.append(
+            {"max_abs": (result.float() - reference.float()).abs().max().item(), "relative_l2": relative_l2.item()}
+        )
+    return launches, errors
+
+
+def benchmark(torch, moe, kernels, tokens, args):
+    data = make_inputs(torch, tokens, args)
+    x, probabilities, indices, gate, up, down = data
+    launches, errors = check_and_capture(torch, moe, kernels, data, args)
+    packed_w13 = moe._pack_bf16_w13(gate, up)
+    route_count = tokens * args.heads * args.top_k
+    route_buffers = moe._allocate_bf16_route_buffers(route_count, args.heads * args.experts, 128, x.device)
+
+    def reference():
+        gather, weights, offsets = moe.global_sort_routes(probabilities, indices, args.experts)
+        return moe.triton_mh_moe_forward(x, gather, weights, offsets, gate, up, down, deterministic=True)
+
+    calls = {
+        "reference_deterministic_routed": reference,
+        "candidate_routed": lambda: moe._bf16_fused_moe_forward(
+            x, probabilities, indices, packed_w13, down, route_buffers
+        ),
+    }
+    if args.mode == "dispatch":
+        config = moe.Magi2MultiHeadMoEConfig(
+            args.heads * 256, args.heads, args.experts, args.top_k, 1280, torch.bfloat16
+        )
+        layer = moe.Magi2MultiHeadMoE(config).to(x.device)
+        for parameter, value in ((layer.W_gate, gate), (layer.W_up, up), (layer.W_down, down)):
+            parameter.copy_(value)
+        layer.gate.normal_(std=1 / 16)
+
+        flat_x = x.reshape(tokens, -1)
+
+        def native_dispatch():
+            x_heads = flat_x.view(tokens, args.heads, 256)
+            probs, idx = layer._route(x_heads)
+            gather, weights, offsets = moe.global_sort_routes(probs, idx, args.experts)
+            return moe.triton_mh_moe_forward(
+                x_heads,
+                gather,
+                weights,
+                offsets,
+                layer.W_gate,
+                layer.W_up,
+                layer.W_down,
+                deterministic=args.native_deterministic,
+            ).reshape(tokens, -1)
+
+        calls = {"native_dispatch": native_dispatch, "fused_dispatch": lambda: layer(flat_x)}
+        torch.testing.assert_close(calls["fused_dispatch"](), calls["native_dispatch"](), rtol=2e-2, atol=0.5)
+    timings = measure(torch, calls, args)
+    names = list(calls)
+    result = {"status": "pass", "tokens": tokens, "mode": args.mode, "correctness": errors, "timings": timings}
+    result["speedup_p50_wall"] = (
+        timings[names[0]]["synchronized_wall"]["p50_us"] / timings[names[1]]["synchronized_wall"]["p50_us"]
+    )
+    if args.stages:
+        route_ids = (indices.int() + torch.arange(args.heads, device=x.device)[:, None, None] * args.experts).reshape(
+            -1, args.top_k
+        )
+
+        def gems():
+            for params, kwargs in launches:
+                moe.invoke_fused_moe_bf16(*params, **kwargs)
+
+        result["standalone_stages_not_additive"] = measure(
+            torch,
+            {
+                "reference_sort": lambda: moe.global_sort_routes(probabilities, indices, args.experts),
+                "candidate_align": lambda: moe._align_bf16_routes(route_ids, args.heads * args.experts, 128),
+                "candidate_pack_w13": lambda: torch.stack((gate.transpose(1, 2), up.transpose(1, 2)), dim=2).reshape(
+                    args.heads * args.experts, 2560, 256
+                ),
+                "candidate_two_gemms_prepared": gems,
+            },
+            args,
+        )
+    return result
+
+
+def main():
+    args = parse_args()
+    import torch
+
+    if hasattr(torch.version, "musa") and torch.version.musa is not None:
+        importlib.import_module("torchada")  # before importing vLLM / capturing CUDA APIs
+    cuda = getattr(torch, "cuda")
+    has_gpu = cuda.is_available() or (
+        hasattr(torch.version, "musa") and torch.version.musa is not None and torch.musa.is_available()
+    )
+    if not has_gpu:
+        print(json.dumps({"status": "skip", "reason": "No visible CUDA/MUSA device"}))
+        return
+    import vllm_omni.diffusion.models.magi2.fused_moe_kernels as kernels
+    import vllm_omni.diffusion.models.magi2.mh_moe as moe
+
+    root = Path(__file__).resolve().parents[2]
+    try:
+        sha = subprocess.check_output(
+            ["git", "-C", str(root), "rev-parse", "HEAD"], text=True, stderr=subprocess.DEVNULL
+        ).strip()
+    except (OSError, subprocess.CalledProcessError):
+        sha = os.environ.get("MAGI2_BENCH_SOURCE_SHA", "unknown")
+    report = {
+        "device": cuda.get_device_name(),
+        "sm_count": cuda.get_device_properties(0).multi_processor_count,
+        "source_sha": sha,
+        "module_path": moe.__file__,
+        "torch": torch.__version__,
+        "args": {**vars(args), "output": str(args.output)},
+        "cache": "512 MiB flush outside each region",
+        "capture": "eager; JIT warmed; no graph/compile",
+        "results": [],
+    }
+    with torch.inference_mode():
+        for tokens in args.tokens:
+            result = benchmark(torch, moe, kernels, tokens, args)
+            report["results"].append(result)
+            print(json.dumps(result), flush=True)
+    if args.output:
+        args.output.parent.mkdir(parents=True, exist_ok=True)
+        args.output.write_text(json.dumps(report, indent=2) + "\n")
+
+
+if __name__ == "__main__":
+    main()

benchmarks/kernels/benchmark_magi2_mhc.py

@@ -0,0 +1,141 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
+"""Compare native and fused MAGI-2 mHC post-processing on CUDA or MUSA.
+
+Run from the checkout root with PyTorch and the selected backend dependencies.
+torchada is loaded automatically only on MUSA::
+
+    python -m benchmarks.kernels.benchmark_magi2_mhc \
+        --tokens 1 4 16 64 129 4096 --hidden-size 3072 --num-tests 30
+
+The released config supplies hidden=3072, four streams and 20 Sinkhorn steps.
+Token counts are synthetic; supply per-rank sizes from the target trace when
+comparing production shapes. FP32 post/residual logits are strided views of the
+same [tokens, 24] projection, as in MHCHandler.compute_logits. Mix inputs use
+the selected dtype. Both mix implementations consume identical coefficients.
+
+The three comparisons cover coefficient preparation, stream mixing, and their
+combined pipeline (including intermediate allocations, excluding projection,
+normalization and apply_pre). This is an eager operator diagnostic, without
+compile or graph capture. It does not measure full model performance.
+No CUDA/MUSA device (or no PyTorch) produces a JSON skip record and exit status zero.
+"""
+
+import argparse
+import math
+from functools import partial
+from typing import Any
+
+from benchmarks.kernels._magi2_gpu_bench import (
+    add_arguments,
+    check_pair,
+    emit,
+    load_musa,
+    measure_pair,
+    report_environment,
+)
+
+
+def benchmark_shape(torch: Any, mhc: Any, args: argparse.Namespace, tokens: int) -> None:
+    dtype = getattr(torch, args.dtype)
+    device = torch.device("cuda")
+    packed = torch.randn(tokens, 24, device=device, dtype=torch.float32)
+    # Preserve the token stride of the model's shared projection allocation.
+    post_inputs = (
+        packed[:, 4:8],
+        packed[:, 8:].view(tokens, 4, 4),
+        torch.tensor([0.8], device=device),
+        torch.randn(4, device=device),
+        torch.tensor([1.1], device=device),
+        torch.randn(4, 4, device=device),
+    )
+    streams = torch.randn(tokens, 4, args.hidden_size, device=device, dtype=dtype)
+    branch = torch.randn(tokens, args.hidden_size, device=device, dtype=dtype)
+    post_op, mix_op = mhc.MHCPostResidual(), mhc.MHCMix()
+    scale = 1 / math.sqrt(4 * args.hidden_size)
+    kwargs = dict(scale=scale, iterations=args.sinkhorn_iterations, epsilon=1e-12, out_dtype=dtype)
+    prepare_reference = partial(post_op.forward_native, *post_inputs, **kwargs)
+    prepare_fused = partial(post_op, *post_inputs, **kwargs)
+    coefficients = prepare_reference()
+    mix_reference = partial(mix_op.forward_native, streams, branch, *coefficients)
+    mix_fused = partial(mix_op, streams, branch, *coefficients)
+    post_kernel, mix_kernel = "_mhc_post_residual_kernel", "_mhc_mix_kernel"
+    cases = {
+        "post_residual": (prepare_reference, prepare_fused, (post_kernel,)),
+        "mix": (mix_reference, mix_fused, (mix_kernel,)),
+        "pipeline": (
+            lambda: mix_op.forward_native(streams, branch, *prepare_reference()),
+            lambda: mix_op(streams, branch, *prepare_fused()),
+            (post_kernel, mix_kernel),
+        ),
+    }
+    rtol = {torch.bfloat16: 1.6e-2, torch.float16: 2e-3, torch.float32: 1e-5}[dtype]
+    for name, (reference, fused, kernels) in cases.items():
+        if args.op != "all" and args.op != name:
+            continue
+        # BF16 MUSA custom ops can quantize values near zero to the adjacent
+        # representable value; retain the strict relative gate while allowing
+        # one BF16 ulp in the absolute comparison.
+        atol = (
+            (1e-2 if dtype == torch.bfloat16 else 1e-6)
+            if name == "post_residual"
+            else (1e-2 if dtype == torch.bfloat16 else 1e-5)
+        )
+        parity = check_pair(torch, reference, fused, mhc, kernels, rtol=rtol, atol=atol)
+        timing = measure_pair(torch, reference, fused, args)
+        # Logical minimum traffic: input reads plus output writes. Native
+        # intermediates and repeat parameter loads can add actual DRAM traffic.
+        element_bytes = streams.element_size()
+        prepare_bytes = tokens * 20 * (4 + element_bytes) + 22 * 4
+        mix_bytes = tokens * (9 * args.hidden_size + 20) * element_bytes
+        minimum_bytes = {"post_residual": prepare_bytes, "mix": mix_bytes, "pipeline": prepare_bytes + mix_bytes}[name]
+        # Sigmoid/exp costs and max reductions are not represented as FLOPs.
+        prepare_flops = tokens * (72 + 64 * args.sinkhorn_iterations)
+        mix_flops = tokens * args.hidden_size * 36
+        scalar_flops = {"post_residual": prepare_flops, "mix": mix_flops, "pipeline": prepare_flops + mix_flops}[name]
+        emit(
+            status="pass",
+            op=name,
+            tokens=tokens,
+            hidden_size=args.hidden_size,
+            num_streams=4,
+            dtype=args.dtype,
+            logits_dtype="float32",
+            packed_shape=list(packed.shape),
+            post_stride=list(post_inputs[0].stride()),
+            residual_stride=list(post_inputs[1].stride()),
+            scale=scale,
+            sinkhorn_iterations=args.sinkhorn_iterations,
+            epsilon=1e-12,
+            minimum_io_bytes=minimum_bytes,
+            scalar_flops_excluding_exp_and_max=scalar_flops,
+            effective_fused_gbps=minimum_bytes / timing["fused"]["p50_us"] / 1000,
+            **parity,
+            **timing,
+        )
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    add_arguments(parser)
+    parser.add_argument("--hidden-size", type=int, default=3072)
+    parser.add_argument("--sinkhorn-iterations", type=int, default=20)
+    parser.add_argument("--op", choices=["all", "post_residual", "mix", "pipeline"], default="all")
+    args = parser.parse_args()
+    if args.hidden_size <= 0 or not 0 <= args.sinkhorn_iterations <= 64:
+        parser.error("hidden-size must be positive; sinkhorn-iterations must be in [0, 64]")
+    torch = load_musa(args, parser)
+    if torch is None:
+        return
+    cuda = getattr(torch, "cuda")
+    from vllm_omni.diffusion.layers import mhc
+
+    report_environment(torch, args, mhc)
+    with torch.inference_mode():
+        for tokens in args.tokens:
+            benchmark_shape(torch, mhc, args, tokens)
+            cuda.empty_cache()
+
+
+if __name__ == "__main__":
+    main()

benchmarks/kernels/benchmark_magi2_swiglu7.py

@@ -0,0 +1,87 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
+"""Compare native and fused MAGI-2 SwiGLU7 operators on CUDA or MUSA.
+
+Run from the checkout root in an environment with PyTorch and the selected
+backend dependencies.  torchada is loaded automatically only on MUSA::
+
+    python -m benchmarks.kernels.benchmark_magi2_swiglu7 \
+        --tokens 1 4 16 64 129 4096 --intermediate-size 1280 --num-tests 30
+
+The released MAGI-2 config supplies the 1280 intermediate width. Token counts
+are a synthetic sweep; pass per-rank sizes from the target trace for production
+comparisons. Input is contiguous interleaved gate/up [tokens, 2 * intermediate].
+Both paths evaluate FP32 arithmetic with one cast to the requested output dtype.
+This measures eager operator latency, not compiled serving performance.
+No CUDA/MUSA device (or no PyTorch) produces a JSON skip record and exit status zero.
+"""
+
+import argparse
+from functools import partial
+
+from benchmarks.kernels._magi2_musa_bench import (
+    add_arguments,
+    check_pair,
+    emit,
+    load_musa,
+    measure_pair,
+    report_environment,
+)
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    add_arguments(parser)
+    parser.add_argument("--intermediate-size", type=int, default=1280)
+    parser.add_argument("--out-dtype", choices=["bfloat16", "float16", "float32"], default=None)
+    args = parser.parse_args()
+    if args.intermediate_size <= 0:
+        parser.error("intermediate-size must be positive")
+    torch = load_musa(args, parser)
+    if torch is None:
+        return
+    cuda = getattr(torch, "cuda")
+    from vllm_omni.diffusion.layers import swiglu7
+
+    report_environment(torch, args, swiglu7)
+    dtype = getattr(torch, args.dtype)
+    out_dtype = getattr(torch, args.out_dtype or args.dtype)
+    device = torch.device("cuda")
+    rtol = {torch.bfloat16: 1.6e-2, torch.float16: 2e-3, torch.float32: 1e-5}[out_dtype]
+    atol = 1e-5 if out_dtype == torch.float32 else 2e-3
+    op = swiglu7.SwiGLU7()
+    with torch.inference_mode():
+        for tokens in args.tokens:
+            # Values outside both clamp boundaries exercise the released formula.
+            x = (torch.randn(tokens, 2 * args.intermediate_size, device=device) * 10).to(dtype)
+            reference = partial(op.forward_native, x, out_dtype=out_dtype)
+            fused = partial(op, x, out_dtype=out_dtype)
+            parity = check_pair(torch, reference, fused, swiglu7, ("_swiglu7_kernel",), rtol=rtol, atol=atol)
+            timing = measure_pair(torch, reference, fused, args)
+            output_elements = tokens * args.intermediate_size
+            minimum_io_bytes = (
+                x.numel() * x.element_size() + output_elements * torch.empty((), dtype=out_dtype).element_size()
+            )
+            emit(
+                status="pass",
+                op="swiglu7",
+                input_shape=list(x.shape),
+                input_stride=list(x.stride()),
+                dtype=args.dtype,
+                out_dtype=args.out_dtype or args.dtype,
+                alpha=1.702,
+                limit=7.0,
+                minimum_io_bytes=minimum_io_bytes,
+                # Six scalar arithmetic operations and one exp per output;
+                # clamps and transcendental costs are not modeled as FLOPs.
+                scalar_flops_excluding_exp_and_clamps=6 * output_elements,
+                effective_fused_gbps=minimum_io_bytes / timing["fused"]["p50_us"] / 1000,
+                **parity,
+                **timing,
+            )
+            del x, reference, fused
+            cuda.empty_cache()
+
+
+if __name__ == "__main__":
+    main()

benchmarks/mammoth_moda2/README.md

@@ -0,0 +1,124 @@
+# MammothModa2 startup benchmark
+
+Measures `Omni()` initialization and first/subsequent image requests for
+MammothModa2-Preview. This is a measurement baseline for
+[#7075](https://github.com/vllm-project/vllm-omni/issues/7075); it changes no runtime code.
+
+## Run
+
+From the repository root, with the model downloaded and vLLM-Omni installed:
+
+```bash
+set -o pipefail
+python benchmarks/mammoth_moda2/bench_startup.py \
+    --model /path/MammothModa2-Preview \
+    --deploy-config vllm_omni/deploy/mammoth_moda2.yaml \
+    --height 512 --width 512 --seed 42 \
+    --extra-body '{"text_guidance_scale": 4.0, "cfg_range": [0.0, 1.0], "num_inference_steps": 8}' \
+    --repeat 1 --label baseline --output-json baseline.json --save-image baseline.png \
+    2>&1 | tee baseline.log
+python benchmarks/mammoth_moda2/parse_startup_log.py baseline.log --require-complete --markdown
+```
+
+Use `OMP_NUM_THREADS=16` to set CPU threads and `--parallel-stage-init` to
+initialize stages concurrently. For first/subsequent requests, use
+`--height 1024 --width 1024 --repeat 3` and 50 inference steps. Memory polling
+is opt-in (`--sample-memory`); collect it in separate runs to avoid perturbing
+the timing comparison. `--save-image` saves only the first request's image.
+Compare that image across runs; a fixed seed alone does not guarantee equality
+across execution modes or software versions.
+
+## Measurement boundaries
+
+| Metric | Boundary / limitation |
+| --- | --- |
+| Imports | Before torch import → benchmark imports complete; excludes interpreter startup |
+| Time to ready | `Omni()` call → return; excludes client imports and requests |
+| Time to first image | Benchmark clock start → first `generate()` return, including request setup |
+| First / subsequent request | Each `generate()` call → return; same prompt and settings |
+| Stage spawn / device init | Log timestamps: launch → `world_size=` → `Starting to load model`; 1 s resolution |
+| Weight load | Engine-reported `Loading weights took`; read/cast/copy are merged |
+| Model setup estimate | Reported model-load time minus weight-load time; not independently instrumented |
+| Profile + KV + warmup | One merged interval after loading; compilation/capture may be nested within it |
+| Compile / graph capture | Explicit engine log durations when present; N/A for eager execution |
+| Memory | Optional 1 s samples: sum of process-tree RSS and maximum device memory used across GPUs reported by `nvidia-smi`; includes other GPU users and can miss transient peaks |
+
+Stage timelines overlap with parallel initialization; do not sum them to
+calculate time to ready. Missing log fields stay unmeasured. The parser marks
+logs without `BENCH_JSON` as incomplete; `--require-complete` makes this an error.
+
+## Reference results
+
+A800-SXM4-80GB; Ubuntu 22.04; 16-CPU cgroup quota on a 128-core host;
+vLLM 0.28.0+cu129, torch 2.13.0+cu129, vLLM-Omni `34aa1d2a`.
+Checkpoint: 34.5 GiB, 8 shards, mostly fp32, on local NVMe.
+Committed eager deploy config, GPU memory fractions 0.5 / 0.3.
+
+The 2026-09-10 rerun used the corrected benchmark: one warm-up process,
+then three rounds of four configurations with order rotated between rounds.
+The checkpoint page cache was warmed before the matrix; no cache eviction
+was requested during it. Each fresh process sent one 512×512, 8-step request
+(guidance 4.0, seed 42). Memory polling was disabled. All 12 measured runs
+are retained in [the results](reference_results.json).
+Values below are `Omni()` seconds, mean ± sample standard deviation, n=3;
+client imports took approximately another 10 s.
+
+| CPU threads | Serial init | Parallel init |
+| --- | --- | --- |
+| Default (64) | 60.1 ± 0.2 | 41.1 ± 1.0 |
+| 16 | 56.5 ± 0.9 | 36.7 ± 0.2 |
+
+On this configuration, parallel initialization reduced time to ready by about
+32%; combining it with 16 threads reduced it by about 39%. This supports
+trying these settings on similar deployments, not changing defaults for all
+models or hardware. Fixed 512×512 samples matched across all 12 measured runs;
+this is a smoke check, not a broad quality evaluation.
+
+For context, in the earlier serial baseline AR / DiT spent approximately 11 / 11 s in
+spawn and imports, 6 / 6 s in device setup, 6 / 4 s loading weights, and
+5 / 2 s in profile + KV + warmup. About 6 s followed the last stage becoming
+ready. These coarse log intervals locate work; they are not independent
+operator timings.
+
+Separate observations from the earlier experiment (not rerun in this matrix):
+
+- Requested page-cache eviction: 80.8 s; FlashInfer JIT cache removed:
+  129.1 s (means, n=3 each). These are separate cache conditions, not guaranteed
+  cold-machine startup or isolated disk/JIT costs.
+- Three 1024×1024, 50-step eager requests in one process took
+  95.2 / 94.9 / 94.6 s. No large first-request penalty was observed in that run.
+- With AR `enforce_eager: false` and serial init, time to ready was
+  62.5 ± 0.1 s (warm caches, n=3); logged graph capture was 4 s / 0.20 GiB.
+  The model reported torch.compile unsupported. KV capacity fell from about
+  15.7 to 2.0 GiB, and one parallel-init trial failed. The reported non-KV
+  profiling difference is not a measurement of graph allocation. Graph-mode
+  samples differed from eager. This needs separate memory and quality analysis
+  before recommending the configuration.
+
+## Loading diagnostic and storage scenarios
+
+```bash
+OMP_NUM_THREADS=16 python benchmarks/mammoth_moda2/raw_load_bench.py \
+    --model /path/MammothModa2-Preview --shards 6,7,8 --mode cpu_cast
+# Compare --mode gpu_cast in a fresh process with the same cache conditions.
+
+MODEL_LOCAL=/local/MammothModa2-Preview OUT_DIR=./startup-bench \
+    bash benchmarks/mammoth_moda2/bench_storage_scenarios.sh
+# Optional MODEL_NFS=/nfs/MammothModa2-Preview; SCENARIOS="local-cold local-warm".
+```
+
+The diagnostic clones each mmap-backed tensor into owned CPU memory before
+casting. `materialize_s` includes page faults, CPU allocation and copying;
+`cast_s` measures dtype conversion; `h2d_s` includes GPU allocation and copy.
+CUDA context/operation warmup is outside the timer. Total also includes file
+opening and bookkeeping. This extra CPU copy differs from the engine loader,
+so these numbers do not decompose its loading time or measure disk bandwidth.
+The revised diagnostic was run three times each for CPU cast with 64 / 16
+threads and GPU cast with 16 threads; per-run values are in the results file.
+Both CPU materialization and conversion were faster with 16 threads here; this
+does not isolate how much of the engine-level improvement comes from either.
+
+Storage scenarios explicitly read shards before warm runs and request
+`posix_fadvise(DONTNEED)` before cold runs. Eviction is not guaranteed, and
+server-side NFS caches are uncontrolled. The script stops on failure and
+summarizes only logs from the current invocation.

benchmarks/mammoth_moda2/__init__.py

@@ -1,4 +1,2 @@
 # SPDX-License-Identifier: Apache-2.0
 # SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
-
-"""Qwen3-Omni turn-based browser call UI."""

benchmarks/mammoth_moda2/bench_startup.py

@@ -0,0 +1,338 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
+"""Startup / loading benchmark for multi-stage omni models (MammothModa2 first).
+
+Measures, from a single fresh process:
+  * python import time (torch + vllm_omni)
+  * Omni() construction wall time (all stages: weight load, profiling, KV cache, warmup)
+  * first request latency (includes JIT / lazy init that only happens on first run)
+  * steady-state request latency (repeat N-1 more requests)
+  * optional host RSS / device memory samples (--sample-memory, once a second)
+
+Per-stage breakdown (weight loading, model-load memory, engine init) is emitted by the
+stage engine-core subprocesses into the log; use parse_startup_log.py on the captured
+stdout/stderr to merge them with the JSON written here.
+
+Usage (keep the log: the per-stage breakdown is parsed from it afterwards):
+  python benchmarks/mammoth_moda2/bench_startup.py --model /path/MammothModa2-Preview \
+      --deploy-config vllm_omni/deploy/mammoth_moda2.yaml \
+      --height 1024 --width 1024 --seed 42 \
+      --extra-body '{"text_guidance_scale": 4.0, "cfg_range": [0.0, 1.0], "num_inference_steps": 50}' \
+      --repeat 2 --label nfs-warm --output-json out.json 2>&1 | tee run.log
+  python benchmarks/mammoth_moda2/parse_startup_log.py run.log --markdown
+"""
+
+import argparse
+import json
+import os
+import platform
+import subprocess
+import threading
+import time
+
+T_PROCESS_START = time.perf_counter()
+
+import torch  # noqa: E402
+
+from vllm_omni.diffusion.utils.param_utils import apply_declared_extra_args  # noqa: E402
+from vllm_omni.entrypoints.omni import Omni  # noqa: E402
+from vllm_omni.entrypoints.openai.stage_params import clone_sampling_params  # noqa: E402
+from vllm_omni.inputs.data import OmniDiffusionSamplingParams  # noqa: E402
+from vllm_omni.model_extras import (  # noqa: E402
+    build_text_to_image_prompt as build_model_text_to_image_prompt,
+)
+from vllm_omni.model_extras import (  # noqa: E402
+    get_extra_body_params,
+    get_model_class_name,
+    should_init_extra_args_for_non_diffusion_stages,
+)
+from vllm_omni.platforms import current_omni_platform  # noqa: E402
+
+T_IMPORTS_DONE = time.perf_counter()
+
+
+def parse_args() -> argparse.Namespace:
+    p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    p.add_argument("--model", required=True)
+    p.add_argument("--deploy-config", default=None)
+    p.add_argument("--prompt", default="A stylish woman riding a motorcycle in NYC, movie poster style")
+    p.add_argument("--negative-prompt", default=None)
+    p.add_argument("--height", type=int, default=1024)
+    p.add_argument("--width", type=int, default=1024)
+    p.add_argument("--seed", type=int, default=42)
+    p.add_argument("--extra-body", type=json.loads, default=None, help="JSON object forwarded as model extra_body")
+    p.add_argument(
+        "--repeat", type=int, default=2, help="number of generate() calls; 1st = first-request, rest = steady"
+    )
+    p.add_argument("--label", default="run", help="free-form label, e.g. nfs-cold / nfs-warm / local-nvme")
+    p.add_argument("--output-json", default=None)
+    p.add_argument("--save-image", default=None, help="optional path to save the first request's image")
+    p.add_argument("--sample-memory", action="store_true", help="sample memory once a second (adds overhead)")
+    p.add_argument(
+        "--parallel-stage-init",
+        action="store_true",
+        help="initialize stages that share a GPU concurrently (VllmOmniOrchestratorConfig.parallel_stage_init)",
+    )
+    args = p.parse_args()
+    if args.repeat < 1:
+        p.error("--repeat must be at least 1")
+    return args
+
+
+def gpu_info() -> dict:
+    info = {"torch": torch.__version__, "cuda": torch.version.cuda, "python": platform.python_version()}
+    try:
+        import vllm
+
+        info["vllm"] = vllm.__version__
+    except Exception:  # pragma: no cover
+        pass
+    try:
+        import vllm_omni
+
+        info["vllm_omni"] = getattr(vllm_omni, "__version__", "src")
+    except Exception:  # pragma: no cover
+        pass
+    try:
+        out = subprocess.run(
+            ["nvidia-smi", "--query-gpu=name,driver_version,memory.total", "--format=csv,noheader"],
+            capture_output=True,
+            text=True,
+            timeout=10,
+        )
+        info["gpu"] = out.stdout.strip()
+    except Exception:
+        info["gpu"] = "n/a"
+    return info
+
+
+class PeakSampler:
+    """Poll host RSS of this process tree and GPU memory in use once a second (benchmark-side only)."""
+
+    def __init__(self, interval_s: float = 1.0):
+        self.interval_s = interval_s
+        self.host_rss_peak_gib = 0.0
+        self.gpu_used_peak_gib = 0.0
+        self._stop = threading.Event()
+        self._thread = threading.Thread(target=self._run, daemon=True)
+
+    def _sample(self) -> None:
+        try:
+            import psutil
+
+            root = psutil.Process()
+            rss = root.memory_info().rss + sum(
+                c.memory_info().rss for c in root.children(recursive=True) if c.is_running()
+            )
+            self.host_rss_peak_gib = max(self.host_rss_peak_gib, rss / 2**30)
+        except Exception:
+            pass
+        try:
+            out = subprocess.run(
+                ["nvidia-smi", "--query-gpu=memory.used", "--format=csv,noheader,nounits"],
+                capture_output=True,
+                text=True,
+                timeout=5,
+            )
+            used = max(float(x) for x in out.stdout.split() if x.strip())
+            self.gpu_used_peak_gib = max(self.gpu_used_peak_gib, used / 1024)
+        except Exception:
+            pass
+
+    def _run(self) -> None:
+        while not self._stop.is_set():
+            self._sample()
+            self._stop.wait(self.interval_s)
+
+    def start(self) -> "PeakSampler":
+        self._thread.start()
+        return self
+
+    def stop(self) -> dict:
+        self._stop.set()
+        self._thread.join(timeout=10)
+        self._sample()
+        return {
+            "host_rss_peak_gib": round(self.host_rss_peak_gib, 2),
+            "gpu_used_peak_gib": round(self.gpu_used_peak_gib, 2),
+        }
+
+
+def fs_type(path: str) -> str:
+    try:
+        out = subprocess.run(["stat", "-f", "-c", "%T", path], capture_output=True, text=True, timeout=10)
+        return out.stdout.strip()
+    except Exception:
+        return "unknown"
+
+
+def build_request(omni: Omni, args: argparse.Namespace, generator: torch.Generator):
+    model_class_name = get_model_class_name(omni)
+    declared = get_extra_body_params(model_class_name)
+
+    prompt_dict = {"prompt": args.prompt}
+    if args.negative_prompt:
+        prompt_dict["negative_prompt"] = args.negative_prompt
+    prompt_dict = build_model_text_to_image_prompt(
+        model_class_name=model_class_name, prompt=prompt_dict, height=args.height, width=args.width
+    )
+
+    diffusion_params = OmniDiffusionSamplingParams(
+        height=args.height,
+        width=args.width,
+        seed=args.seed,
+        generator=generator,
+    )
+    user_extra = dict(args.extra_body or {})
+    if declared:
+        apply_declared_extra_args(diffusion_params, declared, user_extra)
+    else:
+        diffusion_params.extra_args.update({k: v for k, v in user_extra.items() if v is not None})
+
+    init_non_diffusion = should_init_extra_args_for_non_diffusion_stages(model_class_name)
+    sampling_params_list = [clone_sampling_params(p) for p in (omni.default_sampling_params_list or [])]
+    if not sampling_params_list:
+        sampling_params_list = [diffusion_params]
+
+    replaced = False
+    for idx, params in enumerate(sampling_params_list):
+        if isinstance(params, OmniDiffusionSamplingParams):
+            sampling_params_list[idx] = diffusion_params
+            replaced = True
+        elif init_non_diffusion and hasattr(params, "extra_args"):
+            params.extra_args = {**(params.extra_args or {}), **(diffusion_params.extra_args or {})}
+            if hasattr(params, "seed"):
+                params.seed = args.seed
+            # MammothModa2 AR stage: one visual token per grid cell + one EOL per row + 1 look-ahead.
+            info = prompt_dict.get("additional_information", {})
+            if idx == 0 and info.get("omni_task") == ["t2i"]:
+                ar_w = int(info.get("ar_width", [0])[0])
+                ar_h = int(info.get("ar_height", [0])[0])
+                if ar_w > 0 and ar_h > 0:
+                    params.max_tokens = ar_h * (ar_w + 1) + 1
+    if not replaced and len(sampling_params_list) == 1:
+        sampling_params_list = [diffusion_params]
+    return prompt_dict, sampling_params_list
+
+
+def stage_metrics(outputs) -> dict:
+    """Pull vllm-omni's per-stage timing out of the request output when present."""
+    res = {}
+    for out in outputs or []:
+        metrics = getattr(out, "metrics", None)
+        if isinstance(metrics, dict) and "stage_metrics" in metrics:
+            for sid, m in metrics["stage_metrics"].items():
+                if not isinstance(m, dict):
+                    continue
+                res[str(sid)] = {
+                    k: m.get(k)
+                    for k in ("stage_gen_time_ms", "num_tokens_in", "num_tokens_out", "vllm_ttft_ms", "vllm_tpot_ms")
+                    if k in m
+                }
+    return res
+
+
+def main() -> None:
+    args = parse_args()
+    result = {
+        "label": args.label,
+        "model": args.model,
+        "prompt": args.prompt,
+        "negative_prompt": args.negative_prompt,
+        "sample_memory": args.sample_memory,
+        "model_fs": fs_type(args.model),
+        "deploy_config": args.deploy_config,
+        "height": args.height,
+        "width": args.width,
+        "seed": args.seed,
+        "extra_body": args.extra_body,
+        "parallel_stage_init": args.parallel_stage_init,
+        "omp_num_threads": os.environ.get("OMP_NUM_THREADS"),
+        "env": gpu_info(),
+        "phases_s": {"imports": round(T_IMPORTS_DONE - T_PROCESS_START, 3)},
+        "requests": [],
+    }
+
+    omni_kwargs = {"model": args.model, "mode": "text-to-image", "log_stats": True}
+    if args.deploy_config:
+        omni_kwargs["deploy_config"] = args.deploy_config
+    if args.parallel_stage_init:
+        omni_kwargs["parallel_stage_init"] = True
+
+    sampler = PeakSampler().start() if args.sample_memory else None
+    t0 = time.perf_counter()
+    omni = None
+    try:
+        omni = Omni(**omni_kwargs)
+        t1 = time.perf_counter()
+        result["peak_mem_startup"] = sampler.stop() if sampler else {}
+        sampler = None
+        result["phases_s"]["engine_init"] = round(t1 - t0, 3)
+        result["phases_s"]["process_to_engine_ready"] = round(t1 - T_PROCESS_START, 3)
+        print(
+            f"BENCH engine_init={t1 - t0:.2f}s process_to_engine_ready={t1 - T_PROCESS_START:.2f}s "
+            f"peak_mem_startup={result['peak_mem_startup']}",
+            flush=True,
+        )
+        sampler = PeakSampler().start() if args.sample_memory else None
+
+        generator = torch.Generator(device=current_omni_platform.device_type).manual_seed(args.seed)
+        outputs = None
+        first_outputs = None
+        first_image_elapsed = 0.0
+        for i in range(args.repeat):
+            prompt_dict, sampling_params_list = build_request(omni, args, generator)
+            tr0 = time.perf_counter()
+            outputs = omni.generate(prompt_dict, sampling_params_list=sampling_params_list)
+            tr1 = time.perf_counter()
+            if first_outputs is None:
+                first_outputs = outputs
+                first_image_elapsed = tr1 - T_PROCESS_START
+            rec = {
+                "index": i,
+                "kind": "first" if i == 0 else "steady",
+                "latency_s": round(tr1 - tr0, 3),
+                "stages": stage_metrics(outputs),
+            }
+            result["requests"].append(rec)
+            print(f"BENCH request[{i}] {rec['kind']} latency={tr1 - tr0:.2f}s stages={rec['stages']}", flush=True)
+
+        result["peak_mem_requests"] = sampler.stop() if sampler else {}
+        sampler = None
+        steady = [r["latency_s"] for r in result["requests"] if r["kind"] == "steady"]
+        result["summary_s"] = {
+            "imports": result["phases_s"]["imports"],
+            "engine_init": result["phases_s"]["engine_init"],
+            "first_request": result["requests"][0]["latency_s"],
+            "steady_request_avg": round(sum(steady) / len(steady), 3) if steady else None,
+            "time_to_first_image_from_process_start": round(first_image_elapsed, 3),
+        }
+
+        for key in ("host_rss_peak_gib", "gpu_used_peak_gib"):
+            result["summary_s"][key] = (
+                max(result["peak_mem_startup"][key], result["peak_mem_requests"][key]) if args.sample_memory else None
+            )
+
+        from vllm_omni.diffusion.utils.image_output import extract_images_from_outputs
+
+        images = extract_images_from_outputs(first_outputs)
+        if not images:
+            raise RuntimeError("First request returned no image")
+        if args.save_image:
+            os.makedirs(os.path.dirname(os.path.abspath(args.save_image)), exist_ok=True)
+            images[0].save(args.save_image)
+
+        print("BENCH_JSON " + json.dumps(result), flush=True)
+        if args.output_json:
+            with open(args.output_json, "w") as f:
+                json.dump(result, f, indent=2)
+            print(f"BENCH wrote {args.output_json}", flush=True)
+    finally:
+        if sampler:
+            sampler.stop()
+        if omni is not None:
+            omni.close()
+
+
+if __name__ == "__main__":
+    main()

benchmarks/mammoth_moda2/bench_storage_scenarios.sh

@@ -0,0 +1,67 @@
+#!/bin/bash
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
+# MODEL_LOCAL=/local/model MODEL_NFS=/nfs/model OUT_DIR=./startup-bench bash "$0"
+# Optional SCENARIOS: local-cold local-warm nfs-cold nfs-warm.
+set -euo pipefail
+HERE=$(cd "$(dirname "$0")" && pwd)
+OUT_DIR=${OUT_DIR:-./startup-bench}
+DEPLOY_CONFIG=${DEPLOY_CONFIG:-vllm_omni/deploy/mammoth_moda2.yaml}
+STEPS=${STEPS:-50}
+SIZE=${SIZE:-1024}
+REPEAT=${REPEAT:-2}
+SEED=${SEED:-42}
+EXTRA=${EXTRA:-'{"text_guidance_scale": 4.0, "cfg_range": [0.0, 1.0], "num_inference_steps": '$STEPS'}'}
+mkdir -p "$OUT_DIR/logs" "$OUT_DIR/json" "$OUT_DIR/images"
+rm -f "$OUT_DIR/summary.json" "$OUT_DIR/summary.md"
+
+prepare_cache() {
+  python - "$1" "$2" <<'PY'
+import os
+import sys
+from pathlib import Path
+
+files = sorted(Path(sys.argv[1]).glob("*.safetensors"))
+if not files:
+    raise SystemExit("no checkpoint shards found")
+for path in files:
+    with path.open("rb") as f:
+        if sys.argv[2] == "cold":
+            os.posix_fadvise(f.fileno(), 0, 0, os.POSIX_FADV_DONTNEED)
+        else:
+            while f.read(8 * 1024 * 1024):
+                pass
+print("[bench]", "requested eviction of" if sys.argv[2] == "cold" else "read", len(files), "shards")
+PY
+}
+
+default_scenarios=""
+if [ -n "${MODEL_LOCAL:-}" ]; then default_scenarios="local-cold local-warm"; fi
+if [ -n "${MODEL_NFS:-}" ]; then default_scenarios="$default_scenarios nfs-cold nfs-warm"; fi
+read -r -a scenarios <<< "${SCENARIOS:-$default_scenarios}"
+if [ "${#scenarios[@]}" -eq 0 ]; then echo "Set MODEL_LOCAL or MODEL_NFS" >&2; exit 1; fi
+logs=()
+for label in "${scenarios[@]}"; do
+  case "$label" in
+    local-cold|local-warm) model=${MODEL_LOCAL:?Set MODEL_LOCAL} ;;
+    nfs-cold|nfs-warm) model=${MODEL_NFS:?Set MODEL_NFS} ;;
+    *) echo "Unknown scenario: $label" >&2; exit 1 ;;
+  esac
+  prepare_cache "$model" "${label##*-}"
+  log="$OUT_DIR/logs/$label.log"
+  # Remove previous outputs for this label so a failed rerun cannot leave stale results.
+  rm -f "$OUT_DIR/json/$label.json" "$OUT_DIR/images/$label.png"
+  if python "$HERE/bench_startup.py" --model "$model" --deploy-config "$DEPLOY_CONFIG" \
+    --height "$SIZE" --width "$SIZE" --seed "$SEED" --extra-body "$EXTRA" --repeat "$REPEAT" \
+    --label "$label" --output-json "$OUT_DIR/json/$label.json" --save-image "$OUT_DIR/images/$label.png" \
+    > "$log" 2>&1; then
+    logs+=("$log")
+  else
+    rc=$?
+    echo "[bench] $label failed (exit $rc); see $log" >&2
+    exit "$rc"
+  fi
+done
+python "$HERE/parse_startup_log.py" "${logs[@]}" --require-complete --markdown \
+  --json-out "$OUT_DIR/summary.json" | tee "$OUT_DIR/summary.md"
+echo BENCH_ALL_DONE

benchmarks/mammoth_moda2/parse_startup_log.py

@@ -0,0 +1,245 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
+"""Merge per-stage startup numbers from a bench_startup.py log with its BENCH_JSON line.
+
+The stage engine cores run in subprocesses and report weight-loading / init timings
+only through log lines such as:
+
+  (StageEngineCoreProc_stage0_replica0 pid=1) INFO ... Loading weights took 39.66 seconds
+  (StageEngineCoreProc_stage0_replica0 pid=1) INFO ... Model loading took 21.45 GiB memory and 40.59 seconds
+  (StageEngineCoreProc_stage0_replica0 pid=1) INFO ... init engine (profile, create kv cache, warmup model) took 7.00 s
+  (StageEngineCoreProc_stage0_replica0 pid=1) INFO ... Filesystem type for checkpoints: NFS4.
+      Checkpoint size: 34.52 GiB. Available RAM: 22.76 GiB.
+  INFO ... [Omni] AsyncOmniEngine initialized in 159.74 seconds
+
+Usage:
+  python benchmarks/mammoth_moda2/parse_startup_log.py run.log [more.log ...] [--markdown] [--json-out summary.json]
+"""
+
+import argparse
+import json
+import re
+import sys
+from datetime import datetime
+from typing import Any
+
+STAGE = r"\(StageEngineCoreProc_stage(?P<stage>\d+)_replica\d+ pid=\d+\)"
+PATTERNS = {
+    "weights_load_s": re.compile(STAGE + r".*Loading weights took (?P<v>[\d.]+) seconds"),
+    "model_load_s": re.compile(STAGE + r".*Model loading took (?P<mem>[\d.]+) GiB memory and (?P<v>[\d.]+) seconds"),
+    "engine_init_s": re.compile(
+        STAGE + r".*init engine \(profile, create kv cache, warmup model\) took (?P<v>[\d.]+) s"
+    ),
+    # Only present when the stage does not run with enforce_eager.
+    "graph_capture_s": re.compile(
+        STAGE + r".*Graph capturing finished in (?P<v>[\d.]+) secs, took (?P<mem>[\d.]+) GiB"
+    ),
+    "compile_s": re.compile(STAGE + r".*torch\.compile takes (?P<v>[\d.]+) s in total"),
+    "ckpt_fs": re.compile(
+        STAGE
+        + r".*Filesystem type for checkpoints: (?P<fs>\w+)\. Checkpoint size: (?P<size>[\d.]+) GiB\."
+        + r" Available RAM: (?P<ram>[\d.]+) GiB"
+    ),
+}
+ENGINE_READY = re.compile(r"AsyncOmniEngine initialized in (?P<v>[\d.]+) seconds")
+PREFETCH_SKIP = re.compile(STAGE + r".*exceeds 90% of available RAM")
+COMPILE_UNSUPPORTED = re.compile(STAGE + r".*`torch\.compile` is turned on, but the model .* does not support it")
+
+# Startup timeline: (milestone key, regex). Timestamps in vllm logs are "MM-DD HH:MM:SS" (1 s resolution).
+TS = re.compile(r"(?:INFO|WARNING|ERROR) (?P<ts>\d\d-\d\d \d\d:\d\d:\d\d)")
+MILESTONES = [
+    ("omni_init_start", re.compile(r"\[Omni\] Initializing with model")),
+    ("stage{stage}_launch", re.compile(r"Stage-(?P<stage>\d+) set runtime devices")),
+    ("stage{stage}_proc_up", re.compile(STAGE + r".*world_size=\d+ rank=\d+")),
+    ("stage{stage}_load_start", re.compile(STAGE + r".*Starting to load model")),
+    ("stage{stage}_load_done", re.compile(STAGE + r".*Loading weights took")),
+    ("stage{stage}_init_done", re.compile(STAGE + r".*init engine \(profile, create kv cache, warmup model\) took")),
+    ("stage{stage}_ready", re.compile(r"\[StageRuntime\] Stage (?P<stage>\d+) initialized")),
+    ("engine_ready", re.compile(r"AsyncOmniEngine initialized in")),
+]
+
+
+def _ts_seconds(ts: str) -> int:
+    # Use a leap year because engine timestamps do not include the year.
+    value = datetime.strptime("2000-" + ts, "%Y-%m-%d %H:%M:%S")
+    return int((value - datetime(2000, 1, 1)).total_seconds())
+
+
+def timeline(lines: list[str]) -> dict:
+    """Return milestone timestamps (s, relative to omni_init_start) and derived per-stage phase durations."""
+    marks: dict[str, int] = {}
+    for line in lines:
+        tsm = TS.search(line)
+        if not tsm:
+            continue
+        for key, pat in MILESTONES:
+            m = pat.search(line)
+            if m:
+                k = key.format(stage=m.group("stage")) if "{stage}" in key else key
+                marks.setdefault(k, _ts_seconds(tsm.group("ts")))
+                break
+    if "omni_init_start" not in marks:
+        return {}
+    t0 = marks["omni_init_start"]
+    rel = {k: (v - t0) % (366 * 86400) for k, v in marks.items()}
+    phases: dict[str, Any] = {}
+    names = ("launch", "proc_up", "load_start", "load_done", "init_done", "ready")
+    for s in sorted({k.split("_")[0] for k in rel if k.startswith("stage")}):
+        found = {name: rel.get(f"{s}_{name}") for name in names}
+        if any(v is None for v in found.values()):
+            continue
+        ts = {name: int(v) for name, v in found.items() if v is not None}
+        phases[s] = {
+            "spawn_import_config_s": ts["proc_up"] - ts["launch"],
+            "device_dist_init_s": ts["load_start"] - ts["proc_up"],
+            "model_and_weights_load_s": ts["load_done"] - ts["load_start"],
+            "profile_kv_warmup_s": ts["init_done"] - ts["load_done"],
+            "ready_handoff_s": ts["ready"] - ts["init_done"],
+            "total_s": ts["ready"] - ts["launch"],
+        }
+    if "engine_ready" in rel:
+        last_ready = max((v for k, v in rel.items() if k.endswith("_ready") and k != "engine_ready"), default=0)
+        phases["post_stages_wiring_s"] = rel["engine_ready"] - last_ready
+        phases["engine_ready_s"] = rel["engine_ready"]
+    return {"marks_rel_s": rel, "phases": phases}
+
+
+def parse(path: str) -> dict:
+    stages: dict[str, dict] = {}
+    rec: dict[str, Any] = {"log": path, "stages": stages}
+    with open(path, errors="replace") as f:
+        lines = f.readlines()
+    rec["timeline"] = timeline(lines)
+    for line in lines:
+        if line.startswith("BENCH_JSON "):
+            rec["bench"] = json.loads(line[len("BENCH_JSON ") :])
+            continue
+        m = ENGINE_READY.search(line)
+        if m:
+            rec["engine_ready_s_reported"] = float(m.group("v"))
+            continue
+        m = PREFETCH_SKIP.search(line)
+        if m:
+            stages.setdefault(m.group("stage"), {})["page_cache_prefetch_skipped"] = True
+            continue
+        m = COMPILE_UNSUPPORTED.search(line)
+        if m:
+            stages.setdefault(m.group("stage"), {})["torch_compile_unsupported"] = True
+            continue
+        for key, pat in PATTERNS.items():
+            m = pat.search(line)
+            if not m:
+                continue
+            st = stages.setdefault(m.group("stage"), {})
+            if key == "ckpt_fs":
+                st["checkpoint_fs"] = m.group("fs")
+                st["checkpoint_size_gib"] = float(m.group("size"))
+                st["available_ram_gib"] = float(m.group("ram"))
+            elif key == "graph_capture_s":
+                st["graph_capture_s"] = float(m.group("v"))
+                st["graph_capture_mem_gib"] = float(m.group("mem"))
+            elif key == "model_load_s":
+                st["model_load_s"] = float(m.group("v"))
+                st["model_load_mem_gib"] = float(m.group("mem"))
+                # "Model loading took" covers model construction + weight loading; "Loading weights took" only
+                # the weights. The difference also includes loader setup/teardown overhead.
+                if "weights_load_s" in st:
+                    st["model_setup_estimate_s"] = round(st["model_load_s"] - st["weights_load_s"], 2)
+            else:
+                st[key] = float(m.group("v"))
+            break
+    rec["status"] = "complete" if "bench" in rec else "incomplete"
+    return rec
+
+
+def _table(columns: dict[str, str], records: list[dict]) -> str:
+    def row(values):
+        return "| " + " | ".join("" if v is None else str(v) for v in values) + " |"
+
+    lines = [row(columns.values()), row(["---"] * len(columns))]
+    lines.extend(row(record.get(key) for key in columns) for record in records)
+    return "\n".join(lines)
+
+
+def to_markdown(recs: list[dict]) -> str:
+    summary, stages, phases, warnings = [], [], [], []
+    for r in recs:
+        b = r.get("bench", {})
+        label = b.get("label", r["log"])
+        if r.get("status") == "incomplete":
+            warnings.append(f"Incomplete log: {r['log']} (no BENCH_JSON).")
+        summary.append({"label": label, **b.get("summary_s", {})})
+        for sid, st in sorted(r["stages"].items()):
+            stages.append({"label": label, "stage": sid, **st})
+            if st.get("torch_compile_unsupported"):
+                stages[-1]["compile_s"] = "unsupported"
+        ph = r.get("timeline", {}).get("phases", {})
+        for sid, values in sorted(ph.items()):
+            if sid.startswith("stage"):
+                phases.append({"label": label, "stage": sid, **values})
+        if "post_stages_wiring_s" in ph:
+            phases.append({"label": label, "stage": "after last stage", "total_s": ph["post_stages_wiring_s"]})
+    tables = [
+        (
+            {
+                "label": "label",
+                "imports": "imports (s)",
+                "engine_init": "engine ready (s)",
+                "first_request": "first request (s)",
+                "steady_request_avg": "subsequent avg (s)",
+                "time_to_first_image_from_process_start": "process→first image (s)",
+                "host_rss_peak_gib": "sampled host RSS (GiB)",
+                "gpu_used_peak_gib": "sampled GPU used (GiB)",
+            },
+            summary,
+        ),
+        (
+            {
+                "label": "label",
+                "stage": "stage",
+                "model_setup_estimate_s": "model setup estimate (s)",
+                "weights_load_s": "weights load (s)",
+                "compile_s": "compile (s)",
+                "graph_capture_s": "graph capture (s)",
+                "engine_init_s": "profile/KV/warmup reported (s)",
+            },
+            stages,
+        ),
+        (
+            {
+                "label": "label",
+                "stage": "stage",
+                "spawn_import_config_s": "spawn/import (s)",
+                "device_dist_init_s": "device init (s)",
+                "model_and_weights_load_s": "model+weights load (s)",
+                "profile_kv_warmup_s": "profile/KV/warmup interval (s)",
+                "total_s": "total (s)",
+            },
+            phases,
+        ),
+    ]
+    return "\n\n".join(warnings + [_table(columns, rows) for columns, rows in tables])
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("logs", nargs="+")
+    ap.add_argument("--markdown", action="store_true")
+    ap.add_argument("--json-out", default=None)
+    ap.add_argument("--require-complete", action="store_true", help="fail if any log has no BENCH_JSON")
+    args = ap.parse_args()
+    recs = [parse(p) for p in args.logs]
+    if args.require_complete and any(r["status"] != "complete" for r in recs):
+        ap.error("incomplete benchmark log: missing BENCH_JSON")
+    if args.json_out:
+        with open(args.json_out, "w") as f:
+            json.dump(recs, f, indent=2)
+    if args.markdown:
+        print(to_markdown(recs))
+    else:
+        json.dump(recs, sys.stdout, indent=2)
+        print()
+
+
+if __name__ == "__main__":
+    main()

benchmarks/mammoth_moda2/raw_load_bench.py

@@ -0,0 +1,127 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
+"""Raw safetensors -> GPU micro-benchmark, without vLLM's loader.
+
+Compares CPU and GPU dtype conversion after materializing each mmap-backed tensor.
+Materialization includes page faults, CPU allocation and copying; it is not a disk
+bandwidth measurement. CUDA setup is excluded; transfer includes GPU allocation.
+
+  --mode cpu_cast         materialize -> cast to bf16 on the CPU -> copy to GPU
+  --mode gpu_cast         materialize -> copy source dtype to GPU -> cast on the GPU
+
+  python benchmarks/mammoth_moda2/raw_load_bench.py --model /path/MammothModa2-Preview --shards 6,7,8
+  OMP_NUM_THREADS=4 python benchmarks/mammoth_moda2/raw_load_bench.py --model /path --shards 6,7,8 --mode cpu_cast
+"""
+
+import argparse
+import json
+import os
+import re
+import time
+
+import torch
+from safetensors import safe_open
+
+SHARD_NO = re.compile(r"-(\d+)-of-\d+\.safetensors$")
+
+
+def shard_files(model: str, shards: str) -> list[str]:
+    """Resolve ``--shards`` to files, refusing empty or partial matches so a typo cannot pass as a fast load."""
+    files = sorted(f for f in os.listdir(model) if f.endswith(".safetensors"))
+    if not files:
+        raise SystemExit(f"no .safetensors files under {model}")
+    if shards != "all":
+        want = {int(x) for x in shards.split(",")}
+        found = {int(m.group(1)): f for f in files if (m := SHARD_NO.search(f))}
+        missing = sorted(want - found.keys())
+        if missing:
+            raise SystemExit(f"shard(s) {missing} not found under {model}; available: {sorted(found)}")
+        files = [found[n] for n in sorted(want)]
+    return [os.path.join(model, f) for f in files]
+
+
+def sync() -> None:
+    if torch.cuda.is_available():
+        torch.accelerator.synchronize()
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("--model", required=True)
+    ap.add_argument("--shards", default="all", help="comma list of 1-based shard numbers, or 'all'")
+    ap.add_argument("--mode", choices=("cpu_cast", "gpu_cast"), default="cpu_cast")
+    ap.add_argument("--label", default="raw")
+    ap.add_argument("--no-cuda", action="store_true", help="read + CPU cast only (mode must be cpu_cast)")
+    a = ap.parse_args()
+    if a.no_cuda and a.mode != "cpu_cast":
+        ap.error("--no-cuda only makes sense with --mode cpu_cast")
+
+    paths = shard_files(a.model, a.shards)
+    total_bytes = sum(os.path.getsize(p) for p in paths)
+
+    if not a.no_cuda:
+        # Initialize the context and exercise both operations outside the timer.
+        warmup = torch.zeros(1).to("cuda")
+        warmup = warmup.to(torch.bfloat16)
+        sync()
+        del warmup
+
+    t0 = time.perf_counter()
+    n_tensors, cpu_bytes = 0, 0
+    dtypes: dict[str, int] = {}
+    t_materialize = t_cast = t_h2d = 0.0
+    for p in paths:
+        with safe_open(p, framework="pt", device="cpu") as f:
+            for k in f.keys():
+                r0 = time.perf_counter()
+                t = f.get_tensor(k).clone()  # touch every page and own the CPU storage
+                t_materialize += time.perf_counter() - r0
+                dtypes[str(t.dtype)] = dtypes.get(str(t.dtype), 0) + 1
+                cpu_bytes += t.numel() * t.element_size()
+                needs_cast = t.is_floating_point() and t.dtype != torch.bfloat16
+                if a.mode == "cpu_cast":
+                    c0 = time.perf_counter()
+                    if needs_cast:
+                        t = t.to(torch.bfloat16)
+                    t_cast += time.perf_counter() - c0
+                    if not a.no_cuda:
+                        h0 = time.perf_counter()
+                        g = t.to("cuda")
+                        sync()
+                        t_h2d += time.perf_counter() - h0
+                        del g
+                else:
+                    h0 = time.perf_counter()
+                    g = t.to("cuda")
+                    sync()
+                    t_h2d += time.perf_counter() - h0
+                    c0 = time.perf_counter()
+                    if needs_cast:
+                        g = g.to(torch.bfloat16)
+                    sync()
+                    t_cast += time.perf_counter() - c0
+                    del g
+                n_tensors += 1
+                del t
+    total = time.perf_counter() - t0
+    res = {
+        "label": a.label,
+        "mode": a.mode,
+        "files": [os.path.basename(p) for p in paths],
+        "file_bytes_gib": round(total_bytes / 2**30, 2),
+        "tensor_bytes_gib": round(cpu_bytes / 2**30, 2),
+        "n_tensors": n_tensors,
+        "dtypes": dtypes,
+        "torch_threads": torch.get_num_threads(),
+        "cpu_affinity": len(os.sched_getaffinity(0)) if hasattr(os, "sched_getaffinity") else None,
+        "materialize_s": round(t_materialize, 3),
+        "cast_s": round(t_cast, 3),
+        "h2d_s": round(t_h2d, 3),
+        "total_s": round(total, 3),
+        "cuda_setup_included": False,
+    }
+    print("RAWLOAD_JSON " + json.dumps(res))
+
+
+if __name__ == "__main__":
+    main()

benchmarks/mammoth_moda2/reference_results.json

@@ -0,0 +1,39 @@
+{
+  "date": "2026-09-10",
+  "runtime_commit": "34aa1d2afa59547945fc59f88ab6f4f25f52a777",
+  "environment": {"torch": "2.13.0+cu129", "cuda": "12.9", "python": "3.12.14", "vllm": "0.28.0", "vllm_omni": "0.28.1.dev81+g34aa1d2af.d20260908", "gpu": "NVIDIA A800-SXM4-80GB, 580.126.09, 81920 MiB"},
+  "method": "One excluded warm-up/canary; warm page cache; three rounds of the 2x2 eager configurations with rotated order; fresh process for every run; memory polling disabled.",
+  "request": {"height": 512, "width": 512, "seed": 42, "steps": 8, "guidance": 4.0, "prompt": "A stylish woman riding a motorcycle in NYC, movie poster style"},
+  "gpu_memory_utilization": [0.5, 0.3],
+  "cpu_quota": 16,
+  "startup_columns": ["configuration", "round", "engine_init_s", "imports_s", "first_request_s", "time_to_first_image_s", "image_md5"],
+  "startup_runs": [
+    ["default-serial", 1, 60.022, 10.158, 20.827, 91.089, "60c1fe4d464fba8c6ffd1b752fbbd40c"],
+    ["default-serial", 2, 60.015, 10.082, 20.766, 90.909, "60c1fe4d464fba8c6ffd1b752fbbd40c"],
+    ["default-serial", 3, 60.34, 9.911, 21.089, 91.381, "60c1fe4d464fba8c6ffd1b752fbbd40c"],
+    ["omp-serial", 1, 55.45, 9.965, 20.711, 86.169, "60c1fe4d464fba8c6ffd1b752fbbd40c"],
+    ["omp-serial", 2, 57.01, 9.667, 20.697, 87.418, "60c1fe4d464fba8c6ffd1b752fbbd40c"],
+    ["omp-serial", 3, 57.044, 9.707, 20.415, 87.211, "60c1fe4d464fba8c6ffd1b752fbbd40c"],
+    ["default-parallel", 1, 40.258, 10.112, 20.392, 70.829, "60c1fe4d464fba8c6ffd1b752fbbd40c"],
+    ["default-parallel", 2, 42.177, 10.135, 20.692, 73.093, "60c1fe4d464fba8c6ffd1b752fbbd40c"],
+    ["default-parallel", 3, 40.848, 9.997, 20.995, 71.887, "60c1fe4d464fba8c6ffd1b752fbbd40c"],
+    ["omp-parallel", 1, 36.88, 9.666, 20.319, 66.932, "60c1fe4d464fba8c6ffd1b752fbbd40c"],
+    ["omp-parallel", 2, 36.534, 9.6, 20.314, 66.517, "60c1fe4d464fba8c6ffd1b752fbbd40c"],
+    ["omp-parallel", 3, 36.689, 9.582, 20.467, 66.802, "60c1fe4d464fba8c6ffd1b752fbbd40c"]
+  ],
+  "raw_loading_columns": ["configuration", "round", "torch_threads", "n_tensors", "materialize_s", "cast_s", "h2d_s", "total_s"],
+  "raw_loading_runs": [
+    ["cpu64", 1, 64, 620, 3.174, 2.309, 1.217, 7.796],
+    ["cpu64", 2, 64, 620, 2.957, 2.443, 1.433, 7.846],
+    ["cpu64", 3, 64, 620, 2.916, 2.613, 1.067, 7.942],
+    ["cpu16", 1, 16, 620, 0.857, 1.137, 0.713, 3.643],
+    ["cpu16", 2, 16, 620, 0.871, 1.174, 0.746, 3.608],
+    ["cpu16", 3, 16, 620, 0.814, 1.142, 0.724, 3.487],
+    ["gpu16", 1, 16, 620, 0.938, 0.028, 1.41, 3.74],
+    ["gpu16", 2, 16, 620, 0.926, 0.029, 1.467, 3.83],
+    ["gpu16", 3, 16, 620, 0.87, 0.029, 1.395, 3.661]
+  ],
+  "raw_loading_workload": {"shards": [6, 7, 8], "source_dtype": "float32", "tensor_gib": 13.3, "warmup": "CUDA context, copy, cast and synchronize before timer", "order_per_round": ["cpu64", "cpu16", "gpu16"]},
+  "executed_script_sha256": {"parse_startup_log.py": "aa260de2cfcaec2fc4a678b245bb7c16fa464543aaec72b947fd4f769d547011", "raw_load_bench.py": "1f70f6b3c205402b2c2c7c52b224e1be3c04a12d12d11044a86f1b9b6c7a1d0f", "bench_startup.py": "04b1e8b9398b4d9885fbb6fe341e8a9ee98792d0be200c15c57e1fc5183c6ff0", "__init__.py": "b1d617f45f34dbe6f5f58d792d90c84af8d8d0a7e7def77c48170ca93495ca0d"},
+  "notes": ["Standard deviations use ddof=1.", "Only first-request ONGs are compared.", "Raw materialization includes CPU allocation/copy; H2D includes GPU allocation; not engine-loader timings."]
+}

benchmarks/tts/bench_tts.py

@@ -1,4 +1,7 @@
 #!/usr/bin/env python3
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
+
 """Universal TTS benchmark CLI for vllm-omni.
 
 Runs ``vllm bench serve --omni`` with model-aware defaults loaded from
@@ -87,6 +90,7 @@ def build_bench_args(
     served_model_name: str | None = None,
     num_warmups: int = 2,
     request_seed: int | None = None,
+    duration_seconds: float | None = None,
 ) -> list[str]:
     """Build the ``vllm bench serve --omni`` command for one (task, concurrency) run."""
     dataset_name = _TASK_TO_DATASET[task]
@@ -95,7 +99,10 @@ def build_bench_args(
     task_extra_body: dict[str, Any] = dict((model_cfg.get("task_extra_body") or {}).get(task) or {})
     if request_seed is not None:
         task_extra_body["seed"] = request_seed
-
+    if duration_seconds is not None:
+        if not math.isfinite(duration_seconds) or duration_seconds <= 0:
+            raise ValueError("duration_seconds must be finite and positive")
+        task_extra_body["duration_seconds"] = duration_seconds
     # Resolve dataset path
     if dataset_path:
         resolved_dataset_path = dataset_path
@@ -194,7 +201,7 @@ def print_summary_table(results: list[dict[str, Any]]) -> None:
     if not results:
         return
     header = (
-        f"{'Task':<16} {'Concurrency':>11} {'RTF mean':>10} "
+        f"{'Task':<16} {'Task Type':<12} {'Concurrency':>11} {'RTF mean':>10} "
         f"{'TTFP (ms)':>10} {'Throughput':>12} {'WER':>7} {'SIM':>7} {'UTMOS':>7}"
     )
     print(f"\n{'=' * len(header)}")
@@ -204,6 +211,7 @@ def print_summary_table(results: list[dict[str, Any]]) -> None:
     print("-" * len(header))
     for r in results:
         task = r.get("_task", "?")
+        task_type = r.get("_task_type", "-")
         conc = r.get("_concurrency", "?")
         rtf = r.get("mean_audio_rtf", float("nan"))
         ttfp = r.get("mean_audio_ttfp_ms", float("nan"))
@@ -216,7 +224,7 @@ def print_summary_table(results: list[dict[str, Any]]) -> None:
             return f"{v:.{digits}f}" if not math.isnan(v) else "  n/a"
 
         print(
-            f"{task:<16} {str(conc):>11} {fmt(rtf):>10} {fmt(ttfp, 0):>10} "
+            f"{task:<16} {task_type:<12} {str(conc):>11} {fmt(rtf):>10} {fmt(ttfp, 0):>10} "
             f"{fmt(throughput):>12} {fmt(wer):>7} {fmt(sim):>7} {fmt(utmos):>7}"
         )
     print("=" * len(header))
@@ -249,6 +257,12 @@ def main() -> None:
         help="Number of prompts per run. If one value, applied to all concurrency levels.",
     )
     parser.add_argument("--num-warmups", type=int, default=2, help="Warmup requests before each measured run")
+    parser.add_argument(
+        "--duration-seconds",
+        type=float,
+        default=None,
+        help="Target audio duration per request (AuK defaults to 5 seconds). Independent of --output-len.",
+    )
     parser.add_argument(
         "--request-seed",
         type=int,
@@ -277,6 +291,8 @@ def main() -> None:
 
     if args.num_warmups < 0:
         parser.error("--num-warmups cannot be negative")
+    if args.duration_seconds is not None and (not math.isfinite(args.duration_seconds) or args.duration_seconds <= 0):
+        parser.error("--duration-seconds must be finite and positive")
 
     model_configs = load_model_configs(Path(args.model_configs))
     if args.model not in model_configs:
@@ -333,11 +349,16 @@ def main() -> None:
                 served_model_name=args.served_model_name,
                 num_warmups=args.num_warmups,
                 request_seed=args.request_seed,
+                duration_seconds=args.duration_seconds,
             )
             result = run_one_benchmark(cmd)
             if result is not None:
                 result["_task"] = task
                 result["_concurrency"] = concurrency
+                task_body = model_cfg.get("task_extra_body", {}).get(task, {})
+                task_type = task_body.get("task_type")
+                if task_type is not None:
+                    result["_task_type"] = task_type
                 all_results.append(result)
                 # Persist the metadata so plot_results.py can pick it up.
                 if args.output_dir and result_filename:

benchmarks/tts/evidence/breeze_tts_2_20260918.json

@@ -0,0 +1,757 @@
+{
+  "measurement_scope": "Historical measurements for PR7752, not a main-branch comparison or a measurement of subsequent review fixes.",
+  "environment": {
+    "gpu": "NVIDIA H100 80 GB",
+    "gpu_count": 1,
+    "python": "3.12",
+    "pytorch": "2.13.0+cu132",
+    "vllm": "0.29.0",
+    "transformers": "5.14.1",
+    "nsight_systems": "2026.5.1"
+  },
+  "model": {
+    "id": "BreezeBlue/Breeze-TTS-2",
+    "revision": "3e28c5151381a722f1d8661b4118c298caa77aa4"
+  },
+  "workload": {
+    "texts": [
+      "Hello, this is Breeze TTS 2 running on vLLM Omni.",
+      "你好，今天天气很好，我们一起出去走走吧。"
+    ],
+    "alternating_texts": true,
+    "seed": 42,
+    "guidance_scale": 1.0,
+    "repetition_penalty": 1.1,
+    "top_p": 1.0,
+    "sampled": {
+      "temperature": 0.9,
+      "top_k": 50
+    },
+    "greedy": {
+      "temperature": 0.0,
+      "top_k": 0
+    },
+    "stream": false,
+    "response_format": "pcm",
+    "sample_rate": 24000,
+    "channels": 1,
+    "pcm_bits": 16
+  },
+  "cumulative_diagnostic": {
+    "baseline_sha": "5bda85c371e8c0c7728c2f7f5865d3ddd79b9d99",
+    "combined_sha": "37a04897cf8566c63e82eb13c27c064b9bcd4222",
+    "intermediate_variants": "Ordered patch subsets of the combined commit, not independent commits.",
+    "patch_sha256": {
+      "batched_eos": "2f107ea8e890e03cc138ef2f69e693e3474d3aaf0eb4051c00886cbaa9785b3d",
+      "greedy_graph": "4e783313260ddc2e36c8a74ba9d95dc5d78338443612ebf89854e31b8be6933b",
+      "batched_embedding": "90557e4ec73fdedcff51fd717d5e8c6477e71087fd97ac52534542234f5d1d3a"
+    },
+    "sequence_slots": 8,
+    "kv_gib": 2,
+    "codec_chunk_frames": 16,
+    "max_new_tokens": 48,
+    "temperature": 0.0,
+    "cohorts_per_variant": 1,
+    "nsight_injection_present": true,
+    "collection_enabled": false,
+    "warmup": "Two eight-request full-response warmups, then four full-response 64-request cohorts, each preceded by one concurrency-sized warmup.",
+    "variants": [
+      {
+        "variant": "baseline",
+        "makespan_s": 0.9672094583511353,
+        "requests": 8,
+        "concurrency": 8,
+        "audio_s": 30.72,
+        "failures": 0
+      },
+      {
+        "variant": "plus_batched_eos",
+        "makespan_s": 0.9280132218264043,
+        "requests": 8,
+        "concurrency": 8,
+        "audio_s": 30.72,
+        "failures": 0
+      },
+      {
+        "variant": "plus_greedy_graph",
+        "makespan_s": 0.8857334200292826,
+        "requests": 8,
+        "concurrency": 8,
+        "audio_s": 30.72,
+        "failures": 0
+      },
+      {
+        "variant": "plus_batched_embedding",
+        "makespan_s": 0.8435833756811917,
+        "requests": 8,
+        "concurrency": 8,
+        "audio_s": 30.72,
+        "failures": 0
+      }
+    ],
+    "limits": [
+      "Single cumulative diagnostic samples; no repeat spread.",
+      "Nsight injection remains when collection is disabled.",
+      "The final variant used graph tracing configuration; prior variants used node tracing configuration.",
+      "Do not interpret these timings as isolated uninstrumented per-optimization gains."
+    ]
+  },
+  "mechanical_attribution": {
+    "batched_eos": {
+      "stage0_stream_synchronization_calls": [
+        773,
+        438
+      ],
+      "device_to_host_copies": [
+        841,
+        506
+      ],
+      "capture_mode": "node"
+    },
+    "greedy_graph": {
+      "radix_sort_kernels": [
+        735,
+        0
+      ],
+      "capture_mode": "node"
+    },
+    "combined_graph_capture": {
+      "captured_wall_ms": [
+        955.4114495404065,
+        855.6996420957148
+      ],
+      "gpu_envelope_ms": [
+        926.633897,
+        826.396835
+      ],
+      "activity_upper_ms": [
+        651.130806,
+        628.864685
+      ],
+      "idle_lower_ms": [
+        275.50309100000004,
+        197.53215
+      ],
+      "stage0_explicit_kernel_launch_count": [
+        7087,
+        6198
+      ],
+      "stage0_graph_launch_count": [
+        156,
+        156
+      ]
+    },
+    "limits": [
+      "Host API durations overlap GPU activity and are not additional idle time.",
+      "Graph envelopes hide internal gaps: activity is an upper bound and idle is a lower bound.",
+      "Stage 1 has eight versus nine graph launches across the graph-mode traces.",
+      "No separate final-capacity stage breakdown was measured."
+    ]
+  },
+  "combined_uninstrumented_ab": {
+    "baseline_sha": "5bda85c371e8c0c7728c2f7f5865d3ddd79b9d99",
+    "head_sha": "37a04897cf8566c63e82eb13c27c064b9bcd4222",
+    "sequence_slots": 8,
+    "kv_gib": 2,
+    "codec_chunk_frames": 16,
+    "requests_per_group": 64,
+    "max_new_tokens": 384,
+    "baseline_rounds": 2,
+    "head_rounds": 1,
+    "profiler_configured": false,
+    "nsight_injection_present": false,
+    "warmup": "One excluded 64-request full-response wave for each temperature/concurrency family, plus an excluded 2*C-request full-response cohort before every group.",
+    "rows": [
+      {
+        "build": "plain-baseline",
+        "phase": "round-0",
+        "temperature": 0.0,
+        "concurrency": 8,
+        "requests": 64,
+        "makespan_s": 10.77213971177116,
+        "req_per_s": 5.941252314994074,
+        "aggregate_rtf": 0.030157166046391827,
+        "audio_s": 357.2,
+        "caps": 0,
+        "failures": 0,
+        "endpoint_trimmed_audio_s": 305.88,
+        "endpoint_trimmed_aggregate_rtf": 0.03521688149526337,
+        "quiet_tail_at_least_3s": 7,
+        "max_quiet_tail_s": 3.24
+      },
+      {
+        "build": "plain-baseline",
+        "phase": "round-0",
+        "temperature": 0.0,
+        "concurrency": 16,
+        "requests": 64,
+        "makespan_s": 11.076066234149039,
+        "req_per_s": 5.778224745774739,
+        "aggregate_rtf": 0.029092420240988227,
+        "audio_s": 380.72,
+        "caps": 2,
+        "failures": 0,
+        "endpoint_trimmed_audio_s": 315.48,
+        "endpoint_trimmed_aggregate_rtf": 0.035108616185333584,
+        "quiet_tail_at_least_3s": 6,
+        "max_quiet_tail_s": 26.04
+      },
+      {
+        "build": "plain-baseline",
+        "phase": "round-0",
+        "temperature": 0.9,
+        "concurrency": 8,
+        "requests": 64,
+        "makespan_s": 8.603743681218475,
+        "req_per_s": 7.438622345259851,
+        "aggregate_rtf": 0.03389435739528236,
+        "audio_s": 253.84,
+        "caps": 0,
+        "failures": 0,
+        "endpoint_trimmed_audio_s": 239.02,
+        "endpoint_trimmed_aggregate_rtf": 0.035995915325991445,
+        "quiet_tail_at_least_3s": 0,
+        "max_quiet_tail_s": 1.3
+      },
+      {
+        "build": "plain-baseline",
+        "phase": "round-0",
+        "temperature": 0.9,
+        "concurrency": 16,
+        "requests": 64,
+        "makespan_s": 8.66502623911947,
+        "req_per_s": 7.386013409984042,
+        "aggregate_rtf": 0.033143460216950235,
+        "audio_s": 261.44,
+        "caps": 0,
+        "failures": 0,
+        "endpoint_trimmed_audio_s": 242.14,
+        "endpoint_trimmed_aggregate_rtf": 0.03578519137325296,
+        "quiet_tail_at_least_3s": 0,
+        "max_quiet_tail_s": 1.3
+      },
+      {
+        "build": "plain-baseline",
+        "phase": "round-1",
+        "temperature": 0.0,
+        "concurrency": 8,
+        "requests": 64,
+        "makespan_s": 13.904579282738268,
+        "req_per_s": 4.602800178172402,
+        "aggregate_rtf": 0.0392784725501081,
+        "audio_s": 354.0,
+        "caps": 1,
+        "failures": 0,
+        "endpoint_trimmed_audio_s": 285.36,
+        "endpoint_trimmed_aggregate_rtf": 0.048726448285457906,
+        "quiet_tail_at_least_3s": 8,
+        "max_quiet_tail_s": 25.38
+      },
+      {
+        "build": "plain-baseline",
+        "phase": "round-1",
+        "temperature": 0.0,
+        "concurrency": 16,
+        "requests": 64,
+        "makespan_s": 10.060411099344492,
+        "req_per_s": 6.361569061941222,
+        "aggregate_rtf": 0.02935460754944121,
+        "audio_s": 342.72,
+        "caps": 0,
+        "failures": 0,
+        "endpoint_trimmed_audio_s": 289.04,
+        "endpoint_trimmed_aggregate_rtf": 0.03480629359031446,
+        "quiet_tail_at_least_3s": 10,
+        "max_quiet_tail_s": 3.16
+      },
+      {
+        "build": "plain-baseline",
+        "phase": "round-1",
+        "temperature": 0.9,
+        "concurrency": 8,
+        "requests": 64,
+        "makespan_s": 8.535738762002438,
+        "req_per_s": 7.4978864494894575,
+        "aggregate_rtf": 0.033363581777683074,
+        "audio_s": 255.84,
+        "caps": 0,
+        "failures": 0,
+        "endpoint_trimmed_audio_s": 237.9,
+        "endpoint_trimmed_aggregate_rtf": 0.03587952401009852,
+        "quiet_tail_at_least_3s": 0,
+        "max_quiet_tail_s": 2.12
+      },
+      {
+        "build": "plain-baseline",
+        "phase": "round-1",
+        "temperature": 0.9,
+        "concurrency": 16,
+        "requests": 64,
+        "makespan_s": 8.584469985216856,
+        "req_per_s": 7.4553234049642105,
+        "aggregate_rtf": 0.033355882752629996,
+        "audio_s": 257.36,
+        "caps": 0,
+        "failures": 0,
+        "endpoint_trimmed_audio_s": 242.57999999999998,
+        "endpoint_trimmed_aggregate_rtf": 0.035388201769382704,
+        "quiet_tail_at_least_3s": 0,
+        "max_quiet_tail_s": 1.94
+      },
+      {
+        "build": "plain-batched",
+        "phase": "round-0",
+        "temperature": 0.0,
+        "concurrency": 8,
+        "requests": 64,
+        "makespan_s": 12.983129982370883,
+        "req_per_s": 4.929473870083891,
+        "aggregate_rtf": 0.03320153943936907,
+        "audio_s": 391.04,
+        "caps": 2,
+        "failures": 0,
+        "endpoint_trimmed_audio_s": 312.72,
+        "endpoint_trimmed_aggregate_rtf": 0.041516788124747,
+        "quiet_tail_at_least_3s": 11,
+        "max_quiet_tail_s": 28.4
+      },
+      {
+        "build": "plain-batched",
+        "phase": "round-0",
+        "temperature": 0.0,
+        "concurrency": 16,
+        "requests": 64,
+        "makespan_s": 9.772029620129615,
+        "req_per_s": 6.549304749155183,
+        "aggregate_rtf": 0.027698496655696188,
+        "audio_s": 352.8,
+        "caps": 1,
+        "failures": 0,
+        "endpoint_trimmed_audio_s": 286.26,
+        "endpoint_trimmed_aggregate_rtf": 0.034136902187275955,
+        "quiet_tail_at_least_3s": 6,
+        "max_quiet_tail_s": 26.04
+      },
+      {
+        "build": "plain-batched",
+        "phase": "round-0",
+        "temperature": 0.9,
+        "concurrency": 8,
+        "requests": 64,
+        "makespan_s": 8.171920725144446,
+        "req_per_s": 7.831696140061215,
+        "aggregate_rtf": 0.03169376638669115,
+        "audio_s": 257.84,
+        "caps": 0,
+        "failures": 0,
+        "endpoint_trimmed_audio_s": 241.72,
+        "endpoint_trimmed_aggregate_rtf": 0.03380738344011437,
+        "quiet_tail_at_least_3s": 0,
+        "max_quiet_tail_s": 1.22
+      },
+      {
+        "build": "plain-batched",
+        "phase": "round-0",
+        "temperature": 0.9,
+        "concurrency": 16,
+        "requests": 64,
+        "makespan_s": 8.02551719499752,
+        "req_per_s": 7.974563937124526,
+        "aggregate_rtf": 0.030886380830501534,
+        "audio_s": 259.84000000000003,
+        "caps": 0,
+        "failures": 0,
+        "endpoint_trimmed_audio_s": 243.0,
+        "endpoint_trimmed_aggregate_rtf": 0.03302681973250008,
+        "quiet_tail_at_least_3s": 0,
+        "max_quiet_tail_s": 1.78
+      }
+    ],
+    "limits": [
+      "Ranges compare the head sample with two baseline samples; they are not confidence intervals.",
+      "Full-response greedy output lengths and frame-limit incidence vary, so no general greedy throughput gain is claimed.",
+      "No transcription was run on these historical A/B outputs."
+    ]
+  },
+  "capacity": {
+    "runtime_sha": "7bc0b3e3a2e41ef44e8efb30fd927e93bf6a48a7",
+    "main_base_sha": "6bda549ac23a667efbe44a54cf3f6d57e6e24da2",
+    "sequence_slots": [
+      8,
+      16,
+      32
+    ],
+    "kv_gib": [
+      2,
+      4,
+      8
+    ],
+    "codec_graph_buckets": [
+      [
+        1,
+        2,
+        4,
+        8
+      ],
+      [
+        1,
+        2,
+        4,
+        8,
+        16
+      ],
+      [
+        1,
+        2,
+        4,
+        8,
+        16,
+        32
+      ]
+    ],
+    "backbone_capture_sizes": [
+      1,
+      2,
+      4,
+      8,
+      32,
+      64,
+      128,
+      256
+    ],
+    "codec_chunk_frames": 16,
+    "requests_per_group": 128,
+    "cohorts_per_point": 1,
+    "excluded_warmups_per_configuration": {
+      "cohorts": 4,
+      "requests_per_cohort": 64,
+      "covers": "Both client concurrency levels and both sampling settings"
+    },
+    "profiler_configured": false,
+    "nsight_injection_present": false,
+    "rows": [
+      {
+        "slots": 8,
+        "concurrency": 8,
+        "observed_peak_client_requests": 8,
+        "temperature": 0.0,
+        "max_new_tokens": 384,
+        "requests": 128,
+        "failures": 0,
+        "req_per_s": 6.7812081269297915,
+        "makespan_s": 18.87569259107113,
+        "aggregate_rtf": 0.02841355459879445,
+        "audio_s": 664.32,
+        "near_cap_count": 0,
+        "exact_cap_count": 0,
+        "fixed48_verified": null
+      },
+      {
+        "slots": 8,
+        "concurrency": 16,
+        "observed_peak_client_requests": 16,
+        "temperature": 0.0,
+        "max_new_tokens": 384,
+        "requests": 128,
+        "failures": 0,
+        "req_per_s": 5.8798058352164615,
+        "makespan_s": 21.76942633604631,
+        "aggregate_rtf": 0.029975526459636356,
+        "audio_s": 726.24,
+        "near_cap_count": 2,
+        "exact_cap_count": 2,
+        "fixed48_verified": null
+      },
+      {
+        "slots": 8,
+        "concurrency": 8,
+        "observed_peak_client_requests": 8,
+        "temperature": 0.9,
+        "max_new_tokens": 384,
+        "requests": 128,
+        "failures": 0,
+        "req_per_s": 7.72991106982811,
+        "makespan_s": 16.559052082709968,
+        "aggregate_rtf": 0.0320365502296664,
+        "audio_s": 516.88,
+        "near_cap_count": 0,
+        "exact_cap_count": 0,
+        "fixed48_verified": null
+      },
+      {
+        "slots": 8,
+        "concurrency": 16,
+        "observed_peak_client_requests": 16,
+        "temperature": 0.9,
+        "max_new_tokens": 384,
+        "requests": 128,
+        "failures": 0,
+        "req_per_s": 7.945057322591292,
+        "makespan_s": 16.110645248088986,
+        "aggregate_rtf": 0.030958196095482296,
+        "audio_s": 520.4,
+        "near_cap_count": 0,
+        "exact_cap_count": 0,
+        "fixed48_verified": null
+      },
+      {
+        "slots": 8,
+        "concurrency": 8,
+        "observed_peak_client_requests": 8,
+        "temperature": 0.0,
+        "max_new_tokens": 48,
+        "requests": 128,
+        "failures": 0,
+        "req_per_s": 9.850579526557848,
+        "makespan_s": 12.994159344118088,
+        "aggregate_rtf": 0.026436684863521503,
+        "audio_s": 491.52,
+        "near_cap_count": 128,
+        "exact_cap_count": 128,
+        "fixed48_verified": true
+      },
+      {
+        "slots": 8,
+        "concurrency": 16,
+        "observed_peak_client_requests": 16,
+        "temperature": 0.0,
+        "max_new_tokens": 48,
+        "requests": 128,
+        "failures": 0,
+        "req_per_s": 10.665146583658053,
+        "makespan_s": 12.00171033712104,
+        "aggregate_rtf": 0.02441754218978076,
+        "audio_s": 491.52,
+        "near_cap_count": 128,
+        "exact_cap_count": 128,
+        "fixed48_verified": true
+      },
+      {
+        "slots": 16,
+        "concurrency": 16,
+        "observed_peak_client_requests": 16,
+        "temperature": 0.0,
+        "max_new_tokens": 384,
+        "requests": 128,
+        "failures": 0,
+        "req_per_s": 7.754874623816668,
+        "makespan_s": 16.505747186020017,
+        "aggregate_rtf": 0.023979758231665527,
+        "audio_s": 688.32,
+        "near_cap_count": 1,
+        "exact_cap_count": 1,
+        "fixed48_verified": null
+      },
+      {
+        "slots": 16,
+        "concurrency": 32,
+        "observed_peak_client_requests": 32,
+        "temperature": 0.0,
+        "max_new_tokens": 384,
+        "requests": 128,
+        "failures": 0,
+        "req_per_s": 10.104109318018903,
+        "makespan_s": 12.668113138061017,
+        "aggregate_rtf": 0.018577125085143444,
+        "audio_s": 681.92,
+        "near_cap_count": 1,
+        "exact_cap_count": 1,
+        "fixed48_verified": null
+      },
+      {
+        "slots": 16,
+        "concurrency": 16,
+        "observed_peak_client_requests": 16,
+        "temperature": 0.9,
+        "max_new_tokens": 384,
+        "requests": 128,
+        "failures": 0,
+        "req_per_s": 10.920674745703012,
+        "makespan_s": 11.72088748915121,
+        "aggregate_rtf": 0.021831484669108944,
+        "audio_s": 536.88,
+        "near_cap_count": 0,
+        "exact_cap_count": 0,
+        "fixed48_verified": null
+      },
+      {
+        "slots": 16,
+        "concurrency": 32,
+        "observed_peak_client_requests": 32,
+        "temperature": 0.9,
+        "max_new_tokens": 384,
+        "requests": 128,
+        "failures": 0,
+        "req_per_s": 11.210975235811437,
+        "makespan_s": 11.417383172083646,
+        "aggregate_rtf": 0.02120613516360261,
+        "audio_s": 538.4,
+        "near_cap_count": 0,
+        "exact_cap_count": 0,
+        "fixed48_verified": null
+      },
+      {
+        "slots": 16,
+        "concurrency": 16,
+        "observed_peak_client_requests": 16,
+        "temperature": 0.0,
+        "max_new_tokens": 48,
+        "requests": 128,
+        "failures": 0,
+        "req_per_s": 15.14487085057202,
+        "makespan_s": 8.451706274878234,
+        "aggregate_rtf": 0.017195040435543284,
+        "audio_s": 491.52,
+        "near_cap_count": 128,
+        "exact_cap_count": 128,
+        "fixed48_verified": true
+      },
+      {
+        "slots": 16,
+        "concurrency": 32,
+        "observed_peak_client_requests": 32,
+        "temperature": 0.0,
+        "max_new_tokens": 48,
+        "requests": 128,
+        "failures": 0,
+        "req_per_s": 16.563937910502617,
+        "makespan_s": 7.727630995213985,
+        "aggregate_rtf": 0.015721905507840955,
+        "audio_s": 491.52,
+        "near_cap_count": 128,
+        "exact_cap_count": 128,
+        "fixed48_verified": true
+      },
+      {
+        "slots": 32,
+        "concurrency": 32,
+        "observed_peak_client_requests": 32,
+        "temperature": 0.0,
+        "max_new_tokens": 384,
+        "requests": 128,
+        "failures": 0,
+        "req_per_s": 15.246181432777554,
+        "makespan_s": 8.395544849336147,
+        "aggregate_rtf": 0.012390119317202106,
+        "audio_s": 677.6,
+        "near_cap_count": 0,
+        "exact_cap_count": 0,
+        "fixed48_verified": null
+      },
+      {
+        "slots": 32,
+        "concurrency": 64,
+        "observed_peak_client_requests": 64,
+        "temperature": 0.0,
+        "max_new_tokens": 384,
+        "requests": 128,
+        "failures": 0,
+        "req_per_s": 11.166074835908939,
+        "makespan_s": 11.463294119108468,
+        "aggregate_rtf": 0.015826284127331107,
+        "audio_s": 724.32,
+        "near_cap_count": 1,
+        "exact_cap_count": 1,
+        "fixed48_verified": null
+      },
+      {
+        "slots": 32,
+        "concurrency": 32,
+        "observed_peak_client_requests": 32,
+        "temperature": 0.9,
+        "max_new_tokens": 384,
+        "requests": 128,
+        "failures": 0,
+        "req_per_s": 17.17219888865503,
+        "makespan_s": 7.453908543102443,
+        "aggregate_rtf": 0.014336645143680651,
+        "audio_s": 519.92,
+        "near_cap_count": 0,
+        "exact_cap_count": 0,
+        "fixed48_verified": null
+      },
+      {
+        "slots": 32,
+        "concurrency": 64,
+        "observed_peak_client_requests": 64,
+        "temperature": 0.9,
+        "max_new_tokens": 384,
+        "requests": 128,
+        "failures": 0,
+        "req_per_s": 17.514343665613783,
+        "makespan_s": 7.3082955572754145,
+        "aggregate_rtf": 0.013677750331777613,
+        "audio_s": 534.32,
+        "near_cap_count": 0,
+        "exact_cap_count": 0,
+        "fixed48_verified": null
+      },
+      {
+        "slots": 32,
+        "concurrency": 32,
+        "observed_peak_client_requests": 32,
+        "temperature": 0.0,
+        "max_new_tokens": 48,
+        "requests": 128,
+        "failures": 0,
+        "req_per_s": 25.82908453652114,
+        "makespan_s": 4.955653763841838,
+        "aggregate_rtf": 0.010082303393232905,
+        "audio_s": 491.52,
+        "near_cap_count": 128,
+        "exact_cap_count": 128,
+        "fixed48_verified": true
+      },
+      {
+        "slots": 32,
+        "concurrency": 64,
+        "observed_peak_client_requests": 64,
+        "temperature": 0.0,
+        "max_new_tokens": 48,
+        "requests": 128,
+        "failures": 0,
+        "req_per_s": 28.81314277607874,
+        "makespan_s": 4.442417163401842,
+        "aggregate_rtf": 0.00903812085653044,
+        "audio_s": 491.52,
+        "near_cap_count": 128,
+        "exact_cap_count": 128,
+        "fixed48_verified": true
+      }
+    ],
+    "limits": [
+      "This scales slots, KV reservation and codec graph buckets together.",
+      "One cohort per point; no repeated capacity samples or confidence intervals.",
+      "The 48-frame cohorts deliberately truncate speech and are excluded from full-utterance quality claims.",
+      "Peak device memory was not measured.",
+      "No capacity claim extends to cloning, CFG guidance above one, long prompts or other GPUs."
+    ]
+  },
+  "quality": {
+    "full_response_pcm_checked": 1536,
+    "sampled_full_responses": 768,
+    "sampled_frame_limit_hits": 0,
+    "sampled_quiet_tails_at_least_three_seconds": 0,
+    "greedy_frame_limit_hits": 5,
+    "asr_selected_unique_clips": 47,
+    "asr_selected_sampled_clips": 24,
+    "sampled_asr": "All selected sampled transcripts matched after accounting for TTS 2 versus TTS2 spacing.",
+    "endpoint_method": "Trim leading/trailing quiet using 20 ms RMS windows at or below -40 dBFS; keep internal pauses and original makespans.",
+    "limits": [
+      "ASR selects median-duration and longest English/Chinese clips per group, deduplicated by WAV hash.",
+      "Selected ASR and endpoint measurements do not establish perceptual quality parity.",
+      "Capped greedy clips have extra ASR words with unreliable tail alignment.",
+      "No listening or MOS assessment was performed."
+    ]
+  },
+  "metrics": {
+    "requests_per_second": "Completed requests divided by cohort wall time",
+    "aggregate_rtf": "Cohort wall time divided by total returned audio duration",
+    "timing_exclusions": [
+      "PCM validation",
+      "hashing",
+      "WAV file writes",
+      "post-timing audio analysis"
+    ]
+  }
+}

benchmarks/tts/model_configs.yaml

@@ -7,6 +7,33 @@
 # is tracked here.
 
 models:
+  tencent/AuK: &auk
+    supported_tasks: [default_voice, voice_clone]
+    backend: openai-audio-speech
+    endpoint: /v1/audio/speech
+    trust_remote_code: true
+    task_extra_body:
+      default_voice:
+        duration_seconds: 5.0
+        voice: default
+        task_type: CustomVoice
+      voice_clone:
+        duration_seconds: 5.0
+        voice: default
+        task_type: Base
+  tencent/AuK-Flash: *auk
+
+  BreezeBlue/Breeze-TTS-2:
+    supported_tasks: [default_voice, voice_design, voice_clone]
+    backend: openai-audio-speech
+    endpoint: /v1/audio/speech
+    task_extra_body:
+      default_voice: {}
+      voice_design:
+        task_type: VoiceDesign
+      voice_clone:
+        task_type: Base
+
   IndexTeam/IndexTTS-2.5:
     supported_tasks: [voice_clone]
     backend: openai-audio-speech

docker/Dockerfile.ci

@@ -1,10 +1,10 @@
-# Target vLLM version: v0.29.0 (tag @ 98dff2a81d74).
+# Target vLLM version: v0.30.0 (tag @ ced6857afa0e).
 # Use the matching release image and its coherent CUDA/PyTorch/FlashInfer
 # runtime. Do not reinstall a commit wheel or repair dependencies on top of
 # an older base image at a release boundary. For a future unreleased target,
 # restore an explicit wheel pin and validate its complete runtime stack.
 ARG VLLM_BASE_IMAGE=vllm/vllm-openai
-ARG VLLM_BASE_TAG=v0.29.0
+ARG VLLM_BASE_TAG=v0.30.0
 
 # We use registry-based layer caching for the dependencies,
 # where the tag is determined based on the hash of the contents

docker/Dockerfile.cuda

@@ -1,4 +1,4 @@
-ARG BASE_IMAGE=vllm/vllm-openai:v0.29.0
+ARG BASE_IMAGE=vllm/vllm-openai:v0.30.0
 FROM ${BASE_IMAGE}
 
 ARG COMMON_WORKDIR=/app

docker/Dockerfile.npu

@@ -1,5 +1,5 @@
 ARG VLLM_ASCEND_IMAGE=quay.io/atlas-ci/vllm-ascend
-ARG VLLM_ASCEND_VERSION=v0.29.0
+ARG VLLM_ASCEND_VERSION=v0.30.0
 ARG VLLM_ASCEND_DEVICE_SUFFIX=
 FROM ${VLLM_ASCEND_IMAGE}:${VLLM_ASCEND_VERSION}${VLLM_ASCEND_DEVICE_SUFFIX}
 

docker/Dockerfile.npu.ci

@@ -1,5 +1,5 @@
 ARG VLLM_ASCEND_IMAGE=quay.io/atlas-ci/vllm-ascend
-ARG VLLM_ASCEND_VERSION=v0.29.0
+ARG VLLM_ASCEND_VERSION=v0.30.0
 ARG VLLM_ASCEND_DEVICE_SUFFIX=
 FROM ${VLLM_ASCEND_IMAGE}:${VLLM_ASCEND_VERSION}${VLLM_ASCEND_DEVICE_SUFFIX}
 

docker/Dockerfile.rocm

@@ -1,7 +1,7 @@
 # Keep the ROCm runtime aligned with the vLLM release targeted by
 # docker/Dockerfile.ci. A stale base image can build successfully while failing
 # at runtime when vLLM-Omni imports APIs added by the newer vLLM release.
-ARG BASE_IMAGE=vllm/vllm-openai-rocm:v0.29.0
+ARG BASE_IMAGE=vllm/vllm-openai-rocm:v0.30.0
 
 # Keep the dependency-heavy portion independent from the source tree. The
 # registry cache is keyed on these inputs, while the final source copy can be
@@ -16,7 +16,7 @@ FROM ${BASE_IMAGE} AS deps
 # intentional custom source pin must update that regression in the same change,
 # and should return to the stable release as soon as one is available.
 ARG USE_NIGHTLY_BUILD=0
-ARG VLLM_VERSION_OR_COMMIT_HASH=v0.29.0
+ARG VLLM_VERSION_OR_COMMIT_HASH=v0.30.0
 ARG ARG_PYTORCH_ROCM_ARCH
 ENV PYTORCH_ROCM_ARCH=${ARG_PYTORCH_ROCM_ARCH:-${PYTORCH_ROCM_ARCH}}
 

docker/Dockerfile.xpu

@@ -1,4 +1,4 @@
-# Target vLLM version: v0.29.0. Keep VLLM_VERSION aligned with the target vLLM
+# Target vLLM version: v0.30.0. Keep VLLM_VERSION aligned with the target vLLM
 # release on every rebase.
 #
 # This file layers vLLM-Omni on top of the published upstream XPU image, which
@@ -18,7 +18,7 @@
 #
 # .buildkite/intel/scripts/run-xpu-test.sh automates that pull-then-build fallback.
 ARG VLLM_BASE_IMAGE=vllm/vllm-openai-xpu
-ARG VLLM_VERSION=v0.29.0
+ARG VLLM_VERSION=v0.30.0
 ARG VLLM_BASE=${VLLM_BASE_IMAGE}:${VLLM_VERSION}
 
 FROM ${VLLM_BASE} AS vllm-omni

docs/.nav.yml

@@ -15,7 +15,7 @@ nav:
         - Image Edit: serving/image_edit_api.md
         - Videos: serving/videos_api.md
       - Streaming and Realtime:
-        - Realtime Audio: serving/realtime_api.md
+        - Full Duplex OpenAI: serving/realtime_openai_duplex_api.md
         - Full Duplex: serving/full_duplex_api.md
         - Realtime Duplex Reference: serving/realtime_duplex_api.md
         - Streaming Video Input: serving/video_stream_api.md
@@ -47,6 +47,7 @@ nav:
     - configuration/*
   - Models:
     - models/supported_models.md
+    - SeedVR2: models/seedvr2.md
   - Features:
     - Overview: features/README.md
     - Runtime and Stage Execution:
@@ -80,6 +81,8 @@ nav:
       - Cache Acceleration:
         - TeaCache: user_guide/diffusion/cache_acceleration/teacache.md
         - Cache-DiT: user_guide/diffusion/cache_acceleration/cache_dit.md
+      - KV Cache:
+        - Scheduler-Managed Paged KV Cache: user_guide/diffusion/paged_kv_cache.md
       - Parallelism:
         - Overview: user_guide/diffusion/parallelism/overview.md
         - CFG Parallel: user_guide/diffusion/parallelism/cfg_parallel.md
@@ -96,6 +99,7 @@ nav:
         - Hugging Face Hub: user_guide/diffusion/attention_backends/huggingface_hub.md
         - RainFusion: user_guide/diffusion/attention_backends/rainfusion.md
       - Regional Compilation: user_guide/diffusion/regional_compilation.md
+      - Wan VAE Decoder Fast Path: user_guide/diffusion/vae_fast_path.md
       - Frame Interpolation: user_guide/diffusion/frame_interpolation.md
       - Startup and Loading: user_guide/diffusion/startup_and_loading.md
       - LoRA: user_guide/diffusion/lora.md
@@ -148,11 +152,13 @@ nav:
           - Hybrid Sharded Data Parallel (HSDP): design/feature/hsdp.md
           - Pipeline Parallel: design/feature/pipeline_parallel.md
           - Sequence Parallel: design/feature/sequence_parallel.md
+          - Window Sequence Parallel: design/feature/window_sequence_parallel.md
           - Tensor Parallel: design/feature/tensor_parallel.md
           - VAE Patch Parallelism: design/feature/vae_parallel.md
         - Attention Optimization:
           - Backend Selection: design/feature/attention_backend_selection.md
           - Skip-Softmax: design/feature/skip_softmax.md
+          - Scheduler-Managed Paged KV Cache: design/feature/diffusion_paged_kv_cache.md
         - CPU Offloading:
           - Overview: design/feature/offloader/README.md
           - Model-Level: design/feature/offloader/module_offload.md

docs/README.md

@@ -56,9 +56,9 @@ vLLM-Omni is flexible and easy to use with:
 vLLM-Omni seamlessly supports most popular open-source models on HuggingFace, including:
 
 - Omni-modality models (e.g. Qwen3-Omni, MiniCPM-o 4.5, Cosmos3, HunyuanImage, BAGEL)
-- TTS models (e.g. Qwen3-TTS, IndexTTS 2.5, CosyVoice3)
-- Diffusion models — image, video, and audio generation (e.g. MiniMax H3, LTX-2.5, SANA-Video, Wan2.2)
-- Robot-policy and action models (e.g. π0, GR00T-N1.7, DreamZero-DROID, InternVLA-A1)
+- TTS models (e.g. Qwen3-TTS, Tencent AuK, Breeze-TTS-2, CosyVoice3)
+- Diffusion models — image, video, and audio generation (e.g. MiniMax H3, LingBot World, MAGI-2, LTX-2.5, Wan2.2)
+- Robot-policy and action models (e.g. π0.5, GR00T-N1.7, DreamZero-DROID, InternVLA-A1)
 
 For more information, checkout the following:
 

docs/cli/bench/serve.md

@@ -302,6 +302,21 @@ vllm bench serve --omni \
 selects all and oversize values use all available cases. Reference audio is required, and OmniInteract uses the
 `/v1/realtime` endpoint.
 
+To replay an existing sample set (for example `sampled_cases.jsonl` from a prior run), pass
+`--omniinteract-video-list` instead of discovering cases. List order is preserved; `--num-prompts` takes the prefix of
+the list (`0` or a value larger than the list length runs every row). Do not combine video-list mode with `--seed`,
+`--dataset-path`, `--omniinteract-scenario-tags`, or `--omniinteract-scenario-focus`.
+
+Use `--omniinteract-scenario-tags` to steer small runs toward paper-table scenarios (`realtime`, `proactive`,
+`nested`, `interrupted`, `1qna`). By default the sampler covers each requested tag with at least one matching case
+(when available under `--omniinteract-subsets`), then fills the remaining `--num-prompts` budget from other cases.
+Add `--omniinteract-scenario-focus` to run only cases that match those tags. Example smoke coverage:
+
+```bash
+  --num-prompts 5 \
+  --omniinteract-scenario-tags realtime proactive nested interrupted 1qna
+```
+
 Audio is replayed as 16 kHz PCM16 in 200 ms chunks and video at 1 FPS with real-time pacing. All selected media is decoded
 before timing and remains in client memory for the run, so `--max-concurrency` does not limit media preparation memory; use
 explicit `--num-prompts 0` only when the client has enough RAM for the full dataset. Media commands are bounded by
@@ -309,7 +324,9 @@ explicit `--num-prompts 0` only when the client has enough RAM for the full data
 options apply. Use `--omniinteract-require-response` only for functional E2E cases; LISTEN is a valid benchmark result.
 
 Each completed case writes `output.wav`, `wav_transcript.json`, `events.json`, `result.json`, and a final `.done` marker under
-`--omniinteract-output-dir`. The root also contains `batch_summary.json` and `official_eval_manifest.jsonl`; failed cases write
+`--omniinteract-output-dir`. The root also contains `batch_summary.json`, `official_eval_manifest.jsonl` (eligible cases only),
+and `sampled_cases.jsonl` (every sampled case as absolute `video_path` / official `output_name` / `subset` rows for Lucky-Lance
+OmniInteract `batch_inference_minicpmo.py --video_list`); failed cases write
 `.failed.json`. Runs sharing one output root are serialized. Completion validates transport, response lifecycle, and artifacts,
 not answer accuracy. Transcript timestamps are serialized playback-queue times. Playback ACKs report cumulative progress
 incrementally along that serialized clock, like a live listener; the first ack for a response goes out as soon as its audio
@@ -318,6 +335,48 @@ arrives, checkpointing the response's history position so a later committed user
 outputs are ineligible and omitted from the official manifest; `audio_clipped_bytes` records output beyond the rounded video
 horizon.
 
+Accuracy evaluation is opt-in and requires an already-running text judge with an OpenAI-compatible Chat Completions API. The
+benchmark does not launch or stop the judge server. Add the following options to the command above:
+
+```bash
+vllm serve Qwen/Qwen2.5-7B-Instruct \
+  --served-model-name Qwen2.5-7B-Instruct \
+  --port 8001
+```
+
+Then add to the benchmark command:
+
+```bash
+  --omniinteract-evaluate \
+  --omniinteract-judge-base-url http://127.0.0.1:8001 \
+  --omniinteract-judge-model Qwen2.5-7B-Instruct
+```
+
+`--omniinteract-judge-model` must match that judge process's `--served-model-name` (or its `--model`
+string if `--served-model-name` is omitted). It is not a checkpoint path to load, and it is not the
+Omni DUT `--served-model-name`. Point `--omniinteract-judge-base-url` at the judge, not at the Omni
+realtime endpoint.
+
+The early / core / interrupted-partial judge prompts follow the official English
+templates in [Lucky-Lance/OmniInteract](https://github.com/Lucky-Lance/OmniInteract)
+`eval/evaluation/llm_judge.py` (commit `de304cef35fd9a50a5caadb5090c34cfbf0dd868`).
+They correspond to the OmniInteract paper appendix
+([arXiv:2605.26485](https://arxiv.org/abs/2605.26485)) Listing A.1 (early-stage),
+Listing A.2 (interrupted partial quality), and Listing A.3 (core-stage).
+A local OpenAI-compatible judge is not the paper's GPT-4o judge, so reported
+IA-QTF1 numbers are protocol-compatible rather than official paper-table scores.
+
+After artifact publication, the evaluator builds `[start, t_a, end)` slots, judges early and core response text, writes
+per-case details plus `evaluation/unified_eval_summary.json`, and prints IA-QTF1 columns
+(1Q1A realtime / proactive / nested / Global, 1QnA, All Global), interruption diagnostics (NOR / PAQ / CSM), and nested
+metrics (NCCS, inner / outer IA-QTF1, missed outer). Slices and sections with no slots in the evaluated sample are
+omitted from the terminal report. The header lists how many evaluated cases carry each scenario tag
+(`realtime`, `proactive`, `nested`, `interrupted`, `1qna`). Realtime and proactive
+exclude nested inner/outer slots; 1Q1A Global recomputes F1 from those three TP/FP/FN aggregates; All Global includes every
+scored slot plus unmatched-chunk false positives. Plain `wav_transcript.json` timestamps provide chunk-level approximate
+timing. If transcript chunks include `aligned_words`, the evaluator splits boundary-crossing chunks and derives trigger
+timing from word alignment.
+
 Per-response TTFT and TTFP start when the server begins executing the native model-turn request that owns the response. RTF
 continues to use client receipt of `response.created` through the last audio packet, divided by emitted audio duration. Global
 TTFT, TTFP, and RTF cover the complete input-stream window. TPOT/ITL use engine stage-0 timing; ITL is emitted only when every
@@ -325,9 +384,12 @@ token interval is present within a continuous generation segment. Model-unit pac
 excluded from TPOT/ITL. Raw request metrics retain `response_created_to_first_text_ms` and
 `response_created_to_first_audio_ms` as client-envelope diagnostics.
 
-The checked-in local performance configuration measures four deterministic cases from each OmniInteract subset (12 videos
-total), with no benchmark warmups and a maximum concurrency of two. Each subset also sends one readiness request before its
-measured cases. Run it from the repository root with:
+The checked-in local performance configuration measures four deterministic cases from `1q1a`, `1q1a_math`, and
+`1qna` (12 videos total), with no benchmark warmups and a maximum concurrency of two. Each subset also sends one readiness request before its
+measured cases. The runner starts MiniCPM-o 4.5 on the first visible GPU and a text judge
+(`Qwen/Qwen2.5-7B-Instruct`) on the second (`CUDA_VISIBLE_DEVICES=1` relative to the process). After artifacts land it
+scores All Global IA-QTF1 through that local judge. This is protocol-compatible with the paper metric, not a GPT-4o
+paper-table score. Run it from the repository root with two visible GPUs:
 
 ```bash
 export HF_HOME=/path/to/persistent/huggingface-cache
@@ -338,12 +400,14 @@ bash tools/nightly/run_nightly_jobs.sh \
   --label-substr minicpmo_4_5_omniinteract
 ```
 
-The first run downloads the pinned OmniInteract archive into `HF_HOME`; later runs reuse that cache.
+The first run downloads the pinned OmniInteract archive and the judge weights into `HF_HOME`; later runs reuse that cache.
 
 It requires every case to commit its input, complete any emitted response lifecycles, and publish the expected WAV,
 transcript, event, and result artifacts without errors. A valid LISTEN-only case may have no response audio or transcript
 chunks. Official-manifest eligibility is reported separately because clipped or cancelled output is a benchmark-quality
-signal, not a transport failure. This local performance test does not score answer accuracy.
+signal, not a transport failure. Accuracy must finish with `status=ok` on every subset. After all three subsets finish,
+All Global IA-QTF1 is recomputed from pooled `Global_TP` / `Global_FP` / `Global_FN` and must be at or above
+`omniinteract_aggregate_min_ia_qtf1` (checked in as `0.2`).
 
 ### Video-MME Benchmark
 

docs/configuration/README.md

@@ -2,7 +2,7 @@
 
 This section lists the most common options for running vLLM-Omni.
 
-For options within a vLLM Engine, please refer to the [vLLM 0.29 configuration guide](https://docs.vllm.ai/en/v0.29.0/configuration/index.html).
+For options within a vLLM Engine, please refer to the [vLLM 0.30 configuration guide](https://docs.vllm.ai/en/v0.30.0/configuration/index.html).
 
 Each model defines fixed topology in a registered `PipelineConfig` and runtime
 overrides in a deploy YAML.

docs/configuration/environment_variables.md

@@ -64,6 +64,7 @@ The lifecycle labels used below are:
 | Name | Type and default | Applies to and read time | Precedence and invalid values | Lifecycle |
 | --- | --- | --- | --- | --- |
 | `DIFFUSION_ATTENTION_BACKEND` | Backend name or `auto`; default `auto` (platform selection) | Diffusion stages; read when `OmniDiffusionConfig` is constructed | `diffusion_attention_config.default` wins. An unknown backend fails during backend resolution. | Stable fallback |
+| `DIFFUSION_ATTENTION_QUANT` | `<dtype_qk>:<dtype_vo>[:<q_block_size>:<k_block_size>]`; default unset | Diffusion stages; read when `OmniDiffusionConfig` is constructed | Used only when `DIFFUSION_ATTENTION_BACKEND` sets a non-`auto` backend and no explicit attention default exists. Quantization support varies by backend. Malformed fields raise `ValueError`. | Stable fallback |
 | `DIFFUSION_CACHE_BACKEND` | `none`, `cache_dit`, `tea_cache`, `mag_cache`, `step_cache`, `stepcache`, or `step_cache_dit`; default `none` | Diffusion runner startup | Explicit `cache_backend` in config wins. Otherwise this name wins over the deprecated alias. Unsupported values raise `ValueError` during runner setup. | Stable fallback |
 | `DIFFUSION_CACHE_ADAPTER` | Same values as `DIFFUSION_CACHE_BACKEND`; default `none` | Diffusion runner startup | Used only when neither explicit `cache_backend` nor `DIFFUSION_CACHE_BACKEND` is set. Unsupported values raise `ValueError`. | Deprecated; use `DIFFUSION_CACHE_BACKEND` |
 | `OMNI_DIFFUSION_PROMPT_EMBED_CACHE` | Boolean: `1`, `true`, `yes`, `on`, `0`, `false`, `no`, or `off`; default disabled | Each diffusion runner; resolved during model setup | A recognized environment value overrides the explicit enable setting. An unrecognized value is ignored. | Experimental |
@@ -77,6 +78,13 @@ Backend names for `DIFFUSION_ATTENTION_BACKEND` are the members of
 `SAGE_ATTN`, `FLASHINFER_ATTN`, and `TRTLLM_ATTN`. Platform support still
 depends on the installed kernels and model path.
 
+For `DIFFUSION_ATTENTION_QUANT`, the dtype fields accept `float16`,
+`bfloat16`, `int8`, or `fp8_e4m3`; the selected backend may support only a
+subset. For example, `DIFFUSION_ATTENTION_BACKEND=TRTLLM_ATTN` with
+`DIFFUSION_ATTENTION_QUANT=int8:bfloat16:1:16` selects INT8 Q/K SageAttention.
+Explicit `diffusion_attention_config.default` takes precedence over both
+environment variables.
+
 ### Serving and runtime
 
 | Name | Type and default | Applies to and read time | Precedence and invalid values | Lifecycle |
@@ -130,6 +138,22 @@ settings.
 Omni-owned variable, even though vLLM-Omni supplies a persistent default when
 it is unset.
 
+### Torch compilation
+
+| Name | Type and default | Applies to and read time | Precedence and invalid values | Lifecycle |
+| --- | --- | --- | --- | --- |
+| `VLLM_OMNI_TORCH_DYNAMO_RECOMPILE_LIMIT` | Positive integer; unset preserves the current Torch setting | Process default for Torch Dynamo; read when `vllm_omni` is imported | Sets `torch._dynamo.config.recompile_limit` to the requested value. A non-positive or non-integer value raises `ValueError`. Later backend-specific overrides still take precedence. | Diagnostic |
+
+Set this variable before launching Omni, for example:
+
+```bash
+export VLLM_OMNI_TORCH_DYNAMO_RECOMPILE_LIMIT=64
+```
+
+The setting applies across model families and platforms. vLLM compilation
+contexts and backends can subsequently apply their own limits; this variable
+does not replace those backend-specific policies.
+
 ## Per-stage environment
 
 Deploy configurations can set arbitrary environment keys for one stage:
@@ -162,7 +186,7 @@ their keys only.
 ## Inherited vLLM variables
 
 vLLM-Omni also reads variables through its aligned vLLM dependency. Refer to
-the [vLLM 0.29 environment-variable reference](https://docs.vllm.ai/en/v0.29.0/configuration/env_vars.html)
+the [vLLM 0.30 environment-variable reference](https://docs.vllm.ai/en/v0.30.0/configuration/env_vars.html)
 for their definitions. This includes vLLM launch, cache, logging, plugin, ROCm,
 XPU, ModelScope, and FlashInfer workspace settings.
 
@@ -190,7 +214,7 @@ collection, examples, bug reports, or logs.
 
 ## Model-specific variables
 
-The audit found 58 variables read by a single model or pipeline family. They are
+The inventory includes 93 variables read by a single model or pipeline family. They are
 not listed as public usage options here because doing so would turn implementation
 escape hatches into an accidental compatibility contract.
 
@@ -199,11 +223,11 @@ Every audited model-specific name has a migration disposition in the
 
 | Disposition | Count | Required outcome |
 | --- | ---: | --- |
-| Promote | 35 | Move a stable setting into typed stage or model configuration. |
-| Request scope | 6 | Move request-varying behavior into a declared request-option schema. |
+| Promote | 64 | Move a stable setting into typed stage or model configuration. |
+| Request scope | 5 | Move request-varying behavior into a declared request-option schema. |
 | External | 0 | Retain only when a supported third-party library owns the contract. |
-| Internalize | 11 | Keep a debug or diagnostic switch out of public documentation and configuration. |
-| Deprecate/remove | 6 | Remove a compatibility escape hatch that has no continuing contract. |
+| Internalize | 17 | Keep a debug or diagnostic switch out of public documentation and configuration. |
+| Deprecate/remove | 7 | Remove a compatibility escape hatch that has no continuing contract. |
 
 The disposition is a migration target, not a statement that the existing
 environment switch is stable. Promote or request-scope work should land in

docs/configuration/hsdp_pre_sharded_loading.md

@@ -0,0 +1,22 @@
+# Pre-sharded HSDP loading
+
+Use `--use-hsdp --hsdp-shard-size 8 --hsdp-weight-load-strategy pre_sharded`
+to shard transformer parameters before loading checkpoint values. Set the shard
+size to the available GPU count (or configure replication explicitly).
+The default strategy, `full`, retains the existing load-then-shard behavior.
+The same `hsdp_weight_load_strategy` option is available through Python and stage
+engine configuration.
+
+The pre-sharded strategy reads rank-local slices from ordinary Hugging Face
+safetensors checkpoints into FSDP-owned storage. No converted checkpoint is
+required. Cosmos 3 and Cosmos 3 Edge transformer blocks release their initialized
+parameter storage to meta as they are constructed; nonpersistent buffers retain
+their values.
+Encoders and VAEs follow their existing loading paths.
+
+This strategy requires the default diffusion pipeline loader, complete dedicated
+transformer checkpoint sources, and supported checkpoint key mappings. Tensor
+layouts must match the runtime layout; tensor transforms are unsupported.
+Checkpoint tensor dtypes must match runtime tensor dtypes at preflight; dtype conversion is unsupported.
+Quantization and LoRA are unsupported. Every worker needs checkpoint access.
+`num_weight_load_threads` controls the safetensors reader's threads per rank.

docs/configuration/stage_configs.md

@@ -80,6 +80,19 @@ Note: for the diffusion path, an omitted `distributed_executor_backend` selects
 segments) and `mp` when `num_gpus > 1`. Set `mp` explicitly to keep a worker
 subprocess on one GPU. `ray` / `external_launcher` are not fully supported yet.
 
+### Stage-level runner selection
+
+`model_runner: v1` or `v2` at the deploy level sets the default runner.
+A `model_runner` on an individual stage overrides that default, allowing
+one stage to migrate or roll back independently. This does not change
+`PipelineConfig` topology, input adapters, or the connector payload contract.
+An omitted stage override preserves the deploy-level selection.
+
+MRv2 native downstream receivers currently support turn-based requests only;
+streaming sessions and resumable input require the V1 prompt-replacement path.
+Selecting a runner does not add the session capabilities it lacks. Platform
+fallbacks and stage overrides are validated after configuration resolution.
+
 ### Stage fields
 
 Each entry under `stages:` accepts any `StageDeployConfig` field directly (no nested `engine_args:`). Only fields whose value legitimately varies across stages live here; pipeline-wide settings (trust_remote_code, distributed_executor_backend, dtype, quantization, prefix/chunked prefill, DP/PP sizes) are declared at the top level and applied to every stage. Unknown keys fall through to `engine_extras:` and are forwarded to the engine. Frequently used fields are listed below; the source-of-truth schema is `StageDeployConfig` in `vllm_omni/config/stage_config.py`.
@@ -306,7 +319,9 @@ ROCm and MUSA. Select one of these deployment profiles:
 | `qwen3_tts_mrv2.yaml` | V2 | B1 | Explicit MRV2 profile (same runner selection as the default) |
 | `qwen3_tts_high_concurrency_mrv2.yaml` | V2 | B1, B2 | Opt-in throughput tuning |
 | `qwen3_tts_high_concurrency_mrv2_b4.yaml` | V2 | B1, B2, B3, B4 | Experimental throughput / buffered playback |
+| `qwen3_tts_high_concurrency_mrv2_single_gpu.yaml` | V2 | B1–B8 | Experimental two-stage deployment on one GPU, with MPS |
 | `qwen3_tts_high_concurrency.yaml` | V1 | Existing defaults | V1 high-concurrency control |
+| `qwen3_tts_fused_single_gpu.yaml` | V2 | N/A (in-Talker decoder) | Opt-in single-stage CUDA pipeline on one GPU |
 
 ```bash
 # MRV2 is the default; pass a copy with `model_runner: v1` to force V1.
@@ -337,6 +352,102 @@ supply `mtp_sampling_params` and `get_mtp_seed(sampling_params)` for model-local
 (with `mtp_sample_steps` and `mtp_sample_vocab_size`) as needed. Qwen3-TTS retains
 its existing `talker_mtp` entry point for V1.
 
+### Selecting one or two Qwen3-TTS stages
+
+Choose the deployment profile explicitly with `--deploy-config`. The default
+`qwen3_tts` pipeline retains two stages: the Talker publishes codec chunks to
+Code2Wav, which produces audio. Its stage devices can be placed on the same
+GPU or on separate GPUs. `qwen3_tts_fused` is a separate, opt-in pipeline for
+one CUDA GPU: the Talker owns the stateful codec decoder and emits PCM directly,
+without a Code2Wav engine or stage connector.
+
+```bash
+# Two stages on one GPU; the two-stage pipeline remains independently selectable.
+vllm serve Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice --omni \
+  --deploy-config vllm_omni/deploy/qwen3_tts_high_concurrency_mrv2_single_gpu.yaml
+
+# One stage on one GPU; no inter-stage transport or Code2Wav scheduling.
+vllm serve Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice --omni \
+  --deploy-config vllm_omni/deploy/qwen3_tts_fused_single_gpu.yaml
+```
+
+The single-stage profile requires CUDA, V2, asynchronous chunks, TP=PP=1,
+an in-process worker and disabled prefix caching. Select the full profile;
+adding `talker_stream_decode: true` to a two-stage Talker is rejected because
+its latent output belongs to the Code2Wav input contract. Single-stage options
+live in `stages[].additional_config`; two-stage transport and first-frame
+options live in `connectors.*.extra`.
+
+The single-stage profile defaults `talker_stream_first_audio` to `false` for
+throughput. To opt into a separate first-frame delivery queue, save this
+overlay beside the bundled deployment files and pass its path to
+`--deploy-config`:
+
+```yaml
+base_config: qwen3_tts_fused_single_gpu.yaml
+stages:
+  - stage_id: 0
+    additional_config:
+      talker_stream_decode: true
+      talker_stream_first_audio: true
+      ref_code_context_frames: 72
+      code_predictor_kv_cache: true
+      code_predictor_fused_sampling: true
+      code_predictor_fused: true
+```
+
+This queue uses PCM from the single-stage decoder. The two-stage
+`talker_first_audio` option below uses its own first-frame decoder and does
+not require single-stage execution. The two options cannot be enabled
+together. Single-stage prefix-cache reset with
+`reset_running_requests=True` returns `False` while requests are active;
+wait for completion or abort them before retrying.
+
+Reference-cloning context is selected independently by each profile. The
+single-stage profile explicitly uses 72 reference frames, matching the
+`qwen3_tts.yaml` context boundary. A shorter
+`additional_config.ref_code_context_frames` (for example 25) can change
+timbre continuity even when WER is similar;
+the throughput measurements for CustomVoice do not qualify Base voice
+cloning quality. Reference layout, grouping and priming are model hooks;
+the shared worker does not select Qwen's context length or codebook layout.
+
+### Qwen3-TTS first-frame delivery and rollback
+
+The standard `qwen3_tts.yaml`, `qwen3_tts_high_concurrency.yaml`,
+`qwen3_tts_mrv2.yaml` and `qwen3_tts_high_concurrency_mrv2.yaml` profiles
+enable the `talker_first_audio` connector option, as does the experimental
+two-stage `qwen3_tts_high_concurrency_mrv2_single_gpu.yaml` profile.
+This changes the default CUDA MRV2 streaming path:
+residual prediction runs eagerly after the Talker sample, the Talker loads
+an additional first-frame decoder with its weights and CUDA graphs, and the
+orchestrator orders its audio before subsequent Code2Wav chunks.
+
+Code2Wav retains full-audio prefix graphs alongside the optional state-only
+graphs. Each request's delivery marker selects the graph and audio trimming;
+enabling the option alone never suppresses audio or disables prefix batching
+for requests that retain regular codec delivery.
+
+The path requires asynchronous chunks, TP/PP 1, an in-process executor and
+disabled prefix caching. Requests with reference codes and unsupported
+runners or platforms retain regular Code2Wav delivery.
+
+To restore regular codec delivery and avoid loading the Talker's additional
+decoder, use a deploy overlay with the relevant base profile:
+
+```yaml
+base_config: qwen3_tts.yaml
+connectors:
+  connector_of_shared_memory:
+    extra:
+      talker_first_audio: false
+```
+
+First-packet latency measures when PCM starts arriving. Time to first audible
+audio (TTFA) also includes any leading silence in the generated audio. An
+earlier first packet therefore does not necessarily improve TTFA; measure
+both for the intended voice and workload.
+
 ### Included performance work
 
 - Request snapshots have a fast path for immutable scalar leaves, including
@@ -372,14 +483,81 @@ speaker similarity before adopting either batching preset for a production
 workload. Floating-point decoder outputs can differ across batch sizes; this PR
 does not claim bitwise or quality equivalence.
 
-### Optional MPS deployment
+### Experimental MPS deployment
 
-NVIDIA MPS is an optional operator setting for colocated CUDA processes, not a
-YAML option or a library default. This PR does not establish a throughput or
-first-packet latency benefit from MPS. Measure the exact deployment with and
-without MPS before enabling it.
+NVIDIA MPS lets colocated CUDA stage processes share GPU execution resources.
+It is disabled by default. The experimental
+`qwen3_tts_high_concurrency_mrv2_single_gpu.yaml` profile sets `cuda_mps: true`,
+alongside cached residual prediction, fused sampling, time-major codec
+convolutions, first-frame delivery, and larger graph batches.
 
-Use only assigned GPUs and an independent MPS pipe directory. A private MPS
-server does not provide exclusive GPU ownership or MIG isolation. For a
-single-GPU deployment, explicitly place both stages on that GPU; the supplied
-high-concurrency profile places its two stages on different GPUs by default.
+The runtime requires `nvidia-cuda-mps-control` on `PATH` and one explicit CUDA
+GPU per local EngineCore stage, with `parallel_stage_init: false`.
+Use numeric GPU ordinals for stage placement and visibility so initialization
+locks identify the physical GPU before MPS remaps it. It starts a private MPS daemon for each selected
+GPU and stops its own daemon after the stages exit. If
+`CUDA_MPS_PIPE_DIRECTORY` already names an operator-managed daemon, the runtime
+reuses it without stopping it. Diffusion and remote stages are unsupported.
+Set `cuda_mps: false` in a deploy overlay to disable automatic MPS management.
+
+Stage `env.CUDA_MPS_PIPE_DIRECTORY` overrides the parent setting when selecting
+the daemon; an explicit empty string selects a private daemon. Stages sharing
+a GPU reuse the first stage's daemon. Later stages may omit the setting or
+explicitly match that policy; a conflicting explicit setting fails before
+that stage starts. The parent environment is unchanged.
+
+MPS can improve throughput under concurrent load while increasing first-packet
+or first-audible-audio latency, particularly at low request rates. For a
+latency-sensitive workload, compare the same profile with `cuda_mps: false`.
+An inherited operator-managed MPS daemon must also be disabled by its owner
+for that comparison; this switch only controls automatic MPS management.
+
+MPS does not reserve a GPU. Use only assigned GPUs, explicitly place stages on
+the intended GPU, and warm the complete pipeline before measuring performance.
+
+## MOSS-TTS Local 1.5 with Model Runner V2
+
+`OpenMOSS-Team/MOSS-TTS-Local-Transformer-v1.5` can opt into CUDA MRV2 using
+the shared runtime introduced for Qwen3-TTS:
+
+```bash
+vllm serve OpenMOSS-Team/MOSS-TTS-Local-Transformer-v1.5 --omni \
+  --deploy-config vllm_omni/deploy/moss_tts_local_mrv2.yaml
+```
+
+This profile inherits the batching, codec graph buckets and 1-frame/15-frame
+chunk geometry from `moss_tts_local.yaml`, and selects V2 for both the Local
+Talker and codec stages. The Local depth predictor exposes the MRV2 `mtp`
+capabilities while retaining its V1 `talker_mtp` implementation, sampling
+defaults and explicit request-seed handling. Codec chunks use the native data
+plane; the internal Talker retains its final-only orchestrator output policy.
+
+On CUDA, the profile bounds stage-0 prefill work to 512 tokens per iteration.
+The codec retains its own CUDA Graph capture/replay, including batch buckets
+through 64, but disables Inductor compilation (`compilation_config.mode: 0`)
+of the stateful decoder to avoid lengthy compilation at startup. This is
+independent of `enforce_eager`; the profile keeps `enforce_eager: false`.
+It explicitly selects `cudagraph_mode: FULL`, which works without Inductor;
+the default `FULL_AND_PIECEWISE` would otherwise normalize to `NONE` and clear
+the codec's capture buckets when compilation is disabled.
+
+For sustained C128 serving on a large-memory CUDA GPU, select the
+separate `moss_tts_local_mrv2_high_concurrency.yaml` profile. It uses 128
+stream slots per stage, a 32 GiB Talker KV budget, and the codec's
+`triton_slot` backend with Inductor compilation and codec-owned CUDA graphs.
+The bounded KV budget leaves room for codec state and graphs; it does not
+guarantee that 128 maximum-length prompts fit simultaneously. See the
+[MOSS recipe](gh-file:recipes/OpenMOSS/MOSS-TTS.md#local-15-mrv2-and-slot-attention)
+for activation, backend comparisons, memory requirements and benchmark commands.
+
+Omitting `--deploy-config`, or selecting `moss_tts_local.yaml`, retains V1.
+NPU, XPU, ROCm and MUSA overrides also retain V1. This profile does not enable
+MRV2 for MOSS Delay, Realtime or Nano. Local 1.5 outputs 48 kHz stereo audio;
+set `VLLM_OMNI_BENCH_AUDIO_SAMPLE_RATE=48000` and
+`VLLM_OMNI_BENCH_AUDIO_CHANNELS=2` when benchmarking raw PCM.
+
+Event-driven orchestration remains independently selectable with
+`VLLM_OMNI_EVENT_DRIVEN_ORCH=0` or `1`. Keep the runner and deployment identical
+when comparing these modes. Model-runner selection does not change the
+orchestration default or enable experimental reference encoding, chunk ramps,
+generation-output draining or MPS.

docs/contributing/README.md

@@ -295,6 +295,10 @@ If you encounter a bug or have a feature request, please search existing issues
 
 Thank you for your contribution to vLLM-Omni! Before submitting the pull request, please ensure the PR meets the following criteria. This helps vLLM-Omni maintain the code quality and improve the efficiency of the review process.
 
+### Pull Request Limit
+
+To avoid agent spam, vLLM-Omni uses GitHub's [pull request limit](https://github.blog/open-source/maintainers/how-pull-request-limits-are-cutting-down-the-noise/) for contributors without write access. The current cap is 5 open PRs. If this blocks well-intentioned critical work, contact a committer to request bypass list access.
+
 ### DCO and Signed-off-by
 
 When contributing changes to this project, you must agree to the [DCO](https://developercertificate.org/). Commits must include a `Signed-off-by:` header which certifies agreement with the terms of the DCO.