Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 hsliuustc0106
|
8affabd527 |
No files matched your search
@@ -1,64 +0,0 @@
|
||||
---
|
||||
name: 3d-animation-short-generator
|
||||
description: Develop stylized 3D narrative shorts through story, character and environment references, timed shot tables, storyboards, generation, and assembly. Use for coherent animated storytelling or its planning stages, not isolated image edits or photorealistic live action.
|
||||
metadata:
|
||||
compatibility: File-based agent workflow with sibling h3-prompt-writing. Media generation and editing require available tools or a configured backend.
|
||||
---
|
||||
|
||||
# 3D Animation Short Generator
|
||||
|
||||
Read the [portable workflow](../h3-prompt-writing/references/portable-workflow.md).
|
||||
For a complete film, progress from story to references, shot table, text storyboard,
|
||||
clips, and assembly. Keep these as local artifacts, using existing user decisions
|
||||
and authorization. If only a planning stage is requested, deliver that stage.
|
||||
|
||||
## Story and visual contract
|
||||
|
||||
Capture premise, desired emotional outcome, duration, ratio, model, and audio mode.
|
||||
Use warm stylized 3D as a default only when style is unspecified: readable geometric
|
||||
silhouettes, designed hair clumps, tactile materials, soft skin shading where
|
||||
appropriate, expressive brows/eyes, and elastic posing. User-selected 2D or other
|
||||
styles override this preset rather than receiving accidental 3D texture.
|
||||
|
||||
Create an active protagonist with a want, obstacle, meaningful choice, and payoff.
|
||||
Connect the ending to an earlier visual/emotional anchor. Use as many beats as the
|
||||
duration needs; do not pad a tiny story into an eight-beat formula. Dialogue should
|
||||
change a relationship or move the action, and use the user's requested language.
|
||||
|
||||
## Reference design
|
||||
|
||||
- Character references lock name, silhouette, proportions, face, hair, costume,
|
||||
signature props, and color. Label identity in the production document; clean
|
||||
rendering anchors should not carry labels that could leak into the video.
|
||||
- Scene references contain environments only unless the user requested a composite.
|
||||
Record landmarks, lighting direction/time of day, entrances, exits, and prop state.
|
||||
- A reference board with multiple views is for design review. Prefer appropriate
|
||||
clean views for model input so grids and pose sheets do not become final frames.
|
||||
- When an asset changes, update all affected prompts and clips, with a manifest
|
||||
identifying the current revision.
|
||||
|
||||
## Shot planning
|
||||
|
||||
Read [shot-table-spec.md](references/shot-table-spec.md) for the six-column table
|
||||
and continuity checks. Then read
|
||||
[storyboard-guidelines.md](references/storyboard-guidelines.md) for the matching
|
||||
text storyboard and optional pencil previews. The text storyboard is the source
|
||||
of shot timing; preview art does not silently replace it.
|
||||
|
||||
Vary shot size and elastic performance to serve the story. Camera tilts, close-ups,
|
||||
or exaggeration are choices for specific beats, not quotas. Track off-screen
|
||||
characters and spatial landmarks so they do not teleport back into later shots.
|
||||
|
||||
## Generation and assembly
|
||||
|
||||
Before rendering, read [model-selection.md](references/model-selection.md) and
|
||||
[fallback-policy.md](references/fallback-policy.md). Use the chosen model through
|
||||
available tools. Compile prompts using the H3 base or reference guide as appropriate.
|
||||
Remove storyboard-only labels, arrows, panel borders, and sketch styling from
|
||||
rendering inputs. Tie each clip to the current scene and character references.
|
||||
|
||||
Read [qc-checklist.md](references/qc-checklist.md) for assembly and final review.
|
||||
Keep the planned shot order and audio continuity. Use a continuous score when
|
||||
requested, duck it under dialogue/SFX, and do not add subtitles unasked. Deliver
|
||||
the actual film or the requested planning package, with reproducible prompts and
|
||||
any unresolved identity, scene, or timing defects.
|
||||
@@ -1,4 +0,0 @@
|
||||
interface:
|
||||
display_name: "3D Animation Shorts"
|
||||
short_description: "Plan stories, character anchors, shots, and edits"
|
||||
default_prompt: "Use $3d-animation-short-generator to prepare the requested H3 prompts or video with the available tools."
|
||||
@@ -1,21 +0,0 @@
|
||||
# Targeted generation recovery
|
||||
|
||||
Compare the failed result against the shot's actual identity/spatial anchors.
|
||||
Classify the failure: wrong input or API mapping, reference drift, overloaded
|
||||
action, timing, audio, or a render/export failure. Fix transport/configuration
|
||||
errors before spending another generation attempt.
|
||||
|
||||
For drift, quote the relevant anchors and remove unrelated reference roles. For
|
||||
overloaded action, simplify or split the shot while preserving the requested total
|
||||
duration and handoffs. For a scene change, use the new scene reference and avoid
|
||||
conditioning on a previous scene when that would contradict the new setting.
|
||||
|
||||
Use the retry/resource limit from the shared portable workflow. A persistent
|
||||
failure should lead to a concrete choice (simpler action, a supplied reference,
|
||||
an authorized alternate model, or an explicitly incomplete shot), not automatic
|
||||
provider switching or an endless loop. Never present a placeholder as a finished
|
||||
clip or hide an incorrect scene in final assembly.
|
||||
|
||||
For optional pencil previews, reduce annotation or panel count. Use the text
|
||||
storyboard if visualization is not essential; do not rerender correct video
|
||||
solely because a human-review sketch changed.
|
||||
@@ -1,21 +0,0 @@
|
||||
# Model and format selection
|
||||
|
||||
Honor an existing model choice. Otherwise prefer MiniMax H3 when available, and
|
||||
inspect the running backend for partitions, reference modes, audio, resolutions,
|
||||
duration limits, and long-video options. Do not assume hosted and local services
|
||||
have identical limits, quality, prices, or parameter names.
|
||||
|
||||
If another model is requested, use its real documented input contract. A provider
|
||||
switch needs the user's authorization; never silently use a different service
|
||||
because a preset names it as a fallback. For an authorized mixed-model workflow,
|
||||
record model and settings per shot and check cross-shot style differences.
|
||||
|
||||
Keep delivery resolution separate from native generation resolution. Record actual
|
||||
width/height, fps, frame count, and any crop, retiming, or upscaling. Do not claim
|
||||
native 2K when the video was generated at 768p and restored later.
|
||||
|
||||
Compile shot action, camera, identity, scene, and audio into the chosen model's
|
||||
prompt format. For H3 use the bundled writing guides. Strip production labels and
|
||||
preview grids. Prompt for designed silhouettes, elastic action, material and
|
||||
lighting details when these belong to the user's 3D style; do not add text/UI just
|
||||
because a model can generate them.
|
||||
@@ -1,19 +0,0 @@
|
||||
# Assembly and review
|
||||
|
||||
Use current clip revisions in shot-table order. Preserve useful dialogue and SFX;
|
||||
mix a continuous requested score underneath them. Inspect edits at sound bridges
|
||||
and dialogue boundaries, avoiding cut vowels and accidental audio gaps. Only add
|
||||
subtitles, music, or retiming consistent with the brief.
|
||||
|
||||
Review identity, landmarks, screen direction, light/time of day, prop state,
|
||||
emotional payoff, intelligibility, SFX timing, and music balance. Look for leaked
|
||||
storyboard panels, sketch lines, arrows, labels, timecodes, or watermarks. Verify
|
||||
the complete file's duration, dimensions, fps, frame count, audio, and decode.
|
||||
|
||||
Keep planning documents, references, raw clips, and final edits in distinguishable
|
||||
folders. Record the exact assets used in the final edit. After a character, scene,
|
||||
prompt, clip, or score is revised, update its downstream dependents and archive the
|
||||
previous result; do not silently combine stale and current versions.
|
||||
|
||||
Report observed defects and what was repaired, including external editing or
|
||||
upscaling. A successful API response proves completion, not visual correctness.
|
||||
@@ -1,30 +0,0 @@
|
||||
# Six-column shot table
|
||||
|
||||
| Shot ID & Duration | Continuity Handoff | Reference Anchors (Spatial + Identity) | Hook Type | Shot Description (Per-Second Directives) | Audio & Dialogue Track |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
|
||||
Use stable IDs such as S01. Record incoming and outgoing image state, prop
|
||||
position, eyeline, motion, and emotional state in Continuity Handoff.
|
||||
|
||||
Reference Anchors contains named landmarks and screen-relative positions; each
|
||||
character's screen position, facing and pose; recently exited characters' locations;
|
||||
lighting baseline and modifiers; and exact character/scene asset paths or IDs.
|
||||
An explicit scene/time change can replace these anchors; an accidental flip cannot.
|
||||
|
||||
Hook Type names the shot's purpose: setup, reveal, reversal, joke, suspense,
|
||||
tenderness, chase, or callback. Do not force a joke or cut into a quiet scene.
|
||||
|
||||
For each second or meaningful sub-second beat, cover action/pose/expression,
|
||||
camera, spatial/prop state, sound, and the handoff to the next interval. Use compact
|
||||
phrases rather than inventing movement in intentionally still moments. Include
|
||||
anticipation, squash/stretch, overshoot, and follow-through where appropriate.
|
||||
|
||||
Keep the complete dialogue/narration script in Audio & Dialogue Track, with
|
||||
speaker, timing, delivery, and SFX. Distinguish off-screen narration from speaking
|
||||
characters; a visible listener should not lip-sync the narrator's words.
|
||||
|
||||
Before rendering, check full time coverage, supported generation durations, a
|
||||
readable number of active characters, matching scene/identity references, and
|
||||
coherent row-to-row handoffs. Mark intentional cuts/time jumps. Fix contradictory
|
||||
states rather than using a generic continuity assurance. A visual shot may span
|
||||
multiple generation windows when the verified backend supports continuation.
|
||||
@@ -1,29 +0,0 @@
|
||||
# Text storyboard and optional visualization
|
||||
|
||||
Maintain one `storyboard.md` with a section per shot. Each section contains:
|
||||
|
||||
1. Shot ID, title, duration, and narrative purpose.
|
||||
2. Exact scene and character reference IDs.
|
||||
3. Spatial anchors: landmarks, character positions/facing, off-screen characters,
|
||||
key/fill/rim lighting, and important props.
|
||||
4. Incoming and outgoing continuity state.
|
||||
5. Time-ordered beats: pose/expression, camera, audio, and anchor changes.
|
||||
|
||||
Use per-second or finer intervals only where meaningful; cover the complete shot.
|
||||
For narrated moments, specify whether a visible character speaks or only reacts.
|
||||
Optional ASCII staging diagrams can clarify screen direction but are not images
|
||||
to supply as rendering references.
|
||||
|
||||
For heavy revision of a shot, extract its section to a named file and link it from
|
||||
the main document. Choose one authoritative version; reintegrate or update the
|
||||
link after revision so later generation cannot read stale content.
|
||||
|
||||
Only generate pencil storyboards when requested or already authorized as previews.
|
||||
Use equal-sized time-ordered panels, with timecode, pose/expression, camera arrow,
|
||||
and audio/anchor notes. Suggested layouts: three panels in a strip, four in a 2×2
|
||||
grid, six in 2×3. More complex scenes may need fewer representative key poses.
|
||||
Keep shot/character/scene labels readable for human review, but never feed these
|
||||
annotated grids as clean final frames. The text timeline remains authoritative.
|
||||
|
||||
If preview panels fail, simplify annotation or rely on text within the agreed
|
||||
scope. An optional visualization failure need not block a usable text storyboard.
|
||||
@@ -1,53 +0,0 @@
|
||||
# MiniMax H3 skill provenance
|
||||
|
||||
Upstream: https://github.com/MiniMax-AI/MiniMax-H3
|
||||
|
||||
Source snapshot: `d21241f0a4b3acbb34c97dae47fa417b7065e438`, retrieved 2026-09-19.
|
||||
|
||||
Source directory:
|
||||
https://github.com/MiniMax-AI/MiniMax-H3/tree/d21241f0a4b3acbb34c97dae47fa417b7065e438/skills
|
||||
|
||||
The nine upstream skill names are retained. Each local skill corresponds to the
|
||||
same-named upstream `skills/<name>/` directory. Credit for the original creative
|
||||
workflows and templates belongs to MiniMax-AI and the upstream contributors.
|
||||
|
||||
## Files and adaptation
|
||||
|
||||
- `h3-prompt-writing/references/base-format.md` and `ref-format.md` are new,
|
||||
condensed descriptions of the public prompt interface. They preserve necessary
|
||||
field names and notation but do not copy the upstream guides or their examples.
|
||||
The entrypoint adds a portable execution reference and clarifies clip/window
|
||||
duration for verified long-video backends.
|
||||
- The eight style-specific `SKILL.md` files are rewritten portable adaptations of
|
||||
the upstream workflows. Their reference documents retain the relevant creative
|
||||
methods while removing host-specific execution assumptions.
|
||||
- The two `co-op-game-intro-generator/references/` templates retain the upstream
|
||||
creative structure with an added portable contract and explicit user overrides.
|
||||
Their instructions are normalized to English, and the video template expresses
|
||||
visible UI copy as localized variables with English defaults. It also removes
|
||||
an unrequested gender restriction and a fixed yellow/black environment
|
||||
transformation.
|
||||
- The five 3D animation references are condensed adaptations of their upstream
|
||||
counterparts: shot table, storyboards, model selection, fallback, and review.
|
||||
- Product, papercraft, collage, and music references extract reusable direction
|
||||
from their corresponding upstream skill. The shared portable workflow, catalog,
|
||||
and UI metadata are additions for this repository.
|
||||
|
||||
No upstream `LICENSE`, `COPYING`, or `NOTICE` file was present in the Git tree at
|
||||
the pinned snapshot. Its README links to the MiniMax H3 Community License on the
|
||||
model release. Because that license has redistribution conditions, this package
|
||||
does not include the upstream prompt guides verbatim. This provenance record
|
||||
preserves attribution and does not assert that this repository's code license
|
||||
relicenses upstream material.
|
||||
|
||||
## Runtime changes
|
||||
|
||||
Canvas documents become local files and asset manifests; choice-card UI becomes
|
||||
the host's normal clarification mechanism when a decision is actually missing.
|
||||
Available generation/editing tools replace proprietary dispatch calls. Previous
|
||||
user choices and authorization persist across steps. Providers, prices, resolution
|
||||
support, and long-video features are checked rather than hardcoded as guarantees.
|
||||
|
||||
User intent overrides style presets, including instrumental/no-text work, selected
|
||||
language, final morph shape, no-cut sequences, and custom player designs. Native
|
||||
generation and external processing remain separately identified in delivery.
|
||||
@@ -1,96 +0,0 @@
|
||||
# Portable MiniMax H3 Skills
|
||||
|
||||
This directory contains one general prompt-writing skill and eight creative
|
||||
skills adapted from
|
||||
[MiniMax-AI/MiniMax-H3](https://github.com/MiniMax-AI/MiniMax-H3/tree/main/skills).
|
||||
See [H3-SOURCES.md](H3-SOURCES.md) for the source revision, attribution, and
|
||||
adaptation scope.
|
||||
|
||||
## Skill catalog
|
||||
|
||||
| Skill | Intended use |
|
||||
| --- | --- |
|
||||
| [h3-prompt-writing](h3-prompt-writing/SKILL.md) | Structured prompts for T2VA, I2VA, FL2VA, L2VA, and Ref2VA |
|
||||
| [minimalist-product-ad-generator](minimalist-product-ad-generator/SKILL.md) | Minimal product ads, isolated product references, copy, and pacing |
|
||||
| [3d-animation-short-generator](3d-animation-short-generator/SKILL.md) | 3D narrative shorts, character and environment design, shot tables, storyboards, and editing |
|
||||
| [papercraft-stop-motion-explainer](papercraft-stop-motion-explainer/SKILL.md) | Layered paper art, puppets, mechanisms, and miniature educational scenes |
|
||||
| [brand-promo-video-generator](brand-promo-video-generator/SKILL.md) | Brand promos, verified assets and claims, product interaction, and calls to action |
|
||||
| [music-video-subtitle-generator](music-video-subtitle-generator/SKILL.md) | Music videos, instrumental visual narratives, beat planning, and optional lyric typography |
|
||||
| [co-op-game-intro-generator](co-op-game-intro-generator/SKILL.md) | Two-player game menus, confirmation images, and animated world entry |
|
||||
| [paper-collage-explainer-generator](paper-collage-explainer-generator/SKILL.md) | Halftone photo collage, incremental paper assembly, and visual metaphors |
|
||||
| [handdrawn-live-video-generator](handdrawn-live-video-generator/SKILL.md) | Live action with rough luminous drawings, physical contact, continuous morphing, and delayed pursuit |
|
||||
|
||||
## Usage
|
||||
|
||||
Codex discovers repository skills from `.agents/skills/`. Clients that scan
|
||||
skills only when a session starts may need a new session. Claude Code reads the
|
||||
same files through the same-named relative symlinks in `.claude/skills/`. Other
|
||||
agents that support Markdown skills can load the relevant `SKILL.md` directly
|
||||
and follow its links to supporting references.
|
||||
|
||||
Examples:
|
||||
|
||||
```text
|
||||
Use $music-video-subtitle-generator to design a 30-second two-dimensional
|
||||
fluid-motion graphic for this instrumental track. Do not add lyrics or
|
||||
subtitles. Preserve leftward motion and initially return only a storyboard and
|
||||
three H3 prompts.
|
||||
```
|
||||
|
||||
```text
|
||||
Use $co-op-game-intro-generator to create a two-player game intro. The players
|
||||
are Lin and Mei, the style is layered paper sculpture, and the title is
|
||||
Together. Create the confirmation image first and wait for my approval before
|
||||
generating video.
|
||||
```
|
||||
|
||||
## MV smoke test
|
||||
|
||||
Test `music-video-subtitle-generator` together with `h3-prompt-writing` using a
|
||||
user-owned song with timestamped lyrics and an original adult-character
|
||||
reference image:
|
||||
|
||||
```text
|
||||
Use $music-video-subtitle-generator and $h3-prompt-writing to turn this song
|
||||
into a historical-romance music video about a woman and a man. Intercut
|
||||
narrative acting with the woman's singing performance, do not display lyric
|
||||
text, split the work according to the backend's verified clip duration, and
|
||||
preserve the complete original song in the final master.
|
||||
```
|
||||
|
||||
Verify that the global timeline matches the source audio duration, lyrics remain
|
||||
verbatim, and each generation window records its incoming state, outgoing state,
|
||||
and local timing. Ref2VA prompts must preserve the six-section field order and
|
||||
resolve every reference label to a real input. The assembled video must decode
|
||||
completely, and its final audio must come from the supplied source rather than a
|
||||
model reconstruction.
|
||||
|
||||
When moving the collection to another repository or a user-level skill folder,
|
||||
copy all nine skill directories and preserve their sibling layout. The eight
|
||||
creative skills share `h3-prompt-writing/references/portable-workflow.md` and the
|
||||
H3 format guides, so copying only one `SKILL.md` is insufficient. Keep each
|
||||
skill's `references/` and `agents/` directories with it. `agents/openai.yaml`
|
||||
provides optional Codex UI metadata and does not restrict the workflow to an
|
||||
OpenAI-based agent or require a dedicated plugin.
|
||||
|
||||
## Execution boundaries
|
||||
|
||||
Prompt writing, storyboarding, and planning require only an agent that can read
|
||||
and write files. Actual image, video, speech, music, and editing work uses tools
|
||||
or services available in the current environment. When a capability is missing,
|
||||
deliver the complete usable production package and identify the unfinished media
|
||||
step.
|
||||
|
||||
The user's style, language, assets, audio, model selection, and existing
|
||||
authorization take priority. An MV template does not add lyrics to an
|
||||
instrumental track, and a clean motion-graphics request does not inherit film
|
||||
grain. A prompt-only request does not start generation. When the user requests
|
||||
approval of a confirmation image before rendering, preserve that checkpoint.
|
||||
|
||||
Treat vLLM-Omni long-video extensions as capabilities of the checked-out branch
|
||||
and active backend. Do not present a development branch's continuation parameters
|
||||
as a universal H3 API. Preserve real requests and raw outputs, distinguish model
|
||||
results from post-production and upscaling, and inspect actual media properties.
|
||||
|
||||
These skills contain no model weights and do not automatically start a service
|
||||
or invoke a paid generation endpoint.
|
||||
@@ -1,66 +0,0 @@
|
||||
---
|
||||
name: brand-promo-video-generator
|
||||
description: Plan, prompt, or produce brand, product, website, app, and service promotional shorts using traceable assets, accurate claims, timed beats, and clear calls to action. Use for launch and campaign videos, not invented product evidence or long narrative films.
|
||||
metadata:
|
||||
compatibility: File-based workflow with sibling h3-prompt-writing and available research, media generation, inspection, and editing tools.
|
||||
---
|
||||
|
||||
# Brand Promo Video Generator
|
||||
|
||||
Read the [portable workflow](../h3-prompt-writing/references/portable-workflow.md).
|
||||
Use supplied brand assets, source links, and existing session decisions. No
|
||||
proprietary tool allowlist or canvas is required. A completed promo requires real
|
||||
generation/editing capability; otherwise deliver an explicitly incomplete production
|
||||
package with finished prompts and a concrete generation handoff.
|
||||
|
||||
## Establish facts and assets
|
||||
|
||||
Inspect product images, UI screenshots, logo exports, typography, colors, and
|
||||
campaign material. Resolve duration, ratio, audience, campaign focus, exact claims,
|
||||
CTA, copy language, narration, and required deliverable from context. Ask for
|
||||
missing identity-bearing assets when they are essential; do not fabricate logos,
|
||||
product UI, screenshots, results, or performance claims.
|
||||
|
||||
Build a compact truth sheet from user-supplied originals and official sources.
|
||||
Record asset ID, role, local path, source URL/note, and verification status. Do not
|
||||
confuse a retrieved search thumbnail with an original brand asset. Distinguish
|
||||
verified facts from clearly labeled concept copy. Preserve the requested language;
|
||||
chat language alone does not justify translating supplied campaign text.
|
||||
|
||||
## Design a product-specific story
|
||||
|
||||
Choose a spine that exposes real cause and effect:
|
||||
|
||||
- Software: user intent → actual interaction → useful output → evidence → CTA.
|
||||
- Physical product: reveal → interaction → mechanism/detail → use → benefit → CTA.
|
||||
- Service: situation → process → evidence → outcome → promise.
|
||||
- Image-led brand: authentic imagery → repeated visual motif → emotional payoff.
|
||||
|
||||
Plan time/frame ranges, asset IDs, one main action, product evidence, exact copy,
|
||||
readable hold, color state, camera, transition, and rhythm. Use the actual output
|
||||
fps for frame timing. Five to eight major beats can fit a 15-second promo, but
|
||||
readability takes priority over a fixed beat count.
|
||||
|
||||
Let product edges, UI movement, light, cursor paths, or matched geometry motivate
|
||||
transitions. Use a few energy peaks and deliberate settling moments. Preserve logo
|
||||
clear space and readable screenshots. Avoid invented HUDs or metrics that look like
|
||||
product proof. Summarize the truth sheet and plan before rendering; continue under
|
||||
existing authorization unless the user requested approval at that stage.
|
||||
|
||||
## Produce and inspect
|
||||
|
||||
Use authentic still/UI plates for exact content. Generated material can supply
|
||||
conceptual scenes, atmosphere, and transition geometry with clear provenance.
|
||||
Compile actual H3 references and shots using the appropriate bundled guide.
|
||||
Check backend resolution, duration, and audio support before calling it.
|
||||
|
||||
Use native instrumental/SFX direction when appropriate; do not generate a second
|
||||
score automatically. Separate narration or music is useful when requested for
|
||||
control or remixing. Preserve original SFX during assembly when they matter, and
|
||||
duck music under speech. Do not add subtitles unless requested.
|
||||
|
||||
Check claims and exact product names against sources, inspect logo geometry and
|
||||
copy, verify real UI states, and review transitions, aspect ratio, duration, audio,
|
||||
and decoding. For wrong identity-bearing content, use the correct supplied/source
|
||||
asset rather than polishing an imitation. Deliver the video, prompts/timeline,
|
||||
asset manifest, and any remaining factual or visual limitations.
|
||||
@@ -1,4 +0,0 @@
|
||||
interface:
|
||||
display_name: "Brand Promo Videos"
|
||||
short_description: "Build brand promos from verified assets and claims"
|
||||
default_prompt: "Use $brand-promo-video-generator to prepare the requested H3 prompts or video with the available tools."
|
||||
@@ -1,73 +0,0 @@
|
||||
---
|
||||
name: co-op-game-intro-generator
|
||||
description: Design or generate a two-player co-op game menu and opening animation from player names, title, style, and optional identity references. Use for character-led game intros, not playable game implementation or complex multi-page interfaces.
|
||||
metadata:
|
||||
compatibility: File-based workflow with sibling h3-prompt-writing and bundled image/video templates. Rendering uses available generation tools.
|
||||
---
|
||||
|
||||
# Co-op Game Intro Generator
|
||||
|
||||
Read the [portable workflow](../h3-prompt-writing/references/portable-workflow.md).
|
||||
Before writing a confirmation-image prompt, read
|
||||
[h3-confirmation-image-template.md](references/h3-confirmation-image-template.md).
|
||||
Before writing the video prompt, read
|
||||
[h3-video-prompt-template.md](references/h3-video-prompt-template.md).
|
||||
Both templates are required package resources, not remote tools.
|
||||
|
||||
## Resolve player and style bindings
|
||||
|
||||
Reuse or collect the two player names, game title, visual style, desired duration,
|
||||
ratio, and optional identity references. The template defaults to a 15-second,
|
||||
16:9 menu-to-world sequence; adapt it to explicit user requirements. Do not assume
|
||||
the players' gender, body shape, outfits, or equipment from a generic example.
|
||||
|
||||
Use each portrait only for recognizable identity anchors: face silhouette, hair,
|
||||
glasses, proportions, and distinguishing traits. Reinterpret rendering, skin
|
||||
texture, lighting, and costume in the selected visual style. If a portrait or body
|
||||
comparison reference is absent, write a text design rather than inventing an asset
|
||||
label. Maintain player-to-name mapping across all references and frames.
|
||||
|
||||
## Confirmation image
|
||||
|
||||
Fill the template's style, palette, characters, title, and name variables. Keep
|
||||
the two characters central, player cards toward the upper left, menu on the right,
|
||||
and the continue action as the main interaction focus. The title establishes
|
||||
identity without competing with that focus. Buttons, icons, typography, and all
|
||||
player/equipment panels share one palette and style. Keep menu labels on one line.
|
||||
|
||||
The style changes material, shapes, fonts, and decoration; it should not randomly
|
||||
rearrange the menu hierarchy. User-requested layout or ratio changes take priority
|
||||
over template defaults. Preserve the exact supplied title/names and requested copy.
|
||||
|
||||
For image-only/prompt-only work, stop at that deliverable. For video production,
|
||||
generate and inspect the confirmation image if within scope. Honor a request to
|
||||
approve the image first; otherwise use the established end-to-end authorization
|
||||
and continue after reporting the selected image and fixing material defects.
|
||||
|
||||
## Video event plan
|
||||
|
||||
Fill the video template from the current image and player bindings. Its baseline
|
||||
sequence is menu → first player's light equipment → second player's heavy equipment
|
||||
→ shared confirmation → world loading → both enter the world. Equipment, world,
|
||||
body contrast, and timing are editable design choices; preserve requested identity
|
||||
and style instead of forcing a cyberpunk example onto another setting.
|
||||
|
||||
Bind UI reference, each actual portrait, and any real body-comparison image to
|
||||
distinct roles. Compile this creative template into the H3 mode guide's fields;
|
||||
the mixed-language template is not itself an API request. Preserve literal visible
|
||||
UI text in its required language, and remove all unfilled variables before dispatch.
|
||||
Do not treat a composition reference as an exact first-frame anchor unless the
|
||||
selected backend actually supports that role.
|
||||
|
||||
## Review and repair
|
||||
|
||||
Check both names, exact menu strings, layout, player identity, side assignment,
|
||||
equipment distinction, and the transition into the requested world. Match UI
|
||||
hover/click, mechanical locks, loading, and footsteps to the visible events.
|
||||
For identity swaps, strengthen name/position/reference bindings; for weak style,
|
||||
revise palette, material, character, and typography together. Reduce excessive
|
||||
UI text if necessary without silently changing required player names.
|
||||
|
||||
Deliver the requested prompts/assets or the inspected intro video, with reference
|
||||
roles and any text/timing limitations. Missing templates block template execution;
|
||||
missing generation capability still permits a complete prompt handoff.
|
||||
@@ -1,4 +0,0 @@
|
||||
interface:
|
||||
display_name: "Co-op Game Intros"
|
||||
short_description: "Design two-player menus and world-entry animation"
|
||||
default_prompt: "Use $co-op-game-intro-generator to prepare the requested H3 prompts or video with the available tools."
|
||||
-397
@@ -1,397 +0,0 @@
|
||||
# H3 Confirmation Image Template
|
||||
|
||||
## Portable adaptation
|
||||
|
||||
This is a creative template, not a tool call. User requirements override its
|
||||
default ratio, layout, poses, and wording. Fill every relevant variable from the
|
||||
brief; set `{character_count}` to two for this skill. When portraits are absent,
|
||||
replace portrait references with explicit character designs. Do not claim to
|
||||
reference an image that was not supplied. Use the image-generation capability
|
||||
available in the host environment; no specific image provider is required.
|
||||
|
||||
## Core Rule
|
||||
Generate a polished 16:9 landscape confirmation image for a cooperative game main menu.
|
||||
The image consists of two independent systems:
|
||||
• UI Framework (fixed)
|
||||
• Visual Style (dynamic)
|
||||
The UI Framework defines the layout, composition, information hierarchy, interaction logic, reading flow, menu structure, and readability. These elements must remain unchanged.
|
||||
{visual_style} only controls the artistic appearance, including rendering medium, materials, color language, lighting mood, environment style, character design, costume language, UI materials, icon language, typography appearance, decorative motifs, and overall atmosphere.
|
||||
Changing {visual_style} should feel like applying a different art direction to the same professionally designed game menu, never redesigning the interface itself.
|
||||
If any default description conflicts with {visual_style}, prioritize {visual_style} while preserving the UI framework.
|
||||
|
||||
---
|
||||
|
||||
## Design Goal
|
||||
Create a premium commercial console game main menu mockup in 16:9 landscape format. The image should combine the quality of a polished game promotional artwork with the clarity of a professional game UI. Characters and interface must feel naturally integrated, with strong visual hierarchy, excellent readability, clean composition, and a clear interaction focus.
|
||||
|
||||
---
|
||||
|
||||
## Overall Style
|
||||
Follow the user-selected visual style: {visual_style}.
|
||||
The selected style controls the rendering medium, illustration style, materials, palette language, environment, character appearance, outfit design, UI appearance, icon language, typography treatment, decorative elements, and atmosphere.
|
||||
All visual elements must belong to one unified artistic language.
|
||||
Do not mix incompatible visual styles within the same image.
|
||||
The UI framework, layout, hierarchy, composition, and interaction logic must remain consistent regardless of style.
|
||||
|
||||
---
|
||||
|
||||
## Color Palette
|
||||
Derive the entire color system from {visual_style}.
|
||||
Use:
|
||||
• {main_background_color} as the primary background color.
|
||||
• {ui_body_color} as the main UI color.
|
||||
• {text_color} as the primary text color.
|
||||
• {functional_accent_color} as the interaction highlight color.
|
||||
• Red only for warning, danger, or exit actions.
|
||||
Limit the overall palette to five primary colors.
|
||||
All buttons, player cards, icons, borders, highlights, decorative elements, typography, and visual effects must follow the same color system.
|
||||
Avoid introducing unrelated colors outside the selected palette.
|
||||
|
||||
---
|
||||
|
||||
## Composition
|
||||
Use a fixed 16:9 landscape layout.
|
||||
Place {character_count} playable characters in the center as the visual subject.
|
||||
The UI surrounds the characters without blocking them.
|
||||
Place the player information cards in the upper-left corner.
|
||||
Place the main menu vertically on the right.
|
||||
Place a decorative horizontal strip along the bottom, adapting naturally to {visual_style} (such as ribbon, warning strip, wooden plank, stone slab, energy bar, scroll, etc.).
|
||||
Keep a clear Z-shaped reading flow:
|
||||
Player Cards → Characters → Menu → Continue Button.
|
||||
The Continue button must always remain the primary visual focus.
|
||||
Maintain generous negative space and excellent readability.
|
||||
|
||||
---
|
||||
|
||||
## Background
|
||||
Generate a background naturally derived from {visual_style}.
|
||||
The background supports the interface rather than competing with it.
|
||||
Keep enough clean negative space behind all important UI elements.
|
||||
Use subtle textures, environmental details, and decorative motifs only where appropriate for the selected style.
|
||||
Avoid excessive visual noise, dense decoration, heavy dirt, scratches, paint splatter, or distracting textures.
|
||||
The overall background should feel polished, premium, modern, and suitable for a commercial game menu.
|
||||
|
||||
---
|
||||
|
||||
## Character Style
|
||||
All characters strictly follow {visual_style}.
|
||||
The selected style determines:
|
||||
• rendering medium
|
||||
• body proportion
|
||||
• facial stylization
|
||||
• costume language
|
||||
• material treatment
|
||||
• lighting response
|
||||
• animation style
|
||||
• overall character quality
|
||||
Only the character identity anchors and pose remain fixed. Facial rendering, eye design, nose simplification, mouth shape, skin treatment, expression style, and head proportion must be fully reinterpreted in {visual_style}.
|
||||
No default character style may override {visual_style}.
|
||||
|
||||
---
|
||||
|
||||
## Character A / PLAYER 1
|
||||
### Identity
|
||||
Use character reference image 1 only for identity mapping.
|
||||
The reference image donates identity anchors only: recognizable face silhouette, hairstyle, glasses if present, relative facial proportions, and distinctive personal traits.
|
||||
It does not donate photographic realism, real skin texture, real-world lighting, camera quality, or the original image style.
|
||||
Do not lose the character's identity anchors, but the face must be redesigned into {visual_style}.
|
||||
All facial rendering, eye design, nose simplification, mouth shape, skin treatment, expression style, and head proportion must follow {visual_style}.
|
||||
Do not swap identity with other characters.
|
||||
|
||||
### Appearance
|
||||
Character rendering, expression, outfit, materials, and accessories are fully derived from {visual_style} while maintaining a recognizable playable-game-character silhouette.
|
||||
|
||||
### Action
|
||||
Character A sits cross-legged on the ground.
|
||||
Both hands support the body on the floor.
|
||||
The upper body leans slightly backward.
|
||||
Character A looks upward toward the camera.
|
||||
|
||||
---
|
||||
|
||||
## Character B / PLAYER 2
|
||||
### Identity
|
||||
Use character reference image 2 only for identity mapping.
|
||||
The reference image donates identity anchors only: recognizable face silhouette, hairstyle, glasses if present, relative facial proportions, and distinctive personal traits.
|
||||
It does not donate photographic realism, real skin texture, real-world lighting, camera quality, or the original image style.
|
||||
Do not lose the character's identity anchors, but the face must be redesigned into {visual_style}.
|
||||
All facial rendering, eye design, nose simplification, mouth shape, skin treatment, expression style, and head proportion must follow {visual_style}.
|
||||
Do not swap identity with other characters.
|
||||
|
||||
### Appearance
|
||||
Character rendering, expression, outfit, materials, and accessories are fully derived from {visual_style} while maintaining a recognizable playable-game-character silhouette.
|
||||
|
||||
### Action
|
||||
Character B sits cross-legged on the ground.
|
||||
Both hands rest naturally in front of the legs.
|
||||
The upper body leans slightly forward.
|
||||
Character B looks directly toward the camera.
|
||||
|
||||
---
|
||||
|
||||
## Lighting
|
||||
Lighting follows {visual_style} while preserving clear readability.
|
||||
Use a primary top light, secondary fill light, soft ambient bounce, contact shadows, gentle rim lighting, and high-quality global illumination where appropriate.
|
||||
Faces should remain clearly visible.
|
||||
Characters must integrate naturally into the environment while maintaining strong separation from the background.
|
||||
|
||||
---
|
||||
|
||||
## Game UI
|
||||
Create a professional commercial console game main menu.
|
||||
The menu structure, interaction hierarchy, spacing, composition, reading flow, and information architecture remain fixed.
|
||||
{visual_style} only changes the artistic appearance of the interface.
|
||||
The selected style may affect:
|
||||
• UI materials
|
||||
• surface textures
|
||||
• border treatment
|
||||
• corner shape
|
||||
• decorative motifs
|
||||
• color rendering
|
||||
• lighting treatment
|
||||
• icon language
|
||||
• typography appearance
|
||||
• interaction effects
|
||||
It must NOT change:
|
||||
• menu hierarchy
|
||||
• button order
|
||||
• button placement
|
||||
• player-card placement
|
||||
• composition
|
||||
• interaction logic
|
||||
• information architecture
|
||||
• readability
|
||||
The entire interface should feel like one professionally designed game UI system rather than separate elements assembled together.
|
||||
|
||||
---
|
||||
|
||||
## UI Visual Language
|
||||
Derive the complete UI appearance from {visual_style}.
|
||||
The selected style determines:
|
||||
• button materials
|
||||
• border language
|
||||
• panel materials
|
||||
• decorative details
|
||||
• edge treatment
|
||||
• visual weight
|
||||
• highlight style
|
||||
• interaction feedback
|
||||
• shadow treatment
|
||||
• texture language
|
||||
Possible materials include wood, stone, paper, leather, cloth, glass, crystal, hologram, neon, metal, ceramic, carved surfaces, painted surfaces, magical energy, pixel blocks, watercolor paper, ink brush, comic graphics or any other material naturally belonging to {visual_style}.
|
||||
Every UI element must share the same material language and artistic style.
|
||||
|
||||
---
|
||||
|
||||
## Button Layout Rules
|
||||
Place all menu buttons vertically on the right side.
|
||||
Maintain a consistent vertical rhythm.
|
||||
Every button uses the same design family.
|
||||
Button width automatically adapts to text length.
|
||||
Keep all menu titles on a single line.
|
||||
No automatic wrapping.
|
||||
No stacked text.
|
||||
Text remains horizontally centered.
|
||||
Keep consistent padding.
|
||||
Maintain equal spacing between buttons.
|
||||
Avoid visual clutter.
|
||||
|
||||
---
|
||||
|
||||
## Continue Button
|
||||
The Continue button is always the primary visual focus.
|
||||
Make it visually dominant using the emphasis method naturally belonging to {visual_style}.
|
||||
Possible emphasis methods include:
|
||||
• larger size
|
||||
• stronger contrast
|
||||
• brighter color
|
||||
• unique material
|
||||
• distinctive border
|
||||
• lighting
|
||||
• glow
|
||||
• engraving
|
||||
• magical energy
|
||||
• holographic effect
|
||||
• embossed frame
|
||||
• animated visual cue
|
||||
Do not force glow or neon if they conflict with {visual_style}.
|
||||
The Continue button should immediately attract the player's attention.
|
||||
|
||||
---
|
||||
|
||||
## Start New Game Button
|
||||
Place above Continue.
|
||||
Use the standard secondary button appearance.
|
||||
Keep the visual language consistent with the selected style.
|
||||
Add one small style-appropriate icon if suitable.
|
||||
Maintain single-line text.
|
||||
|
||||
---
|
||||
|
||||
## Settings Button
|
||||
Use the standard secondary button appearance.
|
||||
Use one settings icon naturally derived from {visual_style}.
|
||||
Maintain consistent proportions, spacing, and hierarchy.
|
||||
|
||||
---
|
||||
|
||||
## Exit Game Button
|
||||
Use the standard secondary button appearance.
|
||||
Use red only as the danger indicator.
|
||||
Red should remain an accent rather than the dominant color.
|
||||
Add one style-appropriate exit icon if suitable.
|
||||
|
||||
---
|
||||
|
||||
## Player Cards
|
||||
Create an upper-left player information system for {character_count} players.
|
||||
The appearance follows {visual_style}.
|
||||
Card materials, borders, corners, textures, shadows, decorations, and icons are automatically derived from the selected style.
|
||||
Each player card contains:
|
||||
• Player label
|
||||
• Player nickname
|
||||
• READY status
|
||||
Player names:
|
||||
PLAYER 1
|
||||
{player1_name}
|
||||
PLAYER 2
|
||||
{player2_name}
|
||||
READY always uses the functional accent color.
|
||||
The cards should remain highly readable and visually connected to the rest of the interface.
|
||||
|
||||
---
|
||||
|
||||
## Icon System
|
||||
Derive all icons from {visual_style}.
|
||||
Use one unified icon family throughout the entire interface.
|
||||
Possible icon languages include:
|
||||
• minimal outline
|
||||
• solid filled
|
||||
• geometric
|
||||
• comic
|
||||
• pixel
|
||||
• engraved
|
||||
• carved
|
||||
• painted
|
||||
• brush
|
||||
• rune
|
||||
• glyph
|
||||
• holographic
|
||||
• crystal
|
||||
• paper-cut
|
||||
• origami
|
||||
• embossed
|
||||
• low-poly
|
||||
• flat design
|
||||
• stylized 3D
|
||||
All icons must share:
|
||||
• the same artistic style
|
||||
• the same material language
|
||||
• the same perspective
|
||||
• the same lighting treatment
|
||||
• the same stroke language
|
||||
• the same color hierarchy
|
||||
Icons should remain simple, recognizable, and functional.
|
||||
Each interface block may contain at most one horizontal row of icons.
|
||||
No wrapping.
|
||||
No vertical stacking.
|
||||
Icons should support the interface rather than dominate it.
|
||||
|
||||
---
|
||||
|
||||
## Typography
|
||||
Typography hierarchy remains fixed.
|
||||
Typography appearance follows {visual_style}.
|
||||
The selected style controls:
|
||||
• font family
|
||||
• decorative treatment
|
||||
• texture
|
||||
• material
|
||||
• outline
|
||||
• lighting
|
||||
• shadow
|
||||
• wear pattern
|
||||
Maintain:
|
||||
• excellent readability
|
||||
• strong visual hierarchy
|
||||
• single-line menu buttons
|
||||
• consistent spacing
|
||||
• high contrast
|
||||
Button width should always adapt to text length.
|
||||
Never allow menu titles to wrap onto multiple lines.
|
||||
|
||||
---
|
||||
|
||||
## Game Title
|
||||
Display the game title:
|
||||
{game_title}
|
||||
The title should become the primary branding element of the menu.
|
||||
Its material, decorative style, typography, and rendering all follow {visual_style}.
|
||||
Avoid imitating existing commercial game logos.
|
||||
Create an original title treatment inspired by the selected style.
|
||||
Keep the title highly readable.
|
||||
Avoid random symbols or meaningless letters.
|
||||
|
||||
---
|
||||
|
||||
## Interaction States
|
||||
Represent the interface as an active game menu.
|
||||
Buttons may display appropriate interaction feedback naturally belonging to {visual_style}.
|
||||
Possible interaction feedback includes:
|
||||
• hover
|
||||
• pressed state
|
||||
• magical glow
|
||||
• energy pulse
|
||||
• holographic highlight
|
||||
• carved illumination
|
||||
• ink spread
|
||||
• brush bloom
|
||||
• metallic reflection
|
||||
• paper emboss
|
||||
• animated border
|
||||
• pixel flashing
|
||||
Only use interaction effects that naturally belong to the selected style.
|
||||
Avoid forcing modern UI effects into incompatible artistic styles.
|
||||
|
||||
---
|
||||
|
||||
## Rendering
|
||||
Generate a polished AAA-quality commercial game menu artwork.
|
||||
Characters, environment, UI, typography, icons, and decorative elements should feel like one cohesive production.
|
||||
Maintain premium rendering quality.
|
||||
Strong visual hierarchy.
|
||||
Professional commercial presentation.
|
||||
High readability.
|
||||
Balanced composition.
|
||||
Clean finish.
|
||||
|
||||
---
|
||||
|
||||
## Negative Constraints
|
||||
No split screen.
|
||||
No duplicated characters.
|
||||
No identity swap.
|
||||
No loss of identity anchors. No photorealistic face unless {visual_style} explicitly requires photorealism. No realistic photo face pasted onto a stylized body. No mismatched face style.
|
||||
No hairstyle changes.
|
||||
No unreadable UI.
|
||||
No distorted buttons.
|
||||
No overlapping interface.
|
||||
No random text.
|
||||
No random symbols.
|
||||
No watermark.
|
||||
No copied game logo.
|
||||
No branded interface.
|
||||
No inconsistent icon styles.
|
||||
No mixed UI materials.
|
||||
No excessive decoration.
|
||||
No overwhelming background.
|
||||
No visual clutter.
|
||||
No broken layout.
|
||||
No low readability.
|
||||
|
||||
---
|
||||
|
||||
## Medium Lock
|
||||
Render as a polished commercial game key art combined with a professional console game main menu mockup.
|
||||
The interface remains fixed.
|
||||
Only the artistic appearance changes according to {visual_style}.
|
||||
@@ -1,249 +0,0 @@
|
||||
# H3 Video Prompt Template
|
||||
|
||||
## Portable adaptation
|
||||
|
||||
This is a creative planning template. User requirements override its default
|
||||
15-second timeline, body proportions, equipment, environment, ratio, and palette.
|
||||
Retiming must preserve readable event order. Use only supplied reference assets:
|
||||
omit `{height_ref}` or identity-image bindings when absent and replace their role
|
||||
with text design constraints. Compile the filled plan into the H3 base or Ref2VA
|
||||
format before rendering; preserve exact visible UI strings in the user's language.
|
||||
|
||||
Use available tools and the actual backend's parameter schema. Audio directions
|
||||
describe desired content rather than asserting that a particular API supports it.
|
||||
|
||||
## Template completion
|
||||
|
||||
Resolve the palette variables `{background_color}`, `{ui_color}`, `{text_color}`,
|
||||
`{functional_accent_color}`, and `{power_accent_color}` from the selected style.
|
||||
Keep the five values distinct enough for readable UI contrast. Use the power
|
||||
accent for danger, exit, and warning states unless the user supplies a separate
|
||||
semantic-color requirement.
|
||||
|
||||
Fill visible copy in the user's requested language. The following English values
|
||||
are defaults only, not mandatory output:
|
||||
|
||||
- `{player1_label}` / `{player2_label}`: `PLAYER 1` / `PLAYER 2`
|
||||
- `{ready_label}`: `READY`
|
||||
- `{start_label}`, `{continue_label}`, `{settings_label}`, `{exit_label}`:
|
||||
`START NEW GAME`, `CONTINUE`, `SETTINGS`, `EXIT GAME`
|
||||
- `{player1_equipment_label}`: `RIGHT ARM EQUIPMENT`
|
||||
- `{player1_option_a}` / `{player1_option_b}`: `PHANTOM GRIP` / `CHRONOS CLAW`
|
||||
- `{player2_equipment_label}`: `ARMAMENT CUSTOMIZATION`
|
||||
- `{hand_label}`, `{forearm_label}`, `{elbow_label}`, `{upper_arm_label}`:
|
||||
`HAND`, `FOREARM`, `ELBOW`, `UPPER ARM`
|
||||
- `{confirm_label}` / `{loading_label}`: `CONFIRM CONFIG` / `LOADING`
|
||||
|
||||
Before compiling the rendering prompt, search the filled template for every
|
||||
remaining token enclosed in braces. Replace required values. For an absent optional
|
||||
reference, remove its binding and rewrite every dependent clause as a text-only
|
||||
constraint. Do not dispatch a prompt while any template token remains.
|
||||
|
||||
## Prompt principle
|
||||
Use the same method as the GPT confirmation-image prompt:
|
||||
|
||||
**Fixed video event framework + dynamic user-style fill + locked character identity + palette-linked UI system.**
|
||||
|
||||
The video prompt must not blindly preserve the source prompt's default style. It must preserve the timeline, UI events, player positions, equipment logic, and text structure, while dynamically rewriting visual treatment, palette, character rendering, lighting, UI surface, icons, and city-world style according to the user's selected style.
|
||||
|
||||
## Priority order
|
||||
1. User-selected style: {visual_style}
|
||||
2. Confirmed image / UI reference: {ui_ref}
|
||||
3. PLAYER 1 identity reference: {player1_ref}
|
||||
4. PLAYER 2 identity reference: {player2_ref}
|
||||
5. Height/body comparison reference: {height_ref}
|
||||
6. Fixed source video event framework
|
||||
|
||||
## Reference roles
|
||||
- {ui_ref}: confirmed first image. Use it to lock UI layout, menu hierarchy, color system, typography scale, button structure, character-game integration, and overall composition logic.
|
||||
- {player1_ref}: PLAYER 1 identity anchor. Lock exact face, hairstyle, glasses if present, facial proportions, body identity, and nickname mapping to {player1_name}.
|
||||
- {player2_ref}: PLAYER 2 identity anchor. Lock exact face, hairstyle, facial proportions, body identity, and nickname mapping to {player2_name}.
|
||||
- {height_ref}: body comparison anchor. Lock the visible contrast between the two players and prevent identical body proportions.
|
||||
|
||||
## Global style baseline
|
||||
The overall art direction must prioritize the user's selected {visual_style}.
|
||||
Preserve these fixed qualities: a game main-menu interface, premium game-trailer
|
||||
finish, deep integration between the UI and characters, modern commercial game
|
||||
UI design, strong visual impact, clean composition, and restrained decoration.
|
||||
Derive the remaining visual treatment from {visual_style}: character rendering,
|
||||
expressions, costume language, color system, lighting temperature, UI materials,
|
||||
button icons, typography texture, and the world revealed after loading.
|
||||
|
||||
## Palette system
|
||||
Derive the complete video palette from {visual_style}, with linked UI colors:
|
||||
|
||||
- Use {background_color} as the primary background and world color.
|
||||
- Use {ui_color} as the main UI color.
|
||||
- Use {text_color} as the text color.
|
||||
- Use {functional_accent_color} as PLAYER 1's functional accent color.
|
||||
- Use {power_accent_color} as PLAYER 2's power accent and for danger, exit,
|
||||
and warning states.
|
||||
- Limit the complete palette to five colors.
|
||||
- Use a vivid, modern, high-contrast color language consistent with
|
||||
{visual_style}.
|
||||
|
||||
Menus, equipment panels, player cards, buttons, the HUD, loading bars, icons,
|
||||
and text must share this color system. Do not introduce unrelated colors.
|
||||
|
||||
## Character identity and style lock
|
||||
PLAYER 1 follows the facial identity in {player1_ref}. Preserve the face,
|
||||
facial proportions, hairstyle, glasses when present, personal identity, and the
|
||||
mapping to the nickname {player1_name}. Adapt expression, rendering, clothing,
|
||||
and mechanical equipment to {visual_style}. PLAYER 1 always stays on the left,
|
||||
with a taller, slender, agile build. Equipment uses the functional accent color,
|
||||
and the mechanical claw is light, narrow, and flexible.
|
||||
|
||||
PLAYER 2 follows the facial identity in {player2_ref}. Preserve the face,
|
||||
facial proportions, hairstyle, personal identity, and the mapping to the
|
||||
nickname {player2_name}. Adapt expression, rendering, clothing, and mechanical
|
||||
equipment to {visual_style}. PLAYER 2 always stays on the right, with a shorter,
|
||||
broader, power-oriented build. Equipment uses {power_accent_color}, and the
|
||||
mechanical fist is broad, heavy, and visibly weighty.
|
||||
|
||||
Do not swap character identities, merge faces, swap nicknames, or converge the
|
||||
two body types.
|
||||
|
||||
## Fixed timeline framework
|
||||
|
||||
### [0-2 seconds] - Two-player main menu
|
||||
|
||||
Composition and camera: A high-angle extreme wide shot follows the composition
|
||||
logic of {ui_ref}. The camera tilts slightly downward and slowly pushes in.
|
||||
|
||||
Visuals: PLAYER 1 ({player1_name}) and PLAYER 2 ({player2_name}) sit side by
|
||||
side at the center. PLAYER 1 remains on the left and PLAYER 2 on the right. They
|
||||
look up toward the camera with only natural breathing, blinking, and subtle body
|
||||
movement.
|
||||
|
||||
UI: The upper-left player card displays `{player1_label}`, `{player1_name}`, and
|
||||
`{ready_label}` exactly as filled. The second card in the upper-center or
|
||||
upper-left two-card system displays `{player2_label}`, `{player2_name}`, and
|
||||
`{ready_label}` exactly as filled. The vertical menu on the right displays
|
||||
`{start_label}`, `{continue_label}`, `{settings_label}`, and `{exit_label}`
|
||||
exactly as filled. `{continue_label}` is the visual center and primary
|
||||
highlighted button.
|
||||
|
||||
Dynamic style fill: Adapt the menu background, ground texture, button shapes,
|
||||
icons, typography, borders, glow, and sticker treatment to {visual_style}, while
|
||||
preserving the layout and hierarchy in {ui_ref}.
|
||||
|
||||
Sound: Menu ambience, subtle UI-hover sounds, and a low electronic atmosphere
|
||||
before the click. Ignore this direction when generating silent video.
|
||||
|
||||
### [2-4 seconds] - PLAYER 1 right-arm configuration
|
||||
|
||||
Composition and camera: In a medium shot, the camera pushes smoothly from the
|
||||
main menu toward PLAYER 1's right arm. PLAYER 2 remains visible and stable in the
|
||||
background.
|
||||
|
||||
UI: The right-side menu contracts and slides away. A panel with functional-accent
|
||||
guide lines slides in from the left and displays `{player1_label}` and
|
||||
`{player1_equipment_label}` exactly as filled. The equipment list first
|
||||
highlights `{player1_option_a}`, then moves the selection to
|
||||
`{player1_option_b}`.
|
||||
|
||||
Action: PLAYER 1's right cuff opens automatically and a lightweight mechanism
|
||||
unfolds beneath the forearm. The fingers spread as long claw-like joints slide
|
||||
into position and lock one by one, briefly revealing fine wiring, miniature
|
||||
pistons, and metal connectors. Functional-accent LEDs illuminate sequentially
|
||||
when configuration completes.
|
||||
|
||||
Dynamic style fill: Adapt the mechanism, panel material, icons, lines, locking
|
||||
animation, and light effects to {visual_style}. Keep the result light, precise,
|
||||
and flexible, proportioned to PLAYER 1 without changing the face, hairstyle, or
|
||||
main clothing design.
|
||||
|
||||
Sound: Precise mechanical unfolding, light UI switching tones, and small locking
|
||||
clicks.
|
||||
|
||||
### [4-7 seconds] - PLAYER 2 heavy-arm configuration
|
||||
|
||||
Composition and camera: A medium shot tracks smoothly between the players and
|
||||
arcs toward PLAYER 2's left side. PLAYER 1 remains in the background, quietly
|
||||
examining the configured mechanical hand.
|
||||
|
||||
UI: A {power_accent_color} panel slides in and displays `{player2_label}` and
|
||||
`{player2_equipment_label}` exactly as filled. Its grid contains `{hand_label}`,
|
||||
`{forearm_label}`, `{elbow_label}`, and `{upper_arm_label}`, with the selection
|
||||
moving quickly but legibly through all four.
|
||||
|
||||
Action: PLAYER 2's left sleeve opens in sections. Heavy forearm plates spring
|
||||
outward, the previous components detach, and new armor slides into place along
|
||||
guide rails. A thick mechanical bearing replaces the elbow joint, and the broad
|
||||
mechanical hand reassembles and locks. Thick wiring, hydraulic pistons, and a
|
||||
dark metal skeleton appear briefly during the replacement. A restrained warm
|
||||
indicator illuminates as each component locks.
|
||||
|
||||
Dynamic style fill: Adapt the heavy arm, UI panel, component icons, materials,
|
||||
and light effects to {visual_style}. Keep it broad, weighty, and powerful, in
|
||||
clear contrast with PLAYER 1's lightweight claw.
|
||||
|
||||
Sound: Low motor movement, heavy mechanical engagement, and weighty locking
|
||||
feedback.
|
||||
|
||||
### [7-8.5 seconds] - Shared configuration confirmation
|
||||
|
||||
Composition and camera: Pull back smoothly to a medium two-player composition,
|
||||
with PLAYER 1 on the left and PLAYER 2 on the right.
|
||||
|
||||
UI: The two equipment panels converge at center to form a shared button that
|
||||
displays `{confirm_label}` exactly as filled. Adapt its border, glow, icon, and
|
||||
sticker treatment to {visual_style}, while keeping the hierarchy and text legible.
|
||||
|
||||
Action: The cursor clicks the button. A functional-accent energy pulse travels
|
||||
through PLAYER 1's mechanical claw, while a {power_accent_color} pulse travels
|
||||
through PLAYER 2's fist. All UI panels contract inward and disappear. Both
|
||||
players uncross their legs and adjust their posture: PLAYER 1 lifts one knee
|
||||
lightly and articulates the claw fingers in sequence; PLAYER 2 plants one foot
|
||||
firmly and slowly closes the heavy fist.
|
||||
|
||||
Sound: Confirmation tone, two-color energy pulses, and UI contraction.
|
||||
|
||||
### [8.5-10 seconds] - Shared world loading
|
||||
|
||||
Composition and camera: A wide shot reveals a shared loading bar at the bottom.
|
||||
|
||||
UI: The loading bar displays `{loading_label}` exactly as filled and fills
|
||||
rapidly from 0% to 100%. Its left half uses {functional_accent_color} and its
|
||||
right half uses {power_accent_color}. Adapt the HUD and loading-bar shape,
|
||||
border, texture, and typography to {visual_style}, while keeping them legible.
|
||||
|
||||
Environment transformation: Continuously transform the confirmed
|
||||
{visual_style} menu background into a game world in the same style. Menu strips,
|
||||
color blocks, and patterns become corresponding roads, structures, and world
|
||||
elements. Preserve the confirmed palette rather than forcing a yellow-and-black
|
||||
or cyberpunk city.
|
||||
|
||||
Key constraint: The transformation remains continuous and natural, with no hard
|
||||
cut, smoke occlusion, or identity change.
|
||||
|
||||
Sound: Rising loading tone and an ambience transition from menu to game world.
|
||||
|
||||
### [10-15 seconds] - Two-player world entry
|
||||
|
||||
Composition and camera: Move from an extreme wide shot into a third-person
|
||||
tracking view. At 100% loading, both characters stand simultaneously. The camera
|
||||
descends smoothly and arcs behind them into a stable two-player cooperative view.
|
||||
|
||||
World style: Generate the complete world from {visual_style}. Preserve the game
|
||||
opening structure: dense buildings, roads, signal lights, signs, crowds or other
|
||||
moving background elements, passing vehicles, overhead wires, industrial pipes,
|
||||
and a distant skyline. Materials, architecture, signage, lighting, and motion
|
||||
must follow {visual_style}. Do not let the source template's cyberpunk default
|
||||
override the user unless the selected style is cyberpunk.
|
||||
|
||||
Character relationship: Clearly show the players from behind and preserve their
|
||||
body contrast. PLAYER 1 is on the left, tall and slender, with the lightweight
|
||||
claw hanging naturally. PLAYER 2 is on the right, shorter and broader, with the
|
||||
heavy fist slightly raised. PLAYER 1 steps forward first, PLAYER 2 follows, and
|
||||
they enter the street side by side.
|
||||
|
||||
HUD: The HUD fades in. A mini-map appears in the upper right. Separate status
|
||||
bars labeled `{player1_name}` and `{player2_name}` appear in the lower left.
|
||||
The first uses {functional_accent_color}; the second uses {power_accent_color}.
|
||||
A shared objective marker appears in the street ahead.
|
||||
|
||||
Sound: City ambience, distant vehicles, footsteps, and a HUD fade-in cue.
|
||||
|
||||
## Negative constraints
|
||||
No extra player, duplicated character, identity or username swapping, merged bodies, unintended face/hair changes, missing player, unrequested split screen or cuts, random camera shake, floating body parts, gruesome dismemberment, unwanted weapons, style-incompatible neon, unreadable UI, random letters, misspelled usernames, extra menu options, copied game logo, or watermark. Preserve the user's chosen character designs and palette.
|
||||
@@ -1,42 +0,0 @@
|
||||
---
|
||||
name: h3-prompt-writing
|
||||
description: Write MiniMax H3 video generation prompts for T2VA, I2VA, FL2VA, L2VA, and Ref2VA. Use when rewriting multimodal requests into H3 prompt structures, composing integrated_multimodal_description, overall_soundscape, and non_diegetic_music, aligning keyframes, or defining reference labels for images, videos, and audio.
|
||||
metadata:
|
||||
compatibility: Portable to any agent that can read local files — no external API calls, MiniMax Hub tools, or proprietary runtime required. The agents/openai.yaml file only adds optional ChatGPT/Codex UI metadata; it does not restrict the skill to OpenAI agents.
|
||||
---
|
||||
|
||||
# H3 Prompt Writing
|
||||
|
||||
## Workflow
|
||||
|
||||
0. Read [portable-workflow.md](references/portable-workflow.md) for scope, available tools, durable artifacts, and generation handoff. For prompt-only work, no generation tools are required.
|
||||
1. Identify the input mode: T2VA, I2VA, FL2VA, L2VA, or full-reference Ref2VA.
|
||||
2. For base text/keyframe modes, read `references/base-format.md` and follow its final prompt structure.
|
||||
3. For full-reference mode, read `references/ref-format.md` and follow its six-section rewrite format.
|
||||
4. Preserve the exact field names, section order, labels, and timing notation from the selected guide.
|
||||
|
||||
## Base Modes
|
||||
|
||||
- T2VA: build the full audiovisual timeline from text.
|
||||
- I2VA: start from the first frame and develop forward from it.
|
||||
- FL2VA: describe the continuous path between the first and last frames.
|
||||
- L2VA: infer a plausible opening and converge to the supplied last frame.
|
||||
|
||||
Use `integrated_multimodal_description`, `overall_soundscape`, and `non_diegetic_music` in the order shown in `references/base-format.md`.
|
||||
|
||||
## Full-Reference Mode
|
||||
|
||||
Ref2VA rewrites use `subject_definitions`, `summary`, `retention_analysis`, `detailed_description`, `overall_soundscape`, and `non_diegetic_music` in that order. Reference labels stay consistent across all sections.
|
||||
|
||||
Read `references/ref-format.md` for label rules and retention analysis.
|
||||
|
||||
## Output Rules
|
||||
|
||||
- Write rewrite sections in English; preserve dialogue, lyrics, and visible scene text in their original language.
|
||||
- Describe each shot by composition, subjects, environment, actions, camera, sound, and the exact point where referenced content appears.
|
||||
- Avoid plot summaries, unresolved reference labels, and timing that does not match the requested duration.
|
||||
## Tips for Better Results
|
||||
- Match the description to the effective duration of the current clip or generation window. The upstream guides describe 4–15-second clips; longer work requires a verified backend extension or an explicit assembly plan, as described in the portable workflow.
|
||||
- Keep reference labels consistent (e.g. `<Picture 1>`, `<Video 1>`, `<Audio 1>`) across every section.
|
||||
- Prefer concrete visual and audio details over abstract words like "cinematic" or "beautiful".
|
||||
- When using keyframes (I2VA / FL2VA / L2VA), clearly state how the first and/or last frame connects to the timeline.
|
||||
@@ -1,4 +0,0 @@
|
||||
interface:
|
||||
display_name: "H3 Prompt Writing"
|
||||
short_description: "Write structured H3 prompts for each input mode"
|
||||
default_prompt: "Use $h3-prompt-writing to prepare the requested H3 prompts or video with the available tools."
|
||||
@@ -1,86 +0,0 @@
|
||||
# H3 base-mode prompt format
|
||||
|
||||
Use this reference for T2VA, I2VA, FL2VA, and L2VA prompt writing. It is a
|
||||
compact, repository-authored description of the required output contract, not a
|
||||
copy of an upstream prompt guide.
|
||||
|
||||
## Select the anchor mode
|
||||
|
||||
- T2VA has no image anchor. Build the audiovisual sequence from the brief.
|
||||
- I2VA binds `<Picture 1>` to the first frame at 0.00 seconds.
|
||||
- FL2VA binds `<Picture 1>` to the first frame and `<Picture 2>` to the final
|
||||
frame. Describe the observable path between them.
|
||||
- L2VA binds `<Picture 1>` to the final frame. Infer a compatible opening and
|
||||
converge to the supplied image.
|
||||
|
||||
For an image-anchored mode, put a plain-English alignment statement before the
|
||||
structured fields. State each picture label, its shot, and its exact target time.
|
||||
The ending time must match the effective clip duration, formatted to two decimal
|
||||
places. Do not invent a picture label when no image was supplied.
|
||||
|
||||
## Emit three fields in order
|
||||
|
||||
```text
|
||||
integrated_multimodal_description: [Shot 1] ...
|
||||
|
||||
overall_soundscape: ...
|
||||
|
||||
non_diegetic_music: ...
|
||||
```
|
||||
|
||||
`integrated_multimodal_description` is the playback-ordered visual and diegetic
|
||||
audio plan. Establish the initial style, framing, subjects, environment, and
|
||||
spatial relationships, then describe actions and state changes. For keyframe
|
||||
modes, explicitly preserve or approach the relevant composition, identity,
|
||||
clothing, colors, objects, and geometry.
|
||||
|
||||
`overall_soundscape` summarizes ambience, physical effects, and non-verbal human
|
||||
sound. Do not repeat dialogue or music already described elsewhere. Use `N/A`
|
||||
only for explicitly silent output.
|
||||
|
||||
`non_diegetic_music` describes music heard by the audience but not the characters.
|
||||
Specify audible properties such as instrumentation, tempo, rhythm, dynamics, and
|
||||
ending behavior. Put source-visible music, singing, radio, or instrument playing
|
||||
in the main description. Use `N/A` when no score is wanted.
|
||||
|
||||
## Timeline notation
|
||||
|
||||
- Start the opening with `[Shot 1]` and no timestamp.
|
||||
- Mark each later cut with a sequential label and a strictly increasing time,
|
||||
for example `[Shot 2] At 00:03.500, ...`.
|
||||
- Use a new shot for an actual edit or meaningful viewpoint/time change. Keep a
|
||||
continuous action or morph in one shot and describe its internal timing.
|
||||
- Describe camera motion as an action: type, direction, useful amplitude, and
|
||||
speed. Do not append a disconnected list of camera keywords.
|
||||
- Keep every cut and event within the requested clip duration.
|
||||
|
||||
## Speech, text, and sound
|
||||
|
||||
Give each speaking or singing source a stable ID such as `(S1)` or `(S2)` and
|
||||
reuse it across shots. Put only the language tag and literal words inside the
|
||||
dialogue wrapper:
|
||||
|
||||
```text
|
||||
The courier (S1) says softly: <d>[English] The gate is open.</d>
|
||||
```
|
||||
|
||||
Preserve supplied dialogue, lyrics, punctuation, and visible text verbatim. State
|
||||
when a voice is off-screen and keep the visible character's lips closed when that
|
||||
is required. If a line spans a cut, describe the audio continuity on both sides
|
||||
of the cut. Place visible signs, labels, and interface copy in double quotes.
|
||||
|
||||
Tie diegetic sounds to the event that produces them. Avoid describing the same
|
||||
sound as both diegetic and non-diegetic.
|
||||
|
||||
## Anchor-specific checks
|
||||
|
||||
- I2VA: begin from the supplied frame rather than merely borrowing its style.
|
||||
- FL2VA: describe intermediate motion; do not provide two unrelated still-image
|
||||
descriptions. Reach Picture 2 at the actual end time.
|
||||
- L2VA: keep the final picture assigned to the last shot, not automatically to
|
||||
Shot 1, and make the preceding action physically plausible.
|
||||
- All modes: reference labels must map to supplied assets and remain stable.
|
||||
|
||||
Before delivery, verify the field order, shot numbering, timestamps, reference
|
||||
labels, dialogue fidelity, requested language, duration, and absence of unresolved
|
||||
template variables.
|
||||
@@ -1,121 +0,0 @@
|
||||
# Portable H3 production workflow
|
||||
|
||||
This is the runtime contract for the nine H3 skills in this repository. Read it
|
||||
alongside the selected creative skill. The bundle uses local files and available
|
||||
tools; no proprietary canvas, choice-card widget, planner, or executor is required.
|
||||
|
||||
## Scope and decisions
|
||||
|
||||
- Resolve the requested deliverable first: prompt, storyboard, assets, or finished
|
||||
video. A prompt-only request ends with the prompt; it does not start generation.
|
||||
- Reuse the user's duration, language, style, model, audio policy, supplied assets,
|
||||
and existing authorization. State reasonable defaults for optional gaps. Ask
|
||||
only about missing inputs or choices that materially affect the requested work.
|
||||
- Creative checkpoints are reviewable artifacts, not mandatory repeated approvals.
|
||||
Honor an explicit request to approve a preview before rendering. When the user
|
||||
has authorized the complete workflow or delegated creative choices, show the
|
||||
plan and continue within that scope without requiring a special UI control.
|
||||
- A skill never grants permission to publish, send messages, spend outside an
|
||||
authorized budget, or change models against the user's choice. User constraints
|
||||
override a preset's style, language, shot count, timing, or audio defaults.
|
||||
|
||||
## Tools and durable artifacts
|
||||
|
||||
1. Inspect the actual available tools, existing scripts, endpoint documentation,
|
||||
and supplied media. Do not invent a tool name, model parameter, or returned path.
|
||||
2. Write plans and prompts to the user-selected project folder. If none is given,
|
||||
use a new named folder under `outputs/`, outside the skill package. Keep useful
|
||||
artifacts such as `brief.md`, `storyboard.md`, `prompts/`, `references/`,
|
||||
`clips/`, and `final/`; create only those needed for this deliverable.
|
||||
3. Record each reference's role and path: identity, environment, style, typography,
|
||||
first/last frame, or audio. Tie generated clips to the prompt revision, input
|
||||
assets, model, settings, and output paths. Update downstream references when an
|
||||
asset changes; retain previous outputs instead of overwriting user files.
|
||||
4. Use available image generation/editing tools for requested raster assets. Use
|
||||
an available video endpoint or local service for clips, and available audio
|
||||
tooling for requested standalone speech/music. For assembly and inspection,
|
||||
use an installed editor or tools such as FFmpeg, ffprobe, and OpenCV. Follow the
|
||||
host's tool policies, including its preferred image generation mechanism.
|
||||
5. Independent tool jobs may run concurrently when resources permit; this skill
|
||||
does not require separate agents or delegated generation.
|
||||
6. If a required capability is missing, finish the useful in-scope preparation:
|
||||
complete prompts, asset-role manifest, timeline, and verified request example
|
||||
where possible. State the missing capability and unfinished media step. Do not
|
||||
silently substitute another model, claim a render happened, or call a written
|
||||
plan a completed video. Resume generation when the capability becomes available.
|
||||
|
||||
## Compile the creative plan into H3 prompts
|
||||
|
||||
Read [base-format.md](base-format.md) for T2VA / I2VA / FL2VA / L2VA, or
|
||||
[ref-format.md](ref-format.md) for Ref2VA. These are writing modes, not guaranteed API
|
||||
task names. Check the selected backend's input mapping before sending a request.
|
||||
|
||||
- Base prompts use `integrated_multimodal_description`, `overall_soundscape`,
|
||||
`non_diegetic_music`, in that order, with the appropriate keyframe preamble.
|
||||
- Ref2VA uses `subject_definitions`, `summary`, `retention_analysis`,
|
||||
`detailed_description`, `overall_soundscape`, `non_diegetic_music`, in that order.
|
||||
- Every reference label must resolve to a supplied asset, in the same order used
|
||||
by the backend. Omit nonexistent optional references and their dependencies.
|
||||
- Keep the creative brief in the user's language. Structured H3 rendering prompts
|
||||
use English prose and exact field labels, while dialogue, lyrics, and visible
|
||||
text retain their requested language and wording. If the user explicitly wants
|
||||
only a same-language creative prompt, deliver that without silently replacing it
|
||||
with English. Compile a separate rendering prompt when generation is requested.
|
||||
- `[Shot 1]` has no initial timestamp. Later shot labels mark actual cuts. For a
|
||||
continuous morph, describe timed actions inside one shot rather than turning
|
||||
each form into a new shot. Express camera direction, speed, and useful amplitude.
|
||||
- Separate image style from character identity. A style reference must not inject
|
||||
its people, scenery, layout, typography, texture, or lighting into unrelated
|
||||
shots. For literal text fidelity, inspect the rendered result; a prompt is not
|
||||
proof that the exact lettering or beat synchronization was achieved.
|
||||
|
||||
## Longer videos and audio
|
||||
|
||||
Use the actual backend's supported duration, dimensions, frame grid, reference
|
||||
count, audio modes, and model partition. Inspect this checkout's H3 implementation
|
||||
or serving documentation before building requests. Do not assume that a feature
|
||||
branch's long-video extension exists on main or on a hosted service.
|
||||
|
||||
For a verified vLLM-Omni Ref2VA continuation backend, `long_video=true` with
|
||||
`long_video_mode=continuation` can accept one `continuation_prompts` entry per
|
||||
planned window only when the Ref2VA request is executed with its local text
|
||||
encoder. Step execution and stages that use an external text encoder reject
|
||||
`continuation_prompts`; omit the field and use the supported per-step inputs in
|
||||
those configurations. Compute the window count from the aligned total frames,
|
||||
window size, and overlap. Each prompt uses local window time; track global
|
||||
shot/music time separately. Repeated overlap is guidance, not additional output
|
||||
duration. Do not use this mode on a task/partition that does not support it.
|
||||
|
||||
Otherwise split into supported clips and assemble them. Same-scene clips may use
|
||||
tail/head keyframe continuity when supported. Scene changes should have explicit
|
||||
new scene references; a previous scene's latent tail can preserve the wrong
|
||||
setting. Choose cuts, morphing, and transitions from the brief rather than imposing
|
||||
hard cuts whenever a work exceeds 15 seconds. Generation windows and visual shots
|
||||
are different: one window may contain cuts, and one shot may span multiple windows.
|
||||
|
||||
For supplied music, use the selected excerpt as the master timeline unless the
|
||||
user requests replacement. For native audio, write the musical/SFX direction in
|
||||
the prompt and verify that the backend actually returns audio. Do not invent a
|
||||
generic `generate_audio` parameter. If available, locked-source audio latents guide
|
||||
generation but the audio-VAE reconstruction is not bit-identical to the source.
|
||||
Use explicit final audio stream-copy when an exact original track is required.
|
||||
Prompted BPM is a target, not measured timing. Avoid automatically adding lyrics,
|
||||
voiceover, subtitles, or multiple competing scores to an instrumental request.
|
||||
|
||||
## Verification and bounded recovery
|
||||
|
||||
Inspect reference images before using them. After rendering, inspect representative
|
||||
frames, transitions, opening/ending states, text, identities, and scene continuity.
|
||||
Check audio by listening when possible; report any limit on audio review. Check
|
||||
duration, dimensions, fps, frame count, audio presence, and decode validity with
|
||||
media tools. If preserving audio exactly, compare extracted audio stream hashes.
|
||||
|
||||
Correct a demonstrated failure with a targeted change to the affected prompt,
|
||||
reference, or edit. Keep raw generation alongside edited/upscaled versions and
|
||||
label the difference. Use an agreed retry/resource budget; absent one, allow one
|
||||
targeted rerender, then report persistent defects and concrete options rather than
|
||||
entering an unbounded generation loop. Do not switch providers automatically.
|
||||
|
||||
Deliver actual artifact paths, the prompts/settings needed to repeat the work,
|
||||
and material visual/audio limitations. Distinguish requested behavior, observed
|
||||
model output, and post-production repairs.
|
||||
@@ -1,105 +0,0 @@
|
||||
# H3 full-reference prompt format
|
||||
|
||||
Use this reference for Ref2VA prompt writing. It is a compact,
|
||||
repository-authored description of the output contract, not a copy of an
|
||||
upstream prompt guide.
|
||||
|
||||
## Emit six fields in order
|
||||
|
||||
```text
|
||||
subject_definitions: ...
|
||||
|
||||
summary: ...
|
||||
|
||||
retention_analysis: ...
|
||||
|
||||
detailed_description: [Shot 1] ...
|
||||
|
||||
overall_soundscape: ...
|
||||
|
||||
non_diegetic_music: ...
|
||||
```
|
||||
|
||||
Write the structural prose in English. Preserve the requested language and exact
|
||||
wording for dialogue, lyrics, and visible text.
|
||||
|
||||
## Reference labels
|
||||
|
||||
Assign labels only to supplied content and keep every label stable across all six
|
||||
fields:
|
||||
|
||||
- `<Subject N>` identifies reusable visible content such as a person, object,
|
||||
environment, costume, interface, action, or visual treatment.
|
||||
- `<Picture N>` identifies an image used as a concrete frame, composition, or
|
||||
storyboard anchor. If an image only defines a subject, cite it in that subject's
|
||||
definition instead of inventing a separate target-frame role.
|
||||
- `<Video N>` identifies a source video being edited or continued, or a video
|
||||
whose temporal structure is referenced.
|
||||
- `<Audio N>` identifies a supplied audio signal that is copied or referenced.
|
||||
Audio and video label numbers are independent even when they came from one file.
|
||||
|
||||
In `subject_definitions`, give each tracked item a separate line. State its source,
|
||||
role, and the features that later sections must preserve or transform. Do not
|
||||
create labels for assets or roles that do not exist.
|
||||
|
||||
## Summary and retention
|
||||
|
||||
`summary` is one short paragraph describing the target and the principal reference
|
||||
relationships. Use only labels already defined. Distinguish these operations:
|
||||
|
||||
- a concrete image frame anchor;
|
||||
- generation guided by identity, style, action, camera, or audio characteristics;
|
||||
- direct editing of a source video;
|
||||
- continuation from a source video;
|
||||
- copying an audio signal;
|
||||
- referencing audio properties without copying the signal.
|
||||
|
||||
`retention_analysis` has one line per label and states where it is used and what
|
||||
changes. Use consistent machine-readable relationship values:
|
||||
|
||||
- Visual: `fully_preserved`, `partially_preserved`, `attribute_transfer`, or
|
||||
`weak_reference`.
|
||||
- Audio: `fully_copy`, `partially_copy`, `reference`, or `weak_reference`.
|
||||
|
||||
Explain the concrete retained and changed traits after the relationship value.
|
||||
Do not claim full preservation when identity, geometry, timing, or signal content
|
||||
is intentionally altered. Do not count newly requested target action as an
|
||||
unintended loss of reference fidelity.
|
||||
|
||||
## Detailed description
|
||||
|
||||
`detailed_description` is the playback-ordered rendering plan. For each shot,
|
||||
cover the current composition, subject appearance and position, environment,
|
||||
lighting, actions and state changes, camera behavior, diegetic sound, and the
|
||||
point at which each reference becomes relevant.
|
||||
|
||||
- Start with `[Shot 1]` and no timestamp.
|
||||
- Give later cuts sequential labels with strictly increasing times, such as
|
||||
`[Shot 2] At 00:03.500, ...`.
|
||||
- Place relevant reference labels next to the described subject, frame, motion,
|
||||
or sound; do not collect detached labels at the end.
|
||||
- Use stable `(S1)`, `(S2)`, and subsequent IDs for vocal sources. Put literal
|
||||
speech or lyrics inside `<d>[Language] ...</d>`.
|
||||
- Preserve supplied visible text verbatim and place it in double quotes.
|
||||
- Describe continuous transformations inside one shot unless an actual cut is
|
||||
intended. State camera direction, speed, and amplitude when useful.
|
||||
|
||||
For video editing, explicitly identify what stays from the source timeline and
|
||||
what is replaced. For continuation, distinguish the inherited opening state from
|
||||
new content. For style reference, transfer visual properties without accidentally
|
||||
copying people, scenery, layout, or typography.
|
||||
|
||||
## Audio fields
|
||||
|
||||
`overall_soundscape` summarizes ambience, physical action sounds, and non-verbal
|
||||
human sounds. `non_diegetic_music` describes score audible only to the audience.
|
||||
Keep dialogue and diegetic music in `detailed_description`. Use `N/A` only when
|
||||
the corresponding layer is intentionally absent.
|
||||
|
||||
If audio is copied, state whether the whole signal or only a range/layer is used.
|
||||
If it is referenced, describe the specific property—such as voice timbre, tempo,
|
||||
rhythm, or texture—without claiming signal reuse.
|
||||
|
||||
Before delivery, verify the six-field order, one-to-one label definitions,
|
||||
retention coverage, shot timing, dialogue fidelity, requested language, and
|
||||
absence of unresolved template variables.
|
||||
@@ -1,69 +0,0 @@
|
||||
---
|
||||
name: handdrawn-live-video-generator
|
||||
description: Write or generate a continuous live-action scene with a rough luminous hand-drawn entity that contacts real objects, morphs, and escapes a slightly delayed handheld camera. Use for gentle surreal mixed-media shorts, not flat-vector-only animation or horror clips.
|
||||
metadata:
|
||||
compatibility: Portable prompt writing with sibling h3-prompt-writing; actual generation requires an available video backend and any referenced media.
|
||||
---
|
||||
|
||||
# Handdrawn Live-Action Fusion
|
||||
|
||||
Read the [portable workflow](../h3-prompt-writing/references/portable-workflow.md).
|
||||
Keep the user's requested scope and language. A creative-prompt request delivers
|
||||
the prompt, without generating media. For a requested finished video, prepare
|
||||
the prompt and proceed under existing authorization using available tools.
|
||||
|
||||
## Creative contract
|
||||
|
||||
The default is a 15-second, 16:9 continuous scene in an everyday real space. A
|
||||
flat luminous drawn entity visibly contacts a real hand/object early, transforms
|
||||
as one traceable being, escapes through connected space, and the handheld camera
|
||||
reacts slightly late. Duration and ratio follow the user's explicit choices.
|
||||
|
||||
Use rough crayon, chalk, colored pencil, pastel, or brush strokes with uneven fill,
|
||||
frayed edges, line jitter, and frame-by-frame redraw. Keep the live-action setting
|
||||
physically grounded and the drawn presence visibly planar. Uniform neon tubes,
|
||||
plush characters, polished CG, and clean vector strokes would change this style.
|
||||
|
||||
Choose a fresh setting, entity, palette, contact mechanism, transformation chain,
|
||||
escape route, camera reaction, and emotional ending within the user's constraints.
|
||||
Do not ban an otherwise requested motif simply because an upstream example used
|
||||
it. The tone is warm, playful, and gently surprising rather than threatening.
|
||||
|
||||
## Timed action structure
|
||||
|
||||
For a 15-second brief, use these intervals as a starting structure:
|
||||
|
||||
- 0–3s: establish obvious real/drawn contact, such as landing on a palm or slipping
|
||||
around a finger. The real participant reacts.
|
||||
- 3–6s: begin a continuous morph while retaining a color smear, line, curve, or tail
|
||||
from the previous shape. Give it a playful action.
|
||||
- 6–10s: let it escape into an adjacent part of the same space. The camera pans,
|
||||
tilts, or moves only after the entity starts leaving its view.
|
||||
- 10–13s: add another readable interaction or surprise, keeping the same entity.
|
||||
- 13–15s: extend an earlier drawn motif across the space into a gentle larger
|
||||
transformation, then resolve with a small emotional/comic detail.
|
||||
|
||||
Retiming changes the intervals proportionally or reorganizes beats; do not leave
|
||||
15-second timestamps in a shorter prompt. Preserve a user's explicit final form
|
||||
instead of forcing the spatial finale when it conflicts with their ending.
|
||||
|
||||
## Prompt delivery and rendering
|
||||
|
||||
For a same-language creative prompt, begin with duration, ratio, real space, and
|
||||
the live-action/drawing fusion. Then describe phone-camera texture, the timed
|
||||
action intervals, drawn material, camera lag, specific exclusions, and ambient
|
||||
sound. Avoid unrelated headings or an essay when the user asked for copy-ready text.
|
||||
|
||||
For actual H3 generation, create a separate structured rendering prompt using the
|
||||
appropriate base/Ref2VA guide: English prose with literal dialogue/visible text in
|
||||
the requested language. Preserve the creative intent and actual media bindings.
|
||||
Do not send unsupported model settings or promise exact first-frame conditioning
|
||||
merely because an image is present.
|
||||
|
||||
## Review
|
||||
|
||||
Check clear early physical contact, one continuous entity, connected geography,
|
||||
retained shape motifs, delayed camera reaction, rough planar drawing, and a
|
||||
non-threatening ending. Avoid jump scares, sudden blackouts, menacing anatomy, and
|
||||
unrequested extra characters. Deliver the actual prompt/video and state any
|
||||
remaining contact, continuity, texture, or camera-timing defects.
|
||||
@@ -1,4 +0,0 @@
|
||||
interface:
|
||||
display_name: "Handdrawn Live-Action Fusion"
|
||||
short_description: "Direct drawn morphs and real-world interactions"
|
||||
default_prompt: "Use $handdrawn-live-video-generator to prepare the requested H3 prompts or video with the available tools."
|
||||
@@ -1,71 +0,0 @@
|
||||
---
|
||||
name: minimalist-product-ad-generator
|
||||
description: Plan, prompt, or produce minimalist product advertising shorts from product images, preserving variants, material, color, concise typography, and rhythmic camera motion. Use for product launches and ecommerce films, not presenter-led ads or complex UI demonstrations.
|
||||
metadata:
|
||||
compatibility: File-based agent workflow. Requires the sibling h3-prompt-writing package; rendering uses available image, video, audio, and editing tools.
|
||||
---
|
||||
|
||||
# Minimalist Product Ad Generator
|
||||
|
||||
Read the [portable workflow](../h3-prompt-writing/references/portable-workflow.md).
|
||||
Use [product direction](references/product-direction.md) when planning anchors,
|
||||
copy, and timed beats. Respect the requested output: prompt-only work needs no
|
||||
generation tool, while a finished film includes generation, inspection, and assembly.
|
||||
|
||||
## Resolve the product brief
|
||||
|
||||
Inspect supplied product photos, packaging, renders, and detail shots. Record the
|
||||
product category, main variant, actual body/accent colors, material finish, usable
|
||||
mechanisms, and requested copy. Preserve identity and product facts; visual polish
|
||||
does not authorize recoloring the product or inventing a feature.
|
||||
|
||||
Reuse the specified duration, ratio, language, and model. With no constraints,
|
||||
suggest a 10-second H3 film and one of white studio, dark rim light, brand color
|
||||
field, or light lifestyle staging. Ask for a missing product reference when exact
|
||||
identity is required; distinguish a speculative concept from an existing product.
|
||||
If the user delegates direction, record the adopted choices and proceed.
|
||||
|
||||
## Design the film
|
||||
|
||||
1. Choose a product-specific spine: launch reveal, mechanism interaction, or color
|
||||
family. Describe concrete actions such as a lid opening or a crown turning.
|
||||
2. Write short sensory/benefit copy grounded in supplied facts. Preserve the user's
|
||||
exact approved wording and language. For newly requested English copy, 3–5 words
|
||||
usually fit one readable line; do not force English on other-language campaigns.
|
||||
3. Plan three independent anchors: hero silhouette, material/mechanism detail, and
|
||||
final product/copy composition. Generate only the assets needed or requested.
|
||||
Do not combine them into a panel sheet that the video model might reproduce.
|
||||
4. Keep product color, light direction, ratio, and finish consistent across anchors.
|
||||
Original photos establish identity; generated anchors must pass that identity
|
||||
check before they can replace them as rendering references.
|
||||
5. Build the beat table from the reference. Include setup, action, impact, and a
|
||||
stable closing composition; vary shot distance only when it reveals something.
|
||||
|
||||
## Prompt and render
|
||||
|
||||
Compile each beat into the appropriate H3 fields using the sibling writing guide.
|
||||
Bind every anchor to its actual role. Include exact visible copy, entry/exit times,
|
||||
colors, and same-line text motion in the rendering prompt, not only in the plan.
|
||||
Keep typography in intentional negative space without covering the product. A
|
||||
copy-reference image does not mean its line must persist throughout the whole film.
|
||||
|
||||
For a short ad, prefer one coherent full-frame film when the backend supports it.
|
||||
Keep variants ordered: main variant first, supporting variants later, full family
|
||||
only when useful at the close. Avoid product walls, grids, or an anchor-sheet ending.
|
||||
|
||||
Use native audio when appropriate to the brief. A restrained electronic score may
|
||||
use crisp plucks, airy pads, kick/sub accents and tactile wooden percussion. Set
|
||||
tempo and energy from the user's direction. Generate a separate score only if
|
||||
requested or needed for a disclosed audio repair, using an available tool.
|
||||
|
||||
## Review and deliver
|
||||
|
||||
Verify product geometry, variant colors, functional action, final copy, aspect
|
||||
ratio, and closing readability. Listen for excessive bass, dropouts, and unfinished
|
||||
endings. If editing to supplied music, select a suitable continuous excerpt and
|
||||
align visual impacts to observed beats; do not claim synchronization from text alone.
|
||||
|
||||
For bad typography, first reduce simultaneous text or correct its reference. Exact
|
||||
lettering may need a disclosed overlay when consistent with the user's goal; keep
|
||||
the raw output. Deliver the film, prompt/beat plan, used copy and music excerpt,
|
||||
and any remaining product/text fidelity issues.
|
||||
@@ -1,4 +0,0 @@
|
||||
interface:
|
||||
display_name: "Minimalist Product Ads"
|
||||
short_description: "Direct product anchors, copy, motion, and audio"
|
||||
default_prompt: "Use $minimalist-product-ad-generator to prepare the requested H3 prompts or video with the available tools."
|
||||
@@ -1,52 +0,0 @@
|
||||
# Product anchors, typography, and beats
|
||||
|
||||
## Narrative choices
|
||||
|
||||
- Launch: negative space → hero product → material detail → real mechanism action
|
||||
→ supporting variants → stable product and copy.
|
||||
- Feature touch: stillness → interaction trigger → visible mechanism → detail
|
||||
magnification → useful result → closing.
|
||||
- Color family: main variant alone → supporting colors enter → ordered relation
|
||||
→ unified material finish → full set and copy.
|
||||
|
||||
## Independent anchor briefs
|
||||
|
||||
| Anchor | What it locks | Check |
|
||||
| --- | --- | --- |
|
||||
| Hero | Main variant, recognizable silhouette, attractive opening view | Match supplied body color and product structure |
|
||||
| Detail | Material, reflection, texture, visible mechanism | No invented ports, controls, or functionality |
|
||||
| Closing | Product position, negative space, typography treatment | Real copy, readable contrast, no panel/grid layout |
|
||||
|
||||
Each is a standalone full-frame image with the project ratio. Reference roles
|
||||
protect appearance and layout; they do not replace the storyboard or require three
|
||||
sequential static-image segments. Simplify the number of anchors if the backend
|
||||
cannot bind them all, and update the prompt labels accordingly.
|
||||
|
||||
## Typography
|
||||
|
||||
Use one readable line at a time, with a typeface appropriate to the brand. A clean
|
||||
semibold sans-serif is a useful default, not a requirement to copy a specific brand.
|
||||
For a two-part color treatment, use dark text on a light field or light text on a
|
||||
dark field, then emphasize the second part with a specific real product color.
|
||||
Check contrast rather than blindly coloring text like the product.
|
||||
|
||||
The first phrase may enter gently; the second follows while the first shifts
|
||||
slightly within the same line. Do not accidentally turn this into stacked lines.
|
||||
Use actual copy in the prompt and spell out its timing. Avoid arrows, separators,
|
||||
or placeholder marks that the model may reproduce. For a 10-second ad, one middle
|
||||
copy moment and one stable closing line are often enough. Omit copy when requested.
|
||||
|
||||
## Beat table
|
||||
|
||||
| Time range | Shot purpose | Visual lead and camera | Variant state | Exact copy | Text color/motion | Handoff | Rhythm |
|
||||
| --- | --- | --- | --- | --- | --- | --- | --- |
|
||||
|
||||
Use one main action per beat. For 5/10/15-second films, roughly 3–4/5–7/6–9 beats
|
||||
are planning suggestions, not required cuts. Record the ending product position
|
||||
and motion vector so the next beat can inherit them. Ensure the closing line has
|
||||
time to be read rather than using every second for motion.
|
||||
|
||||
If replacing music, keep useful SFX unless replacement of the whole track was
|
||||
requested. Use a continuous excerpt with a clear entry, stable middle, and suitable
|
||||
ending. A short tail fade may resolve a click; do not hide an incomplete phrase
|
||||
behind a long fade. Confirm the entire result remains audible.
|
||||
@@ -1,74 +0,0 @@
|
||||
---
|
||||
name: music-video-subtitle-generator
|
||||
description: Plan, write, audit, or produce music videos with coordinated music, imagery, camera motion, optional performance, and lyric typography. Use for beat-aware MVs and instrumental visual narratives, not ordinary subtitle cleanup or unrelated video edits.
|
||||
metadata:
|
||||
compatibility: File-based workflow with sibling h3-prompt-writing; audio analysis, generation, and final assembly use available tools and backends.
|
||||
---
|
||||
|
||||
# Music Video and Lyric Typography
|
||||
|
||||
Read the [portable workflow](../h3-prompt-writing/references/portable-workflow.md)
|
||||
and, when building a timeline, [music timeline](references/music-timeline.md).
|
||||
Deliver the requested scope: audit, prompt, storyboard, or complete video.
|
||||
|
||||
## Lock the musical and visual intent
|
||||
|
||||
Identify duration, ratio, output resolution, music source/excerpt, visual style,
|
||||
character/scene references, performance mode, and whether text is wanted. Treat a
|
||||
supplied song as the master track unless the user requests replacement. Preserve
|
||||
provided lyrics exactly; do not translate or rewrite them unasked. For instrumental
|
||||
work, do not invent singing, lyrics, a performer, or typography. Generate original
|
||||
lyrics only when requested or clearly within an authorized songwriting brief.
|
||||
|
||||
Measure supplied audio duration and inspect/listen for beat, phrase, breath, drop,
|
||||
and energy changes using available tools. Label estimated or planned timing when
|
||||
measurement is unavailable. If the requested excerpt exceeds the audio, resolve
|
||||
the discrepancy without silently stretching or looping it.
|
||||
|
||||
## Reference roles
|
||||
|
||||
- Character references control identity, silhouette, wardrobe, and presence.
|
||||
- Scene references control environment, time of day, spatial anchors, and lighting.
|
||||
- Typography references control letter treatment, layout, and motion, not people
|
||||
or scenery visible in those references.
|
||||
- Music references control the requested track or sound style; distinguish copying
|
||||
a supplied recording from generating similar instrumentation.
|
||||
|
||||
Do not apply a preset's grain, darkness, hard cuts, or constant close-ups to an
|
||||
incompatible brief. A bright nighttime scene remains night. A continuous flat
|
||||
vector morph keeps smooth color fields and uses elastic shape motion for accents.
|
||||
Use hard cuts, slow holds, or dissolves only when appropriate to the user's intent.
|
||||
|
||||
## Compose the timeline and H3 prompts
|
||||
|
||||
Map visual actions, camera moves, scene changes, and optional text to the music's
|
||||
global time. Build a causal visual progression instead of defaulting to one person
|
||||
singing throughout. Allow room to read a shape, gesture, or lyric before changing it.
|
||||
|
||||
For longer work, choose a supported continuation or clip-assembly approach through
|
||||
the portable workflow. Keep visual shot boundaries distinct from generation
|
||||
windows. Local window prompts must account for overlap without replaying the full
|
||||
previous action. A new scene may require a new reference rather than inheritance
|
||||
of the old scene. Record entry/exit states for each planned segment.
|
||||
|
||||
Compile the timeline into H3's base or Ref2VA fields, preserving dialogue, lyrics,
|
||||
and visible text verbatim. Include concrete instrumentation and rhythmic accents,
|
||||
but treat requested BPM and beat alignment as targets until checked in the output.
|
||||
|
||||
## Typography and finishing
|
||||
|
||||
When requested, give each shot one main readable text event. Keep words away from
|
||||
eyes and critical lip motion. Match sung text to the active phrase, and distinguish
|
||||
spatial typography from subtitle bars. Reduce text complexity if the model cannot
|
||||
render it, or use a disclosed overlay consistent with the requested deliverable.
|
||||
|
||||
For supplied audio, assemble clips to one global master track, maintaining phrase
|
||||
and lip continuity; avoid cuts inside a vowel. For native audio, inspect actual
|
||||
continuity across generation windows. Preserve useful SFX and avoid layered duplicate
|
||||
music. Retiming, grain, grading, interpolation, and upscaling are deliberate editing
|
||||
choices, not automatic finishing requirements.
|
||||
|
||||
Verify transformation/story order, identities, scene changes, requested text,
|
||||
musical continuity, timing, and full media decode. Label raw and processed versions.
|
||||
Deliver the final file and current prompt/timeline files, with measured properties
|
||||
and remaining limitations rather than claims of perfect synchronization.
|
||||
@@ -1,4 +0,0 @@
|
||||
interface:
|
||||
display_name: "Music Video Direction"
|
||||
short_description: "Plan music-driven scenes and optional lyric text"
|
||||
default_prompt: "Use $music-video-subtitle-generator to prepare the requested H3 prompts or video with the available tools."
|
||||
@@ -1,40 +0,0 @@
|
||||
# Global music and local video timing
|
||||
|
||||
Use this planning skeleton, omitting vocal/text fields when not requested:
|
||||
|
||||
```text
|
||||
Global: music/excerpt, duration, ratio, character/style/scene locks, cut policy.
|
||||
|
||||
Segment S01 — global start/end; local generation start/end
|
||||
Incoming state:
|
||||
Music cue: measured beat/phrase timestamp, or clearly labeled planned accent
|
||||
Vocal line and speaker (if present):
|
||||
Visible typography (if requested):
|
||||
Visual action and camera:
|
||||
Outgoing state and transition:
|
||||
References with exact roles:
|
||||
```
|
||||
|
||||
Keep one global music clock. For measured tempo B, the nominal quarter-note spacing
|
||||
is 60/B seconds, but real tracks may have pickups, swing, tempo changes, or inaccurate
|
||||
automatic detections. Validate important cut points against the waveform/listening.
|
||||
An onset detector also finds SFX and melodic attacks; it does not prove drum beats.
|
||||
|
||||
Choose a phrase-complete excerpt when possible. Track breaths, lyric pauses, and
|
||||
vowel continuity for a performance MV. For an instrumental morph, accent the
|
||||
physical preparation, transformation impact, and release rather than introducing
|
||||
unrequested cuts. The final shape or story payoff needs a readable ending.
|
||||
|
||||
For continuation, maintain a table of absolute output frames and local window
|
||||
times. Use the verified backend's native alignment and overlap behavior. Opening
|
||||
overlap continues the inherited state; new actions belong after it. Do not sum
|
||||
overlap twice or describe every window as a fresh opening shot.
|
||||
|
||||
For separately generated clips, use a previous tail frame only when the next shot
|
||||
continues the same scene and the backend supports that keyframe role. For an
|
||||
intentional cut, retain appropriate identity references and select the new scene.
|
||||
Preserve screen direction or a useful match element when it helps the transition.
|
||||
|
||||
When text is requested, track exact string, language, entry/exit time, screen
|
||||
region, and motion. Typography style cards must not donate their depicted characters
|
||||
or settings. Preserve user text; do not silently shorten lyrics to fit a template.
|
||||
@@ -1,69 +0,0 @@
|
||||
---
|
||||
name: paper-collage-explainer-generator
|
||||
description: Translate narration, ideas, or knowledge points into editorial halftone paper-collage metaphors, still prompts, storyboards, or stop-motion clips. Use for photographic cutout collage and tactile assembly, distinct from paper-puppet dioramas or generic cartoons.
|
||||
metadata:
|
||||
compatibility: File-based workflow with sibling h3-prompt-writing; media creation uses available image/video tools and an editor when needed.
|
||||
---
|
||||
|
||||
# Paper Collage Explainer Generator
|
||||
|
||||
Read the [portable workflow](../h3-prompt-writing/references/portable-workflow.md)
|
||||
and [collage design](references/collage-design.md) when constructing scenes.
|
||||
Honor prompt-only and asset-only requests as well as finished video requests.
|
||||
|
||||
## Meaning and media policy
|
||||
|
||||
Extract each source line's meaning, emotion, action verb, visual metaphor, and
|
||||
three to six readable object groups. Preserve the user's topic and wording as
|
||||
context; do not put narration on screen merely because it is available.
|
||||
|
||||
Default to tactile paper SFX, with no BGM, voiceover, or subtitles unless requested.
|
||||
An explainer does not inherently require speech. Existing user requests for those
|
||||
layers already establish the choice; do not ask again. A short 16:9 clip of roughly
|
||||
four seconds is a useful default for one assembly beat, subject to backend limits.
|
||||
|
||||
## Plan the image and motion together
|
||||
|
||||
Create a concise plan of metaphor, palette, object groups, assembly order, duration,
|
||||
and sound cues. For a series, keep paper texture, halftone treatment, keylines,
|
||||
shadows, and rhythmic language consistent; vary the metaphor and color field only
|
||||
where the meaning benefits. A user's no-text or no-music requirement applies to
|
||||
every downstream asset, prompt, clip, and edit.
|
||||
|
||||
Design each still as the completed final composition. Use black-and-white
|
||||
photographic halftone cutouts, selective colored cardstock, warm cream keylines,
|
||||
subtle fiber/torn edges, and soft physical layer shadows on a bold color field.
|
||||
Keep layered depth and negative space; avoid excessive age, dirt, wrinkles, or
|
||||
automatic brown/kraft backgrounds. Exact editable layers require a compositing
|
||||
workflow, not a claim that a generated raster video is a layered project.
|
||||
|
||||
Generate final-composition stills when requested/needed and inspect them. Follow
|
||||
any requested approval gate; otherwise continue within the authorized workflow.
|
||||
|
||||
## Animate the assembly
|
||||
|
||||
Start from a clean paper field matching the planned final color. Base pieces enter,
|
||||
then the main metaphor, then supporting objects. Each piece slides or pops in,
|
||||
bounces slightly, presses flat, pauses, and locks. End with a brief hold on the
|
||||
completed composition. Use stop-motion assembly rather than global fades, smooth
|
||||
digital drifting, fast spinning, or liquid morphs unless explicitly requested.
|
||||
|
||||
Use the supplied final still as a last-frame anchor only when the backend supports
|
||||
it. Otherwise identify it honestly as composition/style guidance; do not describe
|
||||
a reference image as a guaranteed final-frame constraint. Compile the actual H3
|
||||
mode with the sibling guide and bind only real inputs.
|
||||
|
||||
Map short paper slides, pops, taps, rustles, and snaps to motion. Preserve those
|
||||
SFX during assembly. When the user adds music or narration later, mix with the
|
||||
useful SFX unless they request full replacement.
|
||||
|
||||
## Review and deliver
|
||||
|
||||
Check the metaphor without relying on text, object count/readability, matching
|
||||
opening color, visible piece-by-piece construction, final-frame resemblance,
|
||||
controlled paper texture, and no unwanted UI/letters/music/voiceover. If the
|
||||
assembly is weak, simplify the groups and make their arrival order concrete.
|
||||
|
||||
For multiple clips, preserve the planned order and SFX. Deliver prompts, current
|
||||
stills, and actual clips/assembly as requested. Explain any missing generation
|
||||
capability or remaining drift; keep raw output separate from edited fixes.
|
||||
@@ -1,4 +0,0 @@
|
||||
interface:
|
||||
display_name: "Paper Collage Explainers"
|
||||
short_description: "Animate editorial paper metaphors with tactile SFX"
|
||||
default_prompt: "Use $paper-collage-explainer-generator to prepare the requested H3 prompts or video with the available tools."
|
||||
@@ -1,33 +0,0 @@
|
||||
# Editorial collage design
|
||||
|
||||
For each source idea record meaning, emotion, one active verb, a concrete visual
|
||||
metaphor, three to six object groups, and a tactile sound cue. For example, a
|
||||
collaboration concept can become separate cutout hands fitting paper routes
|
||||
together; it need not become a screen full of the word "collaboration".
|
||||
|
||||
Style signature: flat bold color field, black-and-white halftone photographic
|
||||
cutouts, selective colored cardstock accents, warm cream keylines, soft physical
|
||||
paper shadows, fine uncoated fiber, refined irregular torn edges, layered seams.
|
||||
Control texture so the video feels handmade without becoming dirty or distressed.
|
||||
|
||||
Palette choices may support meaning: orange/red for urgency, mustard for tools or
|
||||
warning, green for reflection, purple for memory, teal for collaboration, rose for
|
||||
theatrical contrast. These are suggestions, not universal semantic rules. Use the
|
||||
user's palette and avoid an unrelated kraft-paper opening.
|
||||
|
||||
Still specification: source meaning → metaphor → ratio → background/accent colors
|
||||
→ object roles → foreground/midground/background → final composition → exclusions.
|
||||
|
||||
Motion specification: matching empty color field → base paper structures → main
|
||||
subject → secondary groups → light bounce/press-flat/pause → completed composition.
|
||||
Explicitly describe material boundaries and which piece moves at each beat. A
|
||||
static camera is usually enough to reveal assembly; add movement only when useful.
|
||||
|
||||
Sound specification: synchronize restrained slides, pops, taps, rustles, and snaps.
|
||||
Set `non_diegetic_music: N/A` for the default SFX-only H3 prompt. Do not use silence
|
||||
as a synonym for no background music. Dialogue and subtitles are optional layers.
|
||||
|
||||
Review one established clip as a style/cadence reference for subsequent clips.
|
||||
Change the specific failure: fewer groups for clutter, more physical arrival
|
||||
steps for digital drifting, less aging for dirty paper, stronger final state for
|
||||
composition drift. Do not apply an unrelated aesthetic preset to fix one defect.
|
||||
@@ -1,68 +0,0 @@
|
||||
---
|
||||
name: papercraft-stop-motion-explainer
|
||||
description: Turn knowledge topics into layered papercraft stop-motion explainers, including visual metaphors, miniature sets, prompts, storyboards, motion and sound plans, or rendered video. Use for tactile paper dioramas and mechanisms, not ordinary vector cartoons or photographic collage.
|
||||
metadata:
|
||||
compatibility: Portable file-based workflow with sibling h3-prompt-writing; actual media generation uses tools available in the host environment.
|
||||
---
|
||||
|
||||
# Papercraft Stop-Motion Explainer
|
||||
|
||||
Read the [portable workflow](../h3-prompt-writing/references/portable-workflow.md).
|
||||
For scene/prompt planning read [paper staging](references/paper-staging.md).
|
||||
Route a single image prompt, image series, short animation prompt, or storyboard
|
||||
request directly to that artifact; do not force a full film workflow.
|
||||
|
||||
## Knowledge and direction
|
||||
|
||||
Identify the exact topic, source facts, audience, desired learning outcome, and
|
||||
output type. Preserve technical terms and causal meaning. A visual metaphor helps
|
||||
explain a mechanism; it must not turn into a false scientific claim.
|
||||
|
||||
Reuse requested duration and ratio. If unspecified, a 30-second, 16:9 general-audience
|
||||
concept is a reasonable starting point. Offer distinct directions only when useful:
|
||||
pop-up-book journey, paper laboratory, layered cross-section, miniature nature
|
||||
theater, or mechanism board. Each should identify one learning outcome, a paper
|
||||
mechanism that conveys it, and the signature visual moment.
|
||||
|
||||
## Build the paper world
|
||||
|
||||
Design characters only if they help the explanation. Specify paper construction,
|
||||
silhouette, joint/hinge design, replaceable facial parts, gestures, and repeatable
|
||||
identity cues. Keep the central knowledge object readable.
|
||||
|
||||
Plan foreground occluders, midground mechanism, background scenery, and a distant
|
||||
paper plane when the shot benefits from depth. Record each layer's material,
|
||||
thickness, cut/fold edges, shadow, movement, and educational role. Use real paper
|
||||
mechanics such as pull-tabs, sliders, rotating discs, page folds, and pinned joints.
|
||||
Motion is stepped with small pauses and rebounds, not a smooth CG melt.
|
||||
|
||||
Create an asset list by function: characters, scientific objects, information cards,
|
||||
environment, and moving mechanisms. For a complete production, prepare one to three
|
||||
preview briefs or images when useful and authorized. Respect any preview approval
|
||||
requested by the user; otherwise continue under the existing end-to-end scope.
|
||||
|
||||
## Prompts and timeline
|
||||
|
||||
Write a compact storyboard with columns: time, knowledge beat, visual action,
|
||||
paper movement, camera/transition, sound cue. Each shot explains one knowledge
|
||||
step, with a readable pause for a necessary label. Keep ratio, material, light
|
||||
direction, identity, and domain vocabulary consistent across image-series prompts.
|
||||
|
||||
For a short image-to-video prompt, begin from the actual image and animate a few
|
||||
paper mechanisms with restrained push-in or lateral parallax. Preserve thickness,
|
||||
cut edges, texture, and layer shadows. Compile final rendering prompts using the
|
||||
bundled H3 mode guide, with truthful references and local clip timing.
|
||||
|
||||
## Sound, generation, and review
|
||||
|
||||
Map paper flips, cardboard slides, joint taps, soft pops, and rustles to actual
|
||||
motion. Match any requested music to the topic and cultural context rather than
|
||||
automatically using the same whimsical score. Keep speech intelligible. If narration
|
||||
is requested, preserve its approved wording, measure its duration, and adjust the
|
||||
timeline or script within the user's scope; do not silently speed up the voice.
|
||||
|
||||
Use available generation/editing tools, then inspect both educational accuracy
|
||||
and paper construction: readable mechanism, separate layers/shadows, hinged or
|
||||
stepped motion, plausible camera movement, and a clear ending takeaway. Check
|
||||
narration tails, clicks, audio gaps, and music balance. Deliver requested artifacts
|
||||
and state missing rendering capabilities or unresolved scientific/visual issues.
|
||||
@@ -1,4 +0,0 @@
|
||||
interface:
|
||||
display_name: "Papercraft Explainers"
|
||||
short_description: "Explain ideas with layered paper mechanisms"
|
||||
default_prompt: "Use $papercraft-stop-motion-explainer to prepare the requested H3 prompts or video with the available tools."
|
||||
@@ -1,48 +0,0 @@
|
||||
# Paper staging and prompt building blocks
|
||||
|
||||
## Materials and depth
|
||||
|
||||
Use layered matte cardboard, visible thickness, fibers, folds, seams, tabs, brads,
|
||||
cut edges, and physical shadows. Separate foreground, mechanism, background, and
|
||||
far background with spacing. Roughly 4–7 layers often suffice; remove layers that
|
||||
hide the explanation. Place reading-heavy labels on stable planes. Use miniature
|
||||
macro photography and moderate depth of field without blurring necessary facts.
|
||||
|
||||
| Plane | Useful content | Motion | Purpose |
|
||||
| --- | --- | --- | --- |
|
||||
| Foreground | Paper leaves, frame, curtain, props | Small parallax or hinged reveal | Depth without concealing the mechanism |
|
||||
| Midground | Knowledge model, host, arrow/cards | Pull-tab, slider, joint, rotating disc | Main cause and effect |
|
||||
| Background | Diorama set or schematic layers | Slow paper rail shift | Context and hierarchy |
|
||||
| Far plane | Paper sky/board | Restrained change | Stable visual support |
|
||||
|
||||
## Prompt ingredients
|
||||
|
||||
Image: topic and core object → miniature diorama → layered paper construction →
|
||||
specific material edges → physical light/shadows → clear depth → necessary labels.
|
||||
One frame should communicate one main idea, not the entire lesson at once.
|
||||
|
||||
Image series: repeat the style/identity anchors, then state what changes and why
|
||||
that state explains the next knowledge step. Keep the same ratio and light direction.
|
||||
|
||||
Short animation: preserve the supplied image, make a small paper mechanism move,
|
||||
include tiny pauses/rebounds, and select one restrained camera move. Small puppet
|
||||
gestures and background rails can provide life without overwhelming the model.
|
||||
|
||||
## Camera and transitions
|
||||
|
||||
Use slow push-in, lateral parallax, static demonstration, macro detail, or a slight
|
||||
top-down cross-section view. A full orbit requires a set built for that view.
|
||||
Transitions may use a page flip, pop-up unfold, pull-tab, paper cloud, circular
|
||||
mask, cut-paper door, layer separation, confetti, or sticker reveal. Avoid liquid
|
||||
morphing, neon glitches, and glass shatter unless explicitly requested as contrast.
|
||||
|
||||
## Sound and exclusions
|
||||
|
||||
Use a few motion-specific accents: paper flip, scissors, cardboard slide, joint
|
||||
tap, tape peel, or box opening. A score, when appropriate, can use light percussion,
|
||||
plucked instruments, or topic-specific instrumentation. Do not infer narration
|
||||
merely because the video teaches something.
|
||||
|
||||
Keep constraints concise and relevant: no glossy plastic, smooth CG morphs, metal
|
||||
surfaces replacing paper, missing thickness/shadows, or frantic camera movement.
|
||||
Include them in supported prompt fields; do not invent a negative-prompt API field.
|
||||
@@ -1,299 +0,0 @@
|
||||
#!/bin/bash
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
|
||||
|
||||
# vllm-omni customized version
|
||||
# Based on: https://github.com/vllm-project/ci-infra/blob/main/buildkite/bootstrap-amd.sh
|
||||
# Last synced: 2025-12-15
|
||||
# Modifications: Use local template file instead of downloading from ci-infra
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# The bootstrap runs from the repository root; ShellCheck does not follow
|
||||
# sourced files unless invoked with -x.
|
||||
# shellcheck disable=SC1091
|
||||
source .buildkite/common/scripts/resolve_skip_ci.sh
|
||||
|
||||
AMD_SUITE_SELECTOR=".buildkite/amd/scripts/select_test_suites.py"
|
||||
|
||||
if [[ -z "${RUN_ALL:-}" ]]; then
|
||||
RUN_ALL=0
|
||||
fi
|
||||
|
||||
if [[ -z "${NIGHTLY:-}" ]]; then
|
||||
NIGHTLY=0
|
||||
fi
|
||||
|
||||
if [[ -z "${VLLM_CI_BRANCH:-}" ]]; then
|
||||
VLLM_CI_BRANCH="main"
|
||||
fi
|
||||
|
||||
if [[ -z "${AMD_MIRROR_HW:-}" ]]; then
|
||||
AMD_MIRROR_HW="amdproduction"
|
||||
fi
|
||||
|
||||
PR_LABELS=""
|
||||
# Explicit debug suites do not depend on PR labels.
|
||||
if [[ "${BUILDKITE_PULL_REQUEST:-false}" != "false" && -z "${DEBUG_TEST_YAML:-}" ]]; then
|
||||
if ! PR_LABELS=$(curl -fsSL \
|
||||
"https://api.github.com/repos/vllm-project/vllm-omni/pulls/$BUILDKITE_PULL_REQUEST" \
|
||||
| jq -r '.labels[].name'); then
|
||||
echo "ERROR: Could not read PR labels; refusing to select an incorrect AMD test tier." >&2
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
fail_fast() {
|
||||
if [[ "${BUILDKITE_PULL_REQUEST:-false}" != "false" ]]; then
|
||||
echo true
|
||||
else
|
||||
echo false # not a PR or BUILDKITE_PULL_REQUEST not set
|
||||
fi
|
||||
}
|
||||
|
||||
if [[ -z "${COV_ENABLED:-}" ]]; then
|
||||
COV_ENABLED=0
|
||||
fi
|
||||
|
||||
resolve_test_specs() {
|
||||
local selected_specs
|
||||
if ! selected_specs=$(python3 "$AMD_SUITE_SELECTOR" \
|
||||
--branch "$BUILDKITE_BRANCH" \
|
||||
--labels "$PR_LABELS" \
|
||||
--debug-test-yaml "${DEBUG_TEST_YAML:-}" \
|
||||
--nightly "$NIGHTLY"); then
|
||||
exit 1
|
||||
fi
|
||||
|
||||
TEST_SPECS=()
|
||||
while IFS= read -r suite_spec; do
|
||||
[[ -z "$suite_spec" ]] && continue
|
||||
TEST_SPECS+=("$suite_spec")
|
||||
done <<< "$selected_specs"
|
||||
}
|
||||
|
||||
filter_test_specs_by_skip_ci() {
|
||||
local suite_spec level decision
|
||||
local skip_all=0
|
||||
local -a runnable_specs=()
|
||||
|
||||
for suite_spec in "${TEST_SPECS[@]}"; do
|
||||
case "$suite_spec" in
|
||||
READY_TESTS:*) level="l2" ;;
|
||||
MERGE_TESTS:*) level="l3" ;;
|
||||
NIGHTLY_TESTS:*)
|
||||
# Explicit PR and scheduled nightly selections must survive
|
||||
# L2/L3 diff gating, including docs-only main commits.
|
||||
runnable_specs+=("$suite_spec")
|
||||
continue
|
||||
;;
|
||||
*)
|
||||
echo "ERROR: unknown AMD test suite spec '$suite_spec'" >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
if decision=$(python3 "$SKIP_CI_PY" gate amd "$level"); then
|
||||
echo "Skipping AMD $level suite due to $decision decision."
|
||||
[[ "$decision" == "skip-all" ]] && skip_all=1
|
||||
else
|
||||
runnable_specs+=("$suite_spec")
|
||||
fi
|
||||
done
|
||||
|
||||
TEST_SPECS=("${runnable_specs[@]}")
|
||||
if [[ ${#TEST_SPECS[@]} -eq 0 ]]; then
|
||||
if [[ $skip_all -eq 1 ]]; then
|
||||
buildkite-agent annotate \
|
||||
":memo: CI skipped — docs or pytest skip-mark changes only" \
|
||||
--style "info" 2>/dev/null || true
|
||||
fi
|
||||
echo "No AMD suites remain after skip-ci filtering."
|
||||
exit 0
|
||||
fi
|
||||
}
|
||||
|
||||
upload_pipeline() {
|
||||
echo "Uploading pipeline..."
|
||||
# Install minijinja
|
||||
ls .buildkite || buildkite-agent annotate --style error 'Please merge upstream main branch for buildkite CI'
|
||||
curl -sSfL https://github.com/mitsuhiko/minijinja/releases/download/2.3.1/minijinja-cli-installer.sh | sh
|
||||
# Installed by the minijinja bootstrap above and only present on the CI agent.
|
||||
# shellcheck disable=SC1091
|
||||
source /var/lib/buildkite-agent/.cargo/env
|
||||
|
||||
if [[ $BUILDKITE_PIPELINE_SLUG == "fastcheck" ]]; then
|
||||
AMD_MIRROR_HW="amdtentative"
|
||||
fi
|
||||
|
||||
# Use local template file for vllm-omni
|
||||
cp .buildkite/amd/test-template-amd-omni.j2 .buildkite/amd/test-template.j2
|
||||
|
||||
|
||||
# (WIP) Use pipeline generator instead of jinja template
|
||||
if [ -e ".buildkite/amd/pipeline_generator/pipeline_generator.py" ]; then
|
||||
python -m pip install click pydantic
|
||||
python .buildkite/amd/pipeline_generator/pipeline_generator.py --run_all="$RUN_ALL" --list_file_diff="$LIST_FILE_DIFF" --nightly="$NIGHTLY" --mirror_hw="$AMD_MIRROR_HW"
|
||||
buildkite-agent pipeline upload .buildkite/amd/pipeline.yaml
|
||||
exit 0
|
||||
fi
|
||||
echo "List file diff: $LIST_FILE_DIFF"
|
||||
echo "Run all: $RUN_ALL"
|
||||
echo "Nightly: $NIGHTLY"
|
||||
echo "AMD Mirror HW: $AMD_MIRROR_HW"
|
||||
|
||||
FAIL_FAST=$(fail_fast)
|
||||
|
||||
cd .buildkite/amd
|
||||
|
||||
# Multiple label-selected or debug-selected suites share one image build.
|
||||
if [[ ${#TEST_SPECS[@]} -eq 1 ]]; then
|
||||
TEST_YAML="${TEST_SPECS[0]#*:}"
|
||||
else
|
||||
TEST_YAML=$(mktemp "${TMPDIR:-/tmp}/amd-selected-tests.XXXXXX.yml")
|
||||
python - "$TEST_YAML" "${TEST_SPECS[@]}" <<'PY'
|
||||
import sys
|
||||
|
||||
import yaml
|
||||
|
||||
|
||||
output_path, *suite_specs = sys.argv[1:]
|
||||
combined = {"env": {}, "steps": []}
|
||||
|
||||
for suite_spec in suite_specs:
|
||||
group_name, input_path = suite_spec.split(":", 1)
|
||||
with open(input_path, encoding="utf-8") as test_file:
|
||||
suite = yaml.safe_load(test_file)
|
||||
|
||||
for name, value in (suite.get("env") or {}).items():
|
||||
previous = combined["env"].get(name, value)
|
||||
if previous != value:
|
||||
raise ValueError(
|
||||
f"Conflicting environment value for {name}: {previous!r} != {value!r}"
|
||||
)
|
||||
combined["env"][name] = value
|
||||
|
||||
suite_steps = []
|
||||
for entry in suite.get("steps") or []:
|
||||
if "group" in entry:
|
||||
suite_steps.extend(entry.get("steps") or [])
|
||||
else:
|
||||
suite_steps.append(entry)
|
||||
combined["steps"].append({"group": group_name, "steps": suite_steps})
|
||||
|
||||
with open(output_path, "w", encoding="utf-8") as output_file:
|
||||
yaml.safe_dump(combined, output_file, sort_keys=False)
|
||||
PY
|
||||
fi
|
||||
echo "AMD test suites: ${TEST_SPECS[*]}"
|
||||
|
||||
(
|
||||
set -x
|
||||
# Output pipeline.yaml with all blank lines removed
|
||||
minijinja-cli test-template.j2 "$TEST_YAML" \
|
||||
-D branch="$BUILDKITE_BRANCH" \
|
||||
-D list_file_diff="$LIST_FILE_DIFF" \
|
||||
-D run_all="$RUN_ALL" \
|
||||
-D nightly="$NIGHTLY" \
|
||||
-D mirror_hw="$AMD_MIRROR_HW" \
|
||||
-D fail_fast="$FAIL_FAST" \
|
||||
-D vllm_use_precompiled="$VLLM_USE_PRECOMPILED" \
|
||||
-D vllm_merge_base_commit="$(git merge-base origin/main HEAD)" \
|
||||
-D cov_enabled="$COV_ENABLED" \
|
||||
-D vllm_ci_branch="$VLLM_CI_BRANCH" \
|
||||
| sed '/^[[:space:]]*$/d' \
|
||||
> pipeline.yaml
|
||||
)
|
||||
cat pipeline.yaml
|
||||
if [[ "$TEST_YAML" == "${TMPDIR:-/tmp}/amd-selected-tests."*.yml ]]; then
|
||||
rm -f -- "$TEST_YAML"
|
||||
fi
|
||||
buildkite-agent artifact upload pipeline.yaml
|
||||
buildkite-agent pipeline upload pipeline.yaml
|
||||
exit 0
|
||||
}
|
||||
|
||||
get_diff() {
|
||||
git diff --name-only --diff-filter=ACMDR "$(git merge-base origin/main HEAD)"
|
||||
}
|
||||
|
||||
get_diff_main() {
|
||||
git diff --name-only --diff-filter=ACMDR HEAD~1
|
||||
}
|
||||
|
||||
file_diff=$(get_diff)
|
||||
if [[ $BUILDKITE_BRANCH == "main" ]]; then
|
||||
file_diff=$(get_diff_main)
|
||||
fi
|
||||
|
||||
# Resolve PR tier labels before skip-ci so ready/merge suites can be filtered
|
||||
# independently while an explicitly selected nightly suite passes through.
|
||||
resolve_test_specs
|
||||
filter_test_specs_by_skip_ci
|
||||
|
||||
patterns=(
|
||||
"docker/Dockerfile"
|
||||
"CMakeLists.txt"
|
||||
"requirements/common.txt"
|
||||
"requirements/cuda.txt"
|
||||
"requirements/build.txt"
|
||||
"requirements/test.txt"
|
||||
"setup.py"
|
||||
"csrc/"
|
||||
"cmake/"
|
||||
)
|
||||
|
||||
ignore_patterns=(
|
||||
"docker/Dockerfile."
|
||||
"csrc/cpu"
|
||||
"csrc/rocm"
|
||||
"cmake/hipify.py"
|
||||
"cmake/cpu_extension.cmake"
|
||||
)
|
||||
|
||||
for file in $file_diff; do
|
||||
# First check if file matches any pattern
|
||||
matches_pattern=0
|
||||
for pattern in "${patterns[@]}"; do
|
||||
if [[ $file == "$pattern"* ]] || [[ $file == "$pattern" ]]; then
|
||||
matches_pattern=1
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
# If file matches pattern, check it's not in ignore patterns
|
||||
if [[ $matches_pattern -eq 1 ]]; then
|
||||
matches_ignore=0
|
||||
for ignore in "${ignore_patterns[@]}"; do
|
||||
if [[ $file == "$ignore"* ]] || [[ $file == "$ignore" ]]; then
|
||||
matches_ignore=1
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
if [[ $matches_ignore -eq 0 ]]; then
|
||||
RUN_ALL=1
|
||||
echo "Found changes: $file. Run all tests"
|
||||
break
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
# Decide whether to use precompiled wheels
|
||||
# Relies on existing patterns array as a basis.
|
||||
if [[ -n "${VLLM_USE_PRECOMPILED:-}" ]]; then
|
||||
echo "VLLM_USE_PRECOMPILED is already set to: $VLLM_USE_PRECOMPILED"
|
||||
elif [[ $RUN_ALL -eq 1 ]]; then
|
||||
export VLLM_USE_PRECOMPILED=0
|
||||
echo "Detected critical changes, building wheels from source"
|
||||
else
|
||||
export VLLM_USE_PRECOMPILED=1
|
||||
echo "No critical changes, using precompiled wheels"
|
||||
fi
|
||||
|
||||
|
||||
LIST_FILE_DIFF=$(get_diff | tr ' ' '|')
|
||||
if [[ $BUILDKITE_BRANCH == "main" ]]; then
|
||||
LIST_FILE_DIFF=$(get_diff_main | tr ' ' '|')
|
||||
fi
|
||||
upload_pipeline
|
||||
@@ -1,69 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# Build and push the ROCm CI image with a registry-backed BuildKit cache. The
|
||||
# AMD builders are ephemeral, so a local Docker cache cannot survive between
|
||||
# jobs. Keeping the cache beside the runtime image also reuses the registry
|
||||
# credentials already configured on the amd-cpu agents.
|
||||
|
||||
REGISTRY="${ROCM_CI_IMAGE_REGISTRY:-rocm/vllm-omni}"
|
||||
DOCKERFILE="docker/Dockerfile.rocm"
|
||||
BUILDER_NAME="vllm-omni-rocm-builder"
|
||||
ROCM_ARCH="${PYTORCH_ROCM_ARCH:-gfx942}"
|
||||
|
||||
if [[ -z "${BUILDKITE_COMMIT:-}" ]]; then
|
||||
echo "ERROR: BUILDKITE_COMMIT is not set" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
dependency_files=(
|
||||
"$DOCKERFILE"
|
||||
"${DOCKERFILE}.dockerignore"
|
||||
pyproject.toml
|
||||
setup.py
|
||||
tools/install_torchcodec_rocm.sh
|
||||
)
|
||||
while IFS= read -r dependency_file; do
|
||||
dependency_files+=("$dependency_file")
|
||||
done < <(find requirements -maxdepth 1 -type f -name '*.txt' -print | LC_ALL=C sort)
|
||||
|
||||
for dependency_file in "${dependency_files[@]}"; do
|
||||
if [[ ! -f "$dependency_file" ]]; then
|
||||
echo "ERROR: cache-key input does not exist: $dependency_file" >&2
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
CACHE_KEY=$(
|
||||
{
|
||||
printf 'rocm-arch=%s\n' "$ROCM_ARCH"
|
||||
for dependency_file in "${dependency_files[@]}"; do
|
||||
printf 'path=%s\n' "$dependency_file"
|
||||
sha256sum "$dependency_file"
|
||||
done
|
||||
} | sha256sum | cut -c1-16
|
||||
)
|
||||
CACHE_REF="${REGISTRY}:rocm-deps-cache-${CACHE_KEY}"
|
||||
IMAGE_REF="${REGISTRY}:${BUILDKITE_COMMIT}"
|
||||
|
||||
echo "ROCm dependency cache: ${CACHE_REF}"
|
||||
echo "ROCm CI image: ${IMAGE_REF}"
|
||||
|
||||
# The docker-container driver supports exporting the full cache to a registry;
|
||||
# the default docker driver on CI does not.
|
||||
if ! docker buildx inspect "$BUILDER_NAME" >/dev/null 2>&1; then
|
||||
docker buildx create --name "$BUILDER_NAME" --driver docker-container
|
||||
fi
|
||||
docker buildx use "$BUILDER_NAME"
|
||||
docker buildx inspect --bootstrap
|
||||
|
||||
docker buildx build --push --pull --progress=plain \
|
||||
--target test \
|
||||
--build-arg "ARG_PYTORCH_ROCM_ARCH=${ROCM_ARCH}" \
|
||||
--cache-from "type=registry,ref=${CACHE_REF}" \
|
||||
--cache-to "type=registry,ref=${CACHE_REF},mode=max,compression=zstd,ignore-error=true" \
|
||||
--file "$DOCKERFILE" \
|
||||
--tag "$IMAGE_REF" .
|
||||
@@ -1,78 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
|
||||
|
||||
"""Prebuild the BF16 AITER MHA modules used by Cosmos3 on ROCm."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
import torch
|
||||
|
||||
from vllm_omni.diffusion.attention.backends.utils import fa
|
||||
from vllm_omni.platforms import current_omni_platform
|
||||
|
||||
ATTENTION_VARIANTS = (
|
||||
(True, "mha_fwd_bf16_nbias_mask_nlse_ndropout_nqscale"),
|
||||
(False, "mha_fwd_bf16_nbias_nmask_nlse_ndropout_nqscale"),
|
||||
)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
if not current_omni_platform.is_rocm():
|
||||
raise RuntimeError("The Cosmos3 AITER prewarm must run on ROCm")
|
||||
if fa.flash_attn_func is None:
|
||||
raise RuntimeError("ROCm FlashAttention/AITER is unavailable")
|
||||
|
||||
cache_dir_value = os.environ.get("AITER_JIT_DIR")
|
||||
if not cache_dir_value:
|
||||
raise RuntimeError("AITER_JIT_DIR must identify the job-local cache")
|
||||
cache_dir = Path(cache_dir_value)
|
||||
|
||||
device = torch.device(current_omni_platform.device_type)
|
||||
dtype = torch.bfloat16
|
||||
batch_size = 1
|
||||
sequence_length = 64
|
||||
num_heads = 8
|
||||
head_dim = 64
|
||||
|
||||
torch.manual_seed(42)
|
||||
query = torch.randn(
|
||||
batch_size,
|
||||
sequence_length,
|
||||
num_heads,
|
||||
head_dim,
|
||||
device=device,
|
||||
dtype=dtype,
|
||||
)
|
||||
key = torch.randn_like(query)
|
||||
value = torch.randn_like(query)
|
||||
|
||||
for causal, expected_module in ATTENTION_VARIANTS:
|
||||
output = fa.flash_attn_func(
|
||||
query,
|
||||
key,
|
||||
value,
|
||||
dropout_p=0.0,
|
||||
softmax_scale=head_dim**-0.5,
|
||||
causal=causal,
|
||||
)
|
||||
if isinstance(output, tuple):
|
||||
output = output[0]
|
||||
current_omni_platform.synchronize()
|
||||
|
||||
if output.shape != query.shape:
|
||||
raise RuntimeError(f"Unexpected AITER output shape: {output.shape}")
|
||||
if not torch.isfinite(output).all():
|
||||
raise RuntimeError("AITER prewarm produced non-finite output")
|
||||
|
||||
module_path = cache_dir / f"{expected_module}.so"
|
||||
if not module_path.is_file():
|
||||
raise RuntimeError(f"Expected AITER module was not built: {module_path}")
|
||||
print(f"Prebuilt Cosmos3 AITER module: {module_path}", flush=True)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,103 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Run vLLM-Omni ROCm tests directly in the MI300 Kubernetes pod. The pod's
|
||||
# container image is selected by test-template-amd-omni.j2; MI300 has no DinD.
|
||||
set -euo pipefail
|
||||
|
||||
: "${PYTHONFAULTHANDLER:=1}"
|
||||
: "${HF_HOME:=/home/buildkite-agent/huggingface}"
|
||||
: "${HF_HUB_DOWNLOAD_TIMEOUT:=300}"
|
||||
: "${HF_HUB_ETAG_TIMEOUT:=60}"
|
||||
: "${MIOPEN_DEBUG_CONV_DIRECT:=0}"
|
||||
: "${MIOPEN_DEBUG_CONV_GEMM:=0}"
|
||||
: "${VLLM_ROCM_USE_AITER:=0}"
|
||||
export PYTHONFAULTHANDLER HF_HOME HF_HUB_DOWNLOAD_TIMEOUT HF_HUB_ETAG_TIMEOUT
|
||||
export MIOPEN_DEBUG_CONV_DIRECT MIOPEN_DEBUG_CONV_GEMM VLLM_ROCM_USE_AITER
|
||||
export PYTORCH_ROCM_ARCH=""
|
||||
export PYTHONPATH="${PYTHONPATH:-..}"
|
||||
|
||||
if [[ "${VLLM_CI_DOCKER_DISABLED:-0}" != "1" ]]; then
|
||||
echo "Error: MI300 CI must run natively with Docker disabled." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [[ -n "${TEST_COMMANDS:-}" ]]; then
|
||||
commands="${TEST_COMMANDS}"
|
||||
else
|
||||
commands="$*"
|
||||
fi
|
||||
if [[ -z "${commands}" ]]; then
|
||||
echo "Error: No test commands provided." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "--- Native in-pod ROCm CI"
|
||||
|
||||
job_id="${BUILDKITE_JOB_ID:-${BUILDKITE_PARALLEL_JOB:-local}}"
|
||||
job_id="${job_id//[^A-Za-z0-9_.-]/_}"
|
||||
native_cache_root="/tmp/vllm-omni-native-${job_id}"
|
||||
TMPDIR="${native_cache_root}/tmp"
|
||||
TORCHINDUCTOR_CACHE_DIR="${native_cache_root}/torchinductor"
|
||||
TRITON_CACHE_DIR="${native_cache_root}/triton"
|
||||
VLLM_CACHE_ROOT="${native_cache_root}/vllm"
|
||||
XDG_CACHE_HOME="${native_cache_root}/xdg"
|
||||
HF_DATASETS_CACHE="${native_cache_root}/huggingface/datasets"
|
||||
export TMPDIR TORCHINDUCTOR_CACHE_DIR TRITON_CACHE_DIR VLLM_CACHE_ROOT
|
||||
export XDG_CACHE_HOME HF_DATASETS_CACHE
|
||||
mkdir -p "${TMPDIR}" "${TORCHINDUCTOR_CACHE_DIR}" "${TRITON_CACHE_DIR}" \
|
||||
"${VLLM_CACHE_ROOT}" "${XDG_CACHE_HOME}" "${HF_HOME}" \
|
||||
"${HF_DATASETS_CACHE}"
|
||||
|
||||
if [[ "${VLLM_CI_REQUIRE_PERSISTENT_HF_CACHE:-0}" == "1" ]]; then
|
||||
if ! command -v findmnt >/dev/null 2>&1; then
|
||||
echo "Error: findmnt is required to verify the Hugging Face cache mount." >&2
|
||||
exit 1
|
||||
fi
|
||||
hf_mount=$(findmnt -n -T "${HF_HOME}" -o TARGET 2>/dev/null || true)
|
||||
if [[ -z "${hf_mount}" || "${hf_mount}" == "/" ]]; then
|
||||
echo "Error: MI300 CI requires a persistent volume at or above ${HF_HOME}." >&2
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
rocminfo
|
||||
|
||||
expected_gpus="${VLLM_CI_EXPECTED_GPU_COUNT:-1}"
|
||||
python3 - <<'PY'
|
||||
import os
|
||||
|
||||
import torch
|
||||
|
||||
expected = int(os.environ.get("VLLM_CI_EXPECTED_GPU_COUNT", "1"))
|
||||
assert torch.version.hip, "PyTorch is not a ROCm build"
|
||||
assert torch.cuda.is_available(), "ROCm GPU is not available to PyTorch"
|
||||
actual = torch.cuda.device_count()
|
||||
assert actual == expected, f"Expected {expected} ROCm GPU(s), found {actual}"
|
||||
PY
|
||||
|
||||
echo "Commands:${commands}"
|
||||
if /bin/bash -o pipefail -c '
|
||||
set -E
|
||||
test_status=0
|
||||
trap '\''
|
||||
command_status=$?
|
||||
# Keep running subsequent commands, but retain a failure for the final exit.
|
||||
# Prefer a test failure over pytest "no tests collected" when both occur.
|
||||
if (( test_status == 0 || (test_status == 5 && command_status != 5) )); then
|
||||
test_status=${command_status}
|
||||
fi
|
||||
'\'' ERR
|
||||
eval "$1"
|
||||
exit "${test_status}"
|
||||
' _ "${commands}"; then
|
||||
exit 0
|
||||
else
|
||||
exit_code=$?
|
||||
fi
|
||||
|
||||
if [[ ${exit_code} -eq 5 && "${VLLM_CI_ALLOW_NO_TESTS:-0}" == "1" ]]; then
|
||||
echo "Pytest collected no tests; VLLM_CI_ALLOW_NO_TESTS=1, treating exit code 5 as success."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
exit "${exit_code}"
|
||||
@@ -1,100 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
|
||||
"""Select AMD ready/merge/nightly suites after Buildkite creates a build.
|
||||
|
||||
This repository-side selector does not decide which GitHub label events start
|
||||
the external ``vllm-omni-amd-ci`` pipeline. Its Buildkite PR build condition
|
||||
must admit the relevant label event before this code can inspect the labels.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
from collections.abc import Iterable
|
||||
|
||||
SUITE_SPECS = {
|
||||
"ready": "READY_TESTS:test-amd-ready.yml",
|
||||
"merge": "MERGE_TESTS:test-amd-merge.yml",
|
||||
"nightly": "NIGHTLY_TESTS:test-amd-nightly.yml",
|
||||
}
|
||||
|
||||
|
||||
def _parse_debug_suites(value: str) -> tuple[str, ...]:
|
||||
suites = tuple(item.strip().lower() for item in value.split(",") if item.strip())
|
||||
if not suites:
|
||||
raise ValueError("DEBUG_TEST_YAML did not contain a test suite")
|
||||
|
||||
seen: set[str] = set()
|
||||
for suite in suites:
|
||||
if suite not in SUITE_SPECS:
|
||||
raise ValueError(f"DEBUG_TEST_YAML entries must be 'merge', 'nightly', or 'ready', got {suite!r}")
|
||||
if suite in seen:
|
||||
raise ValueError(f"duplicate DEBUG_TEST_YAML suite {suite!r}")
|
||||
seen.add(suite)
|
||||
return suites
|
||||
|
||||
|
||||
def select_amd_test_suites(
|
||||
*,
|
||||
branch: str,
|
||||
labels: Iterable[str],
|
||||
debug_test_yaml: str = "",
|
||||
nightly: bool = False,
|
||||
) -> tuple[str, ...]:
|
||||
"""Return ordered suite names while preserving legacy unlabeled PRs."""
|
||||
|
||||
if debug_test_yaml:
|
||||
return _parse_debug_suites(debug_test_yaml)
|
||||
if branch == "main":
|
||||
return ("nightly",) if nightly else ("merge",)
|
||||
|
||||
label_set = {label.strip() for label in labels if label.strip()}
|
||||
selected = tuple(
|
||||
suite
|
||||
for label, suite in (
|
||||
("ready", "ready"),
|
||||
("merge-test", "merge"),
|
||||
("nightly-test", "nightly"),
|
||||
)
|
||||
if label in label_set
|
||||
)
|
||||
# AMD historically uploaded the ready suite for every admitted PR build.
|
||||
# Preserve that fallback when no explicit tier label is present.
|
||||
return selected or ("ready",)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--branch", required=True)
|
||||
parser.add_argument("--labels", default="")
|
||||
parser.add_argument("--debug-test-yaml", default="")
|
||||
parser.add_argument("--nightly", type=int, choices=(0, 1), default=0)
|
||||
args = parser.parse_args()
|
||||
|
||||
labels = tuple(args.labels.splitlines())
|
||||
try:
|
||||
suites = select_amd_test_suites(
|
||||
branch=args.branch,
|
||||
labels=labels,
|
||||
debug_test_yaml=args.debug_test_yaml,
|
||||
nightly=bool(args.nightly),
|
||||
)
|
||||
except ValueError as exc:
|
||||
parser.error(str(exc))
|
||||
|
||||
label_set = {label.strip() for label in labels if label.strip()}
|
||||
if args.branch != "main" and not args.debug_test_yaml and not ({"ready", "merge-test", "nightly-test"} & label_set):
|
||||
print(
|
||||
"No AMD ready/merge/nightly label found; preserving the legacy ready-suite fallback.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
|
||||
for suite in suites:
|
||||
print(SUITE_SPECS[suite])
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -1,347 +0,0 @@
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
HF_HUB_DOWNLOAD_TIMEOUT: 300
|
||||
HF_HUB_ETAG_TIMEOUT: 60
|
||||
|
||||
steps:
|
||||
|
||||
- group: ":card_index_dividers: Simple Test"
|
||||
steps:
|
||||
- label: "Simple · Model Executor Test · Shard %N/%t"
|
||||
agent_pool: mi300_1
|
||||
parallelism: 3
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- export VLLM_ROCM_USE_AITER=0
|
||||
# ignore test_teacache_extractors.py because it use rocm gemm kernel from vLLM
|
||||
# that is not supported on CPU
|
||||
- "pytest -sv tests/model_executor -m 'core_model and cpu and not omni' --num-shards=$$BUILDKITE_PARALLEL_JOB_COUNT --shard-id=$$BUILDKITE_PARALLEL_JOB"
|
||||
|
||||
- label: "Simple · Omni Processor Test"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
timeout_in_minutes: 60
|
||||
commands:
|
||||
- export VLLM_ROCM_USE_AITER=0
|
||||
- "pytest -sv tests/model_executor/models/test_omni_processing.py -m 'core_model and cpu and omni'"
|
||||
|
||||
- label: "Simple · Diffusion Test · Shard %N/%t"
|
||||
agent_pool: mi300_1
|
||||
parallelism: 4
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
timeout_in_minutes: 45
|
||||
# Buildkite's status-based rule cannot prove every exit 134 is a GPU hang.
|
||||
# It retries 134 once; the Kubernetes job retry starts a fresh pod.
|
||||
# Assertions (1), timeouts (124), and OOMs (137) remain non-retryable.
|
||||
retry:
|
||||
automatic:
|
||||
- exit_status: 134
|
||||
limit: 1
|
||||
commands:
|
||||
- export VLLM_ROCM_USE_AITER=0
|
||||
# ignore test_teacache_extractors.py because it use rocm gemm kernel from vLLM
|
||||
# that is not supported on CPU
|
||||
# Keep multi-GPU tests out of this one-GPU pool even when their module
|
||||
# also carries the cpu marker for unit tests.
|
||||
# Keep π0.5 CPU units out of these shards: CUDA READY/MERGE already run
|
||||
# the same suite mandatorily, and AMD covers it non-blocking in nightly.
|
||||
- "pytest -sv tests/diffusion -m 'core_model and cpu and not (cards_2 or cards_3 or cards_4 or cards_5 or cards_6 or cards_7 or cards_8)' --ignore=tests/diffusion/cache/test_teacache_extractors.py --ignore=tests/diffusion/models/pi05/test_pi05_units.py --num-shards=$$BUILDKITE_PARALLEL_JOB_COUNT --shard-id=$$BUILDKITE_PARALLEL_JOB"
|
||||
|
||||
- label: "Diffusion · AuK VAE GPU Test"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
timeout_in_minutes: 40
|
||||
commands:
|
||||
- export VLLM_ROCM_USE_AITER=0
|
||||
- >-
|
||||
timeout --signal=TERM --kill-after=2m 35m
|
||||
pytest -sv tests/diffusion/models/auk/test_auk_vae_fastpath.py
|
||||
-m 'core_model and rocm and cards_1' --run-level core_model
|
||||
|
||||
- label: "Simple · Engine&Entrypoints Test"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export VLLM_ROCM_USE_AITER=0
|
||||
- "pytest -sv tests/entrypoints tests/engine -m 'core_model and cpu'"
|
||||
|
||||
- label: "Simple · Other Test"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export VLLM_ROCM_USE_AITER=0
|
||||
# ignore test_teacache_extractors.py because it use rocm gemm kernel from vLLM
|
||||
# that is not supported on CPU
|
||||
- "pytest -sv tests/ -m 'core_model and cpu' --ignore=tests/diffusion --ignore=tests/model_executor --ignore=tests/entrypoints --ignore=tests/engine --ignore=tests/diffusion/cache/test_teacache_extractors.py"
|
||||
|
||||
- group: ":card_index_dividers: Diffusion Test"
|
||||
steps:
|
||||
# AR paged attention is real GPU work, including cold AITER compilation.
|
||||
# Keep its complete numerical matrix out of the CPU-marked Simple shards.
|
||||
- label: "ROCm · AR Diffusion Paged Attention GPU Test"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
timeout_in_minutes: 20
|
||||
commands:
|
||||
- export VLLM_ROCM_USE_AITER=1
|
||||
- export VLLM_OMNI_AR_FA_REQUIRED=1
|
||||
- export AITER_JIT_DIR="/tmp/vllm-omni-ar-aiter-$$BUILDKITE_JOB_ID"
|
||||
- export TORCH_EXTENSIONS_DIR="/tmp/vllm-omni-ar-torch-extensions-$$BUILDKITE_JOB_ID"
|
||||
- mkdir -p "$$AITER_JIT_DIR" "$$TORCH_EXTENSIONS_DIR"
|
||||
- >-
|
||||
timeout --signal=TERM --kill-after=2m 15m
|
||||
pytest -sv tests/diffusion/ar_diffusion/test_paged_attention.py
|
||||
-m 'core_model and rocm and MI325 and cards_1' --run-level core_model
|
||||
|
||||
- label: "Diffusion · LoRA Test"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
artifact_paths:
|
||||
- lora-server.log
|
||||
commands:
|
||||
- pytest -sv tests/diffusion/lora/ -m 'advanced_model and cuda' --run-level "advanced_model" 2>&1 | tee lora-server.log
|
||||
|
||||
- label: "Diffusion Model CPU offloading Test"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- export DIFFUSION_ATTENTION_BACKEND=TORCH_SDPA
|
||||
- pytest -s -v tests/diffusion/offloader/test_diffusion_cpu_offload.py
|
||||
- >-
|
||||
pytest -s -v tests/diffusion/offloader/test_diffusion_layerwise_offload.py
|
||||
--deselect='tests/diffusion/offloader/test_diffusion_layerwise_offload.py::test_layerwise_offload_diffusion_model[stabilityai/stable-audio-open-1.0]'
|
||||
|
||||
# TODO: Diagnose unstable Stable Audio layerwise-offload memory savings on ROCm.
|
||||
- label: "Diffusion Stable Audio Layerwise Offload Test (quarantined)"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: NonBlocking
|
||||
commands:
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- >-
|
||||
pytest -s -v
|
||||
'tests/diffusion/offloader/test_diffusion_layerwise_offload.py::test_layerwise_offload_diffusion_model[stabilityai/stable-audio-open-1.0]'
|
||||
|
||||
- label: "Diffusion Sequence Parallelism Test (Need 4 GPUs) · Shard 1/2"
|
||||
agent_pool: mi300_4
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
timeout_in_minutes: 40
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- pytest -s -v tests/diffusion/distributed/test_sequence_parallel.py
|
||||
|
||||
- label: "Diffusion Sequence Parallelism Test (Need 4 GPUs) · Shard 2/2"
|
||||
agent_pool: mi300_4
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
timeout_in_minutes: 40
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- pytest -s -v tests/diffusion/distributed/test_ulysses_uaa_perf.py
|
||||
|
||||
# merge-only tests
|
||||
- label: "Diffusion Tensor Parallelism Test"
|
||||
agent_pool: mi300_2
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- pytest -s -v tests/diffusion/distributed/test_tensor_parallel.py
|
||||
- >-
|
||||
timeout 20m pytest -s -v
|
||||
tests/diffusion/attention/test_ulysses_uaa.py::test_ulysses_uaa_2d_mask_layout_matches_baseline
|
||||
-m 'core_model and cards_2'
|
||||
--run-level "core_model"
|
||||
|
||||
- label: "Diffusion Model Test"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
# A cold Z-Image run spends about six minutes loading weights before
|
||||
# the first-request AITER compilation, which can itself exceed 23m.
|
||||
- timeout 55m pytest -s -v tests/e2e/offline_inference/test_z_image.py
|
||||
|
||||
- label: "Diffusion GPU Worker Test"
|
||||
agent_pool: mi300_2
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- timeout 20m pytest -s -v tests/diffusion/test_diffusion_worker.py
|
||||
|
||||
- label: "Diffusion · Wan22 Test"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- |
|
||||
timeout 40m bash -c "
|
||||
export VLLM_IMAGE_FETCH_TIMEOUT=60
|
||||
pytest -s -v tests/e2e/offline_inference/test_wan22_t2v.py tests/e2e/online_serving/test_wan22_t2v.py -m 'advanced_model and cuda and diffusion' --run-level 'advanced_model'
|
||||
"
|
||||
|
||||
## ISSUE depends on `diffusers` package: https://github.com/huggingface/diffusers/issues/13274
|
||||
# - label: "Audio Generation Model Test"
|
||||
# agent_pool: mi300_1
|
||||
# depends_on: amd-build
|
||||
# mirror_hardwares: [amdproduction]
|
||||
# grade: Blocking
|
||||
# commands:
|
||||
# - export GPU_ARCHS=gfx942
|
||||
# - export VLLM_LOGGING_LEVEL=DEBUG
|
||||
# - export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
# - timeout 20m pytest -s -v tests/e2e/offline_inference/test_stable_audio_expansion.py -m "advanced_model and diffusion and L4" --run-level advanced_model
|
||||
|
||||
- label: "Engine Test"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- timeout 20m pytest -s -v tests/engine/test_async_omni_engine_abort.py
|
||||
|
||||
# This is from test-nightly.yml.
|
||||
# This model seems to be important, so I am trying to run it here.
|
||||
# Since this is from test-nightly.yml,
|
||||
# I keep this non-blocking.
|
||||
- group: ":card_index_dividers: E2E Test"
|
||||
steps:
|
||||
- label: "Omni Model Test Qwen2-5-Omni"
|
||||
agent_pool: mi300_4
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
commands:
|
||||
- export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
|
||||
- pytest -s -v tests/e2e/*/test_qwen2_5_omni_expansion.py --run-level "core_model"
|
||||
|
||||
- label: "Omni Model Test Qwen3-Omni"
|
||||
agent_pool: mi300_2
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- pytest -s -v tests/e2e/offline_inference/test_qwen3_omni.py tests/e2e/online_serving/test_qwen3_omni.py tests/e2e/offline_inference/test_qwen3_omni_colocate_async.py -m 'advanced_model and cuda' --run-level 'advanced_model'
|
||||
|
||||
- label: "Qwen3-TTS CustomVoice E2E Test · Shard %N/%t"
|
||||
agent_pool: mi300_1
|
||||
parallelism: 3
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
timeout_in_minutes: 80
|
||||
commands:
|
||||
- |
|
||||
timeout --signal=TERM --kill-after=2m 80m bash -c '
|
||||
export VLLM_LOGGING_LEVEL=DEBUG
|
||||
export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
|
||||
TESTS=(
|
||||
"tests/e2e/online_serving/test_qwen3_tts_customvoice.py::test_default_cuda_graph_startup[async_chunk]"
|
||||
"tests/e2e/online_serving/test_qwen3_tts_customvoice.py::test_text_to_audio_002[async_chunk]"
|
||||
"tests/e2e/offline_inference/test_qwen3_tts_customvoice.py::test_text_to_audio_001[no_cuda_graph]"
|
||||
)
|
||||
TEST="$${TESTS[$$BUILDKITE_PARALLEL_JOB]}"
|
||||
pytest -s -v "$$TEST" -m "advanced_model" --run-level "advanced_model"
|
||||
'
|
||||
|
||||
# TODO: Restore this case to the blocking sharded step once its long-running
|
||||
# Whisper validation on AMD CI is resolved.
|
||||
- label: "Qwen3-TTS CustomVoice async_chunk Test (quarantined)"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: NonBlocking
|
||||
timeout_in_minutes: 80
|
||||
commands:
|
||||
- |
|
||||
timeout --signal=TERM --kill-after=2m 75m bash -c '
|
||||
export VLLM_LOGGING_LEVEL=DEBUG
|
||||
export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
|
||||
pytest -s -v "tests/e2e/online_serving/test_qwen3_tts_customvoice.py::test_text_to_audio_001[async_chunk]" -m "advanced_model" --run-level "advanced_model"
|
||||
'
|
||||
|
||||
- label: "Qwen3-TTS Base E2E Test"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
timeout_in_minutes: 50
|
||||
commands:
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
|
||||
- timeout 40m pytest -s -v tests/e2e/online_serving/test_qwen3_tts_base.py tests/e2e/offline_inference/test_qwen3_tts_base.py -m "advanced_model and cuda" --run-level "advanced_model"
|
||||
|
||||
# TODO: Bagel test on ROCm is very unstable. @tjtanaa
|
||||
# Need to debug before reneable numerical changes across large PRs
|
||||
# # split Bagel Model Test with H100 (Real Weights) into three tests
|
||||
# - label: "Bagel Text2Img Model Test (1/3)"
|
||||
# agent_pool: mi300_1
|
||||
# depends_on: amd-build
|
||||
# mirror_hardwares: [amdproduction]
|
||||
# grade: Blocking
|
||||
# commands:
|
||||
# - export GPU_ARCHS=gfx942
|
||||
# - export VLLM_LOGGING_LEVEL=DEBUG
|
||||
# - export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
# - export VLLM_ROCM_USE_AITER_RMSNORM=0
|
||||
# - timeout 30m pytest -s -v tests/e2e/offline_inference/test_bagel.py -m "advanced_model" --run-level "advanced_model" -k "shared_memory" -k "rocm"
|
||||
|
||||
# - label: "Bagel Img2Img Model Test (2/3)"
|
||||
# agent_pool: mi300_1
|
||||
# depends_on: amd-build
|
||||
# mirror_hardwares: [amdproduction]
|
||||
# grade: Blocking
|
||||
# commands:
|
||||
# - export GPU_ARCHS=gfx942
|
||||
# - export VLLM_LOGGING_LEVEL=DEBUG
|
||||
# - export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
# - export VLLM_ROCM_USE_AITER_RMSNORM=0
|
||||
# - timeout 30m pytest -s -v tests/e2e/offline_inference/test_bagel.py -m "advanced_model" --run-level "advanced_model" -k "rocm"
|
||||
|
||||
# - label: "Bagel Online Serving Test (3/3)"
|
||||
# agent_pool: mi300_1
|
||||
# depends_on: amd-build
|
||||
# mirror_hardwares: [amdproduction]
|
||||
# grade: Blocking
|
||||
# commands:
|
||||
# - export GPU_ARCHS=gfx942
|
||||
# - export VLLM_IMAGE_FETCH_TIMEOUT=60
|
||||
# - export VLLM_LOGGING_LEVEL=DEBUG
|
||||
# - export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
# - export VLLM_ROCM_USE_AITER_RMSNORM=0
|
||||
# - timeout 40m pytest -s -v tests/e2e/online_serving/test_bagel.py -m "advanced_model" --run-level "advanced_model" -k "rocm"
|
||||
@@ -1,170 +0,0 @@
|
||||
# AMD MI300 `full_model` nightly coverage during burn-in.
|
||||
# All jobs are non-blocking until the lane has stable runtime evidence.
|
||||
# Historical build #11646 was red in Function Expansion and Accuracy (one
|
||||
# async_chunk similarity mismatch; Seed-TTS WER 0.376480 > 0.35).
|
||||
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
HF_HUB_DOWNLOAD_TIMEOUT: 300
|
||||
HF_HUB_ETAG_TIMEOUT: 60
|
||||
|
||||
steps:
|
||||
# MI325 is the repository's legacy ROCm pytest marker. The AMD template maps
|
||||
# agent_pool to the current MI300 Buildkite queues independently of that mark.
|
||||
- group: ":full_moon: Qwen3-Omni MI300 Nightly"
|
||||
steps:
|
||||
- label: "Qwen3-Omni Function Expansion"
|
||||
agent_pool: mi300_2
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: NonBlocking
|
||||
timeout_in_minutes: 180
|
||||
commands:
|
||||
- export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
|
||||
- pytest -s -v tests/e2e/online_serving/test_qwen3_omni_expansion.py -m "full_model and rocm and MI325 and cards_2" --run-level "full_model"
|
||||
|
||||
- label: "Qwen3-Omni Accuracy"
|
||||
agent_pool: mi300_2
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: NonBlocking
|
||||
timeout_in_minutes: 180
|
||||
artifact_paths:
|
||||
- tests/e2e/accuracy/qwen3_omni/results/qwen_omni_acc/*.json
|
||||
commands:
|
||||
- export SEED_TTS_WER_EVAL=1
|
||||
# PyTorch ROCm uses the torch.cuda device namespace.
|
||||
- export SEED_TTS_EVAL_DEVICE=cuda:1
|
||||
- pytest -s -v tests/e2e/accuracy/qwen3_omni/test_qwen3_omni.py -m "full_model and rocm and MI325 and cards_2" --run-level "full_model"
|
||||
# Native tests run from the image copy under /app/vllm-omni, while
|
||||
# Buildkite's artifact hook searches the checkout workspace.
|
||||
- |
|
||||
if [[ "$$PWD" != "$${BUILDKITE_BUILD_CHECKOUT_PATH:?}" ]]; then
|
||||
artifact_dir="$${BUILDKITE_BUILD_CHECKOUT_PATH}/tests/e2e/accuracy/qwen3_omni/results/qwen_omni_acc"
|
||||
mkdir -p "$$artifact_dir"
|
||||
cp -v tests/e2e/accuracy/qwen3_omni/results/qwen_omni_acc/*.json "$$artifact_dir/"
|
||||
fi
|
||||
|
||||
- label: "Qwen3-Omni Documentation Examples"
|
||||
agent_pool: mi300_2
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: NonBlocking
|
||||
timeout_in_minutes: 120
|
||||
commands:
|
||||
- export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
|
||||
- pytest -s -v tests/examples/online_serving/test_qwen3_omni.py -m "full_model and rocm and MI325 and cards_2" --run-level "full_model"
|
||||
|
||||
- label: "Qwen3-Omni AITER-on Smoke"
|
||||
agent_pool: mi300_2
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: NonBlocking
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- export VLLM_ROCM_USE_AITER=1
|
||||
- pytest -s -v tests/e2e/online_serving/test_qwen3_omni_expansion.py::test_text_to_text_audio_001 -m "full_model and rocm and MI325 and cards_2" --run-level "full_model"
|
||||
|
||||
- group: ":speaker: CosyVoice3 MI300 Nightly"
|
||||
steps:
|
||||
# Exercise all sync/async and Chinese/English cases with the platform's
|
||||
# default attention backend. On ROCm this retains end-to-end AITER
|
||||
# coverage without putting its cold JIT build on the READY critical path.
|
||||
- label: "CosyVoice3-TTS E2E Test"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: NonBlocking
|
||||
timeout_in_minutes: 90
|
||||
retry:
|
||||
automatic:
|
||||
- exit_status: 134
|
||||
limit: 1
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/online_serving/test_cosyvoice3_tts_expansion.py -m "slow" --run-level "core_model"
|
||||
|
||||
# π0.5 CPU units are mandatory on CUDA READY/MERGE. On AMD they exercise the
|
||||
# ROCm container/import path with CPU tensors only, so keep them off the
|
||||
# blocking READY/MERGE critical path and retain non-blocking nightly coverage.
|
||||
- group: ":robot_face: Pi0.5 CPU Nightly"
|
||||
steps:
|
||||
- label: "Simple · Pi0.5 CPU Test"
|
||||
agent_pool: mi300_1
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: NonBlocking
|
||||
timeout_in_minutes: 60
|
||||
commands:
|
||||
- export VLLM_ROCM_USE_AITER=0
|
||||
- "pytest -sv tests/diffusion/models/pi05/test_pi05_units.py -m 'core_model and cpu'"
|
||||
|
||||
- group: ":full_moon: LingBot-Video MI300 Nightly"
|
||||
steps:
|
||||
- label: "LingBot-Video Dense CFG-off Smoke"
|
||||
agent_pool: mi300_1
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: NonBlocking
|
||||
timeout_in_minutes: 90
|
||||
artifact_paths:
|
||||
- artifacts/rocm-lingbot-nightly/**/*
|
||||
commands:
|
||||
- pytest --collect-only -q tests/e2e/online_serving/test_lingbot_video_expansion.py::test_cfg_off -m "full_model and diffusion and rocm and MI325 and cards_1" --run-level full_model
|
||||
- pytest -s -v tests/e2e/online_serving/test_lingbot_video_expansion.py::test_cfg_off -m "full_model and diffusion and rocm and MI325 and cards_1" --run-level full_model --junitxml=/tmp/lingbot-pytest.xml
|
||||
- |
|
||||
artifact_dir="$${BUILDKITE_BUILD_CHECKOUT_PATH:?}/artifacts/rocm-lingbot-nightly"
|
||||
mkdir -p "$$artifact_dir"
|
||||
cp -v /tmp/lingbot-pytest.xml "$$artifact_dir/"
|
||||
|
||||
- group: ":full_moon: Cosmos3 MI300 Nightly"
|
||||
steps:
|
||||
- label: "Cosmos3 T2I Function"
|
||||
agent_pool: mi300_1
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: NonBlocking
|
||||
timeout_in_minutes: 90
|
||||
artifact_paths:
|
||||
- artifacts/rocm-cosmos3-nightly/**/*
|
||||
commands:
|
||||
# Cosmos3 uses both masked and unmasked BF16 AITER MHA modules. Build
|
||||
# them before server startup, then reuse the same job-local cache
|
||||
# during the smoke test.
|
||||
- export VLLM_ROCM_USE_AITER=1
|
||||
- export AITER_JIT_DIR="/tmp/vllm-omni-cosmos3-aiter-$$BUILDKITE_JOB_ID"
|
||||
- export TORCH_EXTENSIONS_DIR="/tmp/vllm-omni-cosmos3-torch-extensions-$$BUILDKITE_JOB_ID"
|
||||
- mkdir -p "$$AITER_JIT_DIR" "$$TORCH_EXTENSIONS_DIR"
|
||||
- pytest --collect-only -q tests/e2e/online_serving/test_cosmos3.py::test_text_to_image_001 -m "core_model and rocm and MI325 and cards_1" --run-level full_model
|
||||
- >-
|
||||
timeout --signal=TERM --kill-after=2m 55m
|
||||
python .buildkite/amd/scripts/prewarm_cosmos3_aiter.py
|
||||
- >-
|
||||
timeout --signal=TERM --kill-after=2m 25m
|
||||
pytest -s -v
|
||||
tests/e2e/online_serving/test_cosmos3.py::test_text_to_image_001
|
||||
-m "core_model and rocm and MI325 and cards_1"
|
||||
--run-level full_model
|
||||
--junitxml=/tmp/cosmos3-pytest.xml
|
||||
- |
|
||||
artifact_dir="$${BUILDKITE_BUILD_CHECKOUT_PATH:?}/artifacts/rocm-cosmos3-nightly"
|
||||
mkdir -p "$$artifact_dir"
|
||||
cp -v /tmp/cosmos3-pytest.xml "$$artifact_dir/"
|
||||
|
||||
- group: ":full_moon: HunyuanImage3 MI300 Nightly"
|
||||
steps:
|
||||
- label: "HunyuanImage3 Offline Pixel Accuracy"
|
||||
agent_pool: mi300_4
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: NonBlocking
|
||||
timeout_in_minutes: 180
|
||||
env:
|
||||
HUNYUAN_IMAGE3_MODEL: "tencent/HunyuanImage-3.0-Instruct"
|
||||
HUNYUAN_IMAGE3_DEVICES: "0,1,2,3"
|
||||
DIFFUSION_ATTENTION_BACKEND: TORCH_SDPA
|
||||
artifact_paths:
|
||||
- artifacts/rocm-hunyuanimage3/**/*
|
||||
commands:
|
||||
- pytest --collect-only -q tests/e2e/accuracy/test_hunyuan_image3_pixel_accuracy.py::test_hunyuan_image3_pixel_accuracy_offline -m "full_model and rocm and MI325 and cards_4" --run-level full_model
|
||||
- pytest -s -v tests/e2e/accuracy/test_hunyuan_image3_pixel_accuracy.py::test_hunyuan_image3_pixel_accuracy_offline -m "full_model and rocm and MI325 and cards_4" --run-level full_model --junitxml=/tmp/hunyuanimage3-pytest.xml
|
||||
- |
|
||||
artifact_dir="$${BUILDKITE_BUILD_CHECKOUT_PATH:?}/artifacts/rocm-hunyuanimage3"
|
||||
mkdir -p "$$artifact_dir"
|
||||
cp -v tests/e2e/accuracy/artifacts/HunyuanImage-3_0-Instruct/*.png "$$artifact_dir/"
|
||||
cp -v /tmp/hunyuanimage3-pytest.xml "$$artifact_dir/"
|
||||
@@ -1,451 +0,0 @@
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
HF_HUB_DOWNLOAD_TIMEOUT: 300
|
||||
HF_HUB_ETAG_TIMEOUT: 60
|
||||
|
||||
steps:
|
||||
|
||||
- group: ":card_index_dividers: Simple Test"
|
||||
steps:
|
||||
- label: "Simple · Model Executor Test · Shard %N/%t"
|
||||
agent_pool: mi300_1
|
||||
parallelism: 3
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- export VLLM_ROCM_USE_AITER=0
|
||||
# ignore test_teacache_extractors.py because it use rocm gemm kernel from vLLM
|
||||
# that is not supported on CPU
|
||||
- "pytest -sv tests/model_executor -m 'core_model and cpu and not omni' --num-shards=$$BUILDKITE_PARALLEL_JOB_COUNT --shard-id=$$BUILDKITE_PARALLEL_JOB"
|
||||
|
||||
- label: "Simple · Omni Processor Test"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
timeout_in_minutes: 60
|
||||
commands:
|
||||
- export VLLM_ROCM_USE_AITER=0
|
||||
- "pytest -sv tests/model_executor/models/test_omni_processing.py -m 'core_model and cpu and omni'"
|
||||
|
||||
- label: "Simple · Diffusion Test · Shard %N/%t"
|
||||
agent_pool: mi300_1
|
||||
parallelism: 4
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
timeout_in_minutes: 45
|
||||
# Buildkite's status-based rule cannot prove every exit 134 is a GPU hang.
|
||||
# It retries 134 once; the Kubernetes job retry starts a fresh pod.
|
||||
# Assertions (1), timeouts (124), and OOMs (137) remain non-retryable.
|
||||
retry:
|
||||
automatic:
|
||||
- exit_status: 134
|
||||
limit: 1
|
||||
commands:
|
||||
- export VLLM_ROCM_USE_AITER=0
|
||||
# ignore test_teacache_extractors.py because it use rocm gemm kernel from vLLM
|
||||
# that is not supported on CPU
|
||||
# Keep multi-GPU tests out of this one-GPU pool even when their module
|
||||
# also carries the cpu marker for unit tests.
|
||||
# Keep π0.5 CPU units out of these shards: CUDA READY/MERGE already run
|
||||
# the same suite mandatorily, and AMD covers it non-blocking in nightly.
|
||||
- "pytest -sv tests/diffusion -m 'core_model and cpu and not (cards_2 or cards_3 or cards_4 or cards_5 or cards_6 or cards_7 or cards_8)' --ignore=tests/diffusion/cache/test_teacache_extractors.py --ignore=tests/diffusion/models/pi05/test_pi05_units.py --num-shards=$$BUILDKITE_PARALLEL_JOB_COUNT --shard-id=$$BUILDKITE_PARALLEL_JOB"
|
||||
|
||||
- label: "Diffusion · AuK VAE GPU Test"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
timeout_in_minutes: 40
|
||||
commands:
|
||||
- export VLLM_ROCM_USE_AITER=0
|
||||
- >-
|
||||
timeout --signal=TERM --kill-after=2m 35m
|
||||
pytest -sv tests/diffusion/models/auk/test_auk_vae_fastpath.py
|
||||
-m 'core_model and rocm and cards_1' --run-level core_model
|
||||
|
||||
- label: "Simple · Engine&Entrypoints Test"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export VLLM_ROCM_USE_AITER=0
|
||||
- "pytest -sv tests/entrypoints tests/engine -m 'core_model and cpu'"
|
||||
|
||||
- label: "Simple · Other Test"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export VLLM_ROCM_USE_AITER=0
|
||||
# ignore test_teacache_extractors.py because it use rocm gemm kernel from vLLM
|
||||
# that is not supported on CPU
|
||||
- "pytest -sv tests/ -m 'core_model and cpu' --ignore=tests/diffusion --ignore=tests/model_executor --ignore=tests/entrypoints --ignore=tests/engine --ignore=tests/diffusion/cache/test_teacache_extractors.py"
|
||||
|
||||
- label: "Custom Pipeline Test"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/features/custom_pipeline/ -m "core_model"
|
||||
|
||||
|
||||
- label: "Voxtral TTS CUDA Unit Test"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- timeout 10m pytest -s -v tests/model_executor/models/voxtral_tts/test_cuda_graph_acoustic_transformer.py
|
||||
|
||||
- group: ":card_index_dividers: Diffusion Test"
|
||||
steps:
|
||||
# Keep real FlashAttention execution out of the CPU-marked Simple lane.
|
||||
# AITER may need to build several kernels on a cold worker, so run one
|
||||
# pytest worker with job-local build directories and a bounded timeout.
|
||||
- label: "ROCm · FlashAttention/AITER GPU Coverage"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: NonBlocking
|
||||
timeout_in_minutes: 60
|
||||
commands:
|
||||
- export VLLM_ROCM_USE_AITER=1
|
||||
- export AITER_JIT_DIR="/tmp/vllm-omni-aiter-$$BUILDKITE_JOB_ID"
|
||||
- export TORCH_EXTENSIONS_DIR="/tmp/vllm-omni-torch-extensions-$$BUILDKITE_JOB_ID"
|
||||
- mkdir -p "$$AITER_JIT_DIR" "$$TORCH_EXTENSIONS_DIR"
|
||||
- >-
|
||||
timeout --signal=TERM --kill-after=2m 55m
|
||||
pytest -s -v -n 1
|
||||
tests/diffusion/attention/
|
||||
-m 'core_model and rocm and MI325 and cards_1'
|
||||
--run-level "core_model"
|
||||
# MammothModa2 exercises the same two AITER MHA modules built by the
|
||||
# preceding suite. Keep its default-backend run in this job so a cold
|
||||
# worker compiles each module once instead of timing out the generic
|
||||
# model job.
|
||||
- >-
|
||||
timeout --signal=TERM --kill-after=2m 10m
|
||||
pytest -s -v -n 1
|
||||
tests/diffusion/models/mammoth_moda2/test_dit_attention_cuda.py
|
||||
--run-level "core_model"
|
||||
|
||||
# AR paged attention is real GPU work, including cold AITER compilation.
|
||||
# Keep its complete numerical matrix out of the CPU-marked Simple shards.
|
||||
- label: "ROCm · AR Diffusion Paged Attention GPU Test"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
timeout_in_minutes: 20
|
||||
commands:
|
||||
- export VLLM_ROCM_USE_AITER=1
|
||||
- export VLLM_OMNI_AR_FA_REQUIRED=1
|
||||
- export AITER_JIT_DIR="/tmp/vllm-omni-ar-aiter-$$BUILDKITE_JOB_ID"
|
||||
- export TORCH_EXTENSIONS_DIR="/tmp/vllm-omni-ar-torch-extensions-$$BUILDKITE_JOB_ID"
|
||||
- mkdir -p "$$AITER_JIT_DIR" "$$TORCH_EXTENSIONS_DIR"
|
||||
- >-
|
||||
timeout --signal=TERM --kill-after=2m 15m
|
||||
pytest -sv tests/diffusion/ar_diffusion/test_paged_attention.py
|
||||
-m 'core_model and rocm and MI325 and cards_1' --run-level core_model
|
||||
|
||||
- label: "Diffusion · Batch Test"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- pytest -sv tests/diffusion/batching -m 'core_model and cuda' --run-level "core_model"
|
||||
|
||||
- label: "Diffusion · Offloader Test"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- pytest -s -v tests/diffusion/offloader -m "core_model and cuda" --run-level "core_model"
|
||||
|
||||
- label: "Diffusion · Cache Test"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export GPU_ARCHS=gfx942
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- pytest -sv tests/diffusion/cache -m 'core_model and cuda' --run-level "core_model"
|
||||
|
||||
- label: "Diffusion Sequence Parallelism Test"
|
||||
agent_pool: mi300_2
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- timeout 20m pytest -s -v tests/diffusion/distributed/test_sequence_parallel.py -m core_model
|
||||
- >-
|
||||
timeout 20m pytest -s -v
|
||||
tests/diffusion/attention/test_ulysses_uaa.py::test_ulysses_uaa_2d_mask_layout_matches_baseline
|
||||
-m 'core_model and cards_2'
|
||||
--run-level "core_model"
|
||||
- >-
|
||||
timeout 20m pytest -s -v
|
||||
tests/diffusion/models/ltx2/test_ltx2_transformer_ulysses.py
|
||||
-m 'core_model and rocm and cards_2'
|
||||
--run-level "core_model"
|
||||
|
||||
- label: "Diffusion GPU Worker Test"
|
||||
agent_pool: mi300_2
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- timeout 20m pytest -s -v tests/diffusion/test_diffusion_worker.py
|
||||
|
||||
- label: "Diffusion · Model Test"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
# Preserve legacy one-GPU tests without cards_1 while excluding every
|
||||
# registered multi-GPU marker from this one-GPU job.
|
||||
- >-
|
||||
timeout 15m pytest -sv tests/diffusion/models/
|
||||
--ignore=tests/diffusion/models/auk/test_auk_vae_fastpath.py
|
||||
--ignore=tests/diffusion/models/mammoth_moda2/test_dit_attention_cuda.py
|
||||
-m 'core_model and cuda and not
|
||||
(cards_2 or cards_3 or cards_4 or cards_5 or cards_6 or cards_7 or cards_8)'
|
||||
--run-level "core_model"
|
||||
# Keep the model-specific numerical coverage blocking without paying
|
||||
# the default ROCm backend's cold AITER compilation cost in this lane.
|
||||
- >-
|
||||
timeout 10m env DIFFUSION_ATTENTION_BACKEND=TORCH_SDPA
|
||||
pytest -sv
|
||||
tests/diffusion/models/mammoth_moda2/test_dit_attention_cuda.py
|
||||
--run-level "core_model"
|
||||
|
||||
- label: "Diffusion · Wan22 Test"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- |
|
||||
timeout 40m bash -c "
|
||||
export VLLM_TEST_CLEAN_GPU_MEMORY=1
|
||||
export VLLM_IMAGE_FETCH_TIMEOUT=60
|
||||
pytest -s -v tests/e2e/online_serving/test_wan22_t2v.py -m 'core_model' --run-level 'core_model'
|
||||
"
|
||||
|
||||
## ISSUE depends on `diffusers` package: https://github.com/huggingface/diffusers/issues/13274
|
||||
# - label: "Audio Generation Model Test"
|
||||
# agent_pool: mi300_1
|
||||
# depends_on: amd-build
|
||||
# mirror_hardwares: [amdproduction]
|
||||
# grade: Blocking
|
||||
# commands:
|
||||
# - export GPU_ARCHS=gfx942
|
||||
# - export VLLM_LOGGING_LEVEL=DEBUG
|
||||
# - export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
# - timeout 20m pytest -s -v tests/e2e/offline_inference/test_stable_audio_expansion.py -m "advanced_model and diffusion and L4" --run-level advanced_model
|
||||
|
||||
- label: "ROCm · Model Executor GPU Coverage (R2-01)"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: NonBlocking
|
||||
timeout_in_minutes: 20
|
||||
artifact_paths:
|
||||
- "artifacts/rocm-r2-01/**/*"
|
||||
commands:
|
||||
# Native AMD tests run from /app/vllm-omni, while Buildkite uploads
|
||||
# artifacts relative to its checkout directory.
|
||||
- export R2_01_ARTIFACT_DIR="$$BUILDKITE_BUILD_CHECKOUT_PATH/artifacts/rocm-r2-01"
|
||||
- mkdir -p "$$R2_01_ARTIFACT_DIR"
|
||||
# The dedicated Engine Test below owns the only selected tests/engine case;
|
||||
# do not duplicate its model startup in this time-bounded coverage lane.
|
||||
# The CUDA ready job also covers tests/worker/test_batched_omni_output.py.
|
||||
# Keep the initial ROCm burn-in scoped to tests/model_executor/.
|
||||
# Verbose pytest output and JUnit retain node IDs without a second
|
||||
# collection-only pass over the same GPU-backed test tree.
|
||||
- >-
|
||||
pytest -s -v -ra
|
||||
tests/model_executor/
|
||||
-m 'core_model and cuda and not nvidia_only and not
|
||||
(cards_2 or cards_3 or cards_4 or cards_5 or cards_6 or cards_7 or cards_8)'
|
||||
--run-level "core_model"
|
||||
--durations=0
|
||||
--junitxml="$$R2_01_ARTIFACT_DIR/pytest.xml"
|
||||
2>&1 | tee "$$R2_01_ARTIFACT_DIR/pytest.log"
|
||||
- >-
|
||||
tail -n 100 "$$R2_01_ARTIFACT_DIR/pytest.log" >
|
||||
"$$R2_01_ARTIFACT_DIR/pytest-summary.txt"
|
||||
|
||||
- label: "Engine Test"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- |
|
||||
timeout 15m bash -c '
|
||||
pytest -s -v tests/engine/test_async_omni_engine_abort.py
|
||||
'
|
||||
|
||||
- label: "ROCm · Entrypoints GPU Coverage (R2-02)"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: NonBlocking
|
||||
timeout_in_minutes: 30
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
artifact_paths:
|
||||
- "artifacts/rocm-r2-02/**/*"
|
||||
commands:
|
||||
# Native AMD tests run from /app/vllm-omni, while Buildkite uploads
|
||||
# artifacts relative to its checkout directory.
|
||||
- export R2_02_ARTIFACT_DIR="$$BUILDKITE_BUILD_CHECKOUT_PATH/artifacts/rocm-r2-02"
|
||||
- mkdir -p "$$R2_02_ARTIFACT_DIR"
|
||||
- >-
|
||||
ps -eo pid=,ppid=,lstart=,stat=,comm= >
|
||||
"$$R2_02_ARTIFACT_DIR/processes-before.txt"
|
||||
# Verbose pytest output and JUnit retain selected node IDs without a
|
||||
# second collection-only pass over the same GPU-backed test tree.
|
||||
# The inner timeout leaves five minutes for teardown verification and
|
||||
# artifact generation before the Buildkite step timeout.
|
||||
- >-
|
||||
timeout --signal=TERM --kill-after=1m 25m
|
||||
pytest -s -v -ra
|
||||
tests/entrypoints/
|
||||
-m 'core_model and cuda and not
|
||||
(cards_2 or cards_3 or cards_4 or cards_5 or cards_6 or cards_7 or cards_8)'
|
||||
--run-level "core_model"
|
||||
--durations=0
|
||||
--junitxml="$$R2_02_ARTIFACT_DIR/pytest.xml"
|
||||
2>&1 | tee "$$R2_02_ARTIFACT_DIR/pytest.log"
|
||||
- >-
|
||||
tail -n 100 "$$R2_02_ARTIFACT_DIR/pytest.log" >
|
||||
"$$R2_02_ARTIFACT_DIR/pytest-summary.txt"
|
||||
- |
|
||||
sleep 5
|
||||
ps -eo pid=,ppid=,lstart=,stat=,comm= > "$$R2_02_ARTIFACT_DIR/processes-after.txt"
|
||||
# Compare PID plus process start time so PID reuse cannot hide a leak.
|
||||
# Double dollars preserve awk field references through Buildkite interpolation.
|
||||
awk '
|
||||
NR == FNR {
|
||||
if ($$9 != "ps") {
|
||||
baseline[$$1 "|" $$3 "|" $$4 "|" $$5 "|" $$6 "|" $$7] = 1
|
||||
}
|
||||
next
|
||||
}
|
||||
$$9 != "ps" {
|
||||
identity = $$1 "|" $$3 "|" $$4 "|" $$5 "|" $$6 "|" $$7
|
||||
if (!(identity in baseline)) {
|
||||
print
|
||||
leaked = 1
|
||||
}
|
||||
}
|
||||
END {
|
||||
if (leaked) {
|
||||
print "status=FAIL leaked_processes_detected=1"
|
||||
exit 1
|
||||
}
|
||||
print "status=PASS leaked_processes=0"
|
||||
}
|
||||
' "$$R2_02_ARTIFACT_DIR/processes-before.txt" \
|
||||
"$$R2_02_ARTIFACT_DIR/processes-after.txt" \
|
||||
2>&1 | tee "$$R2_02_ARTIFACT_DIR/process-cleanup.txt"
|
||||
|
||||
# - label: "Omni Model Test Qwen3-Omni"
|
||||
# agent_pool: mi300_2
|
||||
# depends_on: amd-build
|
||||
# mirror_hardwares: [amdproduction]
|
||||
# grade: Blocking
|
||||
# commands:
|
||||
# - export VLLM_LOGGING_LEVEL=DEBUG
|
||||
# - export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
# - timeout 10m pytest -s -v tests/e2e/offline_inference/test_qwen3_omni.py
|
||||
# - timeout 20m pytest -s -v tests/e2e/online_serving/test_qwen3_omni.py -m "core_model" --run-level "core_model"
|
||||
|
||||
- group: ":card_index_dividers: E2E Test"
|
||||
steps:
|
||||
- label: "Qwen3-TTS E2E Test"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
|
||||
- timeout 30m pytest -s -v tests/e2e/online_serving/test_qwen3_tts_customvoice.py -m "core_model" --run-level "core_model"
|
||||
|
||||
# Keep a fast model/server integration check in the blocking lane. The
|
||||
# full default-backend suite remains in the nightly pipeline because a
|
||||
# cold ROCm AITER MHA build can add more than 25 minutes to this job.
|
||||
- label: "CosyVoice3-TTS E2E Smoke (SDPA)"
|
||||
agent_pool: mi300_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
# A ROCm hardware hang aborts the interpreter with 134. Let Buildkite
|
||||
# create one fresh pod instead of retrying on the poisoned GPU process.
|
||||
retry:
|
||||
automatic:
|
||||
- exit_status: 134
|
||||
limit: 1
|
||||
commands:
|
||||
- export DIFFUSION_ATTENTION_BACKEND=TORCH_SDPA
|
||||
- pytest -s -v tests/e2e/online_serving/test_cosyvoice3_tts_expansion.py::test_voice_clone_zh_002 -m "slow" --run-level "core_model"
|
||||
|
||||
# TODO: Bagel test on ROCm is very unstable. @tjtanaa
|
||||
# Need to debug before reneable numerical changes across large PRs
|
||||
# - label: "Bagel Text2Img Model Test"
|
||||
# agent_pool: mi300_1
|
||||
# depends_on: amd-build
|
||||
# mirror_hardwares: [amdproduction]
|
||||
# grade: Blocking
|
||||
# commands:
|
||||
# - export GPU_ARCHS=gfx942
|
||||
# - export VLLM_LOGGING_LEVEL=DEBUG
|
||||
# - export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
# - export VLLM_ROCM_USE_AITER_RMSNORM=0
|
||||
# - timeout 30m pytest -s -v tests/e2e/offline_inference/test_bagel.py -m "core_model" --run-level "core_model" -k "rocm"
|
||||
|
||||
# - label: "Bagel Img2Img Model Test"
|
||||
# agent_pool: mi300_1
|
||||
# depends_on: amd-build
|
||||
# mirror_hardwares: [amdproduction]
|
||||
# grade: Blocking
|
||||
# commands:
|
||||
# - export GPU_ARCHS=gfx942
|
||||
# - export VLLM_LOGGING_LEVEL=DEBUG
|
||||
# - export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
# - export VLLM_ROCM_USE_AITER_RMSNORM=0
|
||||
# - timeout 30m pytest -s -v tests/e2e/offline_inference/test_bagel.py -m "core_model" --run-level "core_model" -k "rocm"
|
||||
|
||||
# - label: "Bagel Online Serving Test"
|
||||
# agent_pool: mi300_1
|
||||
# depends_on: amd-build
|
||||
# mirror_hardwares: [amdproduction]
|
||||
# grade: Blocking
|
||||
# commands:
|
||||
# - export GPU_ARCHS=gfx942
|
||||
# - export VLLM_IMAGE_FETCH_TIMEOUT=60
|
||||
# - export VLLM_LOGGING_LEVEL=DEBUG
|
||||
# - export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
# - export VLLM_ROCM_USE_AITER_RMSNORM=0
|
||||
# - timeout 40m pytest -s -v tests/e2e/online_serving/test_bagel.py -m "core_model" --run-level "core_model" -k "rocm"
|
||||
@@ -1,255 +0,0 @@
|
||||
{# vllm-omni customized version
|
||||
Based on: https://github.com/vllm-project/ci-infra/blob/main/buildkite/test-template-amd.j2
|
||||
Last synced: 2026-08-04
|
||||
Modifications: Use the shared MI300 queues and Omni command-passing contract
|
||||
#}
|
||||
{% set docker_registry_amd = "rocm/vllm-omni" %}
|
||||
{% set docker_image_amd = docker_registry_amd ~ ":$BUILDKITE_COMMIT" %}
|
||||
{% set default_working_dir = "/app/vllm-omni" %}
|
||||
|
||||
{# MI300 runs tests natively in Kubernetes; Docker-in-Docker is unavailable. #}
|
||||
{% macro amd_native_k8s_plugin(indent, step) -%}
|
||||
{{ indent }}plugins:
|
||||
{{ indent }} - kubernetes:
|
||||
{{ indent }} podSpecPatch:
|
||||
{{ indent }} automountServiceAccountToken: false
|
||||
{{ indent }} securityContext:
|
||||
{{ indent }} seccompProfile:
|
||||
{{ indent }} type: RuntimeDefault
|
||||
{{ indent }} imagePullSecrets:
|
||||
{{ indent }} - name: docker-config
|
||||
{{ indent }} containers:
|
||||
{{ indent }} - name: container-0
|
||||
{{ indent }} image: {{ docker_image_amd }}
|
||||
{{ indent }} imagePullPolicy: Always
|
||||
{{ indent }} securityContext:
|
||||
{{ indent }} allowPrivilegeEscalation: false
|
||||
{{ indent }} capabilities:
|
||||
{{ indent }} add:
|
||||
{{ indent }} - IPC_LOCK
|
||||
{{ indent }} resources:
|
||||
{{ indent }} limits:
|
||||
{{ indent }} amd.com/gpu: "{{ step.agent_pool[-1:] }}"
|
||||
{{ indent }} requests:
|
||||
{{ indent }} amd.com/gpu: "{{ step.agent_pool[-1:] }}"
|
||||
{{ indent }} volumeMounts:
|
||||
{{ indent }} - name: devshm
|
||||
{{ indent }} mountPath: /dev/shm
|
||||
{{ indent }} env:
|
||||
{{ indent }} - name: AMD_CI_RUNTIME
|
||||
{{ indent }} value: native
|
||||
{{ indent }} - name: NATIVE_CI
|
||||
{{ indent }} value: "true"
|
||||
{{ indent }} - name: VLLM_CI_DOCKER_DISABLED
|
||||
{{ indent }} value: "1"
|
||||
{{ indent }} - name: VLLM_CI_EXPECTED_GPU_COUNT
|
||||
{{ indent }} value: "{{ step.agent_pool[-1:] }}"
|
||||
{{ indent }} - name: HF_HOME
|
||||
{{ indent }} value: /home/buildkite-agent/huggingface
|
||||
{{ indent }} - name: MIOPEN_DEBUG_CONV_DIRECT
|
||||
{{ indent }} value: "0"
|
||||
{{ indent }} - name: MIOPEN_DEBUG_CONV_GEMM
|
||||
{{ indent }} value: "0"
|
||||
{{ indent }} - name: VLLM_ROCM_USE_AITER
|
||||
{{ indent }} value: "0"
|
||||
{{ indent }} - name: PYTORCH_ROCM_ARCH
|
||||
{{ indent }} value: ""
|
||||
{{ indent }} volumes:
|
||||
{{ indent }} - name: devshm
|
||||
{{ indent }} emptyDir:
|
||||
{{ indent }} medium: Memory
|
||||
{{ indent }} sizeLimit: 16Gi
|
||||
{%- endmacro %}
|
||||
|
||||
{% if env %}
|
||||
env:
|
||||
{% for name in env %}
|
||||
{{ name }}: "{{ env[name] }}"
|
||||
{% endfor %}
|
||||
{% endif %}
|
||||
steps:
|
||||
- group: "AMD Tests"
|
||||
depends_on: ~
|
||||
steps:
|
||||
- label: "AMD: :docker: build image"
|
||||
depends_on: ~
|
||||
soft_fail: false
|
||||
commands:
|
||||
- "bash .buildkite/amd/scripts/build-ci-image.sh"
|
||||
key: "amd-build"
|
||||
env:
|
||||
DOCKER_BUILDKIT: "1"
|
||||
PYTORCH_ROCM_ARCH: "gfx942"
|
||||
ROCM_CI_IMAGE_REGISTRY: "{{ docker_registry_amd }}"
|
||||
retry:
|
||||
automatic:
|
||||
- exit_status: -1 # Agent was lost
|
||||
limit: 2
|
||||
- exit_status: -10 # Agent was lost
|
||||
limit: 2
|
||||
- exit_status: 128 # Git connectivity issues
|
||||
limit: 2
|
||||
- exit_status: 1 # Machine occasionally fail
|
||||
limit: 1
|
||||
agents:
|
||||
queue: amd-cpu
|
||||
|
||||
{% for step_group in steps %}
|
||||
{% if step_group.group %}
|
||||
- group: "{{ step_group.group }}"
|
||||
{% if step_group.depends_on %}
|
||||
depends_on: {{ step_group.depends_on }}
|
||||
{% endif %}
|
||||
steps:
|
||||
{% for step in step_group.steps %}
|
||||
{% if step.mirror_hardwares and mirror_hw in step.mirror_hardwares %}
|
||||
- label: "{{ step.agent_pool }}: {{ step.label }}"
|
||||
depends_on: amd-build
|
||||
agents:
|
||||
{% if step.agent_pool %}
|
||||
queue: amd_{{ step.agent_pool }}
|
||||
{% else %}
|
||||
queue: amd_mi300_1
|
||||
{% endif %}
|
||||
{{ amd_native_k8s_plugin(" ", step) }}
|
||||
{% set cmd_body = (step.command or (step.commands | join("\n"))) | trim %}
|
||||
{% set indented_cmd = cmd_body | replace("\n", "\n ") %}
|
||||
command: bash .buildkite/amd/scripts/run-amd-test.sh
|
||||
env:
|
||||
AMD_CI_RUNTIME: "native"
|
||||
NATIVE_CI: "true"
|
||||
VLLM_CI_DOCKER_DISABLED: "1"
|
||||
VLLM_CI_EXPECTED_GPU_COUNT: "{{ step.agent_pool[-1:] }}"
|
||||
VLLM_CI_REQUIRE_PERSISTENT_HF_CACHE: "1"
|
||||
HF_HOME: "/home/buildkite-agent/huggingface"
|
||||
HF_HUB_DOWNLOAD_TIMEOUT: "300"
|
||||
HF_HUB_ETAG_TIMEOUT: "60"
|
||||
MIOPEN_DEBUG_CONV_DIRECT: "0"
|
||||
MIOPEN_DEBUG_CONV_GEMM: "0"
|
||||
VLLM_ROCM_USE_AITER: "0"
|
||||
PYTORCH_ROCM_ARCH: ""
|
||||
{% if step.env %}
|
||||
{% for name in step.env %}
|
||||
{{ name }}: "{{ step.env[name] }}"
|
||||
{% endfor %}
|
||||
{% endif %}
|
||||
TEST_COMMANDS: |-
|
||||
(command rocm-smi || true) && cd {{ (step.working_dir or default_working_dir) | safe }}
|
||||
{% if env %}
|
||||
{% for name in env %}
|
||||
export {{ name }}="{{ env[name] }}"
|
||||
{% endfor %}
|
||||
{% endif %}
|
||||
{% if step.env %}
|
||||
{% for name in step.env %}
|
||||
export {{ name }}="{{ step.env[name] }}"
|
||||
{% endfor %}
|
||||
{% endif %}
|
||||
{% if "mi250" in step.agent_pool %}
|
||||
python3 -m pip uninstall -y amd-aiter
|
||||
{% endif %}
|
||||
{{ indented_cmd | safe }}
|
||||
priority: 100
|
||||
{% if step.parallelism %}
|
||||
parallelism: {{ step.parallelism }}
|
||||
{% endif %}
|
||||
{% if step.artifact_paths %}
|
||||
artifact_paths:
|
||||
{% for path in step.artifact_paths %}
|
||||
- "{{ path }}"
|
||||
{% endfor %}
|
||||
{% endif %}
|
||||
{% if step.retry %}
|
||||
retry:
|
||||
automatic:
|
||||
{% for retry_rule in step.retry.automatic %}
|
||||
- exit_status: {{ retry_rule.exit_status }}
|
||||
limit: {{ retry_rule.limit }}
|
||||
{% endfor %}
|
||||
{% endif %}
|
||||
{% if step.timeout_in_minutes %}
|
||||
timeout_in_minutes: {{ step.timeout_in_minutes }}
|
||||
{% endif %}
|
||||
{% if step.grade and step.grade == "Blocking" %}
|
||||
soft_fail: false
|
||||
{% else %}
|
||||
soft_fail: true
|
||||
{% endif%}
|
||||
{% endif %}
|
||||
{% endfor %}
|
||||
{% elif step_group.mirror_hardwares and mirror_hw in step_group.mirror_hardwares %}
|
||||
{% set step = step_group %}
|
||||
- label: "{{ step.agent_pool }}: {{ step.label }}"
|
||||
depends_on: amd-build
|
||||
agents:
|
||||
{% if step.agent_pool %}
|
||||
queue: amd_{{ step.agent_pool }}
|
||||
{% else %}
|
||||
queue: amd_mi300_1
|
||||
{% endif %}
|
||||
{{ amd_native_k8s_plugin(" ", step) }}
|
||||
{% set cmd_body = (step.command or (step.commands | join("\n"))) | trim %}
|
||||
{% set indented_cmd = cmd_body | replace("\n", "\n ") %}
|
||||
command: bash .buildkite/amd/scripts/run-amd-test.sh
|
||||
env:
|
||||
AMD_CI_RUNTIME: "native"
|
||||
NATIVE_CI: "true"
|
||||
VLLM_CI_DOCKER_DISABLED: "1"
|
||||
VLLM_CI_EXPECTED_GPU_COUNT: "{{ step.agent_pool[-1:] }}"
|
||||
VLLM_CI_REQUIRE_PERSISTENT_HF_CACHE: "1"
|
||||
HF_HOME: "/home/buildkite-agent/huggingface"
|
||||
HF_HUB_DOWNLOAD_TIMEOUT: "300"
|
||||
HF_HUB_ETAG_TIMEOUT: "60"
|
||||
MIOPEN_DEBUG_CONV_DIRECT: "0"
|
||||
MIOPEN_DEBUG_CONV_GEMM: "0"
|
||||
VLLM_ROCM_USE_AITER: "0"
|
||||
PYTORCH_ROCM_ARCH: ""
|
||||
{% if step.env %}
|
||||
{% for name in step.env %}
|
||||
{{ name }}: "{{ step.env[name] }}"
|
||||
{% endfor %}
|
||||
{% endif %}
|
||||
TEST_COMMANDS: |-
|
||||
(command rocm-smi || true) && cd {{ (step.working_dir or default_working_dir) | safe }}
|
||||
{% if env %}
|
||||
{% for name in env %}
|
||||
export {{ name }}="{{ env[name] }}"
|
||||
{% endfor %}
|
||||
{% endif %}
|
||||
{% if step.env %}
|
||||
{% for name in step.env %}
|
||||
export {{ name }}="{{ step.env[name] }}"
|
||||
{% endfor %}
|
||||
{% endif %}
|
||||
{% if "mi250" in step.agent_pool %}
|
||||
python3 -m pip uninstall -y amd-aiter
|
||||
{% endif %}
|
||||
{{ indented_cmd | safe }}
|
||||
priority: 100
|
||||
{% if step.parallelism %}
|
||||
parallelism: {{ step.parallelism }}
|
||||
{% endif %}
|
||||
{% if step.artifact_paths %}
|
||||
artifact_paths:
|
||||
{% for path in step.artifact_paths %}
|
||||
- "{{ path }}"
|
||||
{% endfor %}
|
||||
{% endif %}
|
||||
{% if step.retry %}
|
||||
retry:
|
||||
automatic:
|
||||
{% for retry_rule in step.retry.automatic %}
|
||||
- exit_status: {{ retry_rule.exit_status }}
|
||||
limit: {{ retry_rule.limit }}
|
||||
{% endfor %}
|
||||
{% endif %}
|
||||
{% if step.timeout_in_minutes %}
|
||||
timeout_in_minutes: {{ step.timeout_in_minutes }}
|
||||
{% endif %}
|
||||
{% if step.grade and step.grade == "Blocking" %}
|
||||
soft_fail: false
|
||||
{% else %}
|
||||
soft_fail: true
|
||||
{% endif%}
|
||||
{% endif %}
|
||||
{% endfor %}
|
||||
Executable
+246
@@ -0,0 +1,246 @@
|
||||
#!/bin/bash
|
||||
# vllm-omni customized version
|
||||
# Based on: https://github.com/vllm-project/ci-infra/blob/main/buildkite/bootstrap-amd.sh
|
||||
# Last synced: 2025-12-15
|
||||
# Modifications: Use local template file instead of downloading from ci-infra
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
if [[ -z "${RUN_ALL:-}" ]]; then
|
||||
RUN_ALL=0
|
||||
fi
|
||||
|
||||
if [[ -z "${NIGHTLY:-}" ]]; then
|
||||
NIGHTLY=0
|
||||
fi
|
||||
|
||||
if [[ -z "${VLLM_CI_BRANCH:-}" ]]; then
|
||||
VLLM_CI_BRANCH="main"
|
||||
fi
|
||||
|
||||
if [[ -z "${AMD_MIRROR_HW:-}" ]]; then
|
||||
AMD_MIRROR_HW="amdproduction"
|
||||
fi
|
||||
|
||||
if [[ -z "${DOCS_ONLY_DISABLE:-}" ]]; then
|
||||
DOCS_ONLY_DISABLE=0
|
||||
fi
|
||||
|
||||
fail_fast() {
|
||||
DISABLE_LABEL="ci-no-fail-fast"
|
||||
# If BUILDKITE_PULL_REQUEST != "false", then we check the PR labels using curl and jq
|
||||
if [ "$BUILDKITE_PULL_REQUEST" != "false" ]; then
|
||||
PR_LABELS=$(curl -s "https://api.github.com/repos/vllm-project/vllm-omni/pulls/$BUILDKITE_PULL_REQUEST" | jq -r '.labels[].name')
|
||||
if [[ $PR_LABELS == *"$DISABLE_LABEL"* ]]; then
|
||||
echo false
|
||||
else
|
||||
echo true
|
||||
fi
|
||||
else
|
||||
echo false # not a PR or BUILDKITE_PULL_REQUEST not set
|
||||
fi
|
||||
}
|
||||
|
||||
check_run_all_label() {
|
||||
RUN_ALL_LABEL="ready-run-all-tests"
|
||||
# If BUILDKITE_PULL_REQUEST != "false", then we check the PR labels using curl and jq
|
||||
if [ "$BUILDKITE_PULL_REQUEST" != "false" ]; then
|
||||
PR_LABELS=$(curl -s "https://api.github.com/repos/vllm-project/vllm-omni/pulls/$BUILDKITE_PULL_REQUEST" | jq -r '.labels[].name')
|
||||
if [[ $PR_LABELS == *"$RUN_ALL_LABEL"* ]]; then
|
||||
echo true
|
||||
else
|
||||
echo false
|
||||
fi
|
||||
else
|
||||
echo false # not a PR or BUILDKITE_PULL_REQUEST not set
|
||||
fi
|
||||
}
|
||||
|
||||
if [[ -z "${COV_ENABLED:-}" ]]; then
|
||||
COV_ENABLED=0
|
||||
fi
|
||||
|
||||
upload_pipeline() {
|
||||
echo "Uploading pipeline..."
|
||||
# Install minijinja
|
||||
ls .buildkite || buildkite-agent annotate --style error 'Please merge upstream main branch for buildkite CI'
|
||||
curl -sSfL https://github.com/mitsuhiko/minijinja/releases/download/2.3.1/minijinja-cli-installer.sh | sh
|
||||
source /var/lib/buildkite-agent/.cargo/env
|
||||
|
||||
if [[ $BUILDKITE_PIPELINE_SLUG == "fastcheck" ]]; then
|
||||
AMD_MIRROR_HW="amdtentative"
|
||||
fi
|
||||
|
||||
# Use local template file for vllm-omni
|
||||
cp .buildkite/test-template-amd-omni.j2 .buildkite/test-template.j2
|
||||
|
||||
|
||||
# (WIP) Use pipeline generator instead of jinja template
|
||||
if [ -e ".buildkite/pipeline_generator/pipeline_generator.py" ]; then
|
||||
python -m pip install click pydantic
|
||||
python .buildkite/pipeline_generator/pipeline_generator.py --run_all=$RUN_ALL --list_file_diff="$LIST_FILE_DIFF" --nightly="$NIGHTLY" --mirror_hw="$AMD_MIRROR_HW"
|
||||
buildkite-agent pipeline upload .buildkite/pipeline.yaml
|
||||
exit 0
|
||||
fi
|
||||
echo "List file diff: $LIST_FILE_DIFF"
|
||||
echo "Run all: $RUN_ALL"
|
||||
echo "Nightly: $NIGHTLY"
|
||||
echo "AMD Mirror HW: $AMD_MIRROR_HW"
|
||||
|
||||
FAIL_FAST=$(fail_fast)
|
||||
|
||||
cd .buildkite
|
||||
|
||||
# Select test definition file: merge suite for main, ready suite for PRs
|
||||
if [[ $BUILDKITE_BRANCH == "main" ]]; then
|
||||
TEST_YAML="test-amd-merge.yml"
|
||||
else
|
||||
TEST_YAML="test-amd-ready.yaml"
|
||||
fi
|
||||
|
||||
(
|
||||
set -x
|
||||
# Output pipeline.yaml with all blank lines removed
|
||||
minijinja-cli test-template.j2 "$TEST_YAML" \
|
||||
-D branch="$BUILDKITE_BRANCH" \
|
||||
-D list_file_diff="$LIST_FILE_DIFF" \
|
||||
-D run_all="$RUN_ALL" \
|
||||
-D nightly="$NIGHTLY" \
|
||||
-D mirror_hw="$AMD_MIRROR_HW" \
|
||||
-D fail_fast="$FAIL_FAST" \
|
||||
-D vllm_use_precompiled="$VLLM_USE_PRECOMPILED" \
|
||||
-D vllm_merge_base_commit="$(git merge-base origin/main HEAD)" \
|
||||
-D cov_enabled="$COV_ENABLED" \
|
||||
-D vllm_ci_branch="$VLLM_CI_BRANCH" \
|
||||
| sed '/^[[:space:]]*$/d' \
|
||||
> pipeline.yaml
|
||||
)
|
||||
cat pipeline.yaml
|
||||
buildkite-agent artifact upload pipeline.yaml
|
||||
buildkite-agent pipeline upload pipeline.yaml
|
||||
exit 0
|
||||
}
|
||||
|
||||
get_diff() {
|
||||
$(git add .)
|
||||
echo $(git diff --name-only --diff-filter=ACMDR $(git merge-base origin/main HEAD))
|
||||
}
|
||||
|
||||
get_diff_main() {
|
||||
$(git add .)
|
||||
echo $(git diff --name-only --diff-filter=ACMDR HEAD~1)
|
||||
}
|
||||
|
||||
file_diff=$(get_diff)
|
||||
if [[ $BUILDKITE_BRANCH == "main" ]]; then
|
||||
file_diff=$(get_diff_main)
|
||||
fi
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# Early exit start: skip pipeline if conditions are met
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
# skip pipeline if all changed files are under docs/
|
||||
if [[ "${DOCS_ONLY_DISABLE}" != "1" ]]; then
|
||||
if [[ -n "${file_diff:-}" ]]; then
|
||||
docs_only=1
|
||||
# Robust iteration over newline-separated file_diff
|
||||
while IFS= read -r f; do
|
||||
[[ -z "$f" ]] && continue
|
||||
# **Policy:** only skip if *every* path starts with docs/
|
||||
if [[ "$f" != docs/* ]]; then
|
||||
docs_only=0
|
||||
break
|
||||
fi
|
||||
done < <(printf '%s\n' "$file_diff" | tr ' ' '\n' | tr -d '\r')
|
||||
|
||||
if [[ "$docs_only" -eq 1 ]]; then
|
||||
buildkite-agent annotate ":memo: CI skipped — docs/** only changes detected
|
||||
|
||||
\`\`\`
|
||||
${file_diff}
|
||||
\`\`\`" --style "info" || true
|
||||
echo "[docs-only] All changes are under docs/. Exiting before pipeline upload."
|
||||
exit 0
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# Early exit end
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
patterns=(
|
||||
"docker/Dockerfile"
|
||||
"CMakeLists.txt"
|
||||
"requirements/common.txt"
|
||||
"requirements/cuda.txt"
|
||||
"requirements/build.txt"
|
||||
"requirements/test.txt"
|
||||
"setup.py"
|
||||
"csrc/"
|
||||
"cmake/"
|
||||
)
|
||||
|
||||
ignore_patterns=(
|
||||
"docker/Dockerfile."
|
||||
"csrc/cpu"
|
||||
"csrc/rocm"
|
||||
"cmake/hipify.py"
|
||||
"cmake/cpu_extension.cmake"
|
||||
)
|
||||
|
||||
for file in $file_diff; do
|
||||
# First check if file matches any pattern
|
||||
matches_pattern=0
|
||||
for pattern in "${patterns[@]}"; do
|
||||
if [[ $file == $pattern* ]] || [[ $file == $pattern ]]; then
|
||||
matches_pattern=1
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
# If file matches pattern, check it's not in ignore patterns
|
||||
if [[ $matches_pattern -eq 1 ]]; then
|
||||
matches_ignore=0
|
||||
for ignore in "${ignore_patterns[@]}"; do
|
||||
if [[ $file == $ignore* ]] || [[ $file == $ignore ]]; then
|
||||
matches_ignore=1
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
if [[ $matches_ignore -eq 0 ]]; then
|
||||
RUN_ALL=1
|
||||
echo "Found changes: $file. Run all tests"
|
||||
break
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
# Check for ready-run-all-tests label
|
||||
LABEL_RUN_ALL=$(check_run_all_label)
|
||||
if [[ $LABEL_RUN_ALL == true ]]; then
|
||||
RUN_ALL=1
|
||||
NIGHTLY=1
|
||||
echo "Found 'ready-run-all-tests' label. Running all tests including optional tests."
|
||||
fi
|
||||
|
||||
# Decide whether to use precompiled wheels
|
||||
# Relies on existing patterns array as a basis.
|
||||
if [[ -n "${VLLM_USE_PRECOMPILED:-}" ]]; then
|
||||
echo "VLLM_USE_PRECOMPILED is already set to: $VLLM_USE_PRECOMPILED"
|
||||
elif [[ $RUN_ALL -eq 1 ]]; then
|
||||
export VLLM_USE_PRECOMPILED=0
|
||||
echo "Detected critical changes, building wheels from source"
|
||||
else
|
||||
export VLLM_USE_PRECOMPILED=1
|
||||
echo "No critical changes, using precompiled wheels"
|
||||
fi
|
||||
|
||||
|
||||
LIST_FILE_DIFF=$(get_diff | tr ' ' '|')
|
||||
if [[ $BUILDKITE_BRANCH == "main" ]]; then
|
||||
LIST_FILE_DIFF=$(get_diff_main | tr ' ' '|')
|
||||
fi
|
||||
upload_pipeline
|
||||
Executable
+51
@@ -0,0 +1,51 @@
|
||||
#!/bin/bash
|
||||
# vllm-omni Intel bootstrap
|
||||
# Uses static pipeline-intel.yaml for Intel XPU tests
|
||||
# Based on: bootstrap-amd-omni.sh
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
if [[ -z "${DOCS_ONLY_DISABLE:-}" ]]; then
|
||||
DOCS_ONLY_DISABLE=0
|
||||
fi
|
||||
|
||||
upload_pipeline() {
|
||||
echo "--- 🛠 Preparing Intel pipeline"
|
||||
buildkite-agent pipeline upload .buildkite/pipeline-intel.yaml
|
||||
}
|
||||
|
||||
get_diff() {
|
||||
$(git add .)
|
||||
echo $(git diff --name-only --diff-filter=ACMDR $(git merge-base origin/main HEAD))
|
||||
}
|
||||
|
||||
get_diff_main() {
|
||||
$(git add .)
|
||||
echo $(git diff --name-only --diff-filter=ACMDR HEAD~1)
|
||||
}
|
||||
|
||||
file_diff=$(get_diff)
|
||||
if [[ $BUILDKITE_BRANCH == "main" ]]; then
|
||||
file_diff=$(get_diff_main)
|
||||
fi
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# Early exit: skip pipeline if all changed files are under docs/
|
||||
# ----------------------------------------------------------------------
|
||||
if [[ "${DOCS_ONLY_DISABLE}" != "1" ]] && [[ -n "${file_diff:-}" ]]; then
|
||||
docs_only=1
|
||||
while IFS= read -r f; do
|
||||
[[ -z "$f" ]] && continue
|
||||
if [[ "$f" != docs/* ]]; then
|
||||
docs_only=0
|
||||
break
|
||||
fi
|
||||
done < <(printf '%s\n' "$file_diff" | tr ' ' '\n' | tr -d '\r')
|
||||
|
||||
if [[ "$docs_only" -eq 1 ]]; then
|
||||
buildkite-agent annotate ":memo: CI skipped — docs only" --style "info"
|
||||
exit 0
|
||||
fi
|
||||
fi
|
||||
|
||||
upload_pipeline
|
||||
@@ -1,702 +0,0 @@
|
||||
# Uploader-only hardware presets for CUDA / NPU test YAML.
|
||||
# Referenced as ``mirror_hardwares: l4_1`` on a step; ``upload_pipeline.py`` expands to
|
||||
# ``agents`` (+ optional top-level ``image`` for NPU) + ``plugins`` (+ ``retry``
|
||||
# for ``l4_*``), then strips ``mirror_hardwares`` before Buildkite upload.
|
||||
#
|
||||
# Omit the key: compose {chip}_{n} from pytest -m SKU + cards_n (max of several
|
||||
# positives; not cards_1 with no positive cards_* → that chip's highest existing preset).
|
||||
# Unset MIRROR_HW matches H100 or L4 in -m (both → H100); no match skips the step.
|
||||
# MIRROR_HW=b200 matches B200 in -m (else the step is skipped).
|
||||
# MIRROR_HW must be empty or b200 (case-insensitive); unknown values fail the upload.
|
||||
# CUDA preset strings (h100_4, l4_1) are omitted when MIRROR_HW=b200.
|
||||
# Unset MIRROR_HW keeps string presets. NPU presets ignore MIRROR_HW.
|
||||
#
|
||||
# CUDA naming: {chip}_{count}
|
||||
# l4_1, l4_2, l4_3, l4_4, h100_1 … h100_4, b200_1 … b200_4
|
||||
# NPU naming: {queue}_{resource_class}
|
||||
# a2b3_npu_1 — ascend-a2b3, resource_class npu-1 (A2/B3 CI image)
|
||||
# a2b3_npu_4 — ascend-a2b3, resource_class npu-4
|
||||
# a3_npu_2 — ascend-a3, resource_class npu-2 (A3 CI image tag …:$BUILDKITE_COMMIT-a3)
|
||||
# a5_npu_2 — ascend-a5, resource_class npu-2
|
||||
# 310p_npu_1 — ascend-310p, resource_class npu-1
|
||||
#
|
||||
# B200 (b200-k8s): matches vLLM device ``b200-k8s`` from ci-infra
|
||||
# (AgentQueue.B200_K8S + b200_plugin_template). Queue routes the K8s pool;
|
||||
# podSpec uses hostNetwork + /raid|/mnt/shared (no H100-style nodeSelector).
|
||||
#
|
||||
# The ``l4_*`` presets run on the EKS ``l4-k8s`` queue (agent-stack-k8s),
|
||||
# packed onto g6.12xlarge L4 nodes (``vllm.ci/gpu-pool=l4x4``). GPU / CPU /
|
||||
# memory / shm requests scale with card count (1 GPU = 10 CPU / 40Gi / 24Gi shm).
|
||||
# Retry covers lost agents / killed pods, not pytest or OOM (exit 1 / 137).
|
||||
# The Kubernetes HF secret deliberately uses VLLM_CI_HF_TOKEN. Scheduled
|
||||
# build-level HF_TOKEN values can override pod env entries with the same name;
|
||||
# upload_pipeline.py restores the secret-backed value inside the job shell.
|
||||
|
||||
x-npu-k8s-plugins: &npu_k8s_plugins
|
||||
- kubernetes:
|
||||
podSpecPatch:
|
||||
imagePullSecrets:
|
||||
- name: swr-secret
|
||||
|
||||
# Infra-only retry. Omit exit 1 (pytest failed) and 137 (cgroup OOM / SIGKILL
|
||||
# of the test process). ci-infra ``K8S_RETRY`` still retries exit 1.
|
||||
x-l4-k8s-retry: &l4_k8s_retry
|
||||
automatic:
|
||||
- exit_status: -1
|
||||
limit: 1
|
||||
- exit_status: 128
|
||||
limit: 1
|
||||
- signal_reason: agent_stop
|
||||
limit: 1
|
||||
- signal_reason: agent_refused
|
||||
limit: 1
|
||||
|
||||
|
||||
mirror_hardwares:
|
||||
l4_1:
|
||||
agents:
|
||||
queue: l4-k8s
|
||||
retry: *l4_k8s_retry
|
||||
plugins:
|
||||
- kubernetes:
|
||||
metadata:
|
||||
annotations:
|
||||
cluster-autoscaler.kubernetes.io/safe-to-evict: "false"
|
||||
karpenter.sh/do-not-disrupt: "true"
|
||||
podSpec:
|
||||
priorityClassName: ci
|
||||
serviceAccountName: buildkite-gpu-jobs
|
||||
tolerations:
|
||||
- key: nvidia.com/gpu
|
||||
operator: Exists
|
||||
effect: NoSchedule
|
||||
nodeSelector:
|
||||
vllm.ci/gpu-pool: l4x4
|
||||
containers:
|
||||
- image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 1
|
||||
requests:
|
||||
nvidia.com/gpu: 1
|
||||
cpu: "10"
|
||||
memory: 40Gi
|
||||
volumeMounts:
|
||||
- name: devshm
|
||||
mountPath: /dev/shm
|
||||
- name: hf-cache
|
||||
mountPath: /fsx/hf_cache
|
||||
env:
|
||||
- name: HF_HOME
|
||||
value: /fsx/hf_cache
|
||||
- name: VLLM_CI_HF_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: hf-token-secret
|
||||
key: token
|
||||
volumes:
|
||||
- name: devshm
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
sizeLimit: 24Gi
|
||||
- name: hf-cache
|
||||
hostPath:
|
||||
path: /fsx/hf_cache
|
||||
type: DirectoryOrCreate
|
||||
|
||||
l4_2:
|
||||
agents:
|
||||
queue: l4-k8s
|
||||
retry: *l4_k8s_retry
|
||||
plugins:
|
||||
- kubernetes:
|
||||
metadata:
|
||||
annotations:
|
||||
cluster-autoscaler.kubernetes.io/safe-to-evict: "false"
|
||||
karpenter.sh/do-not-disrupt: "true"
|
||||
podSpec:
|
||||
priorityClassName: ci
|
||||
serviceAccountName: buildkite-gpu-jobs
|
||||
tolerations:
|
||||
- key: nvidia.com/gpu
|
||||
operator: Exists
|
||||
effect: NoSchedule
|
||||
nodeSelector:
|
||||
vllm.ci/gpu-pool: l4x4
|
||||
containers:
|
||||
- image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 2
|
||||
requests:
|
||||
nvidia.com/gpu: 2
|
||||
cpu: "20"
|
||||
memory: 80Gi
|
||||
volumeMounts:
|
||||
- name: devshm
|
||||
mountPath: /dev/shm
|
||||
- name: hf-cache
|
||||
mountPath: /fsx/hf_cache
|
||||
env:
|
||||
- name: HF_HOME
|
||||
value: /fsx/hf_cache
|
||||
- name: VLLM_CI_HF_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: hf-token-secret
|
||||
key: token
|
||||
volumes:
|
||||
- name: devshm
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
sizeLimit: 48Gi
|
||||
- name: hf-cache
|
||||
hostPath:
|
||||
path: /fsx/hf_cache
|
||||
type: DirectoryOrCreate
|
||||
|
||||
l4_3:
|
||||
agents:
|
||||
queue: l4-k8s
|
||||
retry: *l4_k8s_retry
|
||||
plugins:
|
||||
- kubernetes:
|
||||
metadata:
|
||||
annotations:
|
||||
cluster-autoscaler.kubernetes.io/safe-to-evict: "false"
|
||||
karpenter.sh/do-not-disrupt: "true"
|
||||
podSpec:
|
||||
priorityClassName: ci
|
||||
serviceAccountName: buildkite-gpu-jobs
|
||||
tolerations:
|
||||
- key: nvidia.com/gpu
|
||||
operator: Exists
|
||||
effect: NoSchedule
|
||||
nodeSelector:
|
||||
vllm.ci/gpu-pool: l4x4
|
||||
containers:
|
||||
- image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 3
|
||||
requests:
|
||||
nvidia.com/gpu: 3
|
||||
cpu: "30"
|
||||
memory: 120Gi
|
||||
volumeMounts:
|
||||
- name: devshm
|
||||
mountPath: /dev/shm
|
||||
- name: hf-cache
|
||||
mountPath: /fsx/hf_cache
|
||||
env:
|
||||
- name: HF_HOME
|
||||
value: /fsx/hf_cache
|
||||
- name: VLLM_CI_HF_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: hf-token-secret
|
||||
key: token
|
||||
volumes:
|
||||
- name: devshm
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
sizeLimit: 72Gi
|
||||
- name: hf-cache
|
||||
hostPath:
|
||||
path: /fsx/hf_cache
|
||||
type: DirectoryOrCreate
|
||||
|
||||
l4_4:
|
||||
agents:
|
||||
queue: l4-k8s
|
||||
retry: *l4_k8s_retry
|
||||
plugins:
|
||||
- kubernetes:
|
||||
metadata:
|
||||
annotations:
|
||||
cluster-autoscaler.kubernetes.io/safe-to-evict: "false"
|
||||
karpenter.sh/do-not-disrupt: "true"
|
||||
podSpec:
|
||||
priorityClassName: ci
|
||||
serviceAccountName: buildkite-gpu-jobs
|
||||
tolerations:
|
||||
- key: nvidia.com/gpu
|
||||
operator: Exists
|
||||
effect: NoSchedule
|
||||
nodeSelector:
|
||||
vllm.ci/gpu-pool: l4x4
|
||||
containers:
|
||||
- image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 4
|
||||
requests:
|
||||
nvidia.com/gpu: 4
|
||||
cpu: "40"
|
||||
memory: 160Gi
|
||||
volumeMounts:
|
||||
- name: devshm
|
||||
mountPath: /dev/shm
|
||||
- name: hf-cache
|
||||
mountPath: /fsx/hf_cache
|
||||
env:
|
||||
- name: HF_HOME
|
||||
value: /fsx/hf_cache
|
||||
- name: VLLM_CI_HF_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: hf-token-secret
|
||||
key: token
|
||||
volumes:
|
||||
- name: devshm
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
sizeLimit: 96Gi
|
||||
- name: hf-cache
|
||||
hostPath:
|
||||
path: /fsx/hf_cache
|
||||
type: DirectoryOrCreate
|
||||
|
||||
h100_1:
|
||||
agents:
|
||||
queue: mithril-h100-pool
|
||||
plugins:
|
||||
- kubernetes:
|
||||
podSpec:
|
||||
containers:
|
||||
- image: 936637512419.dkr.ecr.us-west-2.amazonaws.com/vllm-ci-pull-through-cache/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 1
|
||||
volumeMounts:
|
||||
- name: devshm
|
||||
mountPath: /dev/shm
|
||||
- name: hf-cache
|
||||
mountPath: /root/.cache/huggingface
|
||||
env:
|
||||
- name: HF_HOME
|
||||
value: /root/.cache/huggingface
|
||||
- name: VLLM_CI_HF_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: hf-token-secret
|
||||
key: token
|
||||
nodeSelector:
|
||||
node.kubernetes.io/instance-type: gpu-h100-sxm
|
||||
volumes:
|
||||
- name: devshm
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
- name: hf-cache
|
||||
hostPath:
|
||||
path: /mnt/hf-cache
|
||||
type: DirectoryOrCreate
|
||||
|
||||
h100_2:
|
||||
agents:
|
||||
queue: mithril-h100-pool
|
||||
plugins:
|
||||
- kubernetes:
|
||||
podSpec:
|
||||
containers:
|
||||
- image: 936637512419.dkr.ecr.us-west-2.amazonaws.com/vllm-ci-pull-through-cache/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 2
|
||||
volumeMounts:
|
||||
- name: devshm
|
||||
mountPath: /dev/shm
|
||||
- name: hf-cache
|
||||
mountPath: /root/.cache/huggingface
|
||||
env:
|
||||
- name: HF_HOME
|
||||
value: /root/.cache/huggingface
|
||||
- name: VLLM_CI_HF_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: hf-token-secret
|
||||
key: token
|
||||
nodeSelector:
|
||||
node.kubernetes.io/instance-type: gpu-h100-sxm
|
||||
volumes:
|
||||
- name: devshm
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
- name: hf-cache
|
||||
hostPath:
|
||||
path: /mnt/hf-cache
|
||||
type: DirectoryOrCreate
|
||||
|
||||
h100_3:
|
||||
agents:
|
||||
queue: mithril-h100-pool
|
||||
plugins:
|
||||
- kubernetes:
|
||||
podSpec:
|
||||
containers:
|
||||
- image: 936637512419.dkr.ecr.us-west-2.amazonaws.com/vllm-ci-pull-through-cache/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 3
|
||||
volumeMounts:
|
||||
- name: devshm
|
||||
mountPath: /dev/shm
|
||||
- name: hf-cache
|
||||
mountPath: /root/.cache/huggingface
|
||||
env:
|
||||
- name: HF_HOME
|
||||
value: /root/.cache/huggingface
|
||||
- name: VLLM_CI_HF_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: hf-token-secret
|
||||
key: token
|
||||
nodeSelector:
|
||||
node.kubernetes.io/instance-type: gpu-h100-sxm
|
||||
volumes:
|
||||
- name: devshm
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
- name: hf-cache
|
||||
hostPath:
|
||||
path: /mnt/hf-cache
|
||||
type: DirectoryOrCreate
|
||||
|
||||
h100_4:
|
||||
agents:
|
||||
queue: mithril-h100-pool
|
||||
plugins:
|
||||
- kubernetes:
|
||||
podSpec:
|
||||
containers:
|
||||
- image: 936637512419.dkr.ecr.us-west-2.amazonaws.com/vllm-ci-pull-through-cache/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 4
|
||||
volumeMounts:
|
||||
- name: devshm
|
||||
mountPath: /dev/shm
|
||||
- name: hf-cache
|
||||
mountPath: /root/.cache/huggingface
|
||||
env:
|
||||
- name: HF_HOME
|
||||
value: /root/.cache/huggingface
|
||||
- name: VLLM_CI_HF_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: hf-token-secret
|
||||
key: token
|
||||
nodeSelector:
|
||||
node.kubernetes.io/instance-type: gpu-h100-sxm
|
||||
volumes:
|
||||
- name: devshm
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
- name: hf-cache
|
||||
hostPath:
|
||||
path: /mnt/hf-cache
|
||||
type: DirectoryOrCreate
|
||||
|
||||
# vLLM device: b200-k8s → agents.queue b200-k8s + Buildkite K8s agent-stack podSpec
|
||||
b200_1:
|
||||
agents:
|
||||
queue: b200-k8s
|
||||
plugins:
|
||||
- kubernetes:
|
||||
podSpec:
|
||||
runtimeClassName: nvidia
|
||||
hostNetwork: true
|
||||
dnsPolicy: ClusterFirstWithHostNet
|
||||
imagePullSecrets:
|
||||
- name: k8s-ecr-login-renew-docker-secret
|
||||
containers:
|
||||
- image: 936637512419.dkr.ecr.us-west-2.amazonaws.com/vllm-ci-pull-through-cache/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 1
|
||||
securityContext:
|
||||
capabilities:
|
||||
add:
|
||||
- IPC_LOCK
|
||||
- SYS_RESOURCE
|
||||
volumeMounts:
|
||||
- name: devshm
|
||||
mountPath: /dev/shm
|
||||
- name: raid
|
||||
mountPath: /raid
|
||||
- name: shared
|
||||
mountPath: /mnt/shared
|
||||
env:
|
||||
- name: NCCL_CUMEM_HOST_ENABLE
|
||||
value: "0"
|
||||
- name: HF_HOME
|
||||
value: /raid/hf_cache
|
||||
- name: VLLM_CI_HF_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: hf-token-secret
|
||||
key: token
|
||||
volumes:
|
||||
- name: devshm
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
- name: raid
|
||||
hostPath:
|
||||
path: /raid
|
||||
type: DirectoryOrCreate
|
||||
- name: shared
|
||||
hostPath:
|
||||
path: /mnt/shared
|
||||
type: DirectoryOrCreate
|
||||
|
||||
b200_2:
|
||||
agents:
|
||||
queue: b200-k8s
|
||||
plugins:
|
||||
- kubernetes:
|
||||
podSpec:
|
||||
runtimeClassName: nvidia
|
||||
hostNetwork: true
|
||||
dnsPolicy: ClusterFirstWithHostNet
|
||||
imagePullSecrets:
|
||||
- name: k8s-ecr-login-renew-docker-secret
|
||||
containers:
|
||||
- image: 936637512419.dkr.ecr.us-west-2.amazonaws.com/vllm-ci-pull-through-cache/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 2
|
||||
securityContext:
|
||||
capabilities:
|
||||
add:
|
||||
- IPC_LOCK
|
||||
- SYS_RESOURCE
|
||||
volumeMounts:
|
||||
- name: devshm
|
||||
mountPath: /dev/shm
|
||||
- name: raid
|
||||
mountPath: /raid
|
||||
- name: shared
|
||||
mountPath: /mnt/shared
|
||||
env:
|
||||
- name: NCCL_CUMEM_HOST_ENABLE
|
||||
value: "0"
|
||||
- name: HF_HOME
|
||||
value: /raid/hf_cache
|
||||
- name: VLLM_CI_HF_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: hf-token-secret
|
||||
key: token
|
||||
volumes:
|
||||
- name: devshm
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
- name: raid
|
||||
hostPath:
|
||||
path: /raid
|
||||
type: DirectoryOrCreate
|
||||
- name: shared
|
||||
hostPath:
|
||||
path: /mnt/shared
|
||||
type: DirectoryOrCreate
|
||||
|
||||
b200_3:
|
||||
agents:
|
||||
queue: b200-k8s
|
||||
plugins:
|
||||
- kubernetes:
|
||||
podSpec:
|
||||
runtimeClassName: nvidia
|
||||
hostNetwork: true
|
||||
dnsPolicy: ClusterFirstWithHostNet
|
||||
imagePullSecrets:
|
||||
- name: k8s-ecr-login-renew-docker-secret
|
||||
containers:
|
||||
- image: 936637512419.dkr.ecr.us-west-2.amazonaws.com/vllm-ci-pull-through-cache/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 3
|
||||
securityContext:
|
||||
capabilities:
|
||||
add:
|
||||
- IPC_LOCK
|
||||
- SYS_RESOURCE
|
||||
volumeMounts:
|
||||
- name: devshm
|
||||
mountPath: /dev/shm
|
||||
- name: raid
|
||||
mountPath: /raid
|
||||
- name: shared
|
||||
mountPath: /mnt/shared
|
||||
env:
|
||||
- name: NCCL_CUMEM_HOST_ENABLE
|
||||
value: "0"
|
||||
- name: HF_HOME
|
||||
value: /raid/hf_cache
|
||||
- name: VLLM_CI_HF_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: hf-token-secret
|
||||
key: token
|
||||
volumes:
|
||||
- name: devshm
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
- name: raid
|
||||
hostPath:
|
||||
path: /raid
|
||||
type: DirectoryOrCreate
|
||||
- name: shared
|
||||
hostPath:
|
||||
path: /mnt/shared
|
||||
type: DirectoryOrCreate
|
||||
|
||||
b200_4:
|
||||
agents:
|
||||
queue: b200-k8s
|
||||
plugins:
|
||||
- kubernetes:
|
||||
podSpec:
|
||||
runtimeClassName: nvidia
|
||||
hostNetwork: true
|
||||
dnsPolicy: ClusterFirstWithHostNet
|
||||
imagePullSecrets:
|
||||
- name: k8s-ecr-login-renew-docker-secret
|
||||
containers:
|
||||
- image: 936637512419.dkr.ecr.us-west-2.amazonaws.com/vllm-ci-pull-through-cache/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 4
|
||||
securityContext:
|
||||
capabilities:
|
||||
add:
|
||||
- IPC_LOCK
|
||||
- SYS_RESOURCE
|
||||
volumeMounts:
|
||||
- name: devshm
|
||||
mountPath: /dev/shm
|
||||
- name: raid
|
||||
mountPath: /raid
|
||||
- name: shared
|
||||
mountPath: /mnt/shared
|
||||
env:
|
||||
- name: NCCL_CUMEM_HOST_ENABLE
|
||||
value: "0"
|
||||
- name: HF_HOME
|
||||
value: /raid/hf_cache
|
||||
- name: VLLM_CI_HF_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: hf-token-secret
|
||||
key: token
|
||||
volumes:
|
||||
- name: devshm
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
- name: raid
|
||||
hostPath:
|
||||
path: /raid
|
||||
type: DirectoryOrCreate
|
||||
- name: shared
|
||||
hostPath:
|
||||
path: /mnt/shared
|
||||
type: DirectoryOrCreate
|
||||
|
||||
a2b3_npu_1:
|
||||
agents:
|
||||
queue: "ascend-a2b3"
|
||||
resource_class: "npu-1"
|
||||
image: "swr.cn-southwest-2.myhuaweicloud.com/modelfoundry/vllm-omni-ci-npu:${BUILDKITE_COMMIT}"
|
||||
plugins: *npu_k8s_plugins
|
||||
|
||||
a2b3_npu_4:
|
||||
agents:
|
||||
queue: "ascend-a2b3"
|
||||
resource_class: "npu-4"
|
||||
image: "swr.cn-southwest-2.myhuaweicloud.com/modelfoundry/vllm-omni-ci-npu:${BUILDKITE_COMMIT}"
|
||||
plugins: *npu_k8s_plugins
|
||||
|
||||
|
||||
a2b3_npu_8:
|
||||
agents:
|
||||
queue: "ascend-a2b3"
|
||||
resource_class: "npu-8"
|
||||
image: "swr.cn-southwest-2.myhuaweicloud.com/modelfoundry/vllm-omni-ci-npu:${BUILDKITE_COMMIT}"
|
||||
plugins: *npu_k8s_plugins
|
||||
|
||||
a3_npu_2:
|
||||
agents:
|
||||
queue: "ascend-a3"
|
||||
resource_class: "npu-2"
|
||||
image: "swr.cn-southwest-2.myhuaweicloud.com/modelfoundry/vllm-omni-ci-npu:${BUILDKITE_COMMIT}-a3"
|
||||
plugins: *npu_k8s_plugins
|
||||
|
||||
a3_npu_4:
|
||||
agents:
|
||||
queue: "ascend-a3"
|
||||
resource_class: "npu-4"
|
||||
image: "swr.cn-southwest-2.myhuaweicloud.com/modelfoundry/vllm-omni-ci-npu:${BUILDKITE_COMMIT}-a3"
|
||||
plugins: *npu_k8s_plugins
|
||||
|
||||
a3_npu_8:
|
||||
agents:
|
||||
queue: "ascend-a3"
|
||||
resource_class: "npu-8"
|
||||
image: "swr.cn-southwest-2.myhuaweicloud.com/modelfoundry/vllm-omni-ci-npu:${BUILDKITE_COMMIT}-a3"
|
||||
plugins: *npu_k8s_plugins
|
||||
|
||||
a3_npu_16:
|
||||
agents:
|
||||
queue: "ascend-a3"
|
||||
resource_class: "npu-16"
|
||||
image: "swr.cn-southwest-2.myhuaweicloud.com/modelfoundry/vllm-omni-ci-npu:${BUILDKITE_COMMIT}-a3"
|
||||
plugins: *npu_k8s_plugins
|
||||
|
||||
a5_npu_2:
|
||||
agents:
|
||||
queue: "ascend-a5"
|
||||
resource_class: "npu-2"
|
||||
image: "swr.cn-southwest-2.myhuaweicloud.com/modelfoundry/vllm-omni-ci-npu:${BUILDKITE_COMMIT}-a5"
|
||||
plugins: *npu_k8s_plugins
|
||||
|
||||
a5_npu_4:
|
||||
agents:
|
||||
queue: "ascend-a5"
|
||||
resource_class: "npu-4"
|
||||
image: "swr.cn-southwest-2.myhuaweicloud.com/modelfoundry/vllm-omni-ci-npu:${BUILDKITE_COMMIT}-a5"
|
||||
plugins: *npu_k8s_plugins
|
||||
|
||||
a5_npu_8:
|
||||
agents:
|
||||
queue: "ascend-a5"
|
||||
resource_class: "npu-8"
|
||||
image: "swr.cn-southwest-2.myhuaweicloud.com/modelfoundry/vllm-omni-ci-npu:${BUILDKITE_COMMIT}-a5"
|
||||
plugins: *npu_k8s_plugins
|
||||
|
||||
a5_npu_16:
|
||||
agents:
|
||||
queue: "ascend-a5"
|
||||
resource_class: "npu-16"
|
||||
image: "swr.cn-southwest-2.myhuaweicloud.com/modelfoundry/vllm-omni-ci-npu:${BUILDKITE_COMMIT}-a5"
|
||||
plugins: *npu_k8s_plugins
|
||||
|
||||
310p_npu_1:
|
||||
agents:
|
||||
queue: "ascend-310p"
|
||||
resource_class: "npu-1"
|
||||
image: "swr.cn-southwest-2.myhuaweicloud.com/modelfoundry/vllm-omni-ci-npu:${BUILDKITE_COMMIT}-310p"
|
||||
plugins: *npu_k8s_plugins
|
||||
|
||||
310p_npu_2:
|
||||
agents:
|
||||
queue: "ascend-310p"
|
||||
resource_class: "npu-2"
|
||||
image: "swr.cn-southwest-2.myhuaweicloud.com/modelfoundry/vllm-omni-ci-npu:${BUILDKITE_COMMIT}-310p"
|
||||
plugins: *npu_k8s_plugins
|
||||
|
||||
310p_npu_4:
|
||||
agents:
|
||||
queue: "ascend-310p"
|
||||
resource_class: "npu-4"
|
||||
image: "swr.cn-southwest-2.myhuaweicloud.com/modelfoundry/vllm-omni-ci-npu:${BUILDKITE_COMMIT}-310p"
|
||||
plugins: *npu_k8s_plugins
|
||||
@@ -1,525 +0,0 @@
|
||||
# Uploader-only path-prefix presets for CUDA L2–L5 and NPU L4 test YAML.
|
||||
# Referenced as ``source_file_dependencies: omni_qwen3_omni_function`` (or a list of
|
||||
# keys) on a step; ``upload_pipeline.py`` expands the key(s) only.
|
||||
#
|
||||
# YAML anchors above the registry hold model business-code paths. Job keys
|
||||
# ({model}_function / _perf / _accuracy / _reliability / _doc / _cov) alias
|
||||
# those anchors and add the scripts the job actually runs. Feature keys below
|
||||
# are unchanged. ``source_filter_fallback`` is not a job key: when no job-key
|
||||
# prefix matches, a matching diff keeps every job for that upload.
|
||||
#
|
||||
# YAML anchors are nested lists; the loader flattens them.
|
||||
|
||||
x-qwen3-tts: &qwen3_tts
|
||||
- vllm_omni/engine/async_omni_engine.py
|
||||
- vllm_omni/entrypoints/omni_base.py
|
||||
- vllm_omni/worker/gpu_model_runner.py
|
||||
- vllm_omni/model_executor/models/qwen3_tts/
|
||||
- vllm_omni/model_executor/models/common/qwen3_code_predictor.py
|
||||
- vllm_omni/model_executor/models/common/fused_sampling.py
|
||||
- vllm_omni/model_executor/models/common/short_kv_attention.py
|
||||
- vllm_omni/model_executor/models/common/talker_first_audio.py
|
||||
- vllm_omni/model_executor/models/common/snake_activation.py
|
||||
- vllm_omni/model_executor/models/common/alias_free_activation.py
|
||||
- vllm_omni/model_executor/stage_input_processors/qwen3_tts.py
|
||||
- vllm_omni/deploy/qwen3_tts.yaml
|
||||
- vllm_omni/deploy/qwen3_tts_high_concurrency_mrv2_single_gpu.yaml
|
||||
- vllm_omni/entrypoints/openai/serving_speech.py
|
||||
- vllm_omni/entrypoints/openai/tts_adapters/
|
||||
- vllm_omni/engine/stage_pool.py
|
||||
- vllm_omni/worker_v2/
|
||||
- vllm_omni/core/sched/
|
||||
- vllm_omni/distributed/omni_connectors/
|
||||
- vllm_omni/config/stage_config.py
|
||||
- vllm_omni/config/omni_config.py
|
||||
- vllm_omni/engine/orchestrator.py
|
||||
- vllm_omni/engine/stage_engine_core_proc.py
|
||||
- vllm_omni/engine/cuda_mps.py
|
||||
- vllm_omni/engine/stage_runtime.py
|
||||
- vllm_omni/engine/stage_init_utils.py
|
||||
- vllm_omni/deploy/qwen3_tts_mrv2.yaml
|
||||
- vllm_omni/deploy/qwen3_tts_high_concurrency_mrv2.yaml
|
||||
- vllm_omni/deploy/qwen3_tts_high_concurrency_mrv2_b4.yaml
|
||||
|
||||
x-minicpmo-4-5: &minicpmo_4_5
|
||||
- vllm_omni/engine/async_omni_engine.py
|
||||
- vllm_omni/entrypoints/omni_base.py
|
||||
- vllm_omni/model_executor/models/minicpmo_4_5/
|
||||
- vllm_omni/model_executor/models/step_audio2/
|
||||
- vllm_omni/model_executor/models/cosyvoice3/code2wav_core/
|
||||
- vllm_omni/model_executor/models/cosyvoice3/utils.py
|
||||
- vllm_omni/model_executor/stage_input_processors/minicpmo_4_5_omni.py
|
||||
- vllm_omni/deploy/minicpmo_4_5.yaml
|
||||
- vllm_omni/transformers_utils/chat_templates/minicpmo45_native.jinja
|
||||
|
||||
x-minicpmo-4-5-duplex: &minicpmo_4_5_duplex
|
||||
- vllm_omni/deploy/minicpmo_4_5_duplex.yaml
|
||||
- vllm_omni/engine/duplex/
|
||||
- vllm_omni/engine/duplex_omni_engine.py
|
||||
- vllm_omni/engine/duplex_orchestrator.py
|
||||
- vllm_omni/engine/omni_engine_base.py
|
||||
- vllm_omni/entrypoints/duplex/
|
||||
- vllm_omni/entrypoints/duplex_omni.py
|
||||
- vllm_omni/entrypoints/async_omni_base.py
|
||||
- vllm_omni/protocol/
|
||||
- vllm_omni/clients/
|
||||
- vllm_omni/outputs/duplex.py
|
||||
- vllm_omni/model_executor/duplex_sampling.py
|
||||
- vllm_omni/experimental/fullduplex/
|
||||
- vllm_omni/core/sched/
|
||||
- vllm_omni/outputs/multimodal_accumulation.py
|
||||
- vllm_omni/engine/stage_init_utils.py
|
||||
- vllm_omni/engine/orchestrator.py
|
||||
- vllm_omni/entrypoints/cli/serve.py
|
||||
- vllm_omni/entrypoints/openai/api_server.py
|
||||
- vllm_omni/entrypoints/openai/duplex_capability.py
|
||||
- vllm_omni/config/stage_config.py
|
||||
|
||||
x-qwen2-5-omni: &qwen2_5_omni
|
||||
- vllm_omni/engine/async_omni_engine.py
|
||||
- vllm_omni/entrypoints/omni_base.py
|
||||
- vllm_omni/model_executor/models/qwen2_5_omni/
|
||||
- vllm_omni/model_executor/stage_input_processors/qwen2_5_omni.py
|
||||
- vllm_omni/deploy/qwen2_5_omni.yaml
|
||||
|
||||
x-qwen3-omni: &qwen3_omni
|
||||
- vllm_omni/engine/async_omni_engine.py
|
||||
- vllm_omni/entrypoints/omni_base.py
|
||||
- vllm_omni/entrypoints/openai/serving_chat.py
|
||||
- vllm_omni/model_executor/models/qwen3_omni/
|
||||
- vllm_omni/model_executor/models/common/qwen3_code_predictor.py
|
||||
- vllm_omni/model_executor/models/common/fused_sampling.py
|
||||
- vllm_omni/model_executor/models/common/short_kv_attention.py
|
||||
- vllm_omni/model_executor/models/common/talker_first_audio.py
|
||||
- vllm_omni/model_executor/models/common/snake_activation.py
|
||||
- vllm_omni/model_executor/models/qwen2_5_omni/qwen2_5_omni_thinker.py
|
||||
- vllm_omni/model_executor/models/qwen3_tts/cuda_graph_decoder_wrapper.py
|
||||
- vllm_omni/model_executor/stage_input_processors/qwen3_omni.py
|
||||
- vllm_omni/deploy/qwen3_omni_moe.yaml
|
||||
- vllm_omni/deploy/qwen3_omni_moe_mrv2.yaml
|
||||
- vllm_omni/deploy/qwen3_omni_moe_mrv2_h200.yaml
|
||||
- vllm_omni/deploy/qwen3_omni_moe_thinking.yaml
|
||||
|
||||
x-wan22: &wan22
|
||||
- vllm_omni/diffusion/models/wan2_2/
|
||||
- vllm_omni/diffusion/models/dmd2/
|
||||
|
||||
x-cosmos3: &cosmos3
|
||||
- vllm_omni/diffusion/models/cosmos3/
|
||||
- vllm_omni/diffusion/distributed/autoencoders/autoencoder_kl_wan.py
|
||||
- vllm_omni/experimental/world_models/
|
||||
- vllm_omni/model_extras/cosmos3.py
|
||||
|
||||
x-qwen-image: &qwen_image
|
||||
- vllm_omni/diffusion/models/qwen_image/
|
||||
- vllm_omni/diffusion/models/dmd2/
|
||||
- vllm_omni/diffusion/distributed/autoencoders/autoencoder_kl_qwenimage.py
|
||||
|
||||
x-joy-image: &joy_image
|
||||
- vllm_omni/diffusion/models/joy_image/
|
||||
- vllm_omni/diffusion/distributed/autoencoders/autoencoder_kl_wan.py
|
||||
|
||||
x-hunyuan-image3: &hunyuan_image3
|
||||
- vllm_omni/diffusion/models/hunyuan_image3/
|
||||
- vllm_omni/model_executor/models/hunyuan_image3/
|
||||
- vllm_omni/deploy/hunyuan_image3_dit.yaml
|
||||
|
||||
x-minimax-h3: &minimax_h3
|
||||
- vllm_omni/diffusion/models/minimax_h3/
|
||||
- vllm_omni/diffusion/layers/indexed_modulation.py
|
||||
|
||||
x-hunyuan-video: &hunyuan_video
|
||||
- vllm_omni/diffusion/models/hunyuan_video/
|
||||
- vllm_omni/model_executor/models/hunyuan_video/
|
||||
|
||||
x-ltx2: <x2
|
||||
- vllm_omni/diffusion/models/ltx2/
|
||||
- vllm_omni/model_extras/ltx2.py
|
||||
|
||||
x-sana-video: &sana_video
|
||||
- vllm_omni/diffusion/models/sana_video/
|
||||
- vllm_omni/model_extras/sana_video.py
|
||||
|
||||
x-z-image: &z_image
|
||||
- vllm_omni/diffusion/models/z_image/
|
||||
|
||||
x-stable-audio: &stable_audio
|
||||
- vllm_omni/diffusion/models/stable_audio/
|
||||
|
||||
x-lingbot-video: &lingbot_video
|
||||
- vllm_omni/diffusion/models/lingbot_video/
|
||||
- vllm_omni/model_extras/lingbot_video.py
|
||||
|
||||
x-lingbot-world: &lingbot_world
|
||||
- vllm_omni/diffusion/models/lingbot_world/
|
||||
|
||||
x-kandinsky6: &kandinsky6
|
||||
- vllm_omni/diffusion/models/kandinsky6/
|
||||
- vllm_omni/model_extras/kandinsky6.py
|
||||
|
||||
|
||||
source_file_dependencies:
|
||||
# Hardware presets and shared test helpers. Not a job key — do not attach this
|
||||
# to a step. Used only when no job-key prefix matches: then a hit keeps every job.
|
||||
source_filter_fallback:
|
||||
- .buildkite/common/ci_mirror_hardwares.yml
|
||||
- tests/helpers/
|
||||
|
||||
# --- features ---
|
||||
diffusion_distributed_attention:
|
||||
- tests/diffusion/distributed/
|
||||
- tests/diffusion/attention/test_attention_sp.py
|
||||
- vllm_omni/diffusion/distributed/
|
||||
- vllm_omni/diffusion/attention/
|
||||
- vllm_omni/diffusion/data.py
|
||||
- vllm_omni/diffusion/models/
|
||||
|
||||
custom_pipeline:
|
||||
- tests/e2e/features/custom_pipeline/
|
||||
- tests/e2e/features/helpers/custom_pipeline.py
|
||||
- vllm_omni/diffusion/
|
||||
- vllm_omni/engine/
|
||||
- vllm_omni/entrypoints/
|
||||
- vllm_omni/inputs/
|
||||
- vllm_omni/outputs/
|
||||
|
||||
rlhf_verl_omni:
|
||||
- tests/e2e/features/rlhf_test/test_verl_omni_e2e.py
|
||||
- tests/e2e/features/helpers/verl_omni_server.py
|
||||
- tests/e2e/features/helpers/custom_pipeline.py
|
||||
- vllm_omni/diffusion/
|
||||
- vllm_omni/engine/
|
||||
- vllm_omni/entrypoints/
|
||||
- vllm_omni/inputs/
|
||||
- vllm_omni/outputs/
|
||||
|
||||
# Tiny-model diffusion jobs (shared across many model trees under models/).
|
||||
diffusion_tiny_model:
|
||||
- tests/model_tests/diffusion/
|
||||
- vllm_omni/diffusion/models/
|
||||
|
||||
diffusion_offloader:
|
||||
- tests/diffusion/offloader/
|
||||
- vllm_omni/diffusion/offloader/
|
||||
|
||||
diffusion_quantization:
|
||||
- tests/diffusion/quantization/
|
||||
- vllm_omni/diffusion/quantization/
|
||||
|
||||
# --- models ---
|
||||
# --- omni / tts jobs ---
|
||||
entrypoints_function:
|
||||
- tests/entrypoints/
|
||||
- vllm_omni/entrypoints/
|
||||
|
||||
omni_qwen3_omni_function:
|
||||
- *qwen3_omni
|
||||
- tests/e2e/offline_inference/test_qwen3_omni.py
|
||||
- tests/e2e/offline_inference/test_qwen3_omni_colocate_async.py
|
||||
- tests/e2e/offline_inference/test_qwen3_omni_autoround_w4a16_expansion.py
|
||||
- tests/e2e/offline_inference/test_qwen3_omni_modelopt_nvfp4_w4a4.py
|
||||
- tests/e2e/online_serving/test_qwen3_omni.py
|
||||
- tests/e2e/online_serving/test_qwen3_omni_expansion.py
|
||||
- tests/e2e/online_serving/test_qwen3_omni_multi_replicas.py
|
||||
|
||||
omni_qwen3_omni_accuracy:
|
||||
- *qwen3_omni
|
||||
- tests/e2e/accuracy/qwen3_omni/
|
||||
|
||||
omni_qwen3_omni_perf:
|
||||
- *qwen3_omni
|
||||
- tests/dfx/perf/scripts/run_benchmark.py
|
||||
- tests/dfx/perf/tests/test_qwen3_omni_vllm_text.json
|
||||
- tests/dfx/perf/tests/test_qwen3_omni_async_chunk.json
|
||||
- tests/dfx/perf/tests/test_qwen3_omni_no_async_chunk.json
|
||||
- tests/dfx/perf/tests/test_qwen3_omni_multi_replicas.json
|
||||
|
||||
omni_qwen3_omni_reliability:
|
||||
- *qwen3_omni
|
||||
- tests/dfx/reliability/test_reliability_qwen3_omni.py
|
||||
|
||||
omni_qwen3_omni_doc:
|
||||
- *qwen3_omni
|
||||
- examples/online_serving/qwen3_omni/
|
||||
- examples/online_serving/openai_chat_completion_client_for_multimodal_generation.py
|
||||
- tests/examples/online_serving/test_qwen3_omni.py
|
||||
|
||||
omni_qwen2_5_omni_doc:
|
||||
- *qwen2_5_omni
|
||||
- examples/online_serving/qwen2_5_omni/
|
||||
- examples/online_serving/openai_chat_completion_client_for_multimodal_generation.py
|
||||
- tests/examples/online_serving/test_qwen2_5_omni.py
|
||||
|
||||
omni_minicpmo_4_5_function:
|
||||
- *minicpmo_4_5
|
||||
- tests/e2e/offline_inference/test_minicpmo_4_5.py
|
||||
- tests/e2e/online_serving/test_minicpmo_4_5.py
|
||||
- tests/e2e/online_serving/test_minicpmo_4_5_expansion.py
|
||||
|
||||
omni_minicpmo_4_5_accuracy:
|
||||
- *minicpmo_4_5
|
||||
- tests/e2e/accuracy/minicpmo_4_5/
|
||||
|
||||
omni_duplex_eval_ci:
|
||||
- *minicpmo_4_5
|
||||
- *minicpmo_4_5_duplex
|
||||
- tests/e2e/accuracy/omni_duplex_eval/**
|
||||
- tests/e2e/online_serving/helpers/minicpmo_4_5_duplex.py
|
||||
- vllm_omni/benchmarks/duplex/**
|
||||
- vllm_omni/entrypoints/cli/benchmark/omni_duplex_eval.py
|
||||
|
||||
omni_minicpmo_4_5_perf:
|
||||
- *minicpmo_4_5
|
||||
- tests/dfx/perf/scripts/run_benchmark.py
|
||||
- tests/dfx/perf/tests/test_minicpmo_4_5_ready.json
|
||||
- tests/dfx/perf/tests/test_minicpmo_4_5.json
|
||||
|
||||
omni_minicpmo_4_5_doc:
|
||||
- *minicpmo_4_5
|
||||
- examples/online_serving/minicpmo/realtime_web/
|
||||
- examples/online_serving/minicpmo/realtime_duplex_demo.py
|
||||
- tests/examples/test_minicpmo_realtime_web_server.py
|
||||
- tests/examples/test_minicpmo_realtime_web_static.py
|
||||
- tests/examples/test_minicpmo_realtime_duplex_simple_demo.py
|
||||
|
||||
omni_minicpmo_4_5_duplex_function:
|
||||
- *minicpmo_4_5
|
||||
- *minicpmo_4_5_duplex
|
||||
- tests/e2e/online_serving/test_minicpmo_4_5_duplex.py
|
||||
- tests/e2e/online_serving/test_minicpmo_4_5_duplex_expansion.py
|
||||
- tests/e2e/online_serving/test_minicpmo_4_5_window.py
|
||||
- tests/e2e/online_serving/helpers/minicpmo_window_e2e.py
|
||||
- tests/e2e/online_serving/test_duplex_client_live.py
|
||||
- tests/e2e/online_serving/test_minicpmo_realtime_duplex_drivers.py
|
||||
- tests/e2e/online_serving/helpers/minicpmo_realtime_duplex_scenarios.py
|
||||
- tests/e2e/online_serving/run_minicpmo_realtime_duplex_multi_session.py
|
||||
- tests/e2e/online_serving/run_minicpmo_realtime_duplex_server_vad.py
|
||||
- tests/e2e/online_serving/run_minicpmo_realtime_duplex_soft_interrupt.py
|
||||
- tests/e2e/online_serving/helpers/minicpmo_4_5_duplex.py
|
||||
- tests/assets/minicpmo_4_5/
|
||||
- tests/clients/
|
||||
- tests/engine/duplex/
|
||||
- tests/engine/test_duplex_omni_engine.py
|
||||
- tests/engine/test_duplex_orchestrator.py
|
||||
- tests/entrypoints/duplex/
|
||||
- tests/entrypoints/test_duplex_omni.py
|
||||
- tests/entrypoints/openai_api/test_duplex_api_server.py
|
||||
- tests/model_executor/models/minicpmo_4_5/duplex/
|
||||
- tests/protocol/
|
||||
|
||||
omni_minicpmo_4_5_duplex_perf:
|
||||
- *minicpmo_4_5
|
||||
- *minicpmo_4_5_duplex
|
||||
- tests/dfx/perf/scripts/run_benchmark.py
|
||||
- tests/dfx/perf/tests/test_minicpmo_4_5_duplex_seed_tts.json
|
||||
|
||||
tts_qwen3_tts_function:
|
||||
- *qwen3_tts
|
||||
- tests/e2e/features/prefill_phase/
|
||||
- tests/e2e/offline_inference/test_qwen3_tts_base.py
|
||||
- tests/e2e/offline_inference/test_qwen3_tts_customvoice.py
|
||||
- tests/e2e/online_serving/test_qwen3_tts_base.py
|
||||
- tests/e2e/online_serving/test_qwen3_tts_base_expansion.py
|
||||
- tests/e2e/online_serving/test_qwen3_tts_customvoice.py
|
||||
- tests/e2e/online_serving/test_qwen3_tts_customvoice_expansion.py
|
||||
|
||||
tts_qwen3_tts_perf:
|
||||
- *qwen3_tts
|
||||
- tests/dfx/perf/scripts/run_benchmark.py
|
||||
- tests/dfx/perf/tests/test_tts.json
|
||||
|
||||
tts_qwen3_tts_cov:
|
||||
- *qwen3_tts
|
||||
- .buildkite/common/scripts/run_cov_split.sh
|
||||
- pyproject.toml
|
||||
- tests/e2e/offline_inference/test_qwen3_tts_base.py
|
||||
- tests/e2e/online_serving/test_qwen3_tts_base.py
|
||||
|
||||
# --- diffusion jobs ---
|
||||
diffusion_wan22_function:
|
||||
- *wan22
|
||||
- tests/e2e/offline_inference/test_wan22_t2v.py
|
||||
- tests/e2e/offline_inference/test_wan22_autoround_w4a16_expansion.py
|
||||
- tests/e2e/offline_inference/test_device_postprocess_equivalence.py
|
||||
- tests/e2e/online_serving/test_wan22_t2v.py
|
||||
- tests/e2e/online_serving/test_wan22_expansion.py
|
||||
|
||||
diffusion_wan22_accuracy:
|
||||
- *wan22
|
||||
- tests/e2e/accuracy/wan22_i2v/
|
||||
- tests/e2e/accuracy/test_diffusers_backend_similarity.py
|
||||
|
||||
diffusion_wan22_perf:
|
||||
- *wan22
|
||||
- tests/dfx/perf/scripts/run_benchmark.py
|
||||
- tests/dfx/perf/tests/test_wan22_i2v_vllm_omni.json
|
||||
|
||||
diffusion_wan22_reliability:
|
||||
- *wan22
|
||||
- tests/dfx/reliability/test_reliability_wan22.py
|
||||
|
||||
diffusion_image_to_video_doc:
|
||||
- *wan22
|
||||
- *hunyuan_video
|
||||
- *ltx2
|
||||
- *sana_video
|
||||
- *lingbot_video
|
||||
- examples/offline_inference/image_to_video/
|
||||
- tests/examples/offline_inference/test_image_to_video.py
|
||||
|
||||
diffusion_cosmos3_function:
|
||||
- *cosmos3
|
||||
- tests/e2e/online_serving/test_cosmos3.py
|
||||
|
||||
diffusion_cosmos3_accuracy:
|
||||
- *cosmos3
|
||||
- tests/e2e/accuracy/test_cosmos3_similarity.py
|
||||
|
||||
diffusion_cosmos3_perf:
|
||||
- *cosmos3
|
||||
- tests/dfx/perf/scripts/run_benchmark.py
|
||||
- tests/dfx/perf/tests/test_cosmos3_vllm_omni.json
|
||||
|
||||
diffusion_qwen_image_function:
|
||||
- *qwen_image
|
||||
- tests/e2e/online_serving/test_qwen_image.py
|
||||
- tests/e2e/online_serving/test_qwen_image_expansion.py
|
||||
- tests/e2e/offline_inference/test_qwen_image_autoround_w4a16_expansion.py
|
||||
|
||||
diffusion_qwen_image_accuracy:
|
||||
- *qwen_image
|
||||
- tests/e2e/accuracy/test_qwen_image.py
|
||||
- tests/e2e/accuracy/test_diffusers_backend_similarity.py
|
||||
|
||||
diffusion_joy_image_accuracy:
|
||||
- *joy_image
|
||||
- tests/diffusion/models/joy_image/
|
||||
- tests/e2e/accuracy/test_joy_image_edit.py
|
||||
- recipes/JD/JoyAI-Image-Edit.md
|
||||
|
||||
diffusion_qwen_image_perf:
|
||||
- *qwen_image
|
||||
- tests/dfx/perf/scripts/run_diffusion_benchmark.py
|
||||
- tests/dfx/perf/tests/test_qwen_image_vllm_omni.json
|
||||
|
||||
diffusion_text_to_image_doc:
|
||||
- *qwen_image
|
||||
- *z_image
|
||||
- examples/offline_inference/text_to_image/
|
||||
- examples/online_serving/text_to_image/
|
||||
- tests/examples/offline_inference/test_text_to_image.py
|
||||
- tests/examples/online_serving/test_text_to_image.py
|
||||
- vllm_omni/diffusion/models/nextstep_1_1/
|
||||
- vllm_omni/diffusion/models/longcat_image/
|
||||
- vllm_omni/diffusion/models/ovis_image/
|
||||
- vllm_omni/diffusion/models/omnigen2/
|
||||
- vllm_omni/diffusion/models/sd3/
|
||||
- vllm_omni/diffusion/models/flux/
|
||||
- vllm_omni/diffusion/models/flux2/
|
||||
- vllm_omni/diffusion/models/flux2_klein/
|
||||
- vllm_omni/diffusion/models/hidream_image/
|
||||
- vllm_omni/diffusion/models/krea2/
|
||||
- vllm_omni/diffusion/models/lingbot_video/
|
||||
|
||||
diffusion_hunyuan_image3_function:
|
||||
- *hunyuan_image3
|
||||
- tests/e2e/offline_inference/test_hunyuanimage3_i2t_expansion.py
|
||||
|
||||
diffusion_hunyuan_image3_accuracy:
|
||||
- *hunyuan_image3
|
||||
- tests/e2e/accuracy/test_hunyuan_image3_pixel_accuracy.py
|
||||
- tests/e2e/accuracy/test_hunyuan_image3_lora.py
|
||||
- vllm_omni/diffusion/lora/
|
||||
|
||||
diffusion_hunyuan_image3_perf:
|
||||
- *hunyuan_image3
|
||||
- tests/dfx/perf/scripts/run_diffusion_benchmark.py
|
||||
- tests/dfx/perf/tests/test_hunyuan_image_tp2_cfgp2.json
|
||||
- tests/dfx/perf/tests/test_hunyuan_image_tp2_sp2.json
|
||||
- tests/dfx/perf/tests/test_hunyuan_image_tp4.json
|
||||
|
||||
diffusion_hunyuan_image3_reliability:
|
||||
- *hunyuan_image3
|
||||
- tests/dfx/reliability/test_reliability_hunyuan_image.py
|
||||
|
||||
diffusion_minimax_h3_function:
|
||||
- *minimax_h3
|
||||
- tests/helpers/assertions.py
|
||||
- tests/e2e/online_serving/test_minimax_h3_dlo_dp2_t2va.py
|
||||
- tests/e2e/online_serving/minimax_h3/
|
||||
- tests/diffusion/models/minimax_h3/
|
||||
- vllm_omni/diffusion/lora/
|
||||
- vllm_omni/diffusion/offloader/
|
||||
- vllm_omni/diffusion/distributed/
|
||||
- vllm_omni/entrypoints/openai/
|
||||
|
||||
diffusion_minimax_h3_accuracy:
|
||||
- *minimax_h3
|
||||
- tests/e2e/accuracy/minimax_h3/
|
||||
|
||||
diffusion_minimax_h3_perf:
|
||||
- *minimax_h3
|
||||
- tests/dfx/perf/scripts/run_benchmark.py
|
||||
- tests/dfx/perf/tests/test_minimax_h3_vllm_omni.json
|
||||
|
||||
diffusion_hunyuan_video_function:
|
||||
- *hunyuan_video
|
||||
- tests/e2e/online_serving/test_hunyuan_video_15_expansion.py
|
||||
|
||||
diffusion_hunyuan_video_accuracy:
|
||||
- *hunyuan_video
|
||||
- tests/e2e/accuracy/hunyuanvideo15_t2v/
|
||||
- tests/e2e/accuracy/hunyuanvideo15_i2v/
|
||||
|
||||
diffusion_hunyuan_video_perf:
|
||||
- *hunyuan_video
|
||||
- tests/dfx/perf/scripts/run_benchmark.py
|
||||
- tests/dfx/perf/tests/test_hunyuanvideo15_t2v_vllm_omni.json
|
||||
- tests/dfx/perf/tests/test_hunyuanvideo15_i2v_vllm_omni.json
|
||||
|
||||
diffusion_z_image_function:
|
||||
- *z_image
|
||||
- tests/e2e/online_serving/test_z_image.py
|
||||
- tests/e2e/online_serving/test_zimage_expansion.py
|
||||
|
||||
diffusion_sana_video_function:
|
||||
- *sana_video
|
||||
- tests/e2e/online_serving/test_sana_video_expansion.py
|
||||
- tests/e2e/accuracy/sana_video/
|
||||
|
||||
diffusion_stable_audio_function:
|
||||
- *stable_audio
|
||||
- tests/e2e/offline_inference/test_stable_audio_expansion.py
|
||||
- tests/e2e/online_serving/test_stable_audio_online_expansion.py
|
||||
|
||||
diffusion_lingbot_function:
|
||||
- *lingbot_video
|
||||
- *lingbot_world
|
||||
- tests/e2e/online_serving/test_lingbot_video.py
|
||||
- tests/e2e/online_serving/test_lingbot_video_expansion.py
|
||||
- tests/e2e/online_serving/test_lingbot_video_moe.py
|
||||
- tests/e2e/offline_inference/test_lingbot_world_v2.py
|
||||
|
||||
diffusion_lingbot_perf:
|
||||
- *lingbot_video
|
||||
- tests/dfx/perf/scripts/run_benchmark.py
|
||||
- tests/dfx/perf/tests/test_lingbot_video_vllm_omni.json
|
||||
|
||||
diffusion_kandinsky6_function:
|
||||
- *kandinsky6
|
||||
- tests/e2e/offline_inference/test_kandinsky6.py
|
||||
- tests/e2e/online_serving/test_kandinsky6.py
|
||||
|
||||
|
||||
# Invalid-param jobs: model business code only, plus the reliability scripts.
|
||||
reliability_invalid_param_h100:
|
||||
- *qwen3_omni
|
||||
- *minicpmo_4_5
|
||||
- *qwen_image
|
||||
- *wan22
|
||||
- tests/dfx/reliability/invalid_param_test/
|
||||
|
||||
reliability_invalid_param_l4:
|
||||
- *qwen3_tts
|
||||
- *stable_audio
|
||||
- tests/dfx/reliability/invalid_param_test/
|
||||
@@ -1,23 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Shared Buildkite skip-ci helpers for AMD / Intel bootstrap scripts.
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
SKIP_CI_PY="${SCRIPT_DIR}/skip_ci.py"
|
||||
|
||||
# Resolve git diff once. Exit 0 when this bootstrap should stop uploading.
|
||||
# Decision details are logged by skip_ci.py; this helper only annotates skip-all.
|
||||
# Usage: gate_bootstrap_ci <platform> <l2|l3>
|
||||
gate_bootstrap_ci() {
|
||||
local platform="$1"
|
||||
local level="$2"
|
||||
local kind
|
||||
if kind="$(python3 "${SKIP_CI_PY}" gate "${platform}" "${level}")"; then
|
||||
if [[ "${kind}" == "skip-all" ]]; then
|
||||
buildkite-agent annotate \
|
||||
":memo: CI skipped — docs or pytest skip-mark changes only" \
|
||||
--style "info" 2>/dev/null || true
|
||||
fi
|
||||
exit 0
|
||||
fi
|
||||
}
|
||||
@@ -1,98 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Run a model's E2E tests once per entry mode as separate coverage runs and upload
|
||||
# the reports, so per-mode attribution survives (see vllm-project/vllm-omni#5332).
|
||||
#
|
||||
# Usage: run_cov_split.sh --model-id <id> [--offline <paths>] [--online <paths>] \
|
||||
# -- <pytest args...>
|
||||
#
|
||||
# --model-id Names the artifacts: coverage-<id>-{offline,online}-<step id>.xml.gz.
|
||||
# Use the model's directory name under vllm_omni/*/models/.
|
||||
# --offline Pytest path(s) for the offline half. Repeatable.
|
||||
# --online Pytest path(s) for the online half. Repeatable.
|
||||
# -- Everything after this is forwarded verbatim to every pytest run,
|
||||
# quoting preserved: the job's own -m expression, --run-level, and
|
||||
# anything else it needs (e.g. --test-config-file).
|
||||
#
|
||||
# At least one of --offline/--online is required, so a job with only one mode runs
|
||||
# just that half. Total runtime is bounded by the step's timeout_in_minutes.
|
||||
#
|
||||
# Coverage + artifact upload run only on scheduled weekly E2E sweeps
|
||||
# (WEEKLY=1 and BUILDKITE_BRANCH=main — same gate as bootstrap --e2e).
|
||||
# All other invocations (PR merge-test, regular main merge, local) run the
|
||||
# same combined pytest as the job used before coverage was added.
|
||||
#
|
||||
# Exits non-zero if any half or the upload failed.
|
||||
set -uo pipefail
|
||||
|
||||
MODEL_ID=""
|
||||
OFFLINE=()
|
||||
ONLINE=()
|
||||
PYTEST_ARGS=()
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--model-id) MODEL_ID="$2"; shift 2 ;;
|
||||
--offline) OFFLINE+=("$2"); shift 2 ;;
|
||||
--online) ONLINE+=("$2"); shift 2 ;;
|
||||
--) shift; PYTEST_ARGS=("$@"); break ;;
|
||||
*) echo "run_cov_split.sh: unknown argument: $1" >&2; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [[ -z "${MODEL_ID}" ]]; then
|
||||
echo "run_cov_split.sh: missing --model-id" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
if ((${#OFFLINE[@]} == 0 && ${#ONLINE[@]} == 0)); then
|
||||
echo "run_cov_split.sh: need at least one of --offline / --online" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
_should_run_coverage() {
|
||||
[[ "${WEEKLY:-}" == "1" && "${BUILDKITE_BRANCH:-}" == "main" ]]
|
||||
}
|
||||
|
||||
if ! _should_run_coverage; then
|
||||
PATHS=("${OFFLINE[@]}" "${ONLINE[@]}")
|
||||
echo "--- pytest (no coverage): ${MODEL_ID}"
|
||||
pytest -s -v "${PATHS[@]}" "${PYTEST_ARGS[@]}"
|
||||
exit $?
|
||||
fi
|
||||
|
||||
# Two steps can test the same model, and artifacts are scoped to the build (a
|
||||
# nightly one carries the ready, merge and nightly tiers at once), so unsuffixed
|
||||
# reports would share a path -- a glob download then races to rename onto one file.
|
||||
# Not BUILDKITE_JOB_ID: the step id survives retries, so attempts collapse under
|
||||
# Buildkite's latest-attempt lookup instead of accumulating. A parallel step would
|
||||
# need BUILDKITE_PARALLEL_JOB as well; no pilot is one.
|
||||
REPORT_SUFFIX="${BUILDKITE_STEP_ID:-local}"
|
||||
OFFLINE_REPORT="coverage-${MODEL_ID}-offline-${REPORT_SUFFIX}.xml"
|
||||
ONLINE_REPORT="coverage-${MODEL_ID}-online-${REPORT_SUFFIX}.xml"
|
||||
|
||||
run_half() {
|
||||
local mode="$1"
|
||||
shift
|
||||
local report="coverage-${MODEL_ID}-${mode}-${REPORT_SUFFIX}.xml"
|
||||
echo "--- coverage: ${MODEL_ID} ${mode}"
|
||||
pytest -s -v "$@" "${PYTEST_ARGS[@]}" --cov=vllm_omni --cov-report="xml:${report}"
|
||||
local status=$?
|
||||
# Cobertura lists every statement in vllm_omni whether or not it ran, so the
|
||||
# report is ~7MB no matter how little the job covers. The repetition is what
|
||||
# dominates it, and gzip takes that to ~400KB.
|
||||
gzip -9 -f "${report}" || status=1
|
||||
return "${status}"
|
||||
}
|
||||
|
||||
# Clear both modes, not just the ones being run: the upload glob matches both, so
|
||||
# a report left by an earlier run would otherwise be published as if the mode it
|
||||
# belongs to had run this time.
|
||||
rm -f "${OFFLINE_REPORT}" "${ONLINE_REPORT}" "${OFFLINE_REPORT}.gz" "${ONLINE_REPORT}.gz"
|
||||
|
||||
EXIT=0
|
||||
((${#OFFLINE[@]})) && { run_half offline "${OFFLINE[@]}" || EXIT=1; }
|
||||
((${#ONLINE[@]})) && { run_half online "${ONLINE[@]}" || EXIT=1; }
|
||||
|
||||
buildkite-agent artifact upload "coverage-${MODEL_ID}-*-${REPORT_SUFFIX}.xml.gz" || EXIT=1
|
||||
|
||||
exit "${EXIT}"
|
||||
@@ -1,585 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
|
||||
"""Shared Buildkite skip-ci and CI-YAML level targeting logic.
|
||||
|
||||
``resolve_ci_decision(changed_files, ...)`` buckets explicit file lists (tests, dry-run).
|
||||
``resolve_ci_context_from_git()`` resolves git diff once and returns decision + changed files
|
||||
for Buildkite bootstrap / upload scripts.
|
||||
|
||||
CLI (for bootstrap scripts; exit code is the signal):
|
||||
python3 skip_ci.py gate <platform> <level>
|
||||
Exit 0 when bootstrap should stop (skip-all, or that L2/L3 target is off).
|
||||
Prints ``skip-all`` or ``skip-l23`` on stdout; logs the decision once.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
from dataclasses import dataclass, field, replace
|
||||
from pathlib import Path
|
||||
|
||||
LOG = "[skip-ci]"
|
||||
ROOT = Path(__file__).resolve().parent.parent.parent.parent
|
||||
|
||||
PLATFORMS = ("cuda", "amd", "intel", "npu")
|
||||
L23_LEVELS = frozenset({"l2", "l3"})
|
||||
|
||||
L2_YAML_FILES: dict[str, str] = {
|
||||
".buildkite/cuda/test-ready.yml": "cuda",
|
||||
".buildkite/npu/test-npu-ready.yml": "npu",
|
||||
".buildkite/amd/test-amd-ready.yml": "amd",
|
||||
".buildkite/intel/pipeline-intel.yml": "intel",
|
||||
}
|
||||
|
||||
L3_YAML_FILES: dict[str, str] = {
|
||||
".buildkite/cuda/test-merge.yml": "cuda",
|
||||
".buildkite/amd/test-amd-merge.yml": "amd",
|
||||
}
|
||||
|
||||
L45_YAML_FILES: dict[str, str] = {
|
||||
".buildkite/cuda/test-nightly.yml": "cuda",
|
||||
".buildkite/cuda/test-weekly.yml": "cuda",
|
||||
".buildkite/amd/test-amd-nightly.yml": "amd",
|
||||
".buildkite/npu/test-npu-nightly.yml": "npu",
|
||||
}
|
||||
|
||||
|
||||
def _empty_platform_buckets() -> dict[str, list[str]]:
|
||||
return {platform: [] for platform in PLATFORMS}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CiDevice:
|
||||
"""Per-platform / per-level skip flags for yaml-gated bootstrap (``skip_l2_l3``)."""
|
||||
|
||||
skip_cuda_l2: bool = False
|
||||
skip_cuda_l3: bool = False
|
||||
skip_amd_l2: bool = False
|
||||
skip_amd_l3: bool = False
|
||||
skip_intel_l2: bool = False
|
||||
skip_intel_l3: bool = False
|
||||
skip_npu_l2: bool = False
|
||||
skip_npu_l3: bool = False
|
||||
|
||||
def is_skip(self, platform: str, level: str) -> bool:
|
||||
return bool(getattr(self, f"skip_{platform}_{level}", True))
|
||||
|
||||
|
||||
@dataclass
|
||||
class DiffBuckets:
|
||||
"""Changed files grouped by doc / skip-mark / CI yaml / other."""
|
||||
|
||||
docs: list[str] = field(default_factory=list)
|
||||
skip_tests: list[str] = field(default_factory=list)
|
||||
other: list[str] = field(default_factory=list)
|
||||
l2: dict[str, list[str]] = field(default_factory=_empty_platform_buckets)
|
||||
l3: dict[str, list[str]] = field(default_factory=_empty_platform_buckets)
|
||||
l45: dict[str, list[str]] = field(default_factory=_empty_platform_buckets)
|
||||
|
||||
@property
|
||||
def has_doc_changes(self) -> bool:
|
||||
return bool(self.docs)
|
||||
|
||||
@property
|
||||
def has_skip_test_changes(self) -> bool:
|
||||
return bool(self.skip_tests)
|
||||
|
||||
@property
|
||||
def has_other_changes(self) -> bool:
|
||||
"""Product code or paths outside skip-ci scope."""
|
||||
return bool(self.other)
|
||||
|
||||
@property
|
||||
def has_l23_yaml_changes(self) -> bool:
|
||||
"""At least one whitelisted L2/L3 CI yaml file changed."""
|
||||
return any(self.l2[platform] or self.l3[platform] for platform in PLATFORMS)
|
||||
|
||||
@property
|
||||
def has_l45_yaml_changes(self) -> bool:
|
||||
"""At least one whitelisted L4/L5 (nightly/weekly) CI yaml file changed."""
|
||||
return any(self.l45[platform] for platform in PLATFORMS)
|
||||
|
||||
@property
|
||||
def has_skip_ci_scope_changes(self) -> bool:
|
||||
"""At least one doc / skip-mark / whitelisted CI yaml change."""
|
||||
return (
|
||||
self.has_doc_changes or self.has_skip_test_changes or self.has_l23_yaml_changes or self.has_l45_yaml_changes
|
||||
)
|
||||
|
||||
def l2_platforms(self) -> frozenset[str]:
|
||||
return frozenset(platform for platform in PLATFORMS if self.l2[platform])
|
||||
|
||||
def l3_platforms(self) -> frozenset[str]:
|
||||
return frozenset(platform for platform in PLATFORMS if self.l3[platform])
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SkipL2L3Basis:
|
||||
skip_all_l2: bool
|
||||
skip_all_l3: bool
|
||||
|
||||
@classmethod
|
||||
def from_buckets(cls, buckets: DiffBuckets) -> SkipL2L3Basis | None:
|
||||
"""L4/L5 yaml → skip both L2 and L3 (L2/L3 buckets may rescue).
|
||||
|
||||
Docs / skip-mark changes do not block this path; they are ignored for
|
||||
L2/L3 targeting when CI YAML is also present.
|
||||
"""
|
||||
if buckets.has_other_changes:
|
||||
return None
|
||||
if not buckets.has_l45_yaml_changes:
|
||||
return None
|
||||
return cls(
|
||||
skip_all_l2=not buckets.l2_platforms(),
|
||||
skip_all_l3=not buckets.l3_platforms(),
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SkipL3Basis:
|
||||
enable_l2_platforms: frozenset[str]
|
||||
|
||||
@classmethod
|
||||
def from_buckets(cls, buckets: DiffBuckets) -> SkipL3Basis | None:
|
||||
"""L2 yaml changed → skip L3; keep L2 on touched platforms."""
|
||||
platforms = buckets.l2_platforms()
|
||||
if not platforms:
|
||||
return None
|
||||
return cls(enable_l2_platforms=platforms)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SkipL2Basis:
|
||||
enable_l3_platforms: frozenset[str]
|
||||
|
||||
@classmethod
|
||||
def from_buckets(cls, buckets: DiffBuckets) -> SkipL2Basis | None:
|
||||
"""L3 yaml changed → skip L2; keep L3 on touched platforms."""
|
||||
platforms = buckets.l3_platforms()
|
||||
if not platforms:
|
||||
return None
|
||||
return cls(enable_l3_platforms=platforms)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CiDecision:
|
||||
"""Final CI skip decision for one diff."""
|
||||
|
||||
skip_all: bool = False
|
||||
skip_l2_l3: bool = False
|
||||
device: CiDevice = field(default_factory=CiDevice)
|
||||
message: str = ""
|
||||
|
||||
@classmethod
|
||||
def skip_all_basis(cls, buckets: DiffBuckets, *, diff_range: str | None) -> bool:
|
||||
"""Docs / pytest skip-mark only → suppress entire default CI."""
|
||||
if buckets.has_other_changes:
|
||||
return False
|
||||
if (buckets.has_l23_yaml_changes or buckets.has_l45_yaml_changes) and (
|
||||
buckets.has_doc_changes or buckets.has_skip_test_changes
|
||||
):
|
||||
return False
|
||||
if not buckets.has_doc_changes and not buckets.has_skip_test_changes:
|
||||
return False
|
||||
if diff_range is None and not buckets.has_doc_changes:
|
||||
return False
|
||||
return True
|
||||
|
||||
@classmethod
|
||||
def from_yaml_basis(
|
||||
cls,
|
||||
l2_l3_basis: SkipL2L3Basis | None,
|
||||
l3_basis: SkipL3Basis | None,
|
||||
l2_basis: SkipL2Basis | None,
|
||||
) -> CiDecision:
|
||||
"""Build a yaml-gated decision by merging L2/L3 skip bases.
|
||||
|
||||
Bases are applied as a union of enables: L2 yaml keeps L2 on touched
|
||||
platforms, L3 yaml keeps L3 on touched platforms. Cross-platform mixes
|
||||
(e.g. CUDA ready + AMD merge) must preserve both enables. The resulting
|
||||
per-platform / per-level matrix is stored on ``device`` as ``skip_*``.
|
||||
"""
|
||||
run = {platform: {"l2": True, "l3": True} for platform in PLATFORMS}
|
||||
|
||||
if l2_l3_basis is not None:
|
||||
if l2_l3_basis.skip_all_l2:
|
||||
for platform in PLATFORMS:
|
||||
run[platform]["l2"] = False
|
||||
if l2_l3_basis.skip_all_l3:
|
||||
for platform in PLATFORMS:
|
||||
run[platform]["l3"] = False
|
||||
|
||||
if l3_basis is not None:
|
||||
for platform in PLATFORMS:
|
||||
run[platform]["l3"] = False
|
||||
run[platform]["l2"] = platform in l3_basis.enable_l2_platforms
|
||||
|
||||
if l2_basis is not None:
|
||||
for platform in PLATFORMS:
|
||||
if platform in l2_basis.enable_l3_platforms:
|
||||
run[platform]["l3"] = True
|
||||
if l3_basis is None or platform not in l3_basis.enable_l2_platforms:
|
||||
run[platform]["l2"] = False
|
||||
elif l3_basis is None:
|
||||
# L3-yaml-only: disable platforms that did not change L3.
|
||||
run[platform]["l2"] = False
|
||||
run[platform]["l3"] = False
|
||||
# else: leave L2 enables from SkipL3Basis; keep this platform's L3 off.
|
||||
|
||||
return cls(
|
||||
skip_l2_l3=True,
|
||||
device=cls._device_from_run(run),
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _device_from_run(run: dict[str, dict[str, bool]]) -> CiDevice:
|
||||
return CiDevice(
|
||||
**{f"skip_{platform}_{level}": not run[platform][level] for platform in PLATFORMS for level in ("l2", "l3")}
|
||||
)
|
||||
|
||||
def is_run(self, platform: str, level: str) -> bool:
|
||||
if self.skip_all:
|
||||
return False
|
||||
if not self.skip_l2_l3:
|
||||
return True
|
||||
if level not in L23_LEVELS:
|
||||
return True
|
||||
return not self.device.is_skip(platform, level)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CiContext:
|
||||
"""Skip-ci decision plus changed files used for pipeline filtering."""
|
||||
|
||||
decision: CiDecision
|
||||
changed_files: list[str] | None
|
||||
|
||||
|
||||
def _log(message: str) -> None:
|
||||
print(f"{LOG}: {message}", file=sys.stderr)
|
||||
|
||||
|
||||
def _git(*args: str) -> subprocess.CompletedProcess[str]:
|
||||
return subprocess.run(
|
||||
["git", *args],
|
||||
check=False,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
|
||||
|
||||
# --- Diff helpers ---
|
||||
|
||||
|
||||
def _resolve_diff_range() -> str | None:
|
||||
"""Return a git diff range for PR or main builds, or None when unavailable."""
|
||||
is_pr = os.environ.get("BUILDKITE_PULL_REQUEST", "false") != "false" and os.environ.get(
|
||||
"BUILDKITE_PULL_REQUEST", ""
|
||||
)
|
||||
commit = os.environ.get("BUILDKITE_COMMIT", "")
|
||||
|
||||
if is_pr:
|
||||
base_branch = os.environ.get("BUILDKITE_PULL_REQUEST_BASE_BRANCH", "main")
|
||||
base_ref = f"origin/{base_branch}"
|
||||
if _git("rev-parse", "--verify", base_ref).returncode != 0:
|
||||
_log(f"origin/{base_branch} not found locally; trying fetch")
|
||||
_git("fetch", "--depth=200", "origin", base_branch)
|
||||
if _git("rev-parse", "--verify", base_ref).returncode != 0:
|
||||
if _git("rev-parse", "--verify", base_branch).returncode == 0:
|
||||
base_ref = base_branch
|
||||
else:
|
||||
_log(f"cannot resolve PR base {base_branch}; using safe defaults")
|
||||
return None
|
||||
return f"{base_ref}...{commit}"
|
||||
|
||||
if os.environ.get("BUILDKITE_BRANCH", "") == "main":
|
||||
if _git("rev-parse", "--verify", f"{commit}^").returncode != 0:
|
||||
_log("main commit has no parent; using safe defaults")
|
||||
return None
|
||||
return f"{commit}^..{commit}"
|
||||
|
||||
_log("not PR/main build; using safe defaults")
|
||||
return None
|
||||
|
||||
|
||||
def _changed_files_for_diff_range(diff_range: str | None) -> list[str] | None:
|
||||
"""Return changed file paths for *diff_range*, or None when unavailable."""
|
||||
if diff_range is None:
|
||||
return None
|
||||
|
||||
result = _git("diff", "--name-only", diff_range)
|
||||
if result.returncode != 0:
|
||||
_log(f"git diff failed ({diff_range}); using safe defaults")
|
||||
return None
|
||||
|
||||
files = [line for line in result.stdout.splitlines() if line.strip()]
|
||||
_log(f"{len(files)} changed file(s)")
|
||||
return files
|
||||
|
||||
|
||||
def resolve_ci_context_from_git() -> CiContext:
|
||||
"""Resolve git diff once and return skip-ci decision plus changed files."""
|
||||
diff_range = _resolve_diff_range()
|
||||
changed_files = _changed_files_for_diff_range(diff_range)
|
||||
decision = resolve_ci_decision(changed_files, diff_range=diff_range)
|
||||
return CiContext(decision=decision, changed_files=changed_files)
|
||||
|
||||
|
||||
# --- Doc / skip-mark helpers ---
|
||||
|
||||
|
||||
_SKIP_MARK_RE = re.compile(r"pytest\.mark\.skip(?:if)?\b|pytest\.skip\s*\(")
|
||||
_PYTESTMARK_SKIP_RE = re.compile(r"pytest\.mark\.skip\b")
|
||||
_PYTEST_MARK_ONLY_RE = re.compile(r"pytest\.mark\.\w+")
|
||||
|
||||
|
||||
def _diff_only_contains_skip_mark_changes(diff_text: str) -> bool:
|
||||
"""Return True when a unified diff only *adds* pytest skip marks.
|
||||
|
||||
Deleting a skip mark re-enables the test and must not suppress CI. Editing an
|
||||
existing skip (``-`` old / ``+`` new) still qualifies when at least one skip
|
||||
mark is added.
|
||||
"""
|
||||
|
||||
def paren_balance(line: str) -> int:
|
||||
return line.count("(") - line.count(")")
|
||||
|
||||
def is_blank_or_comment(line: str) -> bool:
|
||||
stripped = line.strip()
|
||||
return not stripped or stripped.startswith("#")
|
||||
|
||||
def is_skip_mark_line(line: str) -> bool:
|
||||
return _SKIP_MARK_RE.search(line.strip()) is not None
|
||||
|
||||
def is_pytestmark_adjacent_line(line: str) -> bool:
|
||||
stripped = line.strip().rstrip(",")
|
||||
if not stripped or stripped in {"[", "]"} or stripped.startswith("#"):
|
||||
return True
|
||||
if stripped.startswith("pytestmark"):
|
||||
return True
|
||||
return _PYTEST_MARK_ONLY_RE.search(stripped) is not None
|
||||
|
||||
pending_depth = 0
|
||||
saw_change = False
|
||||
has_added_skip_mark = False
|
||||
for raw_line in diff_text.splitlines():
|
||||
if raw_line.startswith("@@"):
|
||||
pending_depth = 0
|
||||
continue
|
||||
if not (raw_line.startswith("+") or raw_line.startswith("-")):
|
||||
continue
|
||||
if raw_line.startswith("+++") or raw_line.startswith("---"):
|
||||
continue
|
||||
|
||||
saw_change = True
|
||||
is_add = raw_line.startswith("+")
|
||||
content = raw_line[1:]
|
||||
|
||||
if pending_depth > 0:
|
||||
pending_depth += paren_balance(content)
|
||||
if pending_depth < 0:
|
||||
pending_depth = 0
|
||||
continue
|
||||
|
||||
if is_blank_or_comment(content):
|
||||
continue
|
||||
|
||||
if is_skip_mark_line(content):
|
||||
if is_add:
|
||||
has_added_skip_mark = True
|
||||
pending_depth = max(0, paren_balance(content))
|
||||
continue
|
||||
|
||||
if is_pytestmark_adjacent_line(content):
|
||||
continue
|
||||
|
||||
return False
|
||||
|
||||
return saw_change and has_added_skip_mark
|
||||
|
||||
|
||||
def _is_skip_testcase_change(file_path: str, *, diff_range: str) -> bool:
|
||||
"""Return True when a test file diff is skip-mark-only (including new-file cases)."""
|
||||
diff_result = _git("diff", diff_range, "--", file_path)
|
||||
if diff_result.returncode != 0:
|
||||
_log(f"skip-mark-only: git diff failed for {file_path}")
|
||||
return False
|
||||
|
||||
diff_text = diff_result.stdout
|
||||
if not diff_text.strip():
|
||||
_log(f"skip-mark-only: empty diff for {file_path}")
|
||||
return False
|
||||
|
||||
if _diff_only_contains_skip_mark_changes(diff_text):
|
||||
return True
|
||||
|
||||
if "new file mode" not in diff_text and "--- /dev/null" not in diff_text:
|
||||
_log(f"skip-mark-only: non-skip changes in {file_path}")
|
||||
return False
|
||||
|
||||
commit = os.environ.get("BUILDKITE_COMMIT", "")
|
||||
if not commit:
|
||||
return False
|
||||
|
||||
content_result = _git("show", f"{commit}:{file_path}")
|
||||
if content_result.returncode != 0:
|
||||
_log(f"skip-mark-only: cannot read {file_path} at {commit}")
|
||||
return False
|
||||
|
||||
content = content_result.stdout
|
||||
match = re.search(r"^pytestmark\s*=\s*\[(.*?)^\s*\]", content, re.MULTILINE | re.DOTALL)
|
||||
if match is not None and _PYTESTMARK_SKIP_RE.search(match.group(1)) is not None:
|
||||
_log(f"skip-mark-only: new test file with module-level pytest.mark.skip: {file_path}")
|
||||
return True
|
||||
if re.search(r"^pytestmark\s*=\s*pytest\.mark\.skip\b", content, re.MULTILINE):
|
||||
_log(f"skip-mark-only: new test file with module-level pytest.mark.skip: {file_path}")
|
||||
return True
|
||||
|
||||
_log(f"skip-mark-only: new test file without module-level pytest.mark.skip: {file_path}")
|
||||
return False
|
||||
|
||||
|
||||
def _classify_changed_files_into_buckets(changed_files: list[str], *, diff_range: str | None) -> DiffBuckets:
|
||||
"""Classify each changed file path into a ``DiffBuckets`` category."""
|
||||
buckets = DiffBuckets()
|
||||
for file_path in changed_files:
|
||||
if not file_path:
|
||||
continue
|
||||
if file_path.startswith("docs/") or file_path.endswith(".md") or file_path == "mkdocs.yml":
|
||||
buckets.docs.append(file_path)
|
||||
continue
|
||||
path = Path(file_path)
|
||||
if path.suffix == ".py" and path.parts and path.parts[0] == "tests":
|
||||
if diff_range is not None and _is_skip_testcase_change(file_path, diff_range=diff_range):
|
||||
buckets.skip_tests.append(file_path)
|
||||
else:
|
||||
buckets.other.append(file_path)
|
||||
continue
|
||||
if file_path in L2_YAML_FILES:
|
||||
platform = L2_YAML_FILES[file_path]
|
||||
buckets.l2[platform].append(file_path)
|
||||
elif file_path in L3_YAML_FILES:
|
||||
platform = L3_YAML_FILES[file_path]
|
||||
buckets.l3[platform].append(file_path)
|
||||
elif file_path in L45_YAML_FILES:
|
||||
platform = L45_YAML_FILES[file_path]
|
||||
buckets.l45[platform].append(file_path)
|
||||
else:
|
||||
buckets.other.append(file_path)
|
||||
return buckets
|
||||
|
||||
|
||||
# --- Decision ---
|
||||
|
||||
|
||||
def _finish(decision: CiDecision, message: str) -> CiDecision:
|
||||
finished = replace(decision, message=message)
|
||||
_log(message)
|
||||
return finished
|
||||
|
||||
|
||||
def _format_yaml_gated_message(buckets: DiffBuckets, decision: CiDecision) -> str:
|
||||
"""Human-readable summary of which CI YAML changed and which L2/L3 targets run."""
|
||||
changed_parts: list[str] = []
|
||||
for level_name, by_platform in (("L2", buckets.l2), ("L3", buckets.l3), ("L4/L5", buckets.l45)):
|
||||
files = [path for platform in PLATFORMS for path in by_platform[platform]]
|
||||
if files:
|
||||
changed_parts.append(f"{level_name}=[{', '.join(files)}]")
|
||||
|
||||
run_targets = [
|
||||
f"{platform}/{level}" for platform in PLATFORMS for level in ("l2", "l3") if decision.is_run(platform, level)
|
||||
]
|
||||
skip_targets = [
|
||||
f"{platform}/{level}"
|
||||
for platform in PLATFORMS
|
||||
for level in ("l2", "l3")
|
||||
if not decision.is_run(platform, level)
|
||||
]
|
||||
|
||||
changed = "; ".join(changed_parts) if changed_parts else "none"
|
||||
run = ", ".join(run_targets) if run_targets else "none"
|
||||
skip = ", ".join(skip_targets) if skip_targets else "none"
|
||||
if buckets.has_doc_changes or buckets.has_skip_test_changes:
|
||||
return (
|
||||
"CI config YAML + docs/skip-mark changed — follow CI YAML gating; "
|
||||
f"changed: {changed}; run: {run}; skip: {skip}"
|
||||
)
|
||||
return f"only CI config YAML changed — changed: {changed}; run: {run}; skip: {skip}"
|
||||
|
||||
|
||||
def resolve_ci_decision(
|
||||
changed_files: list[str] | None = None,
|
||||
*,
|
||||
diff_range: str | None = None,
|
||||
) -> CiDecision:
|
||||
if changed_files is None:
|
||||
return _finish(CiDecision(), "could not resolve changed files; run normal CI")
|
||||
|
||||
buckets = _classify_changed_files_into_buckets(changed_files, diff_range=diff_range)
|
||||
if buckets.has_other_changes or not buckets.has_skip_ci_scope_changes:
|
||||
return _finish(CiDecision(), "product or non-whitelisted changes present; run normal CI")
|
||||
|
||||
if CiDecision.skip_all_basis(buckets, diff_range=diff_range):
|
||||
return _finish(
|
||||
CiDecision(skip_all=True),
|
||||
"CI skipped — docs or pytest skip-mark changes only",
|
||||
)
|
||||
|
||||
# Docs / skip-mark mixed with whitelisted CI YAML → same yaml-gated path
|
||||
# as CI-YAML-only (docs/skip-mark do not widen to normal CI).
|
||||
if not buckets.has_l23_yaml_changes and not buckets.has_l45_yaml_changes:
|
||||
if buckets.has_skip_test_changes and diff_range is None:
|
||||
message = (
|
||||
"only pytest skip-mark test changes detected, but no git diff range "
|
||||
"is available to verify them (skip-all requires diff_range); run normal CI"
|
||||
)
|
||||
elif buckets.has_skip_test_changes:
|
||||
message = "pytest skip-mark test changes did not qualify for skip-all and no ci yaml changed; run normal CI"
|
||||
else:
|
||||
message = "doc/skip-mark changes did not qualify for skip-all; run normal CI"
|
||||
return _finish(CiDecision(), message)
|
||||
|
||||
decision = CiDecision.from_yaml_basis(
|
||||
SkipL2L3Basis.from_buckets(buckets),
|
||||
SkipL3Basis.from_buckets(buckets),
|
||||
SkipL2Basis.from_buckets(buckets),
|
||||
)
|
||||
return _finish(decision, _format_yaml_gated_message(buckets, decision))
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(
|
||||
description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
)
|
||||
subparsers = parser.add_subparsers(dest="command", required=True)
|
||||
|
||||
gate_parser = subparsers.add_parser(
|
||||
"gate",
|
||||
help="exit 0 when bootstrap should stop (skip-all or PLATFORM/LEVEL disabled); logs once",
|
||||
)
|
||||
gate_parser.add_argument("platform", help="cuda, amd, intel, or npu")
|
||||
gate_parser.add_argument("level", choices=("l2", "l3"), help="test level")
|
||||
|
||||
args = parser.parse_args()
|
||||
decision = resolve_ci_context_from_git().decision
|
||||
|
||||
if args.command == "gate":
|
||||
if decision.skip_all:
|
||||
print("skip-all")
|
||||
return 0
|
||||
if not decision.is_run(args.platform, args.level):
|
||||
print("skip-l23")
|
||||
return 0
|
||||
return 1
|
||||
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -1,883 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
|
||||
"""Render and optionally upload Buildkite pipeline YAML with diff-aware logic.
|
||||
|
||||
Bootstrap mode (``bootstrap-upload-steps.yml``):
|
||||
- Hook uploads the entry YAML (``pipeline.yml`` / ``pipeline-npu.yml``) with one step that runs
|
||||
``upload_pipeline.py --upload <platform>/bootstrap-upload-steps.yml``.
|
||||
- Injects ``if`` by step ``key`` from skip-ci and uploads child steps (image build, L2–L5 upload).
|
||||
- Detect docs-only, pytest skip-mark-only, or combined skip-ci from git diff.
|
||||
- When only CI level YAML changes, enable **L2/L3** upload steps for affected levels only.
|
||||
|
||||
Test pipeline mode (e.g. test-merge.yml, test-nightly.yml, test-weekly.yml):
|
||||
- Drop steps whose ``source_file_dependencies`` do not match changed files
|
||||
(string/list keys from ci_source_file_dependencies.yml, or inline path
|
||||
prefixes). Filtering applies on **PR label** uploads and on post-merge
|
||||
``main`` L3 uploads. Scheduled ``main`` uploads that pass ``--all``
|
||||
(NIGHTLY/WEEKLY) or ``--e2e`` keep every selected job and still strip
|
||||
the field. If no job-key prefix matches, a change to the pipeline YAML
|
||||
being uploaded or to a path under the ``source_filter_fallback`` registry
|
||||
key keeps every job so command, env, and hardware edits can be validated
|
||||
before merge. When any job-key prefix already matches, normal filtering
|
||||
wins.
|
||||
- Expand uploader-only ``mirror_hardwares`` into ``agents`` (+ optional ``image``
|
||||
for NPU) + ``plugins`` (see ci_mirror_hardwares.yml).
|
||||
- Omit ``mirror_hardwares`` to compose ``{chip}_{n}`` from pytest ``-m`` SKU
|
||||
markers plus ``cards_n`` (max of several positives; ``not cards_1`` with no
|
||||
positive ``cards_*`` uses that chip's highest existing preset). An explicit
|
||||
preset string (``h100_2``) is used as-is (marks are ignored). Names that
|
||||
are not keys in ``ci_mirror_hardwares.yml`` fail the upload.
|
||||
|
||||
Inferred chip: unset ``MIRROR_HW`` matches ``H100`` or ``L4`` in ``-m``
|
||||
(both → H100); no match skips the step. ``MIRROR_HW=b200`` matches
|
||||
``B200`` in ``-m`` (otherwise skipped). ``-m`` is not rewritten.
|
||||
``MIRROR_HW`` must be empty or ``b200``; unknown values fail the upload.
|
||||
A CUDA preset string such as ``h100_4`` is omitted when ``MIRROR_HW=b200``.
|
||||
|
||||
Usage:
|
||||
python3 upload_pipeline.py [--upload] [--all | --e2e] <pipeline.yml>
|
||||
|
||||
Requires PyYAML (``pip install pyyaml``); installs it automatically when missing.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import copy
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
from functools import lru_cache
|
||||
from pathlib import Path
|
||||
from typing import Any, Literal
|
||||
|
||||
try:
|
||||
import yaml
|
||||
except ModuleNotFoundError:
|
||||
subprocess.run(
|
||||
[sys.executable, "-m", "pip", "install", "-q", "pyyaml"],
|
||||
check=True,
|
||||
)
|
||||
import yaml
|
||||
|
||||
from skip_ci import (
|
||||
ROOT,
|
||||
resolve_ci_context_from_git,
|
||||
)
|
||||
|
||||
if str(ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(ROOT))
|
||||
|
||||
from tests.helpers.mark import ( # noqa: E402
|
||||
get_skus_for_platform,
|
||||
get_supported_card_counts,
|
||||
)
|
||||
|
||||
# --- Constants ---
|
||||
|
||||
LOG = "upload_pipeline"
|
||||
BOOTSTRAP_STEPS_FILENAME = "bootstrap-upload-steps.yml"
|
||||
BOOTSTRAP_IMAGE_BUILD_KEYS = frozenset(
|
||||
{
|
||||
"image-build",
|
||||
"image-build-a2",
|
||||
"image-build-a3",
|
||||
"image-build-a5",
|
||||
"image-build-310p",
|
||||
}
|
||||
)
|
||||
BOOTSTRAP_UPLOAD_IF_KEYS = {
|
||||
"upload-ready-pipeline": "ready",
|
||||
"upload-merge-pipeline": "merge",
|
||||
"upload-nightly-pipeline": "nightly",
|
||||
"upload-weekly-pipeline": "weekly",
|
||||
}
|
||||
E2E_GROUP_MARKER = "E2E Test"
|
||||
CI_MIRROR_HARDWARES_PATH = ROOT / ".buildkite/common/ci_mirror_hardwares.yml"
|
||||
CI_SOURCE_FILE_DEPENDENCIES_PATH = ROOT / ".buildkite/common/ci_source_file_dependencies.yml"
|
||||
# Registry key whose prefixes bypass source filtering. Paths live in
|
||||
# ci_source_file_dependencies.yml; do not attach this key to a job.
|
||||
SOURCE_FILTER_FALLBACK_KEY = "source_filter_fallback"
|
||||
CUDA_HF_TOKEN_ENV = "VLLM_CI_HF_TOKEN"
|
||||
CUDA_HF_TOKEN_EXPORT = f'if [ -n "$${{{CUDA_HF_TOKEN_ENV}:-}}" ]; then export HF_TOKEN="$${{{CUDA_HF_TOKEN_ENV}}}"; fi'
|
||||
CUDA_RUNTIME_CHECK = "python3 .buildkite/cuda/scripts/check_vllm_runtime.py"
|
||||
|
||||
# Bootstrap Buildkite ``if`` expressions.
|
||||
# ``*_MAIN_IF``: main + env schedule. ``*_LABEL_IF``: PR label (and/or composed with MAIN).
|
||||
# ``*_UPLOAD_IF``: full gate for uploading that child pipeline.
|
||||
NIGHTLY_MAIN_IF = 'build.branch == "main" && build.env("NIGHTLY") == "1"'
|
||||
NIGHTLY_LABEL_IF = (
|
||||
f'({NIGHTLY_MAIN_IF}) || (build.branch != "main" && build.pull_request.labels includes "nightly-test")'
|
||||
)
|
||||
WEEKLY_E2E_IF = 'build.branch == "main" && build.env("WEEKLY") == "1"'
|
||||
WEEKLY_MAIN_IF = 'build.branch == "main" && (build.env("WEEKLY") == "1" || build.env("NON_CRITICAL") == "1")'
|
||||
WEEKLY_LABEL_IF = f'({WEEKLY_MAIN_IF}) || (build.branch != "main" && build.pull_request.labels includes "weekly-test")'
|
||||
READY_LABEL_IF = 'build.branch != "main" && build.pull_request.labels includes "ready"'
|
||||
MERGE_LABEL_IF = 'build.branch != "main" && build.pull_request.labels includes "merge-test"'
|
||||
MERGE_MAIN_IF = (
|
||||
'build.branch == "main" && build.env("NIGHTLY") != "1" && '
|
||||
'build.env("WEEKLY") != "1" && build.env("NON_CRITICAL") != "1"'
|
||||
)
|
||||
READY_UPLOAD_IF = f"({WEEKLY_E2E_IF}) || ({READY_LABEL_IF})"
|
||||
MERGE_UPLOAD_IF = f"({WEEKLY_E2E_IF}) || (({MERGE_MAIN_IF}) || ({MERGE_LABEL_IF}))"
|
||||
BOOTSTRAP_DISABLED_IF = "false"
|
||||
BOOTSTRAP_ENABLED_IF = "true"
|
||||
|
||||
|
||||
# --- Logging ---
|
||||
|
||||
|
||||
def _log(message: str) -> None:
|
||||
print(f"{LOG}: {message}", file=sys.stderr)
|
||||
|
||||
|
||||
# --- Bootstrap pipeline (bootstrap-upload-steps.yml) ---
|
||||
|
||||
|
||||
def _get_bootstrap_platform(path: Path) -> str:
|
||||
parts = path.as_posix().split("/")
|
||||
return "npu" if "npu" in parts else "cuda"
|
||||
|
||||
|
||||
def _load_bootstrap_steps(path: Path) -> str:
|
||||
if path.name != BOOTSTRAP_STEPS_FILENAME:
|
||||
raise ValueError(f"expected {BOOTSTRAP_STEPS_FILENAME}, got {path.name}")
|
||||
return path.read_text(encoding="utf-8")
|
||||
|
||||
|
||||
def _format_bootstrap_if(expr: str) -> str:
|
||||
"""Return a Buildkite ``if`` string. Buildkite rejects YAML bool ``if`` values."""
|
||||
if expr in ("true", "false"):
|
||||
return expr
|
||||
return f"({expr})"
|
||||
|
||||
|
||||
def _compute_bootstrap_if_exprs(*, decision, platform: str) -> dict[str, str]:
|
||||
if platform == "npu":
|
||||
ready_upload = READY_LABEL_IF
|
||||
merge_upload = BOOTSTRAP_DISABLED_IF
|
||||
weekly_label_if = BOOTSTRAP_DISABLED_IF
|
||||
else:
|
||||
ready_upload = READY_UPLOAD_IF
|
||||
merge_upload = MERGE_UPLOAD_IF
|
||||
weekly_label_if = WEEKLY_LABEL_IF
|
||||
|
||||
if decision.skip_all:
|
||||
# Docs / skip-mark only: no PR-label escape hatch. Main scheduled
|
||||
# NIGHTLY=1 still runs L4; WEEKLY=1 / NON_CRITICAL=1 still run L5.
|
||||
# main+WEEKLY=1 also uploads L2/L3 (those steps then pass --e2e).
|
||||
image_expr = f"({NIGHTLY_MAIN_IF}) || ({WEEKLY_MAIN_IF})" if platform == "cuda" else NIGHTLY_MAIN_IF
|
||||
ready_expr = WEEKLY_E2E_IF if platform == "cuda" else BOOTSTRAP_DISABLED_IF
|
||||
merge_expr = WEEKLY_E2E_IF if platform == "cuda" else BOOTSTRAP_DISABLED_IF
|
||||
nightly_expr = NIGHTLY_MAIN_IF
|
||||
weekly_expr = WEEKLY_MAIN_IF if platform == "cuda" else BOOTSTRAP_DISABLED_IF
|
||||
elif decision.skip_l2_l3:
|
||||
l2_enabled = decision.is_run("npu", "l2") if platform == "npu" else decision.is_run("cuda", "l2")
|
||||
l3_enabled = platform == "cuda" and decision.is_run("cuda", "l3")
|
||||
|
||||
ready_expr = ready_upload if l2_enabled else BOOTSTRAP_DISABLED_IF
|
||||
merge_expr = merge_upload if l3_enabled else BOOTSTRAP_DISABLED_IF
|
||||
nightly_expr = NIGHTLY_LABEL_IF
|
||||
weekly_expr = weekly_label_if if platform == "cuda" else BOOTSTRAP_DISABLED_IF
|
||||
|
||||
image_parts = [f"({NIGHTLY_LABEL_IF})"]
|
||||
if platform == "cuda":
|
||||
image_parts.append(f"({weekly_label_if})")
|
||||
if l2_enabled:
|
||||
image_parts.insert(0, f"({ready_upload})")
|
||||
if l3_enabled:
|
||||
image_parts.insert(1 if l2_enabled else 0, f"({merge_upload})")
|
||||
image_expr = " || ".join(image_parts)
|
||||
else:
|
||||
image_expr = BOOTSTRAP_ENABLED_IF
|
||||
ready_expr = ready_upload
|
||||
merge_expr = merge_upload if platform == "cuda" else BOOTSTRAP_DISABLED_IF
|
||||
nightly_expr = NIGHTLY_LABEL_IF
|
||||
weekly_expr = weekly_label_if if platform == "cuda" else BOOTSTRAP_DISABLED_IF
|
||||
|
||||
return {
|
||||
"image": _format_bootstrap_if(image_expr),
|
||||
"ready": _format_bootstrap_if(ready_expr),
|
||||
"merge": _format_bootstrap_if(merge_expr),
|
||||
"nightly": _format_bootstrap_if(nightly_expr),
|
||||
"weekly": _format_bootstrap_if(weekly_expr),
|
||||
}
|
||||
|
||||
|
||||
def _apply_bootstrap_if(steps: list[Any], if_exprs: dict[str, str]) -> list[Any]:
|
||||
"""Inject ``if`` by step key; drop steps that are unconditionally disabled."""
|
||||
kept: list[Any] = []
|
||||
for step in steps:
|
||||
if not isinstance(step, dict):
|
||||
kept.append(step)
|
||||
continue
|
||||
nested = step.get("steps")
|
||||
if isinstance(nested, list):
|
||||
nested_kept = _apply_bootstrap_if(nested, if_exprs)
|
||||
if nested_kept:
|
||||
kept.append({**step, "steps": nested_kept})
|
||||
continue
|
||||
key = step.get("key")
|
||||
if key in BOOTSTRAP_IMAGE_BUILD_KEYS:
|
||||
step["if"] = if_exprs["image"]
|
||||
elif key in BOOTSTRAP_UPLOAD_IF_KEYS:
|
||||
step["if"] = if_exprs[BOOTSTRAP_UPLOAD_IF_KEYS[key]]
|
||||
if_expr = step.get("if")
|
||||
if if_expr == "false":
|
||||
_log(f"omit disabled bootstrap step {key!r}")
|
||||
continue
|
||||
if if_expr == "true":
|
||||
# Unconditional step: omit ``if`` (Buildkite requires string ``if``, not YAML bool).
|
||||
step.pop("if", None)
|
||||
kept.append(step)
|
||||
return kept
|
||||
|
||||
|
||||
def _render_bootstrap_pipeline(
|
||||
steps_yaml: str,
|
||||
*,
|
||||
decision,
|
||||
path: Path,
|
||||
) -> str:
|
||||
"""Load bootstrap steps YAML and inject ``if`` expressions from skip-ci decision."""
|
||||
doc = yaml.safe_load(steps_yaml)
|
||||
if not isinstance(doc, dict):
|
||||
raise ValueError(f"invalid bootstrap steps YAML for {path}")
|
||||
steps = doc.get("steps")
|
||||
if not isinstance(steps, list):
|
||||
raise ValueError(f"bootstrap steps YAML must contain steps: list in {path}")
|
||||
|
||||
platform = _get_bootstrap_platform(path)
|
||||
if_exprs = _compute_bootstrap_if_exprs(decision=decision, platform=platform)
|
||||
doc["steps"] = _apply_bootstrap_if(steps, if_exprs)
|
||||
return yaml.safe_dump(doc, sort_keys=False)
|
||||
|
||||
|
||||
# --- Test pipeline (test-ready.yml, test-merge.yml, test-nightly.yml) ---
|
||||
|
||||
|
||||
@lru_cache(maxsize=1)
|
||||
def _load_mirror_hardwares() -> dict[str, dict[str, Any]]:
|
||||
if not CI_MIRROR_HARDWARES_PATH.is_file():
|
||||
raise FileNotFoundError(f"missing CI mirror_hardwares registry: {CI_MIRROR_HARDWARES_PATH}")
|
||||
doc = yaml.safe_load(CI_MIRROR_HARDWARES_PATH.read_text(encoding="utf-8"))
|
||||
if not isinstance(doc, dict):
|
||||
raise ValueError(f"invalid CI mirror_hardwares registry: {CI_MIRROR_HARDWARES_PATH}")
|
||||
presets = doc.get("mirror_hardwares")
|
||||
if not isinstance(presets, dict):
|
||||
raise ValueError(f"mirror_hardwares must be a mapping in {CI_MIRROR_HARDWARES_PATH}")
|
||||
return presets
|
||||
|
||||
|
||||
@lru_cache(maxsize=1)
|
||||
def _load_source_file_dependencies() -> dict[str, list[str]]:
|
||||
def flatten(value: Any, *, key: str) -> list[str]:
|
||||
"""Flatten YAML-anchor nested lists into a de-duplicated prefix list."""
|
||||
if isinstance(value, str):
|
||||
if not value.strip():
|
||||
raise ValueError(f"empty path in source_file_dependencies[{key!r}]")
|
||||
return [value]
|
||||
if isinstance(value, list):
|
||||
flattened: list[str] = []
|
||||
seen: set[str] = set()
|
||||
for item in value:
|
||||
for path in flatten(item, key=key):
|
||||
if path not in seen:
|
||||
seen.add(path)
|
||||
flattened.append(path)
|
||||
return flattened
|
||||
raise ValueError(
|
||||
f"source_file_dependencies[{key!r}] must be a list of path prefixes, got {type(value).__name__}",
|
||||
)
|
||||
|
||||
if not CI_SOURCE_FILE_DEPENDENCIES_PATH.is_file():
|
||||
raise FileNotFoundError(
|
||||
f"missing CI source_file_dependencies registry: {CI_SOURCE_FILE_DEPENDENCIES_PATH}",
|
||||
)
|
||||
doc = yaml.safe_load(CI_SOURCE_FILE_DEPENDENCIES_PATH.read_text(encoding="utf-8"))
|
||||
if not isinstance(doc, dict):
|
||||
raise ValueError(f"invalid CI source_file_dependencies registry: {CI_SOURCE_FILE_DEPENDENCIES_PATH}")
|
||||
presets = doc.get("source_file_dependencies")
|
||||
if not isinstance(presets, dict):
|
||||
raise ValueError(f"source_file_dependencies must be a mapping in {CI_SOURCE_FILE_DEPENDENCIES_PATH}")
|
||||
loaded: dict[str, list[str]] = {}
|
||||
for key, value in presets.items():
|
||||
if not isinstance(key, str) or not key.strip():
|
||||
raise ValueError(
|
||||
f"source_file_dependencies keys must be non-empty strings in {CI_SOURCE_FILE_DEPENDENCIES_PATH}",
|
||||
)
|
||||
loaded[key] = flatten(value, key=key)
|
||||
return loaded
|
||||
|
||||
|
||||
_REGISTRY_KEY_RE = re.compile(r"^[a-z][a-z0-9_]*$")
|
||||
|
||||
|
||||
def _parse_command_text(step: dict[str, Any]) -> str:
|
||||
"""Parse step ``commands`` / ``command`` into a single string."""
|
||||
raw = step.get("commands", step.get("command"))
|
||||
if raw is None:
|
||||
return ""
|
||||
if isinstance(raw, str):
|
||||
return raw
|
||||
if isinstance(raw, list):
|
||||
return "\n".join(str(item) for item in raw if item is not None)
|
||||
return str(raw)
|
||||
|
||||
|
||||
def _resolve_source_file_dependencies(step: dict[str, Any]) -> list[str] | None:
|
||||
"""Expand a registry key (or list of keys / inline path prefixes)."""
|
||||
deps = step.get("source_file_dependencies")
|
||||
if deps is None:
|
||||
return None
|
||||
|
||||
def dedupe(*groups: list[str]) -> list[str]:
|
||||
merged: list[str] = []
|
||||
seen: set[str] = set()
|
||||
for group in groups:
|
||||
for path in group:
|
||||
if path not in seen:
|
||||
seen.add(path)
|
||||
merged.append(path)
|
||||
return merged
|
||||
|
||||
def lookup(key: str) -> list[str]:
|
||||
if key == SOURCE_FILTER_FALLBACK_KEY:
|
||||
raise ValueError(
|
||||
f"source_file_dependencies {key!r} bypasses filtering and cannot be attached to a step "
|
||||
f"({_get_step_label(step)!r})",
|
||||
)
|
||||
registry = _load_source_file_dependencies()
|
||||
paths = registry.get(key)
|
||||
if paths is None:
|
||||
known = ", ".join(sorted(registry))
|
||||
raise ValueError(
|
||||
f"unknown source_file_dependencies {key!r} in step {_get_step_label(step)!r}; known: {known}",
|
||||
)
|
||||
return list(paths)
|
||||
|
||||
prefixes: list[str]
|
||||
if isinstance(deps, str):
|
||||
prefixes = lookup(deps) if _REGISTRY_KEY_RE.fullmatch(deps) else [deps]
|
||||
elif isinstance(deps, list):
|
||||
if not all(isinstance(item, str) for item in deps):
|
||||
raise ValueError(
|
||||
f"source_file_dependencies must be a string key or list of strings in step {_get_step_label(step)!r}",
|
||||
)
|
||||
keyish = [bool(_REGISTRY_KEY_RE.fullmatch(item)) for item in deps]
|
||||
if deps and all(keyish):
|
||||
prefixes = dedupe(*(lookup(key) for key in deps))
|
||||
elif any(keyish):
|
||||
raise ValueError(
|
||||
f"source_file_dependencies in step {_get_step_label(step)!r} mixes registry keys and path prefixes",
|
||||
)
|
||||
else:
|
||||
prefixes = list(deps)
|
||||
else:
|
||||
raise ValueError(
|
||||
f"source_file_dependencies must be a string key or list in step {_get_step_label(step)!r}",
|
||||
)
|
||||
return prefixes
|
||||
|
||||
|
||||
@lru_cache(maxsize=1)
|
||||
def _cuda_mirror_chips() -> tuple[str, ...]:
|
||||
"""Lowercase CUDA SKUs that have at least one ``{chip}_{n}`` preset.
|
||||
|
||||
Longer tokens first so ``h100`` is not matched as a prefix of ``h200``.
|
||||
"""
|
||||
skus = get_skus_for_platform("cuda")
|
||||
known = _load_mirror_hardwares()
|
||||
chips = {
|
||||
sku.lower() for sku in skus if any(name == sku.lower() or name.startswith(f"{sku.lower()}_") for name in known)
|
||||
}
|
||||
return tuple(sorted(chips, key=lambda chip: (-len(chip), chip)))
|
||||
|
||||
|
||||
def _parse_cards_marks(expr: str) -> tuple[set[int], bool]:
|
||||
"""Parse registered ``cards_n`` / ``not cards_n`` from a pytest ``-m`` expr.
|
||||
|
||||
Returns positive card counts and whether any ``not cards_*`` is present.
|
||||
"""
|
||||
counts = get_supported_card_counts()
|
||||
alt = "|".join(str(n) for n in sorted(counts, key=lambda n: (-len(str(n)), -n))) if counts else "(?!)"
|
||||
positives: set[int] = set()
|
||||
has_not_cards = False
|
||||
for match in re.finditer(rf"\b(?:(not)\s+)?cards_({alt})\b", expr):
|
||||
if match.group(1):
|
||||
has_not_cards = True
|
||||
else:
|
||||
positives.add(int(match.group(2)))
|
||||
return positives, has_not_cards
|
||||
|
||||
|
||||
def _parse_hardware_marks(expr: str) -> set[str]:
|
||||
"""Parse lowercase positive CUDA SKUs that have a mirror preset.
|
||||
|
||||
``not H100`` is ignored. SKUs with no ``{chip}_*`` preset (e.g. H200) are dropped.
|
||||
"""
|
||||
skus = sorted(get_skus_for_platform("cuda"), key=lambda s: (-len(s), s))
|
||||
if not skus:
|
||||
return set()
|
||||
alt = "|".join(re.escape(sku) for sku in skus)
|
||||
mirror = set(_cuda_mirror_chips())
|
||||
chips: set[str] = set()
|
||||
for match in re.finditer(rf"\b(?:(not)\s+)?({alt})\b", expr):
|
||||
if match.group(1):
|
||||
continue
|
||||
chip = match.group(2).lower()
|
||||
if chip in mirror:
|
||||
chips.add(chip)
|
||||
return chips
|
||||
|
||||
|
||||
# CI policy: not pytest-mark registry.
|
||||
_SUPPORTED_MIRROR_HW_SELECTORS = frozenset({"b200"})
|
||||
# Unset MIRROR_HW: match H100 then L4 in -m (skip if neither).
|
||||
_DEFAULT_INFER_CHIPS = ("h100", "l4")
|
||||
_PYTEST_MARKER_ARG = re.compile(r"-m\s+(?:\"([^\"]*)\"|'([^']*)'|(\S+))")
|
||||
_CARDS_CHIP_MAX: Literal["max"] = "max"
|
||||
|
||||
|
||||
def _get_mirror_hw_selector() -> str:
|
||||
"""Return lowercase ``MIRROR_HW``, or empty to keep string presets / marker chips.
|
||||
|
||||
A set value must be ``b200`` (case-insensitive). Unknown values fail closed
|
||||
so a typo cannot silently drop the CUDA pipeline.
|
||||
"""
|
||||
selector = os.environ.get("MIRROR_HW", "").strip().lower()
|
||||
if not selector:
|
||||
return ""
|
||||
if selector not in _SUPPORTED_MIRROR_HW_SELECTORS:
|
||||
allowed = " or ".join(repr(s) for s in sorted(_SUPPORTED_MIRROR_HW_SELECTORS))
|
||||
raise ValueError(f"unsupported MIRROR_HW={selector!r}; expected empty or {allowed}")
|
||||
return selector
|
||||
|
||||
|
||||
def _parse_pytest_marks(step: dict[str, Any]) -> tuple[set[str], int | Literal["max"] | None]:
|
||||
"""Parse pytest ``-m`` from step commands: CUDA SKU chips and cards count.
|
||||
|
||||
*chips* are lowercase SKUs from positive markers (``not H100`` is ignored).
|
||||
*cards* is the max positive ``cards_n``, ``"max"`` when only ``not cards_*``
|
||||
is present, or ``None``.
|
||||
"""
|
||||
text = _parse_command_text(step)
|
||||
|
||||
chips: set[str] = set()
|
||||
positives: set[int] = set()
|
||||
has_not_cards = False
|
||||
for match in _PYTEST_MARKER_ARG.finditer(text):
|
||||
expr = match.group(1) or match.group(2) or match.group(3) or ""
|
||||
chips |= _parse_hardware_marks(expr)
|
||||
card_counts, not_cards = _parse_cards_marks(expr)
|
||||
positives.update(card_counts)
|
||||
has_not_cards = has_not_cards or not_cards
|
||||
|
||||
if positives:
|
||||
cards: int | Literal["max"] | None = max(positives)
|
||||
elif has_not_cards:
|
||||
cards = _CARDS_CHIP_MAX
|
||||
else:
|
||||
cards = None
|
||||
return chips, cards
|
||||
|
||||
|
||||
def _compose_mirror_hardware_name(
|
||||
chips: set[str],
|
||||
cards: int | Literal["max"] | None,
|
||||
*,
|
||||
step_label: str,
|
||||
) -> str | None:
|
||||
"""Assemble ``{chip}_{n}`` from pytest ``-m`` chips/cards, or None to skip.
|
||||
|
||||
Unset ``MIRROR_HW``: ``H100`` if in ``-m``, else ``L4``, else skip.
|
||||
Set ``MIRROR_HW``: that chip if in ``-m``, else skip.
|
||||
``cards`` is a registered ``cards_n``, ``"max"`` for that chip's highest
|
||||
existing preset, or None (error after a chip matched — no ``cards_*``).
|
||||
"""
|
||||
selector = _get_mirror_hw_selector()
|
||||
if selector:
|
||||
chip = selector if selector in chips else None
|
||||
skip_reason = f"MIRROR_HW={selector!r} not in -m SKUs {sorted(chips)}"
|
||||
else:
|
||||
chip = next((c for c in _DEFAULT_INFER_CHIPS if c in chips), None)
|
||||
skip_reason = f"no H100 or L4 in -m SKUs {sorted(chips)}"
|
||||
if chip is None:
|
||||
_log(f"skip {step_label}: {skip_reason}")
|
||||
return None
|
||||
if cards is None:
|
||||
raise ValueError(
|
||||
f"step {step_label!r} has a pytest -m SKU marker but no cards_* "
|
||||
f"(or not cards_*); add cards_n or set mirror_hardwares to force a preset",
|
||||
)
|
||||
|
||||
count_source = "not cards_*" if cards == _CARDS_CHIP_MAX else f"cards_{cards}"
|
||||
count = None if cards == _CARDS_CHIP_MAX else cards
|
||||
registered = get_supported_card_counts()
|
||||
if count is not None and count not in registered:
|
||||
supported = ", ".join(f"cards_{n}" for n in sorted(registered)) or "(none)"
|
||||
raise ValueError(
|
||||
f"{count_source} in step {step_label!r} is not a registered cards_* mark; supported: {supported}",
|
||||
)
|
||||
known = _load_mirror_hardwares()
|
||||
if count is None:
|
||||
available = [n for n in sorted(registered) if f"{chip}_{n}" in known]
|
||||
if not available:
|
||||
raise ValueError(
|
||||
f"{count_source} in step {step_label!r} has no CUDA count presets for {chip}",
|
||||
)
|
||||
count = max(available)
|
||||
chosen = f"{chip}_{count}"
|
||||
if chosen not in known:
|
||||
raise ValueError(
|
||||
f"{count_source} in step {step_label!r} has no preset {chosen!r}",
|
||||
)
|
||||
_log(f"{step_label}: {count_source} → {chosen!r}")
|
||||
return chosen
|
||||
|
||||
|
||||
def _resolve_mirror_hardware_name(hardware: Any, *, step_label: str) -> str | None:
|
||||
"""Map ``mirror_hardwares`` to a ``ci_mirror_hardwares.yml`` preset, or None to skip.
|
||||
|
||||
The value is a preset name (``h100_2``). Unknown names fail later when the
|
||||
registry is loaded. CUDA names are omitted when ``MIRROR_HW`` names a
|
||||
different chip; NPU strings ignore ``MIRROR_HW``.
|
||||
"""
|
||||
if isinstance(hardware, bool):
|
||||
raise ValueError(f"mirror_hardwares must not be a boolean in step {step_label!r}")
|
||||
if isinstance(hardware, str):
|
||||
name = hardware.strip()
|
||||
elif isinstance(hardware, int):
|
||||
name = str(hardware)
|
||||
else:
|
||||
raise ValueError(
|
||||
f"mirror_hardwares must be a preset string in step {step_label!r}",
|
||||
)
|
||||
if not name:
|
||||
raise ValueError(f"mirror_hardwares must be a non-empty string in step {step_label!r}")
|
||||
|
||||
selector = _get_mirror_hw_selector()
|
||||
token = name.lower()
|
||||
chip = next(
|
||||
(c for c in _cuda_mirror_chips() if token == c or token.startswith(f"{c}_")),
|
||||
None,
|
||||
)
|
||||
if selector and chip is not None and chip != selector:
|
||||
_log(
|
||||
f"skip {step_label}: preset {name!r} is {chip} hardware; MIRROR_HW={selector!r}",
|
||||
)
|
||||
return None
|
||||
return name
|
||||
|
||||
|
||||
def _expand_mirror_hardwares(step: dict[str, Any]) -> dict[str, Any] | None:
|
||||
"""Replace ``mirror_hardwares`` (or infer it from ``-m``) with preset agents/plugins.
|
||||
|
||||
Explicit preset string: used as-is; pytest marks are ignored.
|
||||
Omitted key: match H100/L4 (or ``MIRROR_HW``) in ``-m``, then compose
|
||||
``{chip}_{n}`` from ``cards_n``. No match skips the step. Steps with no
|
||||
SKU/cards and no ``mirror_hardwares`` are left unchanged (CPU jobs).
|
||||
|
||||
Returns None when the current ``MIRROR_HW`` selector does not apply.
|
||||
"""
|
||||
step_label = _get_step_label(step)
|
||||
has_pool = step.get("agents") is not None or step.get("plugins") is not None or step.get("image") is not None
|
||||
|
||||
if "mirror_hardwares" not in step:
|
||||
if has_pool:
|
||||
return step
|
||||
chips, cards = _parse_pytest_marks(step)
|
||||
if not chips and cards is None:
|
||||
return step
|
||||
preset_name = _compose_mirror_hardware_name(chips, cards, step_label=step_label)
|
||||
else:
|
||||
if has_pool:
|
||||
raise ValueError(
|
||||
f"step {step_label!r} sets mirror_hardwares together with agents/plugins/image; "
|
||||
f"use mirror_hardwares only",
|
||||
)
|
||||
preset_name = _resolve_mirror_hardware_name(
|
||||
step.get("mirror_hardwares"),
|
||||
step_label=step_label,
|
||||
)
|
||||
|
||||
if preset_name is None:
|
||||
return None
|
||||
|
||||
preset = _load_mirror_hardwares().get(preset_name)
|
||||
if preset is None:
|
||||
known = ", ".join(sorted(_load_mirror_hardwares()))
|
||||
raise ValueError(
|
||||
f"unknown mirror_hardwares {preset_name!r} in step {step_label!r}; known: {known}",
|
||||
)
|
||||
|
||||
expanded = copy.deepcopy(preset)
|
||||
merged = {key: value for key, value in step.items() if key != "mirror_hardwares"} | expanded
|
||||
# A scheduled build's HF_TOKEN may override a pod env entry with the same
|
||||
# name. The preset injects the Kubernetes secret under a collision-proof
|
||||
# alias; restore the conventional name after Buildkite has merged env.
|
||||
if any(preset_name == chip or preset_name.startswith(f"{chip}_") for chip in _cuda_mirror_chips()):
|
||||
commands = merged.get("commands")
|
||||
if isinstance(commands, list):
|
||||
merged["commands"] = [CUDA_HF_TOKEN_EXPORT, CUDA_RUNTIME_CHECK, *commands]
|
||||
elif commands is None:
|
||||
merged["commands"] = [CUDA_HF_TOKEN_EXPORT, CUDA_RUNTIME_CHECK]
|
||||
else:
|
||||
merged["commands"] = [CUDA_HF_TOKEN_EXPORT, CUDA_RUNTIME_CHECK, commands]
|
||||
# Preset retry (K8S_RETRY on l4_*) must not clobber a step that opted out.
|
||||
if "retry" in step:
|
||||
merged["retry"] = step["retry"]
|
||||
return merged
|
||||
|
||||
|
||||
def _match_source_file(changed_files: list[str], prefixes: list[str]) -> bool:
|
||||
for path in changed_files:
|
||||
for prefix in prefixes:
|
||||
normalized = prefix.rstrip("/")
|
||||
if path == normalized or path.startswith(f"{normalized}/"):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _get_step_label(step: dict[str, Any]) -> str:
|
||||
return str(step.get("group") or step.get("label") or "<step>")
|
||||
|
||||
|
||||
def _process_test_steps(
|
||||
steps: list[Any],
|
||||
changed_files: list[str] | None,
|
||||
) -> list[Any]:
|
||||
"""Drop steps by ``source_file_dependencies`` when *changed_files* is set; always strip that field."""
|
||||
processed: list[Any] = []
|
||||
for step in steps:
|
||||
if not isinstance(step, dict):
|
||||
processed.append(step)
|
||||
continue
|
||||
|
||||
deps = _resolve_source_file_dependencies(step)
|
||||
if changed_files is not None and deps is not None and not _match_source_file(changed_files, deps):
|
||||
_log(f"skip {_get_step_label(step)!r} (no changes under {deps})")
|
||||
continue
|
||||
|
||||
nested = step.get("steps")
|
||||
if nested is not None:
|
||||
kept_nested = _process_test_steps(nested, changed_files)
|
||||
if not kept_nested:
|
||||
_log(f"omit empty group {_get_step_label(step)!r}")
|
||||
continue
|
||||
new_step = {key: value for key, value in step.items() if key != "source_file_dependencies"}
|
||||
new_step["steps"] = kept_nested
|
||||
processed.append(new_step)
|
||||
continue
|
||||
|
||||
leaf = {key: value for key, value in step.items() if key != "source_file_dependencies"}
|
||||
expanded = _expand_mirror_hardwares(leaf)
|
||||
if expanded is None:
|
||||
continue
|
||||
processed.append(expanded)
|
||||
|
||||
return processed
|
||||
|
||||
|
||||
def _select_e2e_group_steps(steps: list[Any]) -> list[Any]:
|
||||
"""Keep only top-level groups whose name contains ``E2E_GROUP_MARKER``."""
|
||||
selected = [
|
||||
step
|
||||
for step in steps
|
||||
if isinstance(step, dict) and isinstance(step.get("group"), str) and E2E_GROUP_MARKER in step["group"]
|
||||
]
|
||||
if not selected:
|
||||
_log(f"no group matching {E2E_GROUP_MARKER!r} found")
|
||||
else:
|
||||
_log(f"keep {len(selected)} group(s) matching {E2E_GROUP_MARKER!r}")
|
||||
return selected
|
||||
|
||||
|
||||
def _any_source_dependency_match(steps: list[Any], changed_files: list[str]) -> bool:
|
||||
"""True when any step with ``source_file_dependencies`` matches *changed_files*."""
|
||||
for step in steps:
|
||||
if not isinstance(step, dict):
|
||||
continue
|
||||
deps = _resolve_source_file_dependencies(step)
|
||||
if deps is not None and _match_source_file(changed_files, deps):
|
||||
return True
|
||||
nested = step.get("steps")
|
||||
if isinstance(nested, list) and _any_source_dependency_match(nested, changed_files):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _render_test_pipeline(
|
||||
doc: dict[str, Any],
|
||||
changed_files: list[str] | None,
|
||||
*,
|
||||
pipeline_path: Path | None = None,
|
||||
e2e_only: bool = False,
|
||||
) -> dict[str, Any]:
|
||||
"""Filter steps by PR diff and strip uploader-only ``source_file_dependencies`` metadata."""
|
||||
# Validate MIRROR_HW once up front. Per-step skip must not run for typos
|
||||
# (e.g. b20o), or a pipeline can shrink to leftover CPU-only steps and
|
||||
# still upload successfully.
|
||||
_get_mirror_hw_selector()
|
||||
steps = doc.get("steps")
|
||||
if not isinstance(steps, list):
|
||||
return doc
|
||||
if e2e_only:
|
||||
steps = _select_e2e_group_steps(steps)
|
||||
# Bypass is a fallback: only when no job-key prefix matched.
|
||||
if changed_files is not None and pipeline_path is not None:
|
||||
if not _any_source_dependency_match(steps, changed_files):
|
||||
bypass = _source_filter_fallback_reason(changed_files, pipeline_path)
|
||||
if bypass is not None:
|
||||
_log(f"keep all jobs (no source_file_dependencies match; bypassed by {bypass})")
|
||||
changed_files = None
|
||||
steps = _process_test_steps(steps, changed_files)
|
||||
return {**doc, "steps": steps}
|
||||
|
||||
|
||||
# --- Entry (read file → bootstrap or test render → YAML string) ---
|
||||
|
||||
|
||||
def _repo_relative_posix(path: Path) -> str:
|
||||
resolved = path.resolve()
|
||||
try:
|
||||
return resolved.relative_to(ROOT).as_posix()
|
||||
except ValueError:
|
||||
return path.as_posix().replace("\\", "/")
|
||||
|
||||
|
||||
def _source_filter_fallback_prefixes() -> list[str]:
|
||||
"""Shared uploader/registry prefixes from ``source_filter_fallback``. Not a job key."""
|
||||
registry = _load_source_file_dependencies()
|
||||
prefixes = registry.get(SOURCE_FILTER_FALLBACK_KEY)
|
||||
if not prefixes:
|
||||
raise ValueError(
|
||||
f"source_file_dependencies[{SOURCE_FILTER_FALLBACK_KEY!r}] must list shared uploader paths "
|
||||
f"in {CI_SOURCE_FILE_DEPENDENCIES_PATH}",
|
||||
)
|
||||
return prefixes
|
||||
|
||||
|
||||
def _source_filter_fallback_reason(changed_files: list[str] | None, pipeline_path: Path) -> str | None:
|
||||
"""Return the changed path that should disable source filtering, if any."""
|
||||
if not changed_files:
|
||||
return None
|
||||
changed = set(changed_files)
|
||||
pipeline_rel = _repo_relative_posix(pipeline_path)
|
||||
if pipeline_rel in changed:
|
||||
return pipeline_rel
|
||||
prefixes = _source_filter_fallback_prefixes()
|
||||
for path in changed_files:
|
||||
if _match_source_file([path], prefixes):
|
||||
return path
|
||||
return None
|
||||
|
||||
|
||||
def _changed_files_for_source_filter(
|
||||
ctx,
|
||||
*,
|
||||
force_all: bool,
|
||||
e2e_only: bool,
|
||||
) -> list[str] | None:
|
||||
"""Return the diff to filter against, or None to keep every step.
|
||||
|
||||
``--all`` / ``--e2e`` disable filtering (scheduled NIGHTLY/WEEKLY full
|
||||
uploads and weekly E2E sweeps). Post-merge ``main`` L3 and PR-label
|
||||
uploads filter against the commit/PR diff. Pipeline YAML /
|
||||
``source_filter_fallback`` matches are applied later in
|
||||
``_render_test_pipeline`` only when no job-key prefix already matched.
|
||||
"""
|
||||
if force_all or e2e_only:
|
||||
return None
|
||||
return ctx.changed_files
|
||||
|
||||
|
||||
def _render_pipeline(
|
||||
path: Path,
|
||||
*,
|
||||
force_all: bool = False,
|
||||
e2e_only: bool = False,
|
||||
) -> str:
|
||||
if path.name == BOOTSTRAP_STEPS_FILENAME:
|
||||
ctx = resolve_ci_context_from_git()
|
||||
continuation = _load_bootstrap_steps(path)
|
||||
return _render_bootstrap_pipeline(
|
||||
continuation,
|
||||
decision=ctx.decision,
|
||||
path=path,
|
||||
)
|
||||
|
||||
text = path.read_text(encoding="utf-8")
|
||||
ctx = resolve_ci_context_from_git()
|
||||
changed_files = _changed_files_for_source_filter(ctx, force_all=force_all, e2e_only=e2e_only)
|
||||
|
||||
doc = yaml.safe_load(text)
|
||||
if not isinstance(doc, dict):
|
||||
raise ValueError(f"invalid pipeline YAML: {path}")
|
||||
|
||||
doc = _render_test_pipeline(
|
||||
doc,
|
||||
changed_files,
|
||||
pipeline_path=path,
|
||||
e2e_only=e2e_only,
|
||||
)
|
||||
return yaml.safe_dump(doc, sort_keys=False)
|
||||
|
||||
|
||||
def _upload_to_buildkite(content: str) -> None:
|
||||
subprocess.run(
|
||||
["buildkite-agent", "pipeline", "upload"],
|
||||
input=content,
|
||||
text=True,
|
||||
check=True,
|
||||
)
|
||||
|
||||
|
||||
# --- CLI ---
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument(
|
||||
"pipeline",
|
||||
nargs="?",
|
||||
default=".buildkite/cuda/bootstrap-upload-steps.yml",
|
||||
help="Pipeline YAML path (default: .buildkite/cuda/bootstrap-upload-steps.yml)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--upload",
|
||||
action="store_true",
|
||||
help="Pipe rendered YAML to buildkite-agent pipeline upload",
|
||||
)
|
||||
mode = parser.add_mutually_exclusive_group()
|
||||
mode.add_argument(
|
||||
"--all",
|
||||
action="store_true",
|
||||
help="Keep all steps (disable diff-aware skipping)",
|
||||
)
|
||||
mode.add_argument(
|
||||
"--e2e",
|
||||
action="store_true",
|
||||
help="Keep only the E2E Test group",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
path = Path(args.pipeline)
|
||||
if not path.is_absolute():
|
||||
path = ROOT / path
|
||||
if not path.is_file():
|
||||
_log(f"missing pipeline file: {path}")
|
||||
return 1
|
||||
|
||||
rendered = _render_pipeline(path, force_all=args.all, e2e_only=args.e2e)
|
||||
if args.upload:
|
||||
_upload_to_buildkite(rendered)
|
||||
else:
|
||||
sys.stdout.write(rendered)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -1,66 +0,0 @@
|
||||
# Uploaded by upload_pipeline.py --upload (not the Buildkite hook).
|
||||
# ``if`` is injected by step ``key`` from skip-ci at upload time.
|
||||
steps:
|
||||
- label: ":docker: Build image"
|
||||
key: image-build
|
||||
commands:
|
||||
- "bash .buildkite/cuda/scripts/build-ci-image.sh"
|
||||
agents:
|
||||
queue: "cpu_queue_premerge"
|
||||
|
||||
# L2 Test — PR ready label (diff-aware) or main+WEEKLY=1 (--e2e, full E2E group)
|
||||
- label: "Upload Ready Pipeline"
|
||||
depends_on: image-build
|
||||
key: upload-ready-pipeline
|
||||
commands:
|
||||
- |
|
||||
EXTRA=""
|
||||
if [ "${WEEKLY:-}" = "1" ] && [ "${BUILDKITE_BRANCH}" = "main" ]; then
|
||||
EXTRA="--e2e"
|
||||
fi
|
||||
python3 .buildkite/common/scripts/upload_pipeline.py --upload $$EXTRA .buildkite/cuda/test-ready.yml
|
||||
agents:
|
||||
queue: "cpu_queue_premerge"
|
||||
|
||||
# L3 Test — PR merge-test label or post-merge on main (both diff-aware);
|
||||
# main+WEEKLY=1 uploads the E2E group only (--e2e).
|
||||
- label: "Upload Merge Pipeline"
|
||||
depends_on: image-build
|
||||
key: upload-merge-pipeline
|
||||
commands:
|
||||
- |
|
||||
EXTRA=""
|
||||
if [ "${WEEKLY:-}" = "1" ] && [ "${BUILDKITE_BRANCH}" = "main" ]; then
|
||||
EXTRA="--e2e"
|
||||
fi
|
||||
python3 .buildkite/common/scripts/upload_pipeline.py --upload $$EXTRA .buildkite/cuda/test-merge.yml
|
||||
agents:
|
||||
queue: "cpu_queue_premerge"
|
||||
|
||||
# L4 Test — main+NIGHTLY=1 (full pipeline), or PR nightly-test label (diff-aware)
|
||||
- label: "Upload Nightly Pipeline"
|
||||
depends_on: image-build
|
||||
key: upload-nightly-pipeline
|
||||
commands:
|
||||
- |
|
||||
EXTRA=""
|
||||
if [ "${NIGHTLY:-}" = "1" ] && [ "${BUILDKITE_BRANCH}" = "main" ]; then
|
||||
EXTRA="--all"
|
||||
fi
|
||||
python3 .buildkite/common/scripts/upload_pipeline.py --upload $$EXTRA .buildkite/cuda/test-nightly.yml
|
||||
agents:
|
||||
queue: "cpu_queue_premerge"
|
||||
|
||||
# L5 Test — main+WEEKLY=1 / NON_CRITICAL=1 (full pipeline), or PR weekly-test label (diff-aware)
|
||||
- label: "Upload Weekly Pipeline"
|
||||
depends_on: image-build
|
||||
key: upload-weekly-pipeline
|
||||
commands:
|
||||
- |
|
||||
EXTRA=""
|
||||
if [ "${BUILDKITE_BRANCH}" = "main" ]; then
|
||||
EXTRA="--all"
|
||||
fi
|
||||
python3 .buildkite/common/scripts/upload_pipeline.py --upload $$EXTRA .buildkite/cuda/test-weekly.yml
|
||||
agents:
|
||||
queue: "cpu_queue_premerge"
|
||||
@@ -1,9 +0,0 @@
|
||||
# Hook uploads this entry YAML; upload_pipeline.py --upload reads bootstrap-upload-steps.yml
|
||||
# and injects ``if`` by step ``key`` from skip-ci.
|
||||
steps:
|
||||
- label: ":github: Resolve skip-ci & upload pipeline"
|
||||
key: upload-ci-pipeline
|
||||
commands:
|
||||
- "python3 .buildkite/common/scripts/upload_pipeline.py --upload .buildkite/cuda/bootstrap-upload-steps.yml"
|
||||
agents:
|
||||
queue: "cpu_queue_premerge"
|
||||
@@ -1,52 +0,0 @@
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
|
||||
# Build and push the CI Docker image with registry-based layer caching.
|
||||
#
|
||||
# Usage: build-ci-image.sh
|
||||
#
|
||||
# Expects the following environment variables (set by Buildkite):
|
||||
# BUILDKITE_COMMIT - commit SHA used as the image tag
|
||||
ECR_NAMESPACE="public.ecr.aws/q9t5s3a7"
|
||||
REGISTRY="${ECR_NAMESPACE}/vllm-ci-test-repo"
|
||||
REGION="us-east-1"
|
||||
DOCKERFILE="docker/Dockerfile.ci"
|
||||
BUILDER_NAME="vllm-omni-builder"
|
||||
|
||||
# Ensure that the env vars are actually set, otherwise exit early
|
||||
if [ -z "${BUILDKITE_COMMIT:-}" ]; then
|
||||
echo "ERROR: BUILDKITE_COMMIT is not set"
|
||||
exit 1
|
||||
fi
|
||||
echo "BUILDKITE_COMMIT: ${BUILDKITE_COMMIT}"
|
||||
|
||||
# Authenticate to ECR Public
|
||||
aws ecr-public get-login-password --region "$REGION" \
|
||||
| docker login --username AWS --password-stdin "$ECR_NAMESPACE"
|
||||
|
||||
# Compute the cache tag for the dependencies image as the hash of the contents.
|
||||
# This is because dependencies don't change often, so most PRs will have the same hash.
|
||||
#
|
||||
# The Dockerfile is also included to avoid churn in the cached image based on the vLLM version.
|
||||
DEP_FILES="pyproject.toml setup.py requirements/*.txt docker/Dockerfile.ci"
|
||||
CACHE_KEY=$(cat ${DEP_FILES} | sha256sum | cut -c1-16)
|
||||
CACHE_TAG="deps-cache-${CACHE_KEY}"
|
||||
echo "Cache key: ${CACHE_TAG}"
|
||||
|
||||
# Set up buildx with docker-container driver; we need to do this
|
||||
# since cache export is not supported for the default docker driver
|
||||
# that is running in the CI.
|
||||
docker buildx inspect $BUILDER_NAME >/dev/null 2>&1 \
|
||||
|| docker buildx create --name $BUILDER_NAME --driver docker-container
|
||||
docker buildx use $BUILDER_NAME
|
||||
|
||||
echo "Building image tag: ${REGISTRY}:${BUILDKITE_COMMIT}"
|
||||
|
||||
# Build + push the image to the registry, using the hash from the dependencies.
|
||||
# Note that the caching here is layerwise, so we're just making sure we always
|
||||
# can cache hit on everything but the last copy at the moment.
|
||||
docker buildx build --push --progress=plain \
|
||||
--cache-from "type=registry,ref=${REGISTRY}:${CACHE_TAG}" \
|
||||
--cache-to "type=registry,ref=${REGISTRY}:${CACHE_TAG},mode=max,compression=zstd" \
|
||||
--file "$DOCKERFILE" \
|
||||
-t "${REGISTRY}:${BUILDKITE_COMMIT}" .
|
||||
@@ -1,38 +0,0 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
|
||||
"""Verify the pinned vLLM and compiled CUDA imports on a GPU CI worker."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def main() -> None:
|
||||
dockerfile = Path(__file__).resolve().parents[3] / "docker" / "Dockerfile.ci"
|
||||
match = re.search(
|
||||
r"^ARG VLLM_VERSION=([0-9]+\.[0-9]+\.[0-9]+)$",
|
||||
dockerfile.read_text(),
|
||||
re.MULTILINE,
|
||||
)
|
||||
if match is None:
|
||||
raise RuntimeError(f"Missing vLLM release version in {dockerfile}")
|
||||
expected_version = match.group(1)
|
||||
|
||||
import torch
|
||||
import vllm
|
||||
|
||||
assert vllm.__version__ == expected_version, vllm.__version__
|
||||
assert torch.cuda.is_available(), "vLLM runtime check requires a GPU worker"
|
||||
for module in ("flashinfer", "vllm._C_stable_libtorch"):
|
||||
importlib.import_module(module)
|
||||
print(
|
||||
f"Verified vLLM CUDA runtime: vllm={vllm.__version__} "
|
||||
f"expected={expected_version} torch={torch.__version__} "
|
||||
f"cuda={torch.version.cuda} device={torch.cuda.get_device_name()}"
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,265 +0,0 @@
|
||||
# CUDA CI L3 (`advanced_model` e2e) plus L1 CPU Simple Test (`core_model and cpu`).
|
||||
# Uploaded by Upload Merge Pipeline: PR label `merge-test`, or post-merge on main
|
||||
# (both diff-aware via source_file_dependencies).
|
||||
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
HF_HUB_DOWNLOAD_TIMEOUT: 300
|
||||
HF_HUB_ETAG_TIMEOUT: 60
|
||||
|
||||
steps:
|
||||
- group: ":card_index_dividers: Simple Test"
|
||||
depends_on: upload-merge-pipeline
|
||||
steps:
|
||||
- label: "Simple · Model Executor Test"
|
||||
timeout_in_minutes: 20
|
||||
commands:
|
||||
- "pytest -sv tests/model_executor -m 'core_model and cpu'"
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- label: "Simple · Diffusion Test"
|
||||
timeout_in_minutes: 30
|
||||
commands:
|
||||
- "pytest -sv tests/diffusion -m 'core_model and cpu'"
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- label: "Simple · Engine&Entrypoints Test"
|
||||
timeout_in_minutes: 20
|
||||
commands:
|
||||
- "pytest -sv tests/entrypoints tests/engine -m 'core_model and cpu'"
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- label: "Simple · Other Test"
|
||||
timeout_in_minutes: 20
|
||||
commands:
|
||||
- "pytest -sv tests/ -m 'core_model and cpu' --ignore=tests/diffusion --ignore=tests/model_executor --ignore=tests/entrypoints --ignore=tests/engine"
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- group: ":card_index_dividers: Diffusion Test"
|
||||
depends_on: upload-merge-pipeline
|
||||
steps:
|
||||
- label: "Diffusion · AR Paged Attention GPU Test"
|
||||
timeout_in_minutes: 20
|
||||
commands:
|
||||
- export VLLM_OMNI_AR_FA_REQUIRED=1
|
||||
- >-
|
||||
timeout 15m pytest -sv
|
||||
tests/diffusion/ar_diffusion/test_paged_attention.py
|
||||
-m 'core_model and cuda and L4 and cards_1'
|
||||
--run-level "core_model"
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- label: "Diffusion · LoRA Test"
|
||||
timeout_in_minutes: 30
|
||||
commands:
|
||||
- pytest -sv tests/diffusion/lora/ -m 'advanced_model and cuda' --run-level "advanced_model"
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- label: "Distributed Test"
|
||||
depends_on: upload-merge-pipeline
|
||||
timeout_in_minutes: 60
|
||||
commands:
|
||||
- |
|
||||
timeout 55m bash -c "
|
||||
set -e
|
||||
export VLLM_IMAGE_FETCH_TIMEOUT=60
|
||||
pytest -s -v tests/distributed -m 'advanced_model and cuda and H100' --run-level 'advanced_model'
|
||||
"
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- group: ":card_index_dividers: Entrypoints Test"
|
||||
depends_on: upload-merge-pipeline
|
||||
steps:
|
||||
- label: "Entrypoint Test with H100"
|
||||
timeout_in_minutes: 60
|
||||
commands:
|
||||
# Turn-based server VAD over /v1/realtime is deferred to RFC #7181 PR 5
|
||||
# (see the P0 thread on #7413); the test stays in the tree and runs again there.
|
||||
- pytest -s -v tests/entrypoints/ -m "advanced_model and cuda and H100" --run-level "advanced_model" --deselect tests/entrypoints/openai_api/test_qwen3_omni_realtime_websocket.py::TestQwen3OmniRealtimeWebSocket::test_server_vad_multi_turn_without_client_commit
|
||||
mirror_hardwares: h100_2
|
||||
|
||||
- label: "Entrypoint Test with L4"
|
||||
timeout_in_minutes: 60
|
||||
# Workaround https://github.com/vllm-project/vllm-omni/issues/8016: shutdown()
|
||||
# leaves inline UniProc DiffusionWorker VRAM mapped in the pytest parent.
|
||||
# This job does not collect the ready leftover
|
||||
# (test_diffusion_generate_request_id, core_model); isolate the
|
||||
# advanced_model DiT handshake instead. Revert to a single sweep after
|
||||
# the product shutdown fix.
|
||||
commands:
|
||||
- |
|
||||
set -eu
|
||||
pytest -s -v tests/entrypoints/test_omni_sleep_mode.py::TestOmniDiffusionSleepMode::test_diffusion_sleep_handshake -m "advanced_model and cuda and L4 and cards_1" --run-level "advanced_model"
|
||||
pytest -s -v tests/entrypoints/ -m "advanced_model and cuda and L4 and cards_1" --run-level "advanced_model" --deselect tests/entrypoints/test_omni_sleep_mode.py::TestOmniDiffusionSleepMode::test_diffusion_sleep_handshake
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- label: "Entrypoint Test with L4 · 2-GPU"
|
||||
timeout_in_minutes: 60
|
||||
commands:
|
||||
- pytest -s -v tests/entrypoints/ -m "advanced_model and cuda and L4 and cards_2" --run-level "advanced_model"
|
||||
mirror_hardwares: l4_2
|
||||
|
||||
- group: ":card_index_dividers: E2E Test"
|
||||
depends_on: upload-merge-pipeline
|
||||
steps:
|
||||
- label: "TTS · Qwen3-TTS CustomVoice Test"
|
||||
source_file_dependencies: tts_qwen3_tts_function
|
||||
commands:
|
||||
- |
|
||||
timeout 40m bash -c "
|
||||
export VLLM_LOGGING_LEVEL=DEBUG
|
||||
export VLLM_ALLOW_LONG_MAX_MODEL_LEN=1
|
||||
pytest -s -v tests/e2e/online_serving/test_qwen3_tts_customvoice.py tests/e2e/offline_inference/test_qwen3_tts_customvoice.py tests/e2e/features/prefill_phase/test_chunked_prefill_tail.py -m 'advanced_model and cuda' --run-level 'advanced_model'
|
||||
"
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- label: "TTS · Qwen3-TTS Base Test"
|
||||
source_file_dependencies: tts_qwen3_tts_cov
|
||||
# 40 -> 60: the coverage split runs the model twice.
|
||||
timeout_in_minutes: 60
|
||||
commands:
|
||||
- |
|
||||
export VLLM_LOGGING_LEVEL=DEBUG
|
||||
export VLLM_ALLOW_LONG_MAX_MODEL_LEN=1
|
||||
.buildkite/common/scripts/run_cov_split.sh \
|
||||
--model-id qwen3_tts \
|
||||
--offline tests/e2e/offline_inference/test_qwen3_tts_base.py \
|
||||
--online tests/e2e/online_serving/test_qwen3_tts_base.py \
|
||||
-- -m 'advanced_model and cuda' --run-level 'advanced_model'
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- label: "Omni · Qwen3-Omni Test"
|
||||
source_file_dependencies: omni_qwen3_omni_function
|
||||
timeout_in_minutes: 50
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/offline_inference/test_qwen3_omni.py tests/e2e/online_serving/test_qwen3_omni.py tests/e2e/offline_inference/test_qwen3_omni_colocate_async.py -m 'advanced_model and cuda' --run-level 'advanced_model'
|
||||
mirror_hardwares: h100_2
|
||||
|
||||
- label: "Omni · Qwen3-Omni Async-Chunk SHM Leak"
|
||||
source_file_dependencies:
|
||||
- tests/e2e/online_serving/test_qwen3_omni_shm_leak.py
|
||||
- tests/helpers/runtime.py
|
||||
- tests/helpers/stage_config.py
|
||||
- vllm_omni/core/sched/omni_ar_scheduler.py
|
||||
- vllm_omni/distributed/omni_connectors/connectors/shm_connector.py
|
||||
- vllm_omni/distributed/omni_connectors/transfer_adapter/chunk_transfer_adapter.py
|
||||
- vllm_omni/engine/orchestrator.py
|
||||
- vllm_omni/engine/stage_engine_core_proc.py
|
||||
- vllm_omni/engine/stage_pool.py
|
||||
- vllm_omni/entrypoints/openai/serving_chat.py
|
||||
- vllm_omni/model_executor/models/qwen3_omni/
|
||||
- vllm_omni/model_executor/models/common/qwen3_code_predictor.py
|
||||
- vllm_omni/model_executor/models/common/snake_activation.py
|
||||
- vllm_omni/model_executor/models/qwen2_5_omni/qwen2_5_omni_thinker.py
|
||||
- vllm_omni/model_executor/models/qwen3_tts/cuda_graph_decoder_wrapper.py
|
||||
- vllm_omni/model_executor/stage_input_processors/qwen3_omni.py
|
||||
- vllm_omni/deploy/qwen3_omni_moe.yaml
|
||||
timeout_in_minutes: 40
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/online_serving/test_qwen3_omni_shm_leak.py -m 'advanced_model and cuda' --run-level 'advanced_model'
|
||||
mirror_hardwares: h100_2
|
||||
|
||||
- label: "Omni · MiniCPM-o 4.5 Test"
|
||||
source_file_dependencies: omni_minicpmo_4_5_function
|
||||
timeout_in_minutes: 50
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/offline_inference/test_minicpmo_4_5.py tests/e2e/online_serving/test_minicpmo_4_5.py -m 'advanced_model and cuda' --run-level 'advanced_model'
|
||||
# h100_2, not h100_1: advanced_model asserts transcribe every audio
|
||||
# response with Whisper, and tests/helpers/media.py requires at least
|
||||
# 16 GiB free VRAM on a device to run Whisper on GPU. When the model
|
||||
# server leaves insufficient memory, Whisper falls back to CPU:
|
||||
# ~8-10 min per clip vs ~30 s, which overruns the 50-minute job timeout
|
||||
# (build 2956). h100_2 guarantees a second GPU with ample free VRAM.
|
||||
mirror_hardwares: h100_2
|
||||
|
||||
- label: "Omni · MiniCPM-o 4.5 Duplex Test"
|
||||
source_file_dependencies: omni_minicpmo_4_5_duplex_function
|
||||
timeout_in_minutes: 50
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/online_serving/test_minicpmo_4_5_duplex.py tests/e2e/online_serving/test_minicpmo_4_5_window.py -m 'advanced_model and cuda' --run-level 'advanced_model'
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "Diffusion · Qwen Image Test"
|
||||
source_file_dependencies: diffusion_qwen_image_function
|
||||
commands:
|
||||
- |
|
||||
timeout 40m bash -c '
|
||||
export VLLM_IMAGE_FETCH_TIMEOUT=60
|
||||
pytest -s -v tests/e2e/online_serving/test_qwen_image.py -m 'advanced_model and cuda' --run-level 'advanced_model'
|
||||
'
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "Diffusion · Kandinsky6 Test"
|
||||
source_file_dependencies: diffusion_kandinsky6_function
|
||||
commands:
|
||||
- |
|
||||
timeout 40m bash -c "
|
||||
pytest -s -v tests/e2e/offline_inference/test_kandinsky6.py tests/e2e/online_serving/test_kandinsky6.py -m 'advanced_model and cuda and diffusion' --run-level 'advanced_model'
|
||||
"
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "Diffusion · Wan22 Test"
|
||||
source_file_dependencies: diffusion_wan22_function
|
||||
commands:
|
||||
- |
|
||||
timeout 40m bash -c "
|
||||
export VLLM_IMAGE_FETCH_TIMEOUT=60
|
||||
pytest -s -v tests/e2e/offline_inference/test_wan22_t2v.py tests/e2e/online_serving/test_wan22_t2v.py -m 'advanced_model and cuda and diffusion' --run-level 'advanced_model'
|
||||
"
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "Diffusion · HunyuanVideo-1.5 Test"
|
||||
source_file_dependencies: diffusion_hunyuan_video_function
|
||||
commands:
|
||||
- |
|
||||
timeout 40m bash -c "
|
||||
export VLLM_IMAGE_FETCH_TIMEOUT=60
|
||||
pytest -s -v tests/e2e/online_serving/test_hunyuan_video_15_expansion.py -m 'advanced_model and cuda and diffusion' --run-level 'advanced_model'
|
||||
"
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "Diffusion · LingBot-Video Test"
|
||||
source_file_dependencies: diffusion_lingbot_function
|
||||
commands:
|
||||
- |
|
||||
timeout 40m bash -c "
|
||||
export VLLM_IMAGE_FETCH_TIMEOUT=60
|
||||
pytest -s -v tests/e2e/online_serving/test_lingbot_video.py -m 'advanced_model and cuda and diffusion' --run-level 'advanced_model'
|
||||
"
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "Diffusion · MiniMax-H3 L3 DLO DP2 and FastH3"
|
||||
source_file_dependencies: diffusion_minimax_h3_function
|
||||
# Keep four server lifecycles in one Buildkite job, but never in one
|
||||
# pytest process. DLO uses multiprocessing shared-memory queues; the
|
||||
# parent's resource_tracker can retain a stale /psm_* entry during
|
||||
# teardown and strand the next Ref2VA collective.
|
||||
# A per-file process plus a hard timeout also makes a regression fail
|
||||
# instead of leaving the merge queue stuck indefinitely.
|
||||
timeout_in_minutes: 150
|
||||
commands:
|
||||
- |
|
||||
set -eu
|
||||
export VLLM_IMAGE_FETCH_TIMEOUT=60
|
||||
if [ -f vllm_omni/diffusion/models/minimax_h3/lora.py ]; then
|
||||
export VLLM_TEST_MINIMAX_H3_TURBO_LORA=$(python3 -c 'from huggingface_hub import hf_hub_download; print(hf_hub_download(repo_id="lightx2v/Minimax-h3-Turbo", filename="minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors"))')
|
||||
fi
|
||||
if [ -f vllm_omni/diffusion/models/minimax_h3/fasth3.py ]; then
|
||||
export VLLM_TEST_MINIMAX_H3_FASTH3_LORA=$(python3 -c 'from huggingface_hub import hf_hub_download; print(hf_hub_download(repo_id="FastVideo/FastVideo-FastH3-4-step-Preview-v1-LoRA", filename="dense-datafree/adapter_model.safetensors"))')
|
||||
fi
|
||||
timeout 35m pytest -s -v tests/e2e/online_serving/minimax_h3/test_minimax_h3_dlo_dp2_fl2va.py -m 'advanced_model and cuda and diffusion' --run-level 'advanced_model'
|
||||
timeout 35m pytest -s -v tests/e2e/online_serving/minimax_h3/test_minimax_h3_dlo_dp2_ref2va.py -m 'advanced_model and cuda and diffusion' --run-level 'advanced_model'
|
||||
timeout 35m pytest -s -v tests/e2e/online_serving/minimax_h3/test_minimax_h3_turbo_lora.py -m 'advanced_model and cuda and diffusion' --run-level 'advanced_model'
|
||||
timeout 35m pytest -s -v tests/e2e/online_serving/minimax_h3/test_minimax_h3_fasth3_lora.py -m 'advanced_model and cuda and diffusion' --run-level 'advanced_model'
|
||||
mirror_hardwares: h100_2
|
||||
|
||||
- label: "Diffusion · MiniMax-H3 Latent Upscale and Refine"
|
||||
source_file_dependencies: diffusion_minimax_h3_function
|
||||
timeout_in_minutes: 40
|
||||
commands:
|
||||
- |
|
||||
set -eu
|
||||
VLLM_TEST_MINIMAX_H3_LATENT_UPSCALER=$(python3 -c 'from huggingface_hub import hf_hub_download; print(hf_hub_download(repo_id="LBH-123-AI/Minimax_h3_latent_Upscaler", filename="minimax_h3_latent_upscaler_3d_conv_v1/minimax_h3_latent_upscaler_3d_conv_v1_bf16.safetensors"))')
|
||||
export VLLM_TEST_MINIMAX_H3_LATENT_UPSCALER
|
||||
timeout 35m pytest -s -v tests/e2e/online_serving/minimax_h3/test_minimax_h3.py -m 'advanced_model and cuda and diffusion' --run-level 'advanced_model'
|
||||
mirror_hardwares: h100_2
|
||||
@@ -1,560 +0,0 @@
|
||||
# CUDA CI L4 (`full_model`): nightly and release share this file.
|
||||
# Uploaded by Upload Nightly Pipeline: NIGHTLY=1, or PR labels (nightly-test / omni-test / ...).
|
||||
# Hardware: omit mirror_hardwares to compose from -m SKU + cards_n; or set a preset string to force.
|
||||
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
HF_HUB_DOWNLOAD_TIMEOUT: 300
|
||||
HF_HUB_ETAG_TIMEOUT: 60
|
||||
|
||||
steps:
|
||||
# Group: collapses under one heading in the Buildkite UI; child steps still run in parallel.
|
||||
- group: ":card_index_dividers: Omni Model Test"
|
||||
key: nightly-omni-test-group
|
||||
depends_on: upload-nightly-pipeline
|
||||
if: build.env("NIGHTLY") == "1" || build.pull_request.labels includes "nightly-test"
|
||||
steps:
|
||||
|
||||
- label: ":full_moon: Omni · Function Test with H100 · Single-GPU"
|
||||
source_file_dependencies:
|
||||
- omni_qwen3_omni_function
|
||||
- omni_minicpmo_4_5_function
|
||||
timeout_in_minutes: 120
|
||||
commands:
|
||||
- pytest -sv tests/e2e --run-level "full_model" -m "full_model and H100 and B200 and omni and cards_1" --ignore=tests/e2e/accuracy --ignore=tests/e2e/online_serving/test_minicpmo_4_5_duplex_expansion.py --ignore=tests/e2e/online_serving/test_minicpmo_4_5_window.py
|
||||
|
||||
- label: ":full_moon: Omni · Function Test with H100 · 2-GPU"
|
||||
source_file_dependencies:
|
||||
- omni_qwen3_omni_function
|
||||
- omni_minicpmo_4_5_function
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- pytest -sv tests/e2e --run-level "full_model" -m "full_model and H100 and B200 and omni and cards_2" --ignore=tests/e2e/accuracy --ignore=tests/e2e/online_serving/test_qwen3_omni_multi_replicas.py
|
||||
|
||||
- label: ":full_moon: Omni · MiniCPM-o 4.5 Duplex Test"
|
||||
source_file_dependencies: omni_minicpmo_4_5_duplex_function
|
||||
timeout_in_minutes: 120
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/online_serving/test_minicpmo_4_5_duplex_expansion.py -m "full_model and cuda and H100 and B200 and omni and cards_1" --run-level "full_model"
|
||||
- pytest -s -v tests/e2e/online_serving/test_minicpmo_4_5_window.py -m "full_model and cuda and H100 and B200 and omni and cards_1" --run-level "full_model"
|
||||
|
||||
- label: ":full_moon: Omni · Doc Test with L4 · 4-GPU"
|
||||
source_file_dependencies: omni_qwen2_5_omni_doc
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
|
||||
- pytest -s -v tests/examples/ -m "full_model and omni and L4 and B200 and cards_4" --run-level "full_model"
|
||||
|
||||
- label: ":full_moon: Omni · Doc Test with H100 · 2-GPU"
|
||||
source_file_dependencies: omni_qwen3_omni_doc
|
||||
timeout_in_minutes: 90
|
||||
artifact_paths:
|
||||
- audio_*.wav
|
||||
commands:
|
||||
- pytest -s -v tests/examples/ -m "full_model and omni and H100 and B200 and cards_2" --run-level "full_model"
|
||||
|
||||
- label: ":full_moon: Omni · Accuracy Test"
|
||||
source_file_dependencies: omni_qwen3_omni_accuracy
|
||||
timeout_in_minutes: 180
|
||||
artifact_paths:
|
||||
- tests/e2e/accuracy/qwen3_omni/results/qwen_omni_acc/*.json
|
||||
commands:
|
||||
- export SEED_TTS_WER_EVAL=1
|
||||
- export SEED_TTS_EVAL_DEVICE=cuda:1
|
||||
- pytest -s -v tests/e2e/accuracy/qwen3_omni/test_qwen3_omni.py -m "full_model and H100 and B200 and cards_2" --run-level full_model
|
||||
|
||||
- label: ":full_moon: Omni · MiniCPM-o 4.5 · Accuracy Test"
|
||||
source_file_dependencies: omni_minicpmo_4_5_accuracy
|
||||
timeout_in_minutes: 180
|
||||
artifact_paths:
|
||||
- tests/e2e/accuracy/minicpmo_4_5/results/*.json
|
||||
commands:
|
||||
- export SEED_TTS_WER_EVAL=1
|
||||
- export SEED_TTS_EVAL_DEVICE=cuda:1
|
||||
- pytest -s -v tests/e2e/accuracy/minicpmo_4_5/test_minicpmo_4_5.py -m "full_model and H100 and B200 and cuda and (cards_1 or cards_2)" --run-level full_model
|
||||
|
||||
- label: ":full_moon: Omni · Omni-DuplexEval CI Guard"
|
||||
source_file_dependencies: omni_duplex_eval_ci
|
||||
timeout_in_minutes: 240
|
||||
env:
|
||||
VLLM_DUPLEX_EVAL_JUDGE_URL: "http://127.0.0.1:8001"
|
||||
commands:
|
||||
- |
|
||||
set +e
|
||||
pytest -s -v tests/e2e/accuracy/omni_duplex_eval/test_omni_duplex_eval_ci.py \
|
||||
-m "full_model and H100 and B200 and cuda and cards_2" \
|
||||
--run-level full_model \
|
||||
--basetemp /tmp/omni_duplex_ci
|
||||
EXIT=$$?
|
||||
buildkite-agent artifact upload "tests/e2e/accuracy/omni_duplex_eval/results/*.json"
|
||||
exit $$EXIT
|
||||
|
||||
- label: ":full_moon: Omni · Perf Test · No Async Chunk"
|
||||
source_file_dependencies: omni_qwen3_omni_perf
|
||||
key: nightly-omni-performance-no-async-chunk
|
||||
timeout_in_minutes: 300
|
||||
artifact_paths:
|
||||
- tests/dfx/perf/results/*.json
|
||||
commands:
|
||||
- export BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_qwen3_omni_no_async_chunk.json -m "H100 and B200 and cards_2"
|
||||
|
||||
- label: ":full_moon: Omni · Perf Test · Async Chunk"
|
||||
source_file_dependencies: omni_qwen3_omni_perf
|
||||
key: nightly-omni-performance-async-chunk
|
||||
timeout_in_minutes: 180
|
||||
artifact_paths:
|
||||
- tests/dfx/perf/results/*.json
|
||||
commands:
|
||||
- export BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_qwen3_omni_async_chunk.json -m "H100 and B200 and full_model and cards_2"
|
||||
|
||||
- label: ":full_moon: Omni · MiniCPM-o 4.5 · Perf Test"
|
||||
source_file_dependencies: omni_minicpmo_4_5_perf
|
||||
key: nightly-omni-performance-minicpmo-4-5
|
||||
timeout_in_minutes: 180
|
||||
artifact_paths:
|
||||
- tests/dfx/perf/results/*.json
|
||||
commands:
|
||||
- export BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_minicpmo_4_5.json -m "H100 and B200 and cards_1"
|
||||
|
||||
- label: ":full_moon: Omni · MiniCPM-o 4.5 · Duplex Seed-TTS Perf Test"
|
||||
source_file_dependencies: omni_minicpmo_4_5_duplex_perf
|
||||
key: nightly-omni-performance-minicpmo-4-5-duplex-seed-tts
|
||||
timeout_in_minutes: 180
|
||||
artifact_paths:
|
||||
- tests/dfx/perf/results/*.json
|
||||
commands:
|
||||
- export BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_minicpmo_4_5_duplex_seed_tts.json -m "H100 and B200 and cards_1"
|
||||
|
||||
- label: ":full_moon: Omni · Multi-Replica Startup Test with 4x H100"
|
||||
source_file_dependencies: omni_qwen3_omni_function
|
||||
timeout_in_minutes: 45
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/online_serving/test_qwen3_omni_multi_replicas.py -m "full_model and H100 and B200 and cards_4" --run-level "core_model"
|
||||
|
||||
- group: ":card_index_dividers: Entrypoints Test"
|
||||
key: nightly-entrypoints-test-group
|
||||
depends_on: upload-nightly-pipeline
|
||||
if: build.env("NIGHTLY") == "1" || build.pull_request.labels includes "nightly-test"
|
||||
steps:
|
||||
|
||||
- label: ":full_moon: Entrypoint Test with H100"
|
||||
source_file_dependencies: entrypoints_function
|
||||
# Merge collects advanced_model (5 duplex interruptions). Nightly
|
||||
# collects full_model (20 interruptions + BAGEL sleep).
|
||||
timeout_in_minutes: 180
|
||||
commands:
|
||||
- pytest -s -v tests/entrypoints/ -m "full_model and cuda and H100 and B200 and cards_2" --run-level "full_model"
|
||||
|
||||
- group: ":card_index_dividers: TTS Model Test"
|
||||
key: nightly-tts-test-group
|
||||
depends_on: upload-nightly-pipeline
|
||||
if: build.env("NIGHTLY") == "1" || build.pull_request.labels includes "nightly-test"
|
||||
steps:
|
||||
|
||||
- label: ":full_moon: TTS · Function Test with L4"
|
||||
source_file_dependencies: tts_qwen3_tts_function
|
||||
timeout_in_minutes: 120
|
||||
env:
|
||||
VLLM_OMNI_FAILED_SPEECH_AUDIO_DIR: .
|
||||
artifact_paths:
|
||||
- failed_speech_*.wav
|
||||
- failed_speech_*.json
|
||||
commands:
|
||||
- export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
|
||||
- export VLLM_USE_DEEP_GEMM="0"
|
||||
- export VLLM_MOE_USE_DEEP_GEMM="0"
|
||||
- pytest -s -v tests/e2e/ -m "full_model and L4 and B200 and tts and cards_1" --run-level "full_model" --ignore=tests/e2e/accuracy
|
||||
|
||||
- label: ":full_moon: TTS · Perf Test · Single-GPU"
|
||||
source_file_dependencies: tts_qwen3_tts_perf
|
||||
key: nightly-tts-performance-single
|
||||
timeout_in_minutes: 180
|
||||
artifact_paths:
|
||||
- tests/dfx/perf/results/*.json
|
||||
commands:
|
||||
- export BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
|
||||
- pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_tts.json -m "H100 and B200 and cards_1"
|
||||
|
||||
- label: ":full_moon: TTS · Perf Test · 2-GPU"
|
||||
source_file_dependencies: tts_qwen3_tts_perf
|
||||
key: nightly-tts-performance-2gpu
|
||||
timeout_in_minutes: 180
|
||||
artifact_paths:
|
||||
- tests/dfx/perf/results/*.json
|
||||
commands:
|
||||
- export BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
|
||||
- pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_tts.json -m "H100 and B200 and cards_2"
|
||||
|
||||
- group: ":card_index_dividers: Tiny Model Tests [Multi-GPU]"
|
||||
key: nightly-tiny-model-test-group
|
||||
depends_on: upload-nightly-pipeline
|
||||
if: >-
|
||||
build.env("NIGHTLY") == "1" ||
|
||||
build.pull_request.labels includes "nightly-test"
|
||||
steps:
|
||||
|
||||
- label: ":full_moon: Tiny Model Tests · Diffusion (Multi-GPU accelerations) · 2-GPU"
|
||||
source_file_dependencies: diffusion_tiny_model
|
||||
timeout_in_minutes: 30
|
||||
# NOTE: Single-GPU (core_model) tiny model tests run on the ready label,
|
||||
# so nightly only adds the multi-GPU parallelism configurations.
|
||||
#
|
||||
# This is a bit confusing, but `full_model` is used to mark the multigpu tests, and
|
||||
# --run-level `core_model` is used to indicate that we should use tiny model weights
|
||||
# for the CI, since these tests should pass irrespective of whether or not we use
|
||||
# the full weights or not.
|
||||
commands:
|
||||
- pytest -sv tests/model_tests/diffusion -m 'full_model and L4 and B200 and cuda and cards_2' --run-level "core_model"
|
||||
|
||||
- label: ":full_moon: Tiny Model Tests · Diffusion (Multi-GPU accelerations) · 4-GPU"
|
||||
source_file_dependencies: diffusion_tiny_model
|
||||
timeout_in_minutes: 30
|
||||
# CFG+TP extra_test_groups request 2×2=4 devices (see get_required_device_count).
|
||||
commands:
|
||||
- pytest -sv tests/model_tests/diffusion -m 'full_model and L4 and B200 and cuda and cards_4' --run-level "core_model"
|
||||
|
||||
- group: ":card_index_dividers: Diffusion X2I(&A&T) Model Test"
|
||||
key: nightly-diffusion-x2iat-group
|
||||
depends_on: upload-nightly-pipeline
|
||||
if: >-
|
||||
build.env("NIGHTLY") == "1" ||
|
||||
build.pull_request.labels includes "nightly-test"
|
||||
steps:
|
||||
|
||||
- label: ":full_moon: Diffusion X2I(&A&T) · Function Test with H100 · Single-GPU"
|
||||
source_file_dependencies: diffusion_qwen_image_function
|
||||
timeout_in_minutes: 120
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/online_serving/test_qwen_image_expansion.py
|
||||
-m "full_model and diffusion and H100 and B200 and cards_1"
|
||||
--run-level "full_model"
|
||||
|
||||
- label: ":full_moon: Diffusion X2I(&A&T) · Function Test with H100 · 2-GPU"
|
||||
source_file_dependencies: diffusion_qwen_image_function
|
||||
timeout_in_minutes: 120
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/online_serving/test_qwen_image_expansion.py
|
||||
-m "full_model and diffusion and H100 and B200 and cards_2"
|
||||
--run-level "full_model"
|
||||
|
||||
- label: ":full_moon: Diffusion X2I(&A&T) · Function Test with L4 · Single-GPU"
|
||||
source_file_dependencies: diffusion_qwen_image_function
|
||||
timeout_in_minutes: 120
|
||||
commands:
|
||||
- pytest -sv tests/e2e/offline_inference/test_qwen_image_autoround_w4a16_expansion.py -m "full_model and diffusion and L4 and B200 and cards_1" --run-level "full_model"
|
||||
|
||||
|
||||
- label: ":full_moon: Diffusion X2I(&A&T) · Doc Test"
|
||||
source_file_dependencies: diffusion_text_to_image_doc
|
||||
timeout_in_minutes: 60
|
||||
commands:
|
||||
- pytest -s -v tests/examples/*/test_text_to_image.py -m "full_model and example and H100 and B200 and (cards_1 or cards_2)" --run-level "full_model"
|
||||
|
||||
- label: ":full_moon: Diffusion X2I(&A&T) · Accuracy Test"
|
||||
source_file_dependencies: diffusion_qwen_image_accuracy
|
||||
timeout_in_minutes: 180
|
||||
commands:
|
||||
- export VLLM_HTTP_TIMEOUT_KEEP_ALIVE=120
|
||||
- pytest -s -v tests/e2e/accuracy/test_qwen_image.py -m "H100 and B200 and cards_1" --run-level full_model
|
||||
- pytest -s -v tests/e2e/accuracy/test_diffusers_backend_similarity.py -k '2i_matches_diffusers' -m "full_model and H100 and B200 and cards_1" --run-level full_model
|
||||
|
||||
- label: ":full_moon: JoyAI-Image-Edit · Accuracy Test"
|
||||
source_file_dependencies: diffusion_joy_image_accuracy
|
||||
timeout_in_minutes: 180
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/accuracy/test_joy_image_edit.py -m "full_model and diffusion and H100 and B200 and cards_1" --run-level full_model
|
||||
|
||||
- label: ":full_moon: HunyuanImage3-DIT · Accuracy Test"
|
||||
source_file_dependencies: diffusion_hunyuan_image3_accuracy
|
||||
timeout_in_minutes: 180
|
||||
commands:
|
||||
- export HUNYUAN_IMAGE3_MODEL="tencent/HunyuanImage-3.0-Instruct"
|
||||
- export HUNYUAN_IMAGE3_DEVICES="0,1,2,3"
|
||||
- pytest -s -v tests/e2e/accuracy/test_hunyuan_image3_pixel_accuracy.py tests/e2e/accuracy/test_hunyuan_image3_lora.py -m "full_model and H100 and B200 and cards_4" --run-level full_model
|
||||
|
||||
- label: ":full_moon: Diffusion X2I(&A&T) · Perf Test · Qwen-Image · Single-GPU"
|
||||
source_file_dependencies: diffusion_qwen_image_perf
|
||||
key: nightly-diffusion-x2iat-performance-qwen-image-single
|
||||
timeout_in_minutes: 180
|
||||
artifact_paths:
|
||||
- tests/dfx/perf/results/diffusion_result_*.json
|
||||
- tests/dfx/perf/results/logs/*.log
|
||||
commands:
|
||||
- export DIFFUSION_BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- export CACHE_DIT_VERSION=1.5.0
|
||||
- pytest -s -v tests/dfx/perf/scripts/run_diffusion_benchmark.py --test-config-file tests/dfx/perf/tests/test_qwen_image_vllm_omni.json -m "H100 and B200 and cards_1"
|
||||
|
||||
- label: ":full_moon: Diffusion X2I(&A&T) · Perf Test · Qwen-Image · 4-GPU"
|
||||
source_file_dependencies: diffusion_qwen_image_perf
|
||||
key: nightly-diffusion-x2iat-performance-qwen-image-4gpu
|
||||
timeout_in_minutes: 180
|
||||
artifact_paths:
|
||||
- tests/dfx/perf/results/diffusion_result_*.json
|
||||
- tests/dfx/perf/results/logs/*.log
|
||||
commands:
|
||||
- export DIFFUSION_BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- export CACHE_DIT_VERSION=1.5.0
|
||||
# Do not pin DIFFUSION_ATTENTION_BACKEND: H100 auto-selects FLASH_ATTN;
|
||||
# B200 rejects explicit FLASH_ATTN without FA4 and uses CUDNN/TRTLLM.
|
||||
- pytest -s -v tests/dfx/perf/scripts/run_diffusion_benchmark.py --test-config-file tests/dfx/perf/tests/test_qwen_image_vllm_omni.json -m "H100 and B200 and cards_4"
|
||||
|
||||
- group: ":card_index_dividers: Diffusion X2V Model Test"
|
||||
key: nightly-diffusion-x2v-group
|
||||
depends_on: upload-nightly-pipeline
|
||||
if: >-
|
||||
build.env("NIGHTLY") == "1" ||
|
||||
build.pull_request.labels includes "nightly-test"
|
||||
steps:
|
||||
|
||||
- label: ":full_moon: Diffusion X2V · Wan2.2 T2V Function Test · Single-GPU"
|
||||
source_file_dependencies: diffusion_wan22_function
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/online_serving/test_wan22_expansion.py -k "t2v" -m "full_model and cuda and H100 and B200 and cards_1" --run-level "full_model"
|
||||
|
||||
- label: ":full_moon: Diffusion X2V · Wan2.2 T2V Function Test · 2-GPU"
|
||||
source_file_dependencies: diffusion_wan22_function
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/online_serving/test_wan22_expansion.py -k "t2v" -m "full_model and cuda and H100 and B200 and cards_2" --run-level "full_model"
|
||||
|
||||
- label: ":full_moon: Diffusion X2V · Wan2.2 I2V/TI2V Function Test · Single-GPU"
|
||||
source_file_dependencies: diffusion_wan22_function
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/online_serving/test_wan22_expansion.py -k "i2v" -m "full_model and cuda and H100 and B200 and cards_1" --run-level "full_model"
|
||||
|
||||
- label: ":full_moon: Diffusion X2V · Wan2.2 I2V/TI2V Function Test · 2-GPU"
|
||||
source_file_dependencies: diffusion_wan22_function
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/online_serving/test_wan22_expansion.py -k "i2v" -m "full_model and cuda and H100 and B200 and cards_2" --run-level "full_model"
|
||||
|
||||
- label: ":full_moon: Diffusion X2V · HunyuanVideo-1.5 Function Test"
|
||||
source_file_dependencies: diffusion_hunyuan_video_function
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/online_serving/test_hunyuan_video_15_expansion.py -m "full_model and H100 and B200 and cuda and (cards_1 or cards_2)" --run-level "full_model"
|
||||
|
||||
- label: ":full_moon: Diffusion X2V · LingBot Function Test"
|
||||
source_file_dependencies: diffusion_lingbot_function
|
||||
timeout_in_minutes: 120
|
||||
commands:
|
||||
- >-
|
||||
pytest -s -v
|
||||
tests/e2e/online_serving/test_lingbot_video_expansion.py
|
||||
tests/e2e/online_serving/test_lingbot_video_moe.py
|
||||
tests/e2e/offline_inference/test_lingbot_world_v2.py
|
||||
-m "full_model and diffusion and H100 and B200 and cards_1"
|
||||
--run-level "full_model"
|
||||
|
||||
- label: ":full_moon: Diffusion X2V · Doc Test"
|
||||
source_file_dependencies: diffusion_image_to_video_doc
|
||||
timeout_in_minutes: 60
|
||||
commands:
|
||||
- pytest -s -v tests/examples/offline_inference/test_image_to_video.py -m "full_model and example and H100 and B200 and cards_1" --run-level "full_model"
|
||||
|
||||
- label: ":full_moon: Diffusion X2V · Wan2.2 Autoround Function Test"
|
||||
source_file_dependencies: diffusion_wan22_function
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/offline_inference/test_wan22_autoround_w4a16_expansion.py -m "full_model and H100 and B200 and cuda and (cards_1 or cards_2)" --run-level "full_model"
|
||||
|
||||
- label: ":full_moon: Diffusion X2V · Wan2.2 Device-Postprocess Equivalence Test"
|
||||
source_file_dependencies: diffusion_wan22_function
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- |
|
||||
VLLM_OMNI_DEVICE_POSTPROCESS_MODEL=Wan-AI/Wan2.2-TI2V-5B-Diffusers \
|
||||
pytest -s -v tests/e2e/offline_inference/test_device_postprocess_equivalence.py -m "full_model and H100 and B200 and cuda and cards_1" --run-level "full_model"
|
||||
|
||||
- label: ":full_moon: Diffusion X2V · Wan2.2 Accuracy Test"
|
||||
source_file_dependencies: diffusion_wan22_accuracy
|
||||
timeout_in_minutes: 180
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/accuracy/wan22_i2v/test_wan22_i2v_video_similarity.py -m "full_model and H100 and B200 and (cards_1 or cards_2)" --run-level full_model
|
||||
- pytest -s -v tests/e2e/accuracy/test_diffusers_backend_similarity.py -k '2v_matches_diffusers' -m "full_model and H100 and B200 and cards_1" --run-level full_model
|
||||
|
||||
- label: ":full_moon: Diffusion X2V · MiniMax H3 I2VA/Ref2VA Accuracy Test"
|
||||
source_file_dependencies: diffusion_minimax_h3_accuracy
|
||||
timeout_in_minutes: 180
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/accuracy/minimax_h3/test_minimax_h3_i2va_ref2va_similarity.py -m "full_model and H100 and B200 and cards_4" --run-level full_model
|
||||
|
||||
- label: ":full_moon: Diffusion X2V · HunyuanVideo-1.5 Accuracy Test"
|
||||
source_file_dependencies: diffusion_hunyuan_video_accuracy
|
||||
timeout_in_minutes: 180
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/accuracy/hunyuanvideo15_t2v/test_hunyuanvideo15_t2v_video_similarity.py -m "full_model and H100 and B200 and (cards_1 or cards_2)" --run-level full_model
|
||||
- pytest -s -v tests/e2e/accuracy/hunyuanvideo15_i2v/test_hunyuanvideo15_i2v_video_similarity.py -m "full_model and H100 and B200 and (cards_1 or cards_2)" --run-level full_model
|
||||
|
||||
- label: ":full_moon: Diffusion X2V · Perf Test · Single-GPU"
|
||||
source_file_dependencies: diffusion_wan22_perf
|
||||
key: nightly-diffusion-x2v-performance-single
|
||||
timeout_in_minutes: 180
|
||||
artifact_paths:
|
||||
- tests/dfx/perf/results/*.json
|
||||
commands:
|
||||
- export BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_wan22_i2v_vllm_omni.json -m "H100 and B200 and cards_1"
|
||||
|
||||
- label: ":full_moon: Diffusion X2V · Perf Test · 2-GPU"
|
||||
source_file_dependencies:
|
||||
- diffusion_wan22_perf
|
||||
- diffusion_cosmos3_perf
|
||||
key: nightly-diffusion-x2v-performance-2gpu
|
||||
timeout_in_minutes: 180
|
||||
artifact_paths:
|
||||
- tests/dfx/perf/results/*.json
|
||||
commands:
|
||||
- export BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- |
|
||||
set +e
|
||||
pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_wan22_i2v_vllm_omni.json -m "H100 and B200 and cards_2"
|
||||
EXIT1=$$?
|
||||
pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_cosmos3_vllm_omni.json -m "H100 and B200 and cards_2"
|
||||
EXIT2=$$?
|
||||
if [ $$EXIT1 -ne 0 ]; then exit $$EXIT1; fi
|
||||
exit $$EXIT2
|
||||
|
||||
- label: ":full_moon: Diffusion X2V · Perf Test · HunyuanVideo-1.5"
|
||||
source_file_dependencies: diffusion_hunyuan_video_perf
|
||||
key: nightly-diffusion-x2v-performance-hunyuanvideo15
|
||||
timeout_in_minutes: 180
|
||||
artifact_paths:
|
||||
- tests/dfx/perf/results/*.json
|
||||
commands:
|
||||
- export BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- |
|
||||
set +e
|
||||
pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_hunyuanvideo15_t2v_vllm_omni.json -m "H100 and B200 and (cards_1 or cards_2)"
|
||||
EXIT1=$$?
|
||||
pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_hunyuanvideo15_i2v_vllm_omni.json -m "H100 and B200 and (cards_1 or cards_2)"
|
||||
EXIT2=$$?
|
||||
if [ $$EXIT1 -ne 0 ]; then exit $$EXIT1; fi
|
||||
exit $$EXIT2
|
||||
|
||||
- label: ":full_moon: Diffusion X2V · Perf Test · MiniMax-H3"
|
||||
source_file_dependencies: diffusion_minimax_h3_perf
|
||||
key: nightly-diffusion-x2v-performance-minimax-h3
|
||||
timeout_in_minutes: 180
|
||||
artifact_paths:
|
||||
- tests/dfx/perf/results/*.json
|
||||
commands:
|
||||
- export BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_minimax_h3_vllm_omni.json -m "H100 and B200 and cards_4"
|
||||
|
||||
- label: ":full_moon: Diffusion X2V · Perf Test · LingBot"
|
||||
source_file_dependencies: diffusion_lingbot_perf
|
||||
key: nightly-diffusion-x2v-performance-lingbot
|
||||
timeout_in_minutes: 180
|
||||
artifact_paths:
|
||||
- tests/dfx/perf/results/*.json
|
||||
commands:
|
||||
- export BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_lingbot_video_vllm_omni.json -m "H100 and B200 and cards_1"
|
||||
|
||||
- group: ":card_index_dividers: Diffusion Test"
|
||||
key: nightly-diffusion-test-group
|
||||
depends_on: upload-nightly-pipeline
|
||||
if: build.env("NIGHTLY") == "1" || build.pull_request.labels includes "nightly-test"
|
||||
steps:
|
||||
- label: ":full_moon: Diffusion · Distributed Test with L4 · 2-GPU"
|
||||
source_file_dependencies: diffusion_distributed_attention
|
||||
timeout_in_minutes: 60
|
||||
commands:
|
||||
- pytest -sv tests/diffusion/distributed/ -m 'full_model and cuda and L4 and B200 and cards_2' --run-level "full_model"
|
||||
|
||||
- label: ":full_moon: Diffusion · Distributed Test with L4 · 3-GPU"
|
||||
source_file_dependencies: diffusion_distributed_attention
|
||||
timeout_in_minutes: 60
|
||||
commands:
|
||||
- pytest -sv tests/diffusion/distributed/ -m 'full_model and cuda and L4 and B200 and cards_3' --run-level "full_model"
|
||||
|
||||
- label: ":full_moon: Diffusion · Distributed Test with L4 · 4-GPU"
|
||||
source_file_dependencies: diffusion_distributed_attention
|
||||
timeout_in_minutes: 60
|
||||
commands:
|
||||
- pytest -sv tests/diffusion/distributed/ -m 'full_model and cuda and L4 and B200 and cards_4' --run-level "full_model"
|
||||
|
||||
- label: ":full_moon: Diffusion · Distributed Test with H100 · 2-GPU"
|
||||
source_file_dependencies: diffusion_distributed_attention
|
||||
timeout_in_minutes: 60
|
||||
commands:
|
||||
- pytest -sv tests/diffusion/distributed/ -m 'full_model and cuda and H100 and B200 and cards_2' --run-level "full_model"
|
||||
|
||||
- label: ":full_moon: Diffusion · Distributed Test with H100 · 4-GPU"
|
||||
source_file_dependencies: diffusion_distributed_attention
|
||||
timeout_in_minutes: 60
|
||||
commands:
|
||||
- pytest -sv tests/diffusion/distributed/ -m 'full_model and cuda and H100 and B200 and cards_4' --run-level "full_model"
|
||||
|
||||
- label: ":full_moon: Diffusion · Offloader Test"
|
||||
source_file_dependencies: diffusion_offloader
|
||||
timeout_in_minutes: 60
|
||||
commands:
|
||||
- pytest -sv tests/diffusion/offloader -m "full_model and L4 and B200 and cuda and cards_1"
|
||||
|
||||
- label: ":full_moon: Diffusion · Quantization Test with H100 · Single-GPU"
|
||||
source_file_dependencies: diffusion_quantization
|
||||
timeout_in_minutes: 60
|
||||
commands:
|
||||
- pytest -sv tests/diffusion/quantization -m 'full_model and cuda and H100 and B200 and cards_1' --run-level "full_model"
|
||||
|
||||
- label: ":full_moon: Diffusion · MiniMax H3 FP8 Quality Test with H100 · 2-GPU"
|
||||
source_file_dependencies: diffusion_minimax_h3_function
|
||||
timeout_in_minutes: 60
|
||||
commands:
|
||||
- pytest -sv tests/diffusion/models/minimax_h3/test_minimax_h3_quantization_quality.py -m 'full_model and cuda and H100 and B200 and cards_2' --run-level "full_model"
|
||||
|
||||
- label: ":full_moon: Diffusion · Quantization Test with L4"
|
||||
source_file_dependencies: diffusion_quantization
|
||||
timeout_in_minutes: 60
|
||||
commands:
|
||||
- pip install "vllm-gguf-plugin==0.0.4"
|
||||
- pytest -sv tests/diffusion/quantization -m 'full_model and cuda and L4 and B200 and cards_1' --run-level "full_model"
|
||||
|
||||
# NOTE: ":email: Nightly Collection & Email" is deprecated and will be removed soon.
|
||||
# Do not add new depends_on entries or extend this step.
|
||||
- label: ":email: Nightly Collection & Email"
|
||||
key: nightly-perf-distribution
|
||||
depends_on:
|
||||
- nightly-omni-performance-no-async-chunk
|
||||
- nightly-omni-performance-async-chunk
|
||||
- nightly-tts-performance-single
|
||||
- nightly-tts-performance-2gpu
|
||||
- nightly-diffusion-x2iat-performance-qwen-image-single
|
||||
- nightly-diffusion-x2iat-performance-qwen-image-4gpu
|
||||
- nightly-diffusion-x2v-performance-single
|
||||
- nightly-diffusion-x2v-performance-2gpu
|
||||
if: build.env("NIGHTLY") == "1" && build.env("EMAIL_DISTRIBUTION") == "1"
|
||||
artifact_paths:
|
||||
- tests/dfx/perf/results/*.xlsx
|
||||
- tests/dfx/perf/results/*.html
|
||||
commands:
|
||||
- pip install openpyxl
|
||||
- export DEFAULT_INPUT_DIR=tests/dfx/perf/results
|
||||
- export DEFAULT_OUTPUT_DIR=tests/dfx/perf/results
|
||||
- buildkite-agent artifact download "tests/dfx/perf/results/*.json" . --step nightly-tts-performance-single
|
||||
- buildkite-agent artifact download "tests/dfx/perf/results/*.json" . --step nightly-tts-performance-2gpu
|
||||
- buildkite-agent artifact download "tests/dfx/perf/results/*.json" . --step nightly-omni-performance-no-async-chunk
|
||||
- buildkite-agent artifact download "tests/dfx/perf/results/*.json" . --step nightly-omni-performance-async-chunk
|
||||
- buildkite-agent artifact download "tests/dfx/perf/results/*.json" . --step nightly-diffusion-x2iat-performance-qwen-image-single
|
||||
- buildkite-agent artifact download "tests/dfx/perf/results/*.json" . --step nightly-diffusion-x2iat-performance-qwen-image-4gpu
|
||||
- buildkite-agent artifact download "tests/dfx/perf/results/*.json" . --step nightly-diffusion-x2v-performance-single
|
||||
- buildkite-agent artifact download "tests/dfx/perf/results/*.json" . --step nightly-diffusion-x2v-performance-2gpu
|
||||
- python tools/nightly/generate_nightly_perf_excel.py
|
||||
- python tools/nightly/generate_nightly_perf_html.py
|
||||
- python tools/nightly/send_nightly_email.py --report-file "tests/dfx/perf/results/*.xlsx, tests/dfx/perf/results/*.html"
|
||||
agents:
|
||||
queue: "cpu_queue_premerge"
|
||||
@@ -1,297 +0,0 @@
|
||||
# CUDA CI L2 (`core_model` e2e) plus L1 CPU Simple Test (`core_model and cpu`).
|
||||
# Uploaded by Upload Ready Pipeline: PR label `ready`, or main+WEEKLY=1 with --e2e.
|
||||
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
HF_HUB_DOWNLOAD_TIMEOUT: 300
|
||||
HF_HUB_ETAG_TIMEOUT: 60
|
||||
|
||||
steps:
|
||||
- group: ":card_index_dividers: Simple Test"
|
||||
depends_on: upload-ready-pipeline
|
||||
steps:
|
||||
- label: "Simple · Model Executor Test"
|
||||
timeout_in_minutes: 20
|
||||
commands:
|
||||
- "pytest -sv tests/model_executor -m 'core_model and cpu'"
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- label: "Simple · CosyVoice3 Packed Flow Test"
|
||||
timeout_in_minutes: 15
|
||||
source_file_dependencies:
|
||||
- tests/model_executor/models/cosyvoice3/test_cosyvoice3_full_batch.py
|
||||
- vllm_omni/model_executor/models/cosyvoice3/
|
||||
commands:
|
||||
- "pytest -sv tests/model_executor/models/cosyvoice3/test_cosyvoice3_full_batch.py -k packed -m 'core_model and cuda' --run-level=core_model"
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "Simple · MRV2 FA3 Graph Bounds Test"
|
||||
timeout_in_minutes: 15
|
||||
source_file_dependencies:
|
||||
- tests/worker_v2/test_capture_query_bound_gpu.py
|
||||
- tests/worker_v2/test_decode_query_bound_gpu.py
|
||||
- vllm_omni/worker_v2/model_states/omni_model_state.py
|
||||
commands:
|
||||
- "pytest -sv tests/worker_v2/test_capture_query_bound_gpu.py tests/worker_v2/test_decode_query_bound_gpu.py -m 'core_model and cuda and H100' --run-level=core_model"
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "Simple · Diffusion Test"
|
||||
timeout_in_minutes: 30
|
||||
commands:
|
||||
- "pytest -sv tests/diffusion -m 'core_model and cpu'"
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- label: "Simple · Engine&Entrypoints Test"
|
||||
timeout_in_minutes: 20
|
||||
commands:
|
||||
- "pytest -sv tests/entrypoints tests/engine -m 'core_model and cpu'"
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- label: "Simple · Other Test"
|
||||
timeout_in_minutes: 20
|
||||
commands:
|
||||
- "pytest -sv tests/ -m 'core_model and cpu' --ignore=tests/diffusion --ignore=tests/model_executor --ignore=tests/entrypoints --ignore=tests/engine"
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- group: ":card_index_dividers: Diffusion Test"
|
||||
depends_on: upload-ready-pipeline
|
||||
steps:
|
||||
|
||||
- label: "Diffusion · FlashAttention Test"
|
||||
timeout_in_minutes: 20
|
||||
commands:
|
||||
- timeout 15m pytest -s -v tests/diffusion/attention/ -m 'core_model and cuda and L4 and cards_1' --run-level "core_model"
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- label: "Diffusion · Offloader Test"
|
||||
commands:
|
||||
- timeout 40m pytest -s -v tests/diffusion/offloader -m "core_model and cuda" --run-level "core_model"
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- label: "Diffusion · Cache Test"
|
||||
commands:
|
||||
- timeout 15m pytest -sv tests/diffusion/cache -m 'core_model and cuda' --run-level "core_model"
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- label: "Diffusion · Batch Test"
|
||||
commands:
|
||||
- timeout 15m pytest -sv tests/diffusion/batching -m 'core_model and cuda' --run-level "core_model"
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- label: "Diffusion · Distributed Test · Single-GPU"
|
||||
source_file_dependencies: diffusion_distributed_attention
|
||||
commands:
|
||||
- timeout 20m pytest -sv tests/diffusion/distributed -m 'core_model and cuda and L4 and cards_1' --run-level "core_model"
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- label: "Diffusion · Distributed Test · 2-GPU"
|
||||
source_file_dependencies: diffusion_distributed_attention
|
||||
commands:
|
||||
- timeout 20m pytest -sv tests/diffusion/distributed -m 'core_model and cuda and L4 and cards_2' --run-level "core_model"
|
||||
- timeout 10m pytest -sv tests/diffusion/attention/test_ulysses_uaa.py -m 'core_model and cuda and L4 and cards_2' --run-level "core_model"
|
||||
mirror_hardwares: l4_2
|
||||
|
||||
- label: "Diffusion · Distributed & Attention Test · 4-GPU"
|
||||
source_file_dependencies: diffusion_distributed_attention
|
||||
commands:
|
||||
- timeout 20m pytest -sv tests/diffusion/distributed -m 'core_model and cuda and L4 and cards_4' --run-level "core_model"
|
||||
- timeout 20m pytest -sv tests/diffusion/attention/test_attention_sp.py -m 'core_model and cuda and L4 and cards_4' --run-level "core_model"
|
||||
mirror_hardwares: l4_4
|
||||
|
||||
- label: "Diffusion · Model Test"
|
||||
commands:
|
||||
- timeout 15m pytest -sv tests/diffusion/models/ -m 'core_model and cuda' --run-level "core_model"
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- label: "Diffusion · Other Test"
|
||||
commands:
|
||||
- timeout 15m pytest -sv tests/diffusion/*.py tests/diffusion/ar_diffusion tests/diffusion/layers/test_adalayernorm_*.py -m 'core_model and cuda' --run-level "core_model"
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- group: ":card_index_dividers: Tiny Model Tests [Single GPU]"
|
||||
depends_on: upload-ready-pipeline
|
||||
steps:
|
||||
|
||||
- label: "Tiny Model Tests · Diffusion (base)"
|
||||
commands:
|
||||
- timeout 15m pytest -sv -n 4 tests/model_tests/diffusion -m 'core_model and cuda' --run-level "core_model"
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- label: "Engine&Model Executor Test"
|
||||
depends_on: upload-ready-pipeline
|
||||
commands:
|
||||
- |
|
||||
timeout 15m bash -c "
|
||||
pytest -sv tests/engine/ tests/model_executor/ tests/worker/test_batched_omni_output.py tests/worker/test_gpu_generation_async_output.py tests/worker_v2/ -m 'core_model and cuda' --run-level 'core_model'
|
||||
"
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- label: "Distributed Test"
|
||||
depends_on: upload-ready-pipeline
|
||||
commands:
|
||||
- |
|
||||
timeout 20m bash -c "
|
||||
pytest -sv tests/distributed/ -m 'core_model and cuda and L4' --run-level 'core_model'
|
||||
"
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- label: "Entrypoints Test"
|
||||
depends_on: upload-ready-pipeline
|
||||
# Workaround https://github.com/vllm-project/vllm-omni/issues/8016: shutdown()
|
||||
# leaves inline UniProc DiffusionWorker VRAM mapped in the pytest parent.
|
||||
# Isolate that leftover case in its own process so the driver drops the
|
||||
# mapping; the rest stays a directory sweep with --deselect. Revert to a
|
||||
# single sweep after the product shutdown fix.
|
||||
commands:
|
||||
- |
|
||||
timeout 30m bash -c "
|
||||
set -eu
|
||||
export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
pytest -sv tests/entrypoints/test_async_omni.py::test_diffusion_generate_request_id -m 'core_model and cuda' --run-level 'core_model'
|
||||
pytest -sv tests/entrypoints/ -m 'core_model and cuda' --run-level 'core_model' --deselect tests/entrypoints/test_async_omni.py::test_diffusion_generate_request_id
|
||||
"
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- group: ":card_index_dividers: E2E Test"
|
||||
depends_on: upload-ready-pipeline
|
||||
steps:
|
||||
- label: "Custom Pipeline Test"
|
||||
source_file_dependencies: custom_pipeline
|
||||
commands:
|
||||
- timeout 20m pytest -s -v tests/e2e/features/custom_pipeline/ -m "core_model"
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
|
||||
- label: "RLHF · VeRL-Omni E2E Test"
|
||||
source_file_dependencies: rlhf_verl_omni
|
||||
commands:
|
||||
- |
|
||||
timeout 20m bash -c "
|
||||
pytest -s -v tests/e2e/features/rlhf_test/test_verl_omni_e2e.py
|
||||
"
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- label: "Omni · Qwen3-Omni Test"
|
||||
source_file_dependencies: omni_qwen3_omni_function
|
||||
commands:
|
||||
- |
|
||||
timeout 20m bash -c "
|
||||
pytest -s -v tests/e2e/online_serving/test_qwen3_omni.py -m 'core_model' --run-level 'core_model'
|
||||
"
|
||||
mirror_hardwares: h100_2
|
||||
|
||||
- label: "Omni · MiniCPM-o 4.5 Offline Test"
|
||||
source_file_dependencies: omni_minicpmo_4_5_function
|
||||
commands:
|
||||
- |
|
||||
timeout 20m bash -c "
|
||||
pytest -s -v tests/e2e/offline_inference/test_minicpmo_4_5.py -m 'core_model' --run-level 'core_model'
|
||||
"
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "Omni · MiniCPM-o 4.5 Online Test"
|
||||
source_file_dependencies: omni_minicpmo_4_5_function
|
||||
commands:
|
||||
- |
|
||||
timeout 20m bash -c "
|
||||
pytest -s -v tests/e2e/online_serving/test_minicpmo_4_5.py -m 'core_model' --run-level 'core_model'
|
||||
"
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "Omni · MiniCPM-o 4.5 Duplex Test"
|
||||
source_file_dependencies: omni_minicpmo_4_5_duplex_function
|
||||
commands:
|
||||
- |
|
||||
timeout 35m bash -c "
|
||||
pytest -s -v tests/e2e/online_serving/test_minicpmo_4_5_duplex.py tests/e2e/online_serving/test_duplex_client_live.py -m 'core_model and cuda' --run-level 'core_model'
|
||||
"
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "Omni · MiniCPM-o 4.5 Perf Test"
|
||||
timeout_in_minutes: 60
|
||||
source_file_dependencies: omni_minicpmo_4_5_perf
|
||||
commands:
|
||||
- export BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_minicpmo_4_5_ready.json -m 'core_model and cuda'
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "TTS · Higgs v3 Full Response Test"
|
||||
source_file_dependencies:
|
||||
- tests/e2e/online_serving/test_higgs_audio_v3.py
|
||||
- vllm_omni/model_executor/models/higgs_audio_v3/
|
||||
- vllm_omni/model_executor/stage_input_processors/higgs_audio_v3.py
|
||||
- vllm_omni/deploy/higgs_multimodal_qwen3.yaml
|
||||
commands:
|
||||
- timeout 20m pytest -s -v tests/e2e/online_serving/test_higgs_audio_v3.py::test_full_response_v1_preserves_audio -m 'core_model and cuda' --run-level 'core_model'
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "TTS · Qwen3-TTS CustomVoice Test"
|
||||
source_file_dependencies: tts_qwen3_tts_function
|
||||
commands:
|
||||
- |
|
||||
timeout 20m bash -c "
|
||||
export VLLM_ALLOW_LONG_MAX_MODEL_LEN=1
|
||||
pytest -s -v tests/e2e/online_serving/test_qwen3_tts_customvoice.py -m 'core_model' --run-level 'core_model'
|
||||
"
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- label: "Diffusion · Wan22 Test"
|
||||
source_file_dependencies: diffusion_wan22_function
|
||||
commands:
|
||||
- |
|
||||
timeout 40m bash -c "
|
||||
export VLLM_IMAGE_FETCH_TIMEOUT=60
|
||||
pytest -s -v tests/e2e/online_serving/test_wan22_t2v.py -m 'core_model and cuda' --run-level 'core_model'
|
||||
"
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "Diffusion · Cosmos3 Test"
|
||||
source_file_dependencies: diffusion_cosmos3_function
|
||||
commands:
|
||||
- |
|
||||
timeout 40m bash -c "
|
||||
export VLLM_IMAGE_FETCH_TIMEOUT=60
|
||||
pytest -s -v tests/e2e/online_serving/test_cosmos3.py -m 'core_model' --run-level 'core_model'
|
||||
"
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "Diffusion · MiniMax-H3 DLO DP2 T2VA Test"
|
||||
timeout_in_minutes: 90
|
||||
source_file_dependencies: diffusion_minimax_h3_function
|
||||
commands:
|
||||
- |
|
||||
timeout 80m bash -c "
|
||||
export VLLM_IMAGE_FETCH_TIMEOUT=60
|
||||
pytest -s -v tests/e2e/online_serving/test_minimax_h3_dlo_dp2_t2va.py -m 'core_model and cuda and diffusion' --run-level 'core_model'
|
||||
"
|
||||
mirror_hardwares: h100_2
|
||||
|
||||
- label: "Diffusion · Qwen Image Test"
|
||||
source_file_dependencies: diffusion_qwen_image_function
|
||||
commands:
|
||||
- |
|
||||
timeout 40m bash -c "
|
||||
export VLLM_IMAGE_FETCH_TIMEOUT=60
|
||||
pytest -s -v tests/e2e/online_serving/test_qwen_image.py -m 'core_model' --run-level 'core_model'
|
||||
"
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "Diffusion · HunyuanVideo-1.5 Test"
|
||||
source_file_dependencies: diffusion_hunyuan_video_function
|
||||
commands:
|
||||
- |
|
||||
timeout 40m bash -c "
|
||||
export VLLM_IMAGE_FETCH_TIMEOUT=60
|
||||
pytest -s -v tests/e2e/online_serving/test_hunyuan_video_15_expansion.py -m 'core_model and diffusion' --run-level 'core_model'
|
||||
"
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "Diffusion · LingBot-Video Test"
|
||||
source_file_dependencies: diffusion_lingbot_function
|
||||
commands:
|
||||
- |
|
||||
timeout 40m bash -c "
|
||||
export VLLM_IMAGE_FETCH_TIMEOUT=60
|
||||
pytest -s -v tests/e2e/online_serving/test_lingbot_video.py -m 'core_model and cuda' --run-level 'core_model'
|
||||
"
|
||||
mirror_hardwares: h100_1
|
||||
@@ -1,651 +0,0 @@
|
||||
# CUDA CI L5 (`slow` / reliability / extra coverage).
|
||||
# Uploaded by Upload Weekly Pipeline: WEEKLY=1 / NON_CRITICAL=1, or PR label `weekly-test`.
|
||||
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
HF_HUB_DOWNLOAD_TIMEOUT: 300
|
||||
HF_HUB_ETAG_TIMEOUT: 60
|
||||
|
||||
steps:
|
||||
- label: "Simple · CPU Coverage Test"
|
||||
key: weekly-simple-cpu-coverage
|
||||
depends_on: upload-weekly-pipeline
|
||||
if: build.env("WEEKLY") == "1"
|
||||
timeout_in_minutes: 60
|
||||
artifact_paths:
|
||||
- coverage-core-model-cpu-*.xml.gz
|
||||
commands:
|
||||
- |
|
||||
set +e
|
||||
REPORT="coverage-core-model-cpu-$${BUILDKITE_STEP_ID:-local}.xml"
|
||||
pytest -sv tests/ -m 'core_model and cpu' --cov=vllm_omni --cov-report=term-missing:skip-covered --cov-report=xml:$${REPORT}
|
||||
EXIT=$$?
|
||||
gzip -9 -f "$${REPORT}"
|
||||
exit $$EXIT
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- group: "Reliability Test"
|
||||
depends_on: upload-weekly-pipeline
|
||||
if: build.env("WEEKLY") == "1" || build.pull_request.labels includes "weekly-test"
|
||||
steps:
|
||||
|
||||
- label: "Reliability Test · qwen3-omni"
|
||||
source_file_dependencies: omni_qwen3_omni_reliability
|
||||
timeout_in_minutes: 180
|
||||
commands:
|
||||
- pytest -s -v tests/dfx/reliability/test_reliability_qwen3_omni.py -m "slow"
|
||||
mirror_hardwares: h100_2
|
||||
|
||||
- label: "Reliability Test · wan22"
|
||||
source_file_dependencies: diffusion_wan22_reliability
|
||||
timeout_in_minutes: 180
|
||||
commands:
|
||||
- pytest -s -v tests/dfx/reliability/test_reliability_wan22.py -m "slow"
|
||||
mirror_hardwares: h100_2
|
||||
|
||||
- label: "Reliability Test · hunyuan-image"
|
||||
source_file_dependencies: diffusion_hunyuan_image3_reliability
|
||||
timeout_in_minutes: 180
|
||||
commands:
|
||||
- pytest -s -v tests/dfx/reliability/test_reliability_hunyuan_image.py -m "slow"
|
||||
mirror_hardwares: h100_4
|
||||
|
||||
- label: "Reliability Test · Invalid parameters · H100 · Single-GPU"
|
||||
source_file_dependencies: reliability_invalid_param_h100
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- pytest -s -v tests/dfx/reliability/invalid_param_test/ -m "slow and H100 and cards_1"
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "Reliability Test · Invalid parameters · H100 · 2-GPU"
|
||||
source_file_dependencies: reliability_invalid_param_h100
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- pytest -s -v tests/dfx/reliability/invalid_param_test/ -m "slow and H100 and cards_2"
|
||||
mirror_hardwares: h100_2
|
||||
|
||||
- label: "Reliability Test · Invalid parameters · L4"
|
||||
source_file_dependencies: reliability_invalid_param_l4
|
||||
timeout_in_minutes: 120
|
||||
commands:
|
||||
- pytest -s -v tests/dfx/reliability/invalid_param_test/ -m "slow and L4"
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- group: "Perf Test"
|
||||
depends_on: upload-weekly-pipeline
|
||||
if: build.env("WEEKLY") == "1" || build.pull_request.labels includes "weekly-test"
|
||||
steps:
|
||||
|
||||
- label: ":full_moon: Omni · Perf Test · vLLM Text"
|
||||
source_file_dependencies: omni_qwen3_omni_perf
|
||||
key: weekly-omni-performance-vllm-text
|
||||
timeout_in_minutes: 180
|
||||
artifact_paths:
|
||||
- tests/dfx/perf/results/*.json
|
||||
commands:
|
||||
- export BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_qwen3_omni_vllm_text.json
|
||||
mirror_hardwares: h100_2
|
||||
|
||||
- label: ":full_moon: Omni · Perf Test · Async Chunk · Random"
|
||||
key: weekly-omni-performance-async-chunk-random
|
||||
timeout_in_minutes: 180
|
||||
artifact_paths:
|
||||
- tests/dfx/perf/results/*.json
|
||||
commands:
|
||||
- export BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_qwen3_omni_async_chunk.json -m "H100 and slow"
|
||||
mirror_hardwares: h100_2
|
||||
|
||||
- label: ":full_moon: Omni · Perf Test · Multi-Replica"
|
||||
source_file_dependencies: omni_qwen3_omni_perf
|
||||
key: weekly-omni-performance-multi-replicas
|
||||
timeout_in_minutes: 180
|
||||
artifact_paths:
|
||||
- tests/dfx/perf/results/*.json
|
||||
commands:
|
||||
- export BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_qwen3_omni_multi_replicas.json
|
||||
mirror_hardwares: h100_3
|
||||
|
||||
- group: ":card_index_dividers: E2E Tests"
|
||||
depends_on: upload-weekly-pipeline
|
||||
if: build.env("NON_CRITICAL") == "1"
|
||||
steps:
|
||||
|
||||
# Online Qwen2.5 expansion is 2-card; offline expansion + AutoRound are 4-card.
|
||||
- label: "Omni · L4 · 2-GPU"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
|
||||
- pytest -s -v tests/e2e/online_serving/test_qwen2_5_omni_expansion.py -m "slow and L4 and omni and cards_2" --run-level "core_model"
|
||||
mirror_hardwares: l4_2
|
||||
|
||||
- label: "Omni · L4 · 4-GPU"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
|
||||
- pytest -s -v tests/e2e/offline_inference/test_qwen2_5_omni_expansion.py -m "slow and L4 and omni and cards_4" --run-level "core_model"
|
||||
- >-
|
||||
pytest -s -v tests/e2e/
|
||||
--ignore=tests/e2e/online_serving/test_qwen2_5_omni_expansion.py
|
||||
--ignore=tests/e2e/offline_inference/test_qwen2_5_omni_expansion.py
|
||||
-m "slow and L4 and omni and cards_4"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: l4_4
|
||||
|
||||
- label: "Omni · H100 · Single-GPU"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/ -m "slow and H100 and omni and cards_1"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "Omni · H100 · 4-GPU"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/ -m "slow and H100 and omni and cards_4"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: h100_4
|
||||
|
||||
- label: "TTS · Moss · L4"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
|
||||
- export VLLM_USE_DEEP_GEMM="0"
|
||||
- export VLLM_MOE_USE_DEEP_GEMM="0"
|
||||
- |
|
||||
failed=0
|
||||
for f in \
|
||||
tests/e2e/offline_inference/test_moss_tts_expansion.py \
|
||||
tests/e2e/offline_inference/test_moss_tts_nano_expansion.py \
|
||||
tests/e2e/offline_inference/test_moss_tts_realtime.py \
|
||||
tests/e2e/offline_inference/test_moss_tts_realtime_expansion.py \
|
||||
tests/e2e/online_serving/test_moss_tts_expansion.py \
|
||||
tests/e2e/online_serving/test_moss_tts_nano_expansion.py
|
||||
do
|
||||
pytest -s -v "$$f" -m "slow and L4 and tts" --run-level "full_model" || failed=1
|
||||
done
|
||||
exit $$failed
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- label: "TTS · VoxCPM2 · Voxtral · L4"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
|
||||
- export VLLM_USE_DEEP_GEMM="0"
|
||||
- export VLLM_MOE_USE_DEEP_GEMM="0"
|
||||
- |
|
||||
failed=0
|
||||
for f in \
|
||||
tests/e2e/offline_inference/test_voxcpm2_tts.py \
|
||||
tests/e2e/online_serving/test_voxcpm2_tts.py \
|
||||
tests/e2e/online_serving/test_voxcpm2_tts_expansion.py \
|
||||
tests/e2e/online_serving/test_voxcpm2_tts_lora.py \
|
||||
tests/e2e/offline_inference/test_voxtral_tts_expansion.py \
|
||||
tests/e2e/online_serving/test_voxtral_tts_expansion.py
|
||||
do
|
||||
pytest -s -v "$$f" -m "slow and L4 and tts" --run-level "full_model" || failed=1
|
||||
done
|
||||
exit $$failed
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- label: "TTS · Higgs · L4"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
|
||||
- export VLLM_USE_DEEP_GEMM="0"
|
||||
- export VLLM_MOE_USE_DEEP_GEMM="0"
|
||||
- |
|
||||
failed=0
|
||||
for f in \
|
||||
tests/e2e/offline_inference/test_higgs_audio_v2_expansion.py \
|
||||
tests/e2e/online_serving/test_higgs_audio_v2_expansion.py \
|
||||
tests/e2e/online_serving/test_higgs_audio_v3.py
|
||||
do
|
||||
pytest -s -v "$$f" -m "slow and L4 and tts" --run-level "full_model" || failed=1
|
||||
done
|
||||
exit $$failed
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- label: "TTS · OmniVoice · Step-Audio · L4"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
|
||||
- export VLLM_USE_DEEP_GEMM="0"
|
||||
- export VLLM_MOE_USE_DEEP_GEMM="0"
|
||||
- |
|
||||
failed=0
|
||||
for f in \
|
||||
tests/e2e/offline_inference/test_omnivoice_expansion.py \
|
||||
tests/e2e/online_serving/test_omnivoice_expansion.py \
|
||||
tests/e2e/online_serving/test_omnivoice_parity.py \
|
||||
tests/e2e/offline_inference/test_step_audio2_expansion.py
|
||||
do
|
||||
pytest -s -v "$$f" -m "slow and L4 and tts" --run-level "full_model" || failed=1
|
||||
done
|
||||
exit $$failed
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- label: "TTS · Ming · GLM · IndexTTS · Breeze · L4"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
|
||||
- export VLLM_USE_DEEP_GEMM="0"
|
||||
- export VLLM_MOE_USE_DEEP_GEMM="0"
|
||||
- |
|
||||
failed=0
|
||||
for f in \
|
||||
tests/e2e/offline_inference/test_ming_tts_expansion.py \
|
||||
tests/e2e/online_serving/test_ming_tts_expansion.py \
|
||||
tests/e2e/offline_inference/test_glm_tts_expansion.py \
|
||||
tests/e2e/online_serving/test_glm_tts_expansion.py \
|
||||
tests/e2e/offline_inference/test_indextts2_expansion.py \
|
||||
tests/e2e/online_serving/test_indextts2_expansion.py \
|
||||
tests/e2e/online_serving/test_breeze_tts_2.py
|
||||
do
|
||||
pytest -s -v "$$f" -m "slow and L4 and tts" --run-level "full_model" || failed=1
|
||||
done
|
||||
exit $$failed
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- label: "TTS · Audex · CosyVoice · Qwen3-TTS · L4"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
|
||||
- export VLLM_USE_DEEP_GEMM="0"
|
||||
- export VLLM_MOE_USE_DEEP_GEMM="0"
|
||||
- |
|
||||
failed=0
|
||||
for f in \
|
||||
tests/e2e/offline_inference/test_audex_tts.py \
|
||||
tests/e2e/offline_inference/test_audex_s2t.py \
|
||||
tests/e2e/offline_inference/test_audex_tta.py \
|
||||
tests/e2e/offline_inference/test_cosyvoice3_expansion.py \
|
||||
tests/e2e/online_serving/test_qwen3_tts_word_timestamps.py
|
||||
do
|
||||
pytest -s -v "$$f" -m "slow and L4 and tts" --run-level "full_model" || failed=1
|
||||
done
|
||||
exit $$failed
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- label: "TTS · H100"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
|
||||
- export VLLM_USE_DEEP_GEMM="0"
|
||||
- export VLLM_MOE_USE_DEEP_GEMM="0"
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/ -m "slow and H100 and tts"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
# Own step: the image has no NanoCodec, so this file import-skips out of
|
||||
# the mark-selected TTS steps above. The overrides keep nemo_toolkit[tts]
|
||||
# from downgrading transformers; see
|
||||
# examples/offline_inference/text_to_speech/README.md.
|
||||
- label: "TTS · Gepard-1.0 · L4"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
|
||||
- printf '%s\n' "transformers>=5.5.3" "huggingface-hub>=1.0" > /tmp/gepard-overrides.txt
|
||||
- uv pip install --python "$(python3 -c 'import sys; print(sys.executable)')" "nemo_toolkit[tts]==2.7.3" --overrides /tmp/gepard-overrides.txt
|
||||
- >-
|
||||
pytest -s -v tests/e2e/offline_inference/test_gepard_tts.py
|
||||
tests/e2e/online_serving/test_gepard_tts.py
|
||||
-m "tts and cuda"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- label: "Diffusion · L4 · Single-GPU"
|
||||
timeout_in_minutes: 60
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/ -m "slow and diffusion and L4 and cards_1"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: l4_1
|
||||
|
||||
- label: "Diffusion · L4 · 2-GPU"
|
||||
timeout_in_minutes: 60
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/ -m "slow and diffusion and L4 and cards_2"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: l4_2
|
||||
|
||||
- label: "Diffusion · L4 · 4-GPU"
|
||||
timeout_in_minutes: 60
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/ -m "slow and diffusion and L4 and cards_4"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: l4_4
|
||||
|
||||
# Split the old shared H100 Single-GPU sweep: one pytest process collected
|
||||
# every slow+cards_1 diffusion file and exceeded 150 minutes.
|
||||
- label: ":robot_face: Robot Policy OpenPI · π0.5 · H100 · Single-GPU"
|
||||
timeout_in_minutes: 120
|
||||
commands:
|
||||
- pip install --no-deps "openpi-client @ git+https://github.com/Physical-Intelligence/openpi.git@215abfb217dbac7d5f1273282331b9b1866c0479#subdirectory=packages/openpi-client"
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/online_serving/test_pi05_expansion.py
|
||||
-m "slow and diffusion and H100 and cards_1"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "Diffusion · Qwen-Image · H100 · Single-GPU"
|
||||
timeout_in_minutes: 120
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/online_serving/test_qwen_image_edit.py
|
||||
tests/e2e/online_serving/test_qwen_image_edit_expansion.py
|
||||
tests/e2e/online_serving/test_qwen_image_layered.py
|
||||
tests/e2e/online_serving/test_qwen_image_layered_expansion.py
|
||||
tests/e2e/accuracy/test_qwen_image_edit.py
|
||||
tests/e2e/accuracy/test_qwen_image_layered.py
|
||||
-m "slow and diffusion and H100 and cards_1"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "Diffusion · Wan2.1 VACE · H100 · Single-GPU"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/online_serving/test_wan_2_1_vace_expansion.py
|
||||
-m "slow and diffusion and H100 and cards_1"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "Diffusion · LongCat · H100 · Single-GPU"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/online_serving/test_longcat_image_expansion.py
|
||||
tests/e2e/online_serving/test_longcat_image_edit_expansion.py
|
||||
-m "slow and diffusion and H100 and cards_1"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "Diffusion · Bagel · H100 · Single-GPU"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/online_serving/test_bagel.py
|
||||
tests/e2e/online_serving/test_bagel_expansion.py
|
||||
tests/e2e/offline_inference/test_bagel.py
|
||||
tests/e2e/offline_inference/test_bagel_expansion.py
|
||||
-m "slow and diffusion and H100 and cards_1"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "Diffusion · Boogu · H100 · Single-GPU"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/online_serving/test_boogu_image.py
|
||||
tests/e2e/online_serving/test_boogu_image_edit.py
|
||||
tests/e2e/online_serving/test_boogu_image_expansion.py
|
||||
tests/e2e/online_serving/test_boogu_image_edit_expansion.py
|
||||
-m "slow and diffusion and H100 and cards_1"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "Diffusion · Flux · H100 · Single-GPU"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/online_serving/test_flux_2_dev_expansion.py
|
||||
tests/e2e/offline_inference/test_flux1_schnell_expansion.py
|
||||
-m "slow and diffusion and H100 and cards_1"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "Diffusion · LTX · H100 · Single-GPU"
|
||||
timeout_in_minutes: 120
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/online_serving/test_ltx.py
|
||||
tests/e2e/online_serving/test_ltx25.py
|
||||
tests/e2e/accuracy/ltx/test_ltx_official_similarity.py
|
||||
tests/e2e/accuracy/ltx/test_ltx25_official_similarity.py
|
||||
-m "slow and diffusion and H100 and cards_1"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "Diffusion · Krea2 · H100 · Single-GPU"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/online_serving/test_krea2_expansion.py
|
||||
tests/e2e/offline_inference/test_krea2_expansion.py
|
||||
-m "slow and diffusion and H100 and cards_1"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "Diffusion · HiDream-I1 · H100 · Single-GPU"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/online_serving/test_hidream_i1_full_expansion.py
|
||||
-m "slow and diffusion and H100 and cards_1"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "Diffusion · SANA-Video · H100 · Single-GPU"
|
||||
source_file_dependencies: diffusion_sana_video_function
|
||||
timeout_in_minutes: 180
|
||||
commands:
|
||||
- export SANA_VIDEO_GOLDEN_BASE_URL=https://vllm-public-assets.s3.us-west-2.amazonaws.com/omni-assets/sana-video/v2
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/online_serving/test_sana_video_expansion.py
|
||||
tests/e2e/accuracy/sana_video/
|
||||
-m "slow and diffusion and H100 and cards_1"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "Diffusion · SenseNova-U1 · H100 · Single-GPU"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/offline_inference/test_sensenova_u1_expansion.py
|
||||
tests/e2e/offline_inference/test_sensenova_u1_text2img_expansion.py
|
||||
tests/e2e/offline_inference/test_sensenova_u1_img2img_expansion.py
|
||||
-m "slow and diffusion and H100 and cards_1"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "Diffusion · MammothModa2 · H100 · Single-GPU"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/offline_inference/test_mammoth_moda2_expansion.py
|
||||
-m "slow and diffusion and H100 and cards_1"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
- label: "Diffusion · Helios · H100 · Single-GPU"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/accuracy/test_video_streaming_output_similarity.py
|
||||
-m "slow and diffusion and H100 and cards_1"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: h100_1
|
||||
|
||||
# Split the old shared H100 2-GPU sweep: one pytest process collected every
|
||||
# slow+cards_2 diffusion file and exceeded 90 minutes.
|
||||
- label: "Diffusion · Qwen-Image · H100 · 2-GPU"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/online_serving/test_qwen_image_edit_expansion.py
|
||||
tests/e2e/online_serving/test_qwen_image_layered_expansion.py
|
||||
-m "slow and diffusion and H100 and cards_2"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: h100_2
|
||||
|
||||
- label: "Diffusion · Wan2.1 VACE · H100 · 2-GPU"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/online_serving/test_wan_2_1_vace_expansion.py
|
||||
-m "slow and diffusion and H100 and cards_2"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: h100_2
|
||||
|
||||
- label: "Diffusion · LongCat · H100 · 2-GPU"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/online_serving/test_longcat_image_expansion.py
|
||||
tests/e2e/online_serving/test_longcat_image_edit_expansion.py
|
||||
-m "slow and diffusion and H100 and cards_2"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: h100_2
|
||||
|
||||
- label: "Diffusion · Bagel · H100 · 2-GPU"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/online_serving/test_bagel_expansion.py
|
||||
tests/e2e/offline_inference/test_bagel_expansion.py
|
||||
-m "slow and diffusion and H100 and cards_2"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: h100_2
|
||||
|
||||
- label: "Diffusion · Boogu · H100 · 2-GPU"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/online_serving/test_boogu_image_expansion.py
|
||||
tests/e2e/online_serving/test_boogu_image_edit_expansion.py
|
||||
-m "slow and diffusion and H100 and cards_2"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: h100_2
|
||||
|
||||
- label: "Diffusion · Flux · H100 · 2-GPU"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/online_serving/test_flux_2_dev_expansion.py
|
||||
tests/e2e/offline_inference/test_flux1_kontext_expansion.py
|
||||
-m "slow and diffusion and H100 and cards_2"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: h100_2
|
||||
|
||||
- label: "Diffusion · GLM-Image · H100 · 2-GPU"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/online_serving/test_glm_image_expansion.py
|
||||
tests/e2e/offline_inference/test_glm_image_autoround_w4a16_expansion.py
|
||||
-m "slow and diffusion and H100 and cards_2"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: h100_2
|
||||
|
||||
- label: "Diffusion · LTX · H100 · 2-GPU"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/online_serving/test_ltx.py
|
||||
-m "slow and diffusion and H100 and cards_2"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: h100_2
|
||||
|
||||
- label: "Diffusion · Krea2 · H100 · 2-GPU"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/offline_inference/test_krea2_expansion.py
|
||||
-m "slow and diffusion and H100 and cards_2"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: h100_2
|
||||
|
||||
- label: "Diffusion · DreamZero · H100 · 2-GPU"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/online_serving/test_dreamzero_expansion.py
|
||||
tests/e2e/accuracy/test_dreamzero.py
|
||||
-m "slow and diffusion and H100 and cards_2"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: h100_2
|
||||
|
||||
- label: "Diffusion · H100 · 3-GPU"
|
||||
timeout_in_minutes: 120
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/ -m "slow and diffusion and H100 and cards_3"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: h100_3
|
||||
|
||||
- label: "Diffusion · H100 · 4-GPU"
|
||||
timeout_in_minutes: 90
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/ -m "slow and diffusion and H100 and cards_4"
|
||||
--run-level "full_model"
|
||||
--ignore=tests/e2e/online_serving/test_magi2.py
|
||||
mirror_hardwares: h100_4
|
||||
|
||||
# Dedicated: HiDream-O1 accuracy needs a long TP2 run; the shared H100
|
||||
# 2-GPU sweep is only 90 minutes.
|
||||
- label: "Diffusion · HiDream-O1-Image Accuracy · H100 · 2-GPU"
|
||||
timeout_in_minutes: 180
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/accuracy/test_hidream_o1_image.py -m "slow and diffusion and cuda and H100 and cards_2" --run-level "full_model"
|
||||
mirror_hardwares: h100_2
|
||||
|
||||
# Dedicated: Boogu CFG-parallel size 3 needs 3 GPUs; weekly has no H100 3-GPU sweep.
|
||||
- label: "Diffusion · Boogu Edit CFG · H100 · 3-GPU"
|
||||
timeout_in_minutes: 120
|
||||
commands:
|
||||
- >-
|
||||
pytest -sv
|
||||
tests/e2e/online_serving/test_boogu_image_edit_expansion.py
|
||||
-k "test_double_guidance_cfg_parallel and cfg_parallel_3_full"
|
||||
-m "slow and diffusion and cuda and H100 and cards_3"
|
||||
--run-level "full_model"
|
||||
mirror_hardwares: h100_3
|
||||
|
||||
# Dedicated: MAGI-2 Preview serving smoke needs ~30 min server init; the
|
||||
# shared H100 4-GPU sweep is only 90 minutes.
|
||||
- label: "Diffusion · MAGI-2 Preview Serving Smoke · H100 · 4-GPU"
|
||||
timeout_in_minutes: 180
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/online_serving/test_magi2.py -m "slow and diffusion and cuda and H100 and cards_4" --run-level "full_model"
|
||||
mirror_hardwares: h100_4
|
||||
@@ -1,16 +0,0 @@
|
||||
#!/bin/bash
|
||||
# vllm-omni Intel bootstrap
|
||||
# Uses static pipeline-intel.yml for Intel XPU tests
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
source .buildkite/common/scripts/resolve_skip_ci.sh
|
||||
|
||||
upload_pipeline() {
|
||||
echo "--- 🛠 Preparing Intel pipeline"
|
||||
buildkite-agent pipeline upload .buildkite/intel/pipeline-intel.yml
|
||||
}
|
||||
|
||||
gate_bootstrap_ci intel l2
|
||||
|
||||
upload_pipeline
|
||||
@@ -1,127 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# This script build the XPU docker image and run the offline inference inside the container.
|
||||
set -ex
|
||||
|
||||
omni_source_dir=$(git rev-parse --show-toplevel)
|
||||
|
||||
: "${VLLM_VERSION:?VLLM_VERSION must be set}"
|
||||
# Official upstream XPU image. It is published from vllm's own
|
||||
# docker/Dockerfile.xpu, which is exactly what build_vllm_base() clones and
|
||||
# builds below, so pulling it is equivalent to building the fallback base.
|
||||
upstream_base_image="vllm/vllm-openai-xpu:${VLLM_VERSION}"
|
||||
local_base_image="xpu/vllm-omni-ci-base:${VLLM_VERSION}"
|
||||
image_name="xpu/vllm-omni-ci:${BUILDKITE_COMMIT:?BUILDKITE_COMMIT must be set}"
|
||||
container_name="xpu_${BUILDKITE_COMMIT}_$(
|
||||
tr -dc A-Za-z0-9 </dev/urandom | head -c 10
|
||||
echo
|
||||
)"
|
||||
|
||||
cd "${omni_source_dir}"
|
||||
|
||||
# The XPU base image is ~37GB; the default gzip layer exporter is single-threaded
|
||||
# and takes ~16min to compress it. zstd is multi-threaded (uses all cores) at a
|
||||
# similar ratio; set EXPORT_COMPRESSION=uncompressed to skip compression entirely
|
||||
# for local-only images. This requires the containerd image store (buildx docker
|
||||
# driver), which is the default here.
|
||||
EXPORT_COMPRESSION="${EXPORT_COMPRESSION:-zstd}"
|
||||
if [ "${EXPORT_COMPRESSION}" = "uncompressed" ]; then
|
||||
export_args=(--output "type=image,name={{IMAGE}},compression=uncompressed")
|
||||
else
|
||||
export_args=(--output "type=image,name={{IMAGE}},compression=${EXPORT_COMPRESSION},compression-level=3,force-compression=true")
|
||||
fi
|
||||
|
||||
docker_build() {
|
||||
# $1 = image name; remaining args passed through to docker build.
|
||||
local image="$1"
|
||||
shift
|
||||
local out=("${export_args[@]/'{{IMAGE}}'/${image}}")
|
||||
docker build "${out[@]}" "$@" -f docker/Dockerfile.xpu .
|
||||
}
|
||||
|
||||
build_vllm_base() (
|
||||
local vllm_source_dir
|
||||
vllm_source_dir=$(mktemp -d)
|
||||
trap 'rm -rf "${vllm_source_dir}"' EXIT
|
||||
|
||||
git clone --depth 1 --branch "${VLLM_VERSION}" \
|
||||
https://github.com/vllm-project/vllm "${vllm_source_dir}"
|
||||
|
||||
local out=("${export_args[@]/'{{IMAGE}}'/${base_image_name}}")
|
||||
docker build "${out[@]}" \
|
||||
--target vllm-openai \
|
||||
-f "${vllm_source_dir}/docker/Dockerfile.xpu" \
|
||||
"${vllm_source_dir}"
|
||||
)
|
||||
|
||||
# Resolve the vLLM base image, in priority order:
|
||||
# 1. the published upstream image for VLLM_VERSION, pulled fresh (so moving
|
||||
# tags like nightly are refreshed rather than served stale from disk);
|
||||
# 2. a local copy of that same image, when the registry is unreachable;
|
||||
# 3. VLLM_BASE, if that image is already on disk;
|
||||
# 4. otherwise build VLLM_BASE from the matching vLLM source tag (~30min).
|
||||
# VLLM_BASE names the fallback base and defaults to ${local_base_image}; set it
|
||||
# to reuse or produce a base under a different tag.
|
||||
resolve_vllm_base() {
|
||||
if docker pull "${upstream_base_image}"; then
|
||||
base_image_name="${upstream_base_image}"
|
||||
return
|
||||
fi
|
||||
|
||||
# Registry unreachable, but a previous run may have left the image behind.
|
||||
if [ -n "$(docker images -q "${upstream_base_image}")" ]; then
|
||||
echo "WARNING: could not pull ${upstream_base_image}; using local copy" >&2
|
||||
base_image_name="${upstream_base_image}"
|
||||
return
|
||||
fi
|
||||
|
||||
echo "WARNING: ${upstream_base_image} unavailable remotely and locally" >&2
|
||||
base_image_name="${VLLM_BASE:-${local_base_image}}"
|
||||
if [ -z "$(docker images -q "${base_image_name}")" ]; then
|
||||
echo "WARNING: building ${base_image_name} from vLLM ${VLLM_VERSION} source" >&2
|
||||
build_vllm_base
|
||||
fi
|
||||
}
|
||||
|
||||
resolve_vllm_base
|
||||
|
||||
# Try building the docker image
|
||||
docker_build "${image_name}" --build-arg "VLLM_BASE=${base_image_name}" --build-arg "VLLM_VERSION=${VLLM_VERSION}"
|
||||
|
||||
# Setup cleanup
|
||||
remove_docker_container() {
|
||||
docker rm -f "${container_name}" || true
|
||||
docker image rm -f "${image_name}" || true
|
||||
docker system prune -f || true
|
||||
}
|
||||
trap remove_docker_container EXIT
|
||||
|
||||
HF_CACHE="${HF_CACHE:-$(realpath ~)/.cache/huggingface}"
|
||||
mkdir -p "${HF_CACHE}"
|
||||
HF_MOUNT="/root/.cache/huggingface"
|
||||
|
||||
time timeout -k 30 30m docker run \
|
||||
--device /dev/dri:/dev/dri \
|
||||
--net=host \
|
||||
--ipc=host \
|
||||
-v /dev/dri/by-path:/dev/dri/by-path \
|
||||
-v "${HF_CACHE}:${HF_MOUNT}" \
|
||||
--security-opt seccomp=unconfined \
|
||||
--entrypoint="" \
|
||||
-e VLLM_LOGGING_LEVEL \
|
||||
-e VLLM_OMNI_LOGGING_LEVEL \
|
||||
-e HF_TOKEN \
|
||||
-e ZE_AFFINITY_MASK \
|
||||
--name "${container_name}" \
|
||||
"${image_name}" \
|
||||
bash -c '
|
||||
set -e
|
||||
echo $ZE_AFFINITY_MASK
|
||||
pip install tblib==3.1.0
|
||||
cd /workspace/vllm-omni
|
||||
XPU_TEST_PATHS="tests/diffusion tests/dfx tests/e2e"
|
||||
pytest -v -s $XPU_TEST_PATHS -m "core_model and xpu and B60"
|
||||
pytest -v -s tests/diffusion/quantization/test_mxfp8_config.py
|
||||
pytest -v -s $XPU_TEST_PATHS -m "advanced_model and xpu and B60"
|
||||
pytest -v -s $XPU_TEST_PATHS -m "omni and xpu and B60"
|
||||
'
|
||||
@@ -1,112 +0,0 @@
|
||||
# Uploaded by upload_pipeline.py --upload (not the Buildkite hook).
|
||||
# ``if`` is injected by step ``key`` from skip-ci at upload time.
|
||||
steps:
|
||||
- label: ":buildkit: Build and Push NPU Test Image - A2"
|
||||
key: image-build-a2
|
||||
agents: &npu-image-build-agents
|
||||
queue: "ascend-a2b3"
|
||||
resource_class: "npu-1"
|
||||
plugins: &npu-image-build-plugins
|
||||
- kubernetes:
|
||||
metadata:
|
||||
annotations:
|
||||
vault.hashicorp.com/agent-init-first: "true"
|
||||
vault.hashicorp.com/agent-inject: "true"
|
||||
vault.hashicorp.com/agent-inject-perms-ca.pem: "0400"
|
||||
vault.hashicorp.com/agent-inject-perms-cert.pem: "0400"
|
||||
vault.hashicorp.com/agent-inject-perms-config.json: "0400"
|
||||
vault.hashicorp.com/agent-inject-perms-key.pem: "0400"
|
||||
vault.hashicorp.com/agent-inject-secret-ca.pem: internal/data/ascend/buildkitd
|
||||
vault.hashicorp.com/agent-inject-secret-cert.pem: internal/data/ascend/buildkitd
|
||||
vault.hashicorp.com/agent-inject-secret-config.json: internal/data/ascend/buildkitd
|
||||
vault.hashicorp.com/agent-inject-secret-key.pem: internal/data/ascend/buildkitd
|
||||
vault.hashicorp.com/agent-inject-template-ca.pem: "{{- with secret \"internal/data/ascend/buildkitd\" -}}\n{{ .Data.data.RootCA }}\n{{- end }}"
|
||||
vault.hashicorp.com/agent-inject-template-cert.pem: "{{- with secret \"internal/data/ascend/buildkitd\" -}}\n{{ .Data.data.ClientCaCert }}\n{{- end }}"
|
||||
vault.hashicorp.com/agent-inject-template-config.json: "{{- with secret \"internal/data/ascend/buildkitd\" -}}\n{{ .Data.data.dockerConfig }}\n{{- end }}"
|
||||
vault.hashicorp.com/agent-inject-template-key.pem: "{{- with secret \"internal/data/ascend/buildkitd\" -}}\n{{ .Data.data.ClientCaKey }}\n{{- end }}"
|
||||
vault.hashicorp.com/agent-pre-populate-only: "true"
|
||||
vault.hashicorp.com/agent-run-as-group: "1000"
|
||||
vault.hashicorp.com/agent-run-as-user: "1000"
|
||||
vault.hashicorp.com/agent-service-account-token-volume-name: token-vol
|
||||
vault.hashicorp.com/role: ascend-gha-runners
|
||||
vault.hashicorp.com/secret-volume-path: /home/user/.docker/
|
||||
vault.hashicorp.com/tls-skip-verify: "true"
|
||||
podSpecPatch:
|
||||
volumes:
|
||||
- name: token-vol
|
||||
projected:
|
||||
defaultMode: 420
|
||||
sources:
|
||||
- serviceAccountToken:
|
||||
audience: api
|
||||
expirationSeconds: 600
|
||||
path: token
|
||||
env:
|
||||
VLLM_IMAGE_TAG: "${BUILDKITE_COMMIT}"
|
||||
IMAGE_NAME: "vllm-omni-ci-npu"
|
||||
IMAGE_REGISTRY: "swr.cn-southwest-2.myhuaweicloud.com/modelfoundry"
|
||||
BUILDKITD_ADDR: "tcp://buildkitd-service.buildkitd:1234"
|
||||
command: bash .buildkite/npu/scripts/build-npu-image.sh Dockerfile.npu.ci "$${VLLM_IMAGE_TAG}"
|
||||
|
||||
- label: ":buildkit: Build and Push NPU Test Image - A3"
|
||||
key: image-build-a3
|
||||
agents: *npu-image-build-agents
|
||||
plugins: *npu-image-build-plugins
|
||||
env:
|
||||
VLLM_IMAGE_TAG: "${BUILDKITE_COMMIT}-a3"
|
||||
IMAGE_NAME: "vllm-omni-ci-npu"
|
||||
IMAGE_REGISTRY: "swr.cn-southwest-2.myhuaweicloud.com/modelfoundry"
|
||||
BUILDKITD_ADDR: "tcp://buildkitd-service.buildkitd:1234"
|
||||
command: bash .buildkite/npu/scripts/build-npu-image.sh Dockerfile.npu.ci "$${VLLM_IMAGE_TAG}" -a3
|
||||
|
||||
- label: ":buildkit: Build and Push NPU Test Image - A5"
|
||||
key: image-build-a5
|
||||
agents: *npu-image-build-agents
|
||||
plugins: *npu-image-build-plugins
|
||||
env:
|
||||
VLLM_IMAGE_TAG: "${BUILDKITE_COMMIT}-a5"
|
||||
IMAGE_NAME: "vllm-omni-ci-npu"
|
||||
IMAGE_REGISTRY: "swr.cn-southwest-2.myhuaweicloud.com/modelfoundry"
|
||||
BUILDKITD_ADDR: "tcp://buildkitd-service.buildkitd:1234"
|
||||
command: bash .buildkite/npu/scripts/build-npu-image.sh Dockerfile.npu.ci "$${VLLM_IMAGE_TAG}" -a5
|
||||
|
||||
- label: ":buildkit: Build and Push NPU Test Image - 310P"
|
||||
key: image-build-310p
|
||||
agents: *npu-image-build-agents
|
||||
plugins: *npu-image-build-plugins
|
||||
env:
|
||||
VLLM_IMAGE_TAG: "${BUILDKITE_COMMIT}-310p"
|
||||
IMAGE_NAME: "vllm-omni-ci-npu"
|
||||
IMAGE_REGISTRY: "swr.cn-southwest-2.myhuaweicloud.com/modelfoundry"
|
||||
BUILDKITD_ADDR: "tcp://buildkitd-service.buildkitd:1234"
|
||||
command: bash .buildkite/npu/scripts/build-npu-image.sh Dockerfile.npu.ci "$${VLLM_IMAGE_TAG}" -310p
|
||||
|
||||
- label: "Upload Ready Pipeline"
|
||||
depends_on:
|
||||
- image-build-a2
|
||||
- image-build-a3
|
||||
- image-build-a5
|
||||
- image-build-310p
|
||||
key: upload-ready-pipeline
|
||||
agents:
|
||||
queue: "cpu_queue_premerge"
|
||||
command: |
|
||||
python3 .buildkite/common/scripts/upload_pipeline.py --upload .buildkite/npu/test-npu-ready.yml
|
||||
|
||||
# L4 Test — main+NIGHTLY=1 (scheduled), or PR nightly-test label
|
||||
- label: "Upload Nightly Pipeline"
|
||||
depends_on:
|
||||
- image-build-a2
|
||||
- image-build-a3
|
||||
- image-build-a5
|
||||
- image-build-310p
|
||||
key: upload-nightly-pipeline
|
||||
commands:
|
||||
- |
|
||||
EXTRA=""
|
||||
if [ "${NIGHTLY:-}" = "1" ] && [ "${BUILDKITE_BRANCH}" = "main" ]; then
|
||||
EXTRA="--all"
|
||||
fi
|
||||
python3 .buildkite/common/scripts/upload_pipeline.py --upload $$EXTRA .buildkite/npu/test-npu-nightly.yml
|
||||
agents:
|
||||
queue: "cpu_queue_premerge"
|
||||
@@ -1,104 +0,0 @@
|
||||
# steps:
|
||||
# - label: ":buildkit: Build and Push NPU Test Image - A3"
|
||||
# key: image-build-a3
|
||||
# agents:
|
||||
# queue: "ascend-a3"
|
||||
# resource_class: "npu-2"
|
||||
# plugins:
|
||||
# - kubernetes:
|
||||
# metadata:
|
||||
# annotations:
|
||||
# vault.hashicorp.com/agent-init-first: "true"
|
||||
# vault.hashicorp.com/agent-inject: "true"
|
||||
# vault.hashicorp.com/agent-inject-perms-ca.pem: "0400"
|
||||
# vault.hashicorp.com/agent-inject-perms-cert.pem: "0400"
|
||||
# vault.hashicorp.com/agent-inject-perms-config.json: "0400"
|
||||
# vault.hashicorp.com/agent-inject-perms-key.pem: "0400"
|
||||
# vault.hashicorp.com/agent-inject-secret-ca.pem: internal/data/ascend/buildkitd
|
||||
# vault.hashicorp.com/agent-inject-secret-cert.pem: internal/data/ascend/buildkitd
|
||||
# vault.hashicorp.com/agent-inject-secret-config.json: internal/data/ascend/buildkitd
|
||||
# vault.hashicorp.com/agent-inject-secret-key.pem: internal/data/ascend/buildkitd
|
||||
# vault.hashicorp.com/agent-inject-template-ca.pem: "{{- with secret \"internal/data/ascend/buildkitd\" -}}\n{{ .Data.data.RootCA }}\n{{- end }}"
|
||||
# vault.hashicorp.com/agent-inject-template-cert.pem: "{{- with secret \"internal/data/ascend/buildkitd\" -}}\n{{ .Data.data.ClientCaCert }}\n{{- end }}"
|
||||
# vault.hashicorp.com/agent-inject-template-config.json: "{{- with secret \"internal/data/ascend/buildkitd\" -}}\n{{ .Data.data.dockerConfig }}\n{{- end }}"
|
||||
# vault.hashicorp.com/agent-inject-template-key.pem: "{{- with secret \"internal/data/ascend/buildkitd\" -}}\n{{ .Data.data.ClientCaKey }}\n{{- end }}"
|
||||
# vault.hashicorp.com/agent-pre-populate-only: "true"
|
||||
# vault.hashicorp.com/agent-run-as-group: "1000"
|
||||
# vault.hashicorp.com/agent-run-as-user: "1000"
|
||||
# vault.hashicorp.com/agent-service-account-token-volume-name: token-vol
|
||||
# vault.hashicorp.com/role: ascend-gha-runners
|
||||
# vault.hashicorp.com/secret-volume-path: /home/user/.docker/
|
||||
# vault.hashicorp.com/tls-skip-verify: "true"
|
||||
# podSpecPatch:
|
||||
# volumes:
|
||||
# - name: token-vol
|
||||
# projected:
|
||||
# defaultMode: 420
|
||||
# sources:
|
||||
# - serviceAccountToken:
|
||||
# audience: api
|
||||
# expirationSeconds: 600
|
||||
# path: token
|
||||
# env:
|
||||
# VLLM_IMAGE_TAG: "${BUILDKITE_COMMIT}"
|
||||
# IMAGE_NAME: "vllm-omni-ci-npu"
|
||||
# IMAGE_REGISTRY: "swr.cn-southwest-2.myhuaweicloud.com/modelfoundry"
|
||||
# BUILDKITD_ADDR: "tcp://buildkitd-service.buildkitd:1234"
|
||||
# command: |
|
||||
# set -ex
|
||||
|
||||
# echo "--- Building and pushing NPU Test Image"
|
||||
# echo "Image: $${IMAGE_REGISTRY}/$${IMAGE_NAME}:$${VLLM_IMAGE_TAG}"
|
||||
# echo "buildkitd address: $${BUILDKITD_ADDR}"
|
||||
|
||||
# git config --global url."https://gh-proxy.test.osinfra.cn/https://github.com/".insteadOf "https://github.com/"
|
||||
|
||||
# if ! command -v buildctl &> /dev/null; then
|
||||
# echo "Installing buildctl..."
|
||||
# mkdir -p /tmp/buildkit
|
||||
# BUILDKIT_VERSION="v0.29.0"
|
||||
# wget -q "https://gh-proxy.test.osinfra.cn/https://github.com/moby/buildkit/releases/download/$${BUILDKIT_VERSION}/buildkit-$${BUILDKIT_VERSION}.linux-arm64.tar.gz" -O /tmp/buildkit.tar.gz
|
||||
# tar -xzf /tmp/buildkit.tar.gz -C /tmp/buildkit
|
||||
# cp /tmp/buildkit/bin/buildctl /usr/local/bin/
|
||||
# fi
|
||||
|
||||
# export DOCKER_CONFIG=/home/user/.docker
|
||||
# buildctl \
|
||||
# --addr="$${BUILDKITD_ADDR}" \
|
||||
# --tlscacert=/home/user/.docker/ca.pem \
|
||||
# --tlscert=/home/user/.docker/cert.pem \
|
||||
# --tlskey=/home/user/.docker/key.pem \
|
||||
# build \
|
||||
# --frontend dockerfile.v0 \
|
||||
# --local context=. \
|
||||
# --local dockerfile=./docker \
|
||||
# --opt filename=Dockerfile.npu.ci.a3 \
|
||||
# --secret id=dockerconfig,src=/home/user/.docker/config.json \
|
||||
# --output type=image,name=$${IMAGE_REGISTRY}/$${IMAGE_NAME}:$${VLLM_IMAGE_TAG},push=true \
|
||||
# --progress=plain
|
||||
|
||||
# echo "--- Image pushed successfully"
|
||||
# echo "$${IMAGE_REGISTRY}/$${IMAGE_NAME}:$${VLLM_IMAGE_TAG}"
|
||||
|
||||
# # - label: "Upload Ready Pipeline - A3"
|
||||
# # depends_on: image-build-a3
|
||||
# # key: upload-ready-pipeline-a3
|
||||
# # if: build.branch != "main" && build.pull_request.labels includes "npu-test"
|
||||
# # agents:
|
||||
# # queue: "ascend-a3"
|
||||
# # resource_class: "npu-2"
|
||||
# # command: |
|
||||
# # buildkite-agent pipeline upload .buildkite/npu/test-npu-ready.yml
|
||||
|
||||
# # L4 Test — main+NIGHTLY=1 (scheduled), or PR with specific label (e.g. add label then Rebuild)
|
||||
# - label: "Upload Nightly Pipeline"
|
||||
# depends_on: image-build
|
||||
# key: upload-nightly-pipeline
|
||||
# if: >-
|
||||
# (build.branch == "main" && build.env("NIGHTLY") == "1") ||
|
||||
# (build.branch != "main" && build.pull_request.labels includes "nightly-test")
|
||||
# commands:
|
||||
# - buildkite-agent pipeline upload .buildkite/npu/test-npu-nightly.yml
|
||||
# agents:
|
||||
# queue: "ascend-a3"
|
||||
# resource_class: "npu-2"
|
||||
@@ -1,9 +0,0 @@
|
||||
# Hook uploads this entry YAML; upload_pipeline.py --upload reads bootstrap-upload-steps.yml
|
||||
# and injects ``if`` by step ``key`` from skip-ci.
|
||||
steps:
|
||||
- label: ":github: Resolve skip-ci & upload pipeline"
|
||||
key: upload-ci-pipeline
|
||||
commands:
|
||||
- "python3 .buildkite/common/scripts/upload_pipeline.py --upload .buildkite/npu/bootstrap-upload-steps.yml"
|
||||
agents:
|
||||
queue: "cpu_queue_premerge"
|
||||
@@ -1,48 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -ex
|
||||
|
||||
dockerfile=${1:?Dockerfile is required}
|
||||
image_tag=${2:?Image tag is required}
|
||||
device_suffix=${3:-}
|
||||
|
||||
image_name=${IMAGE_NAME:-vllm-omni-ci-npu}
|
||||
image_registry=${IMAGE_REGISTRY:-swr.cn-southwest-2.myhuaweicloud.com/modelfoundry}
|
||||
buildkitd_addr=${BUILDKITD_ADDR:-tcp://buildkitd-service.buildkitd:1234}
|
||||
|
||||
echo "--- Building and pushing NPU Test Image"
|
||||
echo "Image: ${image_registry}/${image_name}:${image_tag}"
|
||||
echo "Dockerfile: ${dockerfile}"
|
||||
|
||||
if ! command -v buildctl &> /dev/null; then
|
||||
mkdir -p /tmp/buildkit
|
||||
buildkit_version=v0.29.0
|
||||
wget -q \
|
||||
"https://github.com/moby/buildkit/releases/download/${buildkit_version}/buildkit-${buildkit_version}.linux-arm64.tar.gz" \
|
||||
-O /tmp/buildkit.tar.gz
|
||||
tar -xzf /tmp/buildkit.tar.gz -C /tmp/buildkit
|
||||
cp /tmp/buildkit/bin/buildctl /usr/local/bin/
|
||||
fi
|
||||
|
||||
build_args=()
|
||||
if [[ -n "${device_suffix}" ]]; then
|
||||
build_args+=(--opt "build-arg:VLLM_ASCEND_DEVICE_SUFFIX=${device_suffix}")
|
||||
fi
|
||||
|
||||
export DOCKER_CONFIG=/home/user/.docker
|
||||
buildctl \
|
||||
--addr="${buildkitd_addr}" \
|
||||
--tlscacert=/home/user/.docker/ca.pem \
|
||||
--tlscert=/home/user/.docker/cert.pem \
|
||||
--tlskey=/home/user/.docker/key.pem \
|
||||
build \
|
||||
--frontend dockerfile.v0 \
|
||||
--local context=. \
|
||||
--local dockerfile=./docker \
|
||||
--opt "filename=${dockerfile}" \
|
||||
"${build_args[@]}" \
|
||||
--secret id=dockerconfig,src=/home/user/.docker/config.json \
|
||||
--output "type=image,name=${image_registry}/${image_name}:${image_tag},push=true" \
|
||||
--progress=plain
|
||||
|
||||
echo "--- Image pushed successfully"
|
||||
echo "${image_registry}/${image_name}:${image_tag}"
|
||||
@@ -1,354 +0,0 @@
|
||||
steps:
|
||||
- group: ":card_index_dividers: Omni Model Test"
|
||||
key: nightly-omni-test-group
|
||||
depends_on: upload-nightly-pipeline
|
||||
if: >-
|
||||
build.env("NIGHTLY") == "1" ||
|
||||
build.pull_request.labels includes "nightly-test"
|
||||
steps:
|
||||
- label: ":full_moon: Omni · Perf Test · No Async Chunk"
|
||||
source_file_dependencies: omni_qwen3_omni_perf
|
||||
key: nightly-omni-performance-no-async-chunk
|
||||
timeout_in_minutes: 180
|
||||
mirror_hardwares: a3_npu_4
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
|
||||
HF_TOKEN: "${HF_TOKEN}"
|
||||
commands:
|
||||
- export BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- |
|
||||
set +e
|
||||
pytest -s -v tests/dfx/perf/scripts/run_benchmark.py -m npu --test-config-file tests/dfx/perf/tests/test_qwen3_omni_no_async_chunk.json
|
||||
EXIT=$$?
|
||||
buildkite-agent artifact upload "tests/dfx/perf/results/*.json"
|
||||
exit $$EXIT
|
||||
- label: ":full_moon: Omni · Perf Test · Async Chunk"
|
||||
source_file_dependencies: omni_qwen3_omni_perf
|
||||
key: nightly-omni-performance-async-chunk
|
||||
timeout_in_minutes: 180
|
||||
mirror_hardwares: a3_npu_4
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
|
||||
HF_TOKEN: "${HF_TOKEN}"
|
||||
commands:
|
||||
- export BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- |
|
||||
set +e
|
||||
pytest -s -v tests/dfx/perf/scripts/run_benchmark.py -m npu --test-config-file tests/dfx/perf/tests/test_qwen3_omni_async_chunk.json
|
||||
EXIT=$$?
|
||||
buildkite-agent artifact upload "tests/dfx/perf/results/*.json"
|
||||
exit $$EXIT
|
||||
- label: ":full_moon: Omni · MiniCPM-o 4.5 Test"
|
||||
source_file_dependencies: omni_minicpmo_4_5_function
|
||||
mirror_hardwares: a3_npu_2
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
|
||||
HF_TOKEN: "${HF_TOKEN}"
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/offline_inference/test_minicpmo_4_5.py tests/e2e/online_serving/test_minicpmo_4_5.py tests/e2e/online_serving/test_minicpmo_4_5_expansion.py -m 'full_model' --run-level 'full_model'
|
||||
- label: ":full_moon: Omni · MiniCPM-o 4.5 · Duplex Test"
|
||||
source_file_dependencies: omni_minicpmo_4_5_duplex_function
|
||||
timeout_in_minutes: 180
|
||||
mirror_hardwares: a3_npu_2
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
|
||||
HF_TOKEN: "${HF_TOKEN}"
|
||||
commands:
|
||||
- >-
|
||||
pytest -s -v
|
||||
tests/e2e/online_serving/test_minicpmo_4_5_duplex.py
|
||||
tests/e2e/online_serving/test_minicpmo_4_5_duplex_expansion.py
|
||||
-m "npu and A3 and omni"
|
||||
--run-level full_model
|
||||
- >-
|
||||
pytest -s -v
|
||||
tests/e2e/online_serving/test_minicpmo_4_5_window.py
|
||||
-m "full_model and npu and A3 and omni"
|
||||
--run-level full_model
|
||||
- label: ":full_moon: Omni · MiniCPM-o 4.5 · Accuracy Test"
|
||||
source_file_dependencies: omni_minicpmo_4_5_accuracy
|
||||
timeout_in_minutes: 180
|
||||
mirror_hardwares: a3_npu_2
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
|
||||
HF_TOKEN: "${HF_TOKEN}"
|
||||
commands:
|
||||
# Pin the Seed-TTS scorer to the second card: _get_eval_device() otherwise
|
||||
# auto-picks npu:0, which the omni server under test already fills with its
|
||||
# KV cache, so Whisper OOMs on `.to(device)` and every clip reports
|
||||
# "ASR / WER failed".
|
||||
- export SEED_TTS_WER_EVAL=1
|
||||
- export SEED_TTS_EVAL_DEVICE=npu:1
|
||||
- |
|
||||
set +e
|
||||
pytest -s -v tests/e2e/accuracy/minicpmo_4_5/test_minicpmo_4_5.py -m "full_model and npu and A3" --run-level full_model
|
||||
EXIT=$$?
|
||||
buildkite-agent artifact upload "tests/e2e/accuracy/minicpmo_4_5/results/*.json"
|
||||
exit $$EXIT
|
||||
- label: ":full_moon: Omni · Omni-DuplexEval CI Guard"
|
||||
source_file_dependencies: omni_duplex_eval_ci
|
||||
timeout_in_minutes: 240
|
||||
mirror_hardwares: a3_npu_2
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
|
||||
HF_TOKEN: "${HF_TOKEN}"
|
||||
VLLM_DUPLEX_EVAL_JUDGE_URL: "http://127.0.0.1:8001"
|
||||
commands:
|
||||
- |
|
||||
set +e
|
||||
pytest -s -v tests/e2e/accuracy/omni_duplex_eval/test_omni_duplex_eval_ci.py \
|
||||
-m "full_model and npu and A3 and cards_2" \
|
||||
--run-level full_model \
|
||||
--basetemp /tmp/omni_duplex_ci
|
||||
EXIT=$$?
|
||||
buildkite-agent artifact upload "tests/e2e/accuracy/omni_duplex_eval/results/*.json"
|
||||
exit $$EXIT
|
||||
- label: ":full_moon: Omni · MiniCPM-o 4.5 · Perf Test"
|
||||
source_file_dependencies: omni_minicpmo_4_5_perf
|
||||
key: nightly-omni-performance-minicpmo-4-5-npu
|
||||
timeout_in_minutes: 180
|
||||
mirror_hardwares: a3_npu_2
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
|
||||
HF_TOKEN: "${HF_TOKEN}"
|
||||
commands:
|
||||
- export BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- |
|
||||
set +e
|
||||
pytest -s -v tests/dfx/perf/scripts/run_benchmark.py -m "npu and A3" --test-config-file tests/dfx/perf/tests/test_minicpmo_4_5.json
|
||||
EXIT=$$?
|
||||
buildkite-agent artifact upload "tests/dfx/perf/results/*.json"
|
||||
exit $$EXIT
|
||||
- label: ":full_moon: Omni · MiniCPM-o 4.5 · Perf Test (A2)"
|
||||
source_file_dependencies: omni_minicpmo_4_5_perf
|
||||
key: nightly-omni-performance-minicpmo-4-5-npu-a2
|
||||
timeout_in_minutes: 180
|
||||
mirror_hardwares: a2b3_npu_1
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
|
||||
HF_TOKEN: "${HF_TOKEN}"
|
||||
HF_HUB_OFFLINE: "1"
|
||||
commands:
|
||||
- export BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- |
|
||||
set +e
|
||||
pytest -s -v tests/dfx/perf/scripts/run_benchmark.py -m "npu and A2" --test-config-file tests/dfx/perf/tests/test_minicpmo_4_5.json
|
||||
EXIT=$$?
|
||||
buildkite-agent artifact upload "tests/dfx/perf/results/*.json"
|
||||
exit $$EXIT
|
||||
- label: ":full_moon: Omni · MiniCPM-o 4.5 · Duplex Seed-TTS Perf Test"
|
||||
source_file_dependencies: omni_minicpmo_4_5_duplex_perf
|
||||
key: nightly-omni-performance-minicpmo-4-5-duplex-seed-tts-npu
|
||||
timeout_in_minutes: 180
|
||||
mirror_hardwares: a3_npu_2
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
|
||||
HF_TOKEN: "${HF_TOKEN}"
|
||||
commands:
|
||||
- export BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- |
|
||||
set +e
|
||||
pytest -s -v tests/dfx/perf/scripts/run_benchmark.py -m "npu and A3" --test-config-file tests/dfx/perf/tests/test_minicpmo_4_5_duplex_seed_tts.json
|
||||
EXIT=$$?
|
||||
buildkite-agent artifact upload "tests/dfx/perf/results/*.json"
|
||||
exit $$EXIT
|
||||
- group: ":card_index_dividers: TTS Model Test"
|
||||
key: nightly-tts-test-group
|
||||
depends_on: upload-nightly-pipeline
|
||||
if: build.env("NIGHTLY") == "1" || build.pull_request.labels includes "nightly-test"
|
||||
steps:
|
||||
- label: ":full_moon: TTS · Perf Test"
|
||||
source_file_dependencies: tts_qwen3_tts_perf
|
||||
timeout_in_minutes: 180
|
||||
mirror_hardwares: a3_npu_2
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
|
||||
HF_TOKEN: "${HF_TOKEN}"
|
||||
commands:
|
||||
- export BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
|
||||
- |
|
||||
set +e
|
||||
pytest -s -v tests/dfx/perf/scripts/run_benchmark.py -m npu --test-config-file tests/dfx/perf/tests/test_tts.json
|
||||
EXIT=$$?
|
||||
buildkite-agent artifact upload "tests/dfx/perf/results/*.json"
|
||||
exit $$EXIT
|
||||
- group: ":card_index_dividers: Diffusion X2I(&A&T) Model Test"
|
||||
key: nightly-diffusion-x2iat-group
|
||||
depends_on: upload-nightly-pipeline
|
||||
if: >-
|
||||
build.env("NIGHTLY") == "1" ||
|
||||
build.pull_request.labels includes "nightly-test"
|
||||
steps:
|
||||
- label: ":full_moon: HunyuanImage3-DIT · Accuracy Test"
|
||||
source_file_dependencies: diffusion_hunyuan_image3_accuracy
|
||||
timeout_in_minutes: 180
|
||||
mirror_hardwares: a3_npu_4
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
HUNYUAN_IMAGE3_MODEL: "tencent/HunyuanImage-3.0-Instruct"
|
||||
HUNYUAN_IMAGE3_DEVICES: "0,1,2,3"
|
||||
DIFFUSION_ATTENTION_BACKEND: TORCH_SDPA
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/accuracy/test_hunyuan_image3_pixel_accuracy.py -m "full_model and A3" --run-level full_model
|
||||
- label: ":full_moon: Diffusion Perf · HunyuanImage3 TP=2 CFGP=2"
|
||||
source_file_dependencies: diffusion_hunyuan_image3_perf
|
||||
timeout_in_minutes: 180
|
||||
mirror_hardwares: a3_npu_4
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
HF_TOKEN: "${HF_TOKEN}"
|
||||
DIFFUSION_ATTENTION_BACKEND: TORCH_SDPA
|
||||
commands:
|
||||
- export DIFFUSION_BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- |
|
||||
set +e
|
||||
pytest -s -v tests/dfx/perf/scripts/run_diffusion_benchmark.py -m A3 --test-config-file tests/dfx/perf/tests/test_hunyuan_image_tp2_cfgp2.json
|
||||
EXIT=$$?
|
||||
buildkite-agent artifact upload "tests/dfx/perf/results/diffusion_result_*.json"
|
||||
buildkite-agent artifact upload "tests/dfx/perf/results/logs/*.log"
|
||||
exit $$EXIT
|
||||
- label: ":full_moon: Diffusion Perf · HunyuanImage3 TP=2 SP=2"
|
||||
source_file_dependencies: diffusion_hunyuan_image3_perf
|
||||
timeout_in_minutes: 180
|
||||
mirror_hardwares: a3_npu_4
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
HF_TOKEN: "${HF_TOKEN}"
|
||||
DIFFUSION_ATTENTION_BACKEND: TORCH_SDPA
|
||||
commands:
|
||||
- export DIFFUSION_BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- |
|
||||
set +e
|
||||
pytest -s -v tests/dfx/perf/scripts/run_diffusion_benchmark.py -m A3 --test-config-file tests/dfx/perf/tests/test_hunyuan_image_tp2_sp2.json
|
||||
EXIT=$$?
|
||||
buildkite-agent artifact upload "tests/dfx/perf/results/diffusion_result_*.json"
|
||||
buildkite-agent artifact upload "tests/dfx/perf/results/logs/*.log"
|
||||
exit $$EXIT
|
||||
- label: ":full_moon: Diffusion Perf · HunyuanImage3 TP=4"
|
||||
source_file_dependencies: diffusion_hunyuan_image3_perf
|
||||
timeout_in_minutes: 180
|
||||
mirror_hardwares: a3_npu_4
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
HF_TOKEN: "${HF_TOKEN}"
|
||||
DIFFUSION_ATTENTION_BACKEND: TORCH_SDPA
|
||||
commands:
|
||||
- export DIFFUSION_BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- |
|
||||
set +e
|
||||
pytest -s -v tests/dfx/perf/scripts/run_diffusion_benchmark.py -m A3 --test-config-file tests/dfx/perf/tests/test_hunyuan_image_tp4.json
|
||||
EXIT=$$?
|
||||
buildkite-agent artifact upload "tests/dfx/perf/results/diffusion_result_*.json"
|
||||
buildkite-agent artifact upload "tests/dfx/perf/results/logs/*.log"
|
||||
exit $$EXIT
|
||||
- label: ":full_moon: Diffusion X2I(&A&T) · Perf Test · Qwen-Image · Single-NPU"
|
||||
source_file_dependencies: diffusion_qwen_image_perf
|
||||
key: nightly-diffusion-x2iat-performance-qwen-image-single-npu
|
||||
timeout_in_minutes: 180
|
||||
mirror_hardwares: a3_npu_2
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
HF_TOKEN: "${HF_TOKEN}"
|
||||
DIFFUSION_ATTENTION_BACKEND: TORCH_SDPA
|
||||
commands:
|
||||
- export DIFFUSION_BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- |
|
||||
set +e
|
||||
pytest -s -v tests/dfx/perf/scripts/run_diffusion_benchmark.py -m "npu and A3 and cards_1" --test-config-file tests/dfx/perf/tests/test_qwen_image_vllm_omni.json
|
||||
EXIT=$$?
|
||||
buildkite-agent artifact upload "tests/dfx/perf/results/diffusion_result_*.json"
|
||||
buildkite-agent artifact upload "tests/dfx/perf/results/logs/*.log"
|
||||
exit $$EXIT
|
||||
- label: ":full_moon: Diffusion X2I(&A&T) · Perf Test · Qwen-Image · 4-NPU"
|
||||
source_file_dependencies: diffusion_qwen_image_perf
|
||||
key: nightly-diffusion-x2iat-performance-qwen-image-4npu
|
||||
timeout_in_minutes: 180
|
||||
mirror_hardwares: a3_npu_4
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
HF_TOKEN: "${HF_TOKEN}"
|
||||
DIFFUSION_ATTENTION_BACKEND: TORCH_SDPA
|
||||
commands:
|
||||
- export DIFFUSION_BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- |
|
||||
set +e
|
||||
pytest -s -v tests/dfx/perf/scripts/run_diffusion_benchmark.py -m "npu and A3 and cards_4" --test-config-file tests/dfx/perf/tests/test_qwen_image_vllm_omni.json
|
||||
EXIT=$$?
|
||||
buildkite-agent artifact upload "tests/dfx/perf/results/diffusion_result_*.json"
|
||||
buildkite-agent artifact upload "tests/dfx/perf/results/logs/*.log"
|
||||
exit $$EXIT
|
||||
- group: ":card_index_dividers: Diffusion X2V Model Test"
|
||||
key: nightly-diffusion-x2v-group
|
||||
depends_on: upload-nightly-pipeline
|
||||
if: >-
|
||||
build.env("NIGHTLY") == "1" ||
|
||||
build.pull_request.labels includes "nightly-test"
|
||||
steps:
|
||||
- label: ":full_moon: Diffusion X2V · HunyuanVideo-1.5 Function Test"
|
||||
source_file_dependencies: diffusion_hunyuan_video_function
|
||||
timeout_in_minutes: 180
|
||||
mirror_hardwares: a2b3_npu_4
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
|
||||
HF_TOKEN: "${HF_TOKEN}"
|
||||
HF_HUB_OFFLINE: "1"
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/online_serving/test_hunyuan_video_15_expansion.py -m "npu and A2" --run-level "full_model"
|
||||
- label: ":full_moon: Diffusion X2V · HunyuanVideo-1.5 Perf Test"
|
||||
source_file_dependencies: diffusion_hunyuan_video_perf
|
||||
key: nightly-diffusion-x2v-performance-hunyuanvideo15
|
||||
timeout_in_minutes: 180
|
||||
mirror_hardwares: a3_npu_2
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
|
||||
HF_TOKEN: "${HF_TOKEN}"
|
||||
HF_HUB_OFFLINE: "1"
|
||||
DIFFUSION_ATTENTION_BACKEND: TORCH_SDPA
|
||||
commands:
|
||||
- export BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- |
|
||||
set +e
|
||||
pytest -s -v tests/dfx/perf/scripts/run_benchmark.py -m "npu and A3" --test-config-file tests/dfx/perf/tests/test_hunyuanvideo15_t2v_vllm_omni.json
|
||||
EXIT=$$?
|
||||
buildkite-agent artifact upload "tests/dfx/perf/results/*.json"
|
||||
exit $$EXIT
|
||||
- label: ":full_moon: Diffusion X2V · Wan 2.2 Function Test"
|
||||
source_file_dependencies: diffusion_wan22_function
|
||||
mirror_hardwares: a2b3_npu_4
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
|
||||
HF_TOKEN: "${HF_TOKEN}"
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/online_serving/test_wan22_expansion.py -k npu_i2v --run-level "full_model"
|
||||
- label: ":full_moon: Diffusion X2V · Wan 2.2 Perf Test"
|
||||
source_file_dependencies: diffusion_wan22_perf
|
||||
key: nightly-diffusion-x2v-performance
|
||||
timeout_in_minutes: 180
|
||||
mirror_hardwares: a3_npu_4
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
|
||||
HF_TOKEN: "${HF_TOKEN}"
|
||||
DIFFUSION_ATTENTION_BACKEND: TORCH_SDPA
|
||||
commands:
|
||||
- export BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- |
|
||||
set +e
|
||||
pytest -s -v tests/dfx/perf/scripts/run_benchmark.py -m npu --test-config-file tests/dfx/perf/tests/test_wan22_i2v_vllm_omni.json
|
||||
EXIT=$$?
|
||||
buildkite-agent artifact upload "tests/dfx/perf/results/*.json"
|
||||
exit $$EXIT
|
||||
@@ -1,156 +0,0 @@
|
||||
steps:
|
||||
- group: ":card_index_dividers: Simple Test"
|
||||
depends_on: upload-ready-pipeline
|
||||
steps:
|
||||
- label: "Platforms Test"
|
||||
key: ready-platforms-test
|
||||
mirror_hardwares: a3_npu_2
|
||||
commands:
|
||||
- >-
|
||||
pytest -s -v
|
||||
tests/platforms/npu/test_minicpmo_code2wav_npugraph.py
|
||||
-m 'core_model and npu and A3 and omni'
|
||||
--run-level 'core_model'
|
||||
|
||||
- label: "Diffusion NPU layer tests - A2"
|
||||
key: ready-diffusion-npu-layer-tests-a2
|
||||
mirror_hardwares: a2b3_npu_1
|
||||
commands:
|
||||
- >-
|
||||
pytest -s -v
|
||||
tests/diffusion/layers/test_activation.py
|
||||
-m 'core_model and npu and A2'
|
||||
--run-level 'core_model'
|
||||
|
||||
- label: "Diffusion NPU layer tests - A3"
|
||||
key: ready-diffusion-npu-layer-tests
|
||||
mirror_hardwares: a3_npu_2
|
||||
commands:
|
||||
- >-
|
||||
pytest -s -v
|
||||
tests/diffusion/layers/test_activation.py
|
||||
tests/diffusion/layers/test_fused_qk_norm_rope_npu.py
|
||||
tests/diffusion/models/minimax_h3/test_minimax_h3_qwen3vl_swiglu.py
|
||||
tests/diffusion/models/minimax_h3/test_minimax_h3_modulation.py
|
||||
-m 'core_model and npu and A3'
|
||||
--run-level 'core_model'
|
||||
|
||||
- label: "Diffusion NPU layer tests - A5"
|
||||
key: ready-diffusion-npu-layer-tests-a5
|
||||
mirror_hardwares: a5_npu_2
|
||||
commands:
|
||||
- >-
|
||||
pytest -s -v
|
||||
tests/diffusion/layers/test_activation.py
|
||||
-m 'core_model and npu and A5'
|
||||
--run-level 'core_model'
|
||||
|
||||
- label: "Diffusion NPU layer tests - 310P"
|
||||
key: ready-diffusion-npu-layer-tests-310p
|
||||
mirror_hardwares: 310p_npu_1
|
||||
commands:
|
||||
- >-
|
||||
pytest -s -v
|
||||
tests/diffusion/layers/test_activation.py
|
||||
-m 'core_model and npu and 310P'
|
||||
--run-level 'core_model'
|
||||
|
||||
- group: ":card_index_dividers: E2E Test"
|
||||
key: ready-e2e-group
|
||||
depends_on: upload-ready-pipeline
|
||||
steps:
|
||||
- label: "Omni · MiniCPM-o 4.5 Test"
|
||||
mirror_hardwares: a3_npu_2
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/offline_inference/test_minicpmo_4_5.py tests/e2e/online_serving/test_minicpmo_4_5.py -m 'core_model and npu and A3 and omni' --run-level 'core_model'
|
||||
|
||||
- label: "Omni · MiniCPM-o 4.5 Duplex Test"
|
||||
mirror_hardwares: a3_npu_2
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/online_serving/test_minicpmo_4_5_duplex.py -m 'core_model and npu and A3 and omni' --run-level 'core_model'
|
||||
|
||||
- label: "Omni · MiniCPM-o 4.5 Perf Test"
|
||||
timeout_in_minutes: 60
|
||||
mirror_hardwares: a3_npu_2
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
|
||||
commands:
|
||||
- export BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_minicpmo_4_5_ready.json -m 'core_model and npu and A3 and omni'
|
||||
|
||||
- label: "Omni · Qwen3-Omni Test"
|
||||
mirror_hardwares: a3_npu_4
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/online_serving/test_qwen3_omni.py -m 'core_model and npu and A3 and omni' --run-level 'core_model'
|
||||
timeout_in_minutes: 60
|
||||
|
||||
- label: ":sound: TTS · Qwen3-TTS CustomVoice Test"
|
||||
mirror_hardwares: a3_npu_2
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
|
||||
HF_TOKEN: "${HF_TOKEN}"
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/online_serving/test_qwen3_tts_customvoice.py -m 'core_model and npu' --run-level 'core_model'
|
||||
|
||||
- label: "Diffusion · HunyuanVideo-1.5 Test"
|
||||
mirror_hardwares: a3_npu_2
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
|
||||
commands:
|
||||
- |
|
||||
timeout 40m bash -c "
|
||||
export VLLM_IMAGE_FETCH_TIMEOUT=60
|
||||
pytest -s -v tests/e2e/online_serving/test_hunyuan_video_15_expansion.py -m 'core_model and diffusion and npu' --run-level 'core_model'
|
||||
"
|
||||
timeout_in_minutes: 60
|
||||
|
||||
- label: "Diffusion · Qwen Image Test"
|
||||
mirror_hardwares: a3_npu_2
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
|
||||
commands:
|
||||
- |
|
||||
timeout 40m bash -c "
|
||||
export VLLM_IMAGE_FETCH_TIMEOUT=60
|
||||
pytest -s -v tests/e2e/online_serving/test_qwen_image.py -m 'core_model and npu' --run-level 'core_model'
|
||||
"
|
||||
timeout_in_minutes: 60
|
||||
|
||||
- label: "Diffusion · Wan22 Test"
|
||||
mirror_hardwares: a3_npu_2
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
|
||||
commands:
|
||||
- |
|
||||
timeout 40m bash -c "
|
||||
export VLLM_IMAGE_FETCH_TIMEOUT=60
|
||||
pytest -s -v tests/e2e/online_serving/test_wan22_t2v.py -m 'core_model and npu' --run-level 'core_model'
|
||||
"
|
||||
timeout_in_minutes: 60
|
||||
|
||||
- label: "Diffusion · Hunyuan Image Test"
|
||||
mirror_hardwares: a3_npu_4
|
||||
env:
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"
|
||||
commands:
|
||||
- |
|
||||
timeout 40m bash -c "
|
||||
export VLLM_IMAGE_FETCH_TIMEOUT=60
|
||||
pytest -s -v tests/e2e/online_serving/test_hunyuan_image.py -m 'core_model and diffusion and npu' --run-level 'core_model'
|
||||
"
|
||||
timeout_in_minutes: 60
|
||||
@@ -1,16 +1,16 @@
|
||||
# .buildkite/intel/pipeline-intel.yml
|
||||
# .buildkite/pipeline-intel.yaml
|
||||
steps:
|
||||
- group: "INTEL Tests"
|
||||
steps:
|
||||
- label: "Intel: XPU Test"
|
||||
command: "bash .buildkite/intel/scripts/run-xpu-test.sh"
|
||||
command: "bash .buildkite/scripts/hardware_ci/run-xpu-test.sh"
|
||||
agents:
|
||||
queue: "intel-gpu-omni"
|
||||
env:
|
||||
DOCKER_BUILDKIT: "1"
|
||||
# Buildkite will automatically replace this with the actual commit hash
|
||||
VLLM_IMAGE_TAG: "${BUILDKITE_COMMIT}"
|
||||
VLLM_VERSION: "v0.31.0"
|
||||
VLLM_VERSION: "v0.18.0"
|
||||
priority: 100
|
||||
timeout_in_minutes: 60
|
||||
soft_fail: false
|
||||
soft_fail: true
|
||||
@@ -0,0 +1,40 @@
|
||||
steps:
|
||||
- label: ":docker: Build image"
|
||||
key: image-build
|
||||
commands:
|
||||
- "aws ecr-public get-login-password --region us-east-1 | docker login --username AWS --password-stdin public.ecr.aws/q9t5s3a7"
|
||||
- "docker build --progress=plain --file docker/Dockerfile.ci -t vllm-omni-ci ."
|
||||
- "docker tag vllm-omni-ci public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT"
|
||||
- "docker push public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT"
|
||||
agents:
|
||||
queue: "cpu_queue_premerge"
|
||||
|
||||
# L2 Test
|
||||
- label: "Upload Ready Pipeline"
|
||||
depends_on: image-build
|
||||
key: upload-ready-pipeline
|
||||
if: build.branch != "main" && build.pull_request.labels includes "ready"
|
||||
commands:
|
||||
- buildkite-agent pipeline upload .buildkite/test-ready.yml
|
||||
agents:
|
||||
queue: "cpu_queue_premerge"
|
||||
|
||||
# L3 Test
|
||||
- label: "Upload Merge Pipeline"
|
||||
depends_on: image-build
|
||||
key: upload-merge-pipeline
|
||||
if: build.branch == "main" && build.env("NIGHTLY") != "1"
|
||||
commands:
|
||||
- buildkite-agent pipeline upload .buildkite/test-merge.yml
|
||||
agents:
|
||||
queue: "cpu_queue_premerge"
|
||||
|
||||
# L4 Test — main+NIGHTLY=1 (scheduled), or PR with label nightly-test (e.g. add label then Rebuild)
|
||||
- label: "Upload Nightly Pipeline"
|
||||
depends_on: image-build
|
||||
key: upload-nightly-pipeline
|
||||
if: '(build.branch == "main" && build.env("NIGHTLY") == "1") || (build.branch != "main" && build.pull_request.labels includes "nightly-test")'
|
||||
commands:
|
||||
- buildkite-agent pipeline upload .buildkite/test-nightly.yml
|
||||
agents:
|
||||
queue: "cpu_queue_premerge"
|
||||
@@ -2,7 +2,10 @@ steps:
|
||||
- label: ":docker: Build image"
|
||||
key: image-build
|
||||
commands:
|
||||
- "bash .buildkite/cuda/scripts/build-ci-image.sh"
|
||||
- "aws ecr-public get-login-password --region us-east-1 | docker login --username AWS --password-stdin public.ecr.aws/q9t5s3a7"
|
||||
- "docker build --progress=plain --file docker/Dockerfile.ci -t vllm-omni-ci ."
|
||||
- "docker tag vllm-omni-ci public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT"
|
||||
- "docker push public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT"
|
||||
agents:
|
||||
queue: "cpu_queue_premerge"
|
||||
|
||||
@@ -11,7 +14,7 @@ steps:
|
||||
depends_on: image-build
|
||||
key: upload-ready-pipeline
|
||||
commands:
|
||||
- python3 .buildkite/common/scripts/upload_pipeline.py --upload --all .buildkite/cuda/test-ready.yml
|
||||
- buildkite-agent pipeline upload .buildkite/test-ready.yml
|
||||
agents:
|
||||
queue: "cpu_queue_premerge"
|
||||
|
||||
@@ -20,7 +23,7 @@ steps:
|
||||
depends_on: image-build
|
||||
key: upload-merge-pipeline
|
||||
commands:
|
||||
- python3 .buildkite/common/scripts/upload_pipeline.py --upload --all .buildkite/cuda/test-merge.yml
|
||||
- buildkite-agent pipeline upload .buildkite/test-merge.yml
|
||||
agents:
|
||||
queue: "cpu_queue_premerge"
|
||||
|
||||
@@ -30,6 +33,6 @@ steps:
|
||||
key: upload-nightly-pipeline
|
||||
if: build.env("NIGHTLY") == "1"
|
||||
commands:
|
||||
- python3 .buildkite/common/scripts/upload_pipeline.py --upload --all .buildkite/cuda/test-nightly.yml
|
||||
- buildkite-agent pipeline upload .buildkite/test-nightly.yml
|
||||
agents:
|
||||
queue: "cpu_queue_premerge"
|
||||
@@ -1,132 +0,0 @@
|
||||
steps:
|
||||
- input: "Provide Release version here"
|
||||
id: input-release-version
|
||||
fields:
|
||||
- text: "What is the release version?"
|
||||
key: release-version
|
||||
|
||||
- label: "Build and upload wheel"
|
||||
depends_on: ~
|
||||
key: "build-wheel"
|
||||
agents:
|
||||
queue: cpu_queue_release
|
||||
commands:
|
||||
- "curl -LsSf https://astral.sh/uv/install.sh | sh"
|
||||
- 'export PATH="$HOME/.local/bin:$PATH"'
|
||||
- "uv venv --python=3.12 && source .venv/bin/activate"
|
||||
- "uv pip install --upgrade build"
|
||||
- "python3 -m build"
|
||||
- "bash .buildkite/release/scripts/upload-nightly-wheels.sh"
|
||||
|
||||
- label: "Generate and upload wheel indices"
|
||||
depends_on: "build-wheel"
|
||||
allow_dependency_failure: true
|
||||
agents:
|
||||
queue: small_cpu_queue_release
|
||||
commands:
|
||||
- "bash .buildkite/release/scripts/generate-and-upload-nightly-index.sh"
|
||||
|
||||
- block: "Upload release wheels to PyPI"
|
||||
depends_on: "build-wheel"
|
||||
key: block-upload-release-wheels
|
||||
|
||||
- label: "Upload release wheels to PyPI"
|
||||
depends_on:
|
||||
- block-upload-release-wheels
|
||||
- input-release-version
|
||||
key: upload-release-wheels
|
||||
agents:
|
||||
queue: small_cpu_queue_release
|
||||
commands:
|
||||
- "bash .buildkite/release/scripts/publish-release-wheels-pypi.sh"
|
||||
|
||||
# Auto-skip when NIGHTLY=1 (scheduled); otherwise require manual unblock.
|
||||
- block: "Unblock to build release Docker images"
|
||||
depends_on: ~
|
||||
key: block-build-release-images
|
||||
if: build.env("NIGHTLY") != "1"
|
||||
|
||||
- group: "Build release Docker images"
|
||||
key: "build-release-images"
|
||||
depends_on: block-build-release-images
|
||||
allow_dependency_failure: true
|
||||
steps:
|
||||
- label: "Build release image - x86_64"
|
||||
key: build-release-image-x86
|
||||
depends_on: ~
|
||||
agents:
|
||||
queue: cpu_queue_release
|
||||
commands:
|
||||
- "aws ecr-public get-login-password --region us-east-1 | docker login --username AWS --password-stdin public.ecr.aws/q9t5s3a7"
|
||||
- |
|
||||
DOCKER_BUILDKIT=1 docker build \
|
||||
--build-arg max_jobs=16 \
|
||||
--tag public.ecr.aws/q9t5s3a7/vllm-omni-release-repo:$BUILDKITE_COMMIT-$(uname -m) \
|
||||
--progress plain \
|
||||
-f docker/Dockerfile.cuda .
|
||||
- "docker push public.ecr.aws/q9t5s3a7/vllm-omni-release-repo:$BUILDKITE_COMMIT-$(uname -m)"
|
||||
# Fallback default tag (no arch suffix) if arm64 build fails.
|
||||
- "docker tag public.ecr.aws/q9t5s3a7/vllm-omni-release-repo:$BUILDKITE_COMMIT-$(uname -m) public.ecr.aws/q9t5s3a7/vllm-omni-release-repo:$BUILDKITE_COMMIT"
|
||||
- "docker push public.ecr.aws/q9t5s3a7/vllm-omni-release-repo:$BUILDKITE_COMMIT"
|
||||
env:
|
||||
DOCKER_BUILDKIT: "1"
|
||||
|
||||
- label: "Build release image - aarch64"
|
||||
key: build-release-image-arm64
|
||||
depends_on: ~
|
||||
agents:
|
||||
queue: arm64_cpu_queue_release
|
||||
commands:
|
||||
- "aws ecr-public get-login-password --region us-east-1 | docker login --username AWS --password-stdin public.ecr.aws/q9t5s3a7"
|
||||
- |
|
||||
DOCKER_BUILDKIT=1 docker build \
|
||||
--build-arg max_jobs=16 \
|
||||
--tag public.ecr.aws/q9t5s3a7/vllm-omni-release-repo:$BUILDKITE_COMMIT-$(uname -m) \
|
||||
--progress plain \
|
||||
-f docker/Dockerfile.cuda .
|
||||
- "docker push public.ecr.aws/q9t5s3a7/vllm-omni-release-repo:$BUILDKITE_COMMIT-$(uname -m)"
|
||||
env:
|
||||
DOCKER_BUILDKIT: "1"
|
||||
|
||||
# NIGHTLY=1 only: promote ECR images to DockerHub nightly tags, then prune.
|
||||
- label: "Publish nightly multi-arch image to DockerHub"
|
||||
depends_on:
|
||||
- build-release-image-x86
|
||||
- build-release-image-arm64
|
||||
if: build.env("NIGHTLY") == "1"
|
||||
key: publish-nightly-images-dockerhub
|
||||
agents:
|
||||
queue: small_cpu_queue_release
|
||||
commands:
|
||||
- "bash .buildkite/release/scripts/publish-release-images.sh nightly"
|
||||
- "bash .buildkite/release/scripts/cleanup-nightly-builds.sh"
|
||||
plugins:
|
||||
- docker-login#v3.0.0:
|
||||
username: vllmbot
|
||||
password-env: DOCKERHUB_TOKEN
|
||||
env:
|
||||
DOCKER_BUILDKIT: "1"
|
||||
DOCKERHUB_USERNAME: "vllmbot"
|
||||
|
||||
- block: "Publish release images to DockerHub"
|
||||
key: block-publish-release-images
|
||||
depends_on:
|
||||
- build-release-image-x86
|
||||
- build-release-image-arm64
|
||||
- input-release-version
|
||||
|
||||
- label: "Publish release images to DockerHub"
|
||||
depends_on:
|
||||
- block-publish-release-images
|
||||
key: publish-release-images-dockerhub
|
||||
agents:
|
||||
queue: small_cpu_queue_release
|
||||
commands:
|
||||
- "bash .buildkite/release/scripts/publish-release-images.sh"
|
||||
plugins:
|
||||
- docker-login#v3.0.0:
|
||||
username: vllmbot
|
||||
password-env: DOCKERHUB_TOKEN
|
||||
env:
|
||||
DOCKER_BUILDKIT: "1"
|
||||
DOCKERHUB_USERNAME: "vllmbot"
|
||||
@@ -1,147 +0,0 @@
|
||||
#!/bin/bash
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
#
|
||||
# Clean up old nightly builds from DockerHub (vllm/vllm-omni), keeping the
|
||||
# newest 14 tags with the nightly- prefix.
|
||||
|
||||
set -ex
|
||||
|
||||
TAG_PREFIX="nightly-"
|
||||
REPO="vllm/vllm-omni"
|
||||
REPO_API_URL="https://hub.docker.com/v2/repositories/${REPO}/tags"
|
||||
|
||||
echo "Cleaning up tags with prefix: $TAG_PREFIX in repository: $REPO"
|
||||
|
||||
if [ -z "$DOCKERHUB_TOKEN" ]; then
|
||||
echo "Error: DOCKERHUB_TOKEN environment variable is not set"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ -z "$DOCKERHUB_USERNAME" ]; then
|
||||
echo "Error: DOCKERHUB_USERNAME environment variable is not set"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Getting DockerHub bearer token..."
|
||||
set +x
|
||||
BEARER_TOKEN=$(curl -s -X POST \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "{\"username\": \"$DOCKERHUB_USERNAME\", \"password\": \"$DOCKERHUB_TOKEN\"}" \
|
||||
"https://hub.docker.com/v2/users/login" | jq -r '.token')
|
||||
set -x
|
||||
|
||||
if [ -z "$BEARER_TOKEN" ] || [ "$BEARER_TOKEN" = "null" ]; then
|
||||
echo "Error: Failed to get DockerHub bearer token"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
get_all_tags() {
|
||||
local page=1
|
||||
local all_tags=""
|
||||
# Hub `name` is a partial match (not a strict prefix). Use it to narrow the
|
||||
# list, then enforce startswith() client-side. Exhaust API pages; do not
|
||||
# stop merely because a page has no matching tags after filtering.
|
||||
local name_query
|
||||
name_query=$(printf '%s' "$TAG_PREFIX" | jq -sRr @uri)
|
||||
|
||||
while true; do
|
||||
set +x
|
||||
local response
|
||||
response=$(curl -s -H "Authorization: Bearer $BEARER_TOKEN" \
|
||||
"$REPO_API_URL?page=$page&page_size=100&name=${name_query}")
|
||||
set -x
|
||||
|
||||
local page_count
|
||||
page_count=$(echo "$response" | jq -r '.results | length')
|
||||
if [ -z "$page_count" ] || [ "$page_count" = "0" ] || [ "$page_count" = "null" ]; then
|
||||
break
|
||||
fi
|
||||
|
||||
local tags
|
||||
tags=$(echo "$response" | jq -r --arg prefix "$TAG_PREFIX" \
|
||||
'.results[] | select(.name | startswith($prefix)) | "\(.last_updated)|\(.name)"')
|
||||
|
||||
if [ -n "$tags" ]; then
|
||||
all_tags="$all_tags$tags"$'\n'
|
||||
fi
|
||||
page=$((page + 1))
|
||||
done
|
||||
|
||||
echo "$all_tags" | sort -r | cut -d'|' -f2
|
||||
}
|
||||
|
||||
delete_tag() {
|
||||
local tag_name="$1"
|
||||
echo "Deleting tag: $tag_name"
|
||||
|
||||
local delete_url="https://hub.docker.com/v2/repositories/${REPO}/tags/$tag_name"
|
||||
set +x
|
||||
local response http_code body
|
||||
response=$(curl -s -w "\n%{http_code}" -X DELETE \
|
||||
-H "Authorization: Bearer $BEARER_TOKEN" "$delete_url")
|
||||
set -x
|
||||
http_code=$(printf '%s\n' "$response" | tail -n1)
|
||||
body=$(printf '%s\n' "$response" | sed '$d')
|
||||
|
||||
if [ "$http_code" -ge 200 ] && [ "$http_code" -lt 300 ]; then
|
||||
echo "Successfully deleted tag: $tag_name (HTTP ${http_code})"
|
||||
return 0
|
||||
fi
|
||||
|
||||
local detail=""
|
||||
if echo "$body" | jq -e '.detail' > /dev/null 2>&1; then
|
||||
detail=$(echo "$body" | jq -r '.detail')
|
||||
elif [ -n "$body" ]; then
|
||||
detail="$body"
|
||||
else
|
||||
detail="(empty response body)"
|
||||
fi
|
||||
echo "Error: Failed to delete tag $tag_name (HTTP ${http_code}): ${detail}"
|
||||
return 1
|
||||
}
|
||||
|
||||
echo "Fetching all tags from DockerHub..."
|
||||
all_tags=$(get_all_tags)
|
||||
|
||||
if [ -z "$all_tags" ]; then
|
||||
echo "No tags found to clean up"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
total_tags=$(echo "$all_tags" | wc -l)
|
||||
echo "Found $total_tags tags"
|
||||
|
||||
tags_to_keep=14
|
||||
tags_to_delete=$((total_tags - tags_to_keep))
|
||||
|
||||
if [ $tags_to_delete -le 0 ]; then
|
||||
echo "No tags need to be deleted (only $total_tags tags found, keeping $tags_to_keep)"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
echo "Will delete $tags_to_delete old tags, keeping the newest $tags_to_keep"
|
||||
|
||||
tags_to_delete_list=$(echo "$all_tags" | tail -n +$((tags_to_keep + 1)))
|
||||
|
||||
if [ -z "$tags_to_delete_list" ]; then
|
||||
echo "No tags to delete"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
echo "Deleting old tags..."
|
||||
delete_failures=0
|
||||
while IFS= read -r tag; do
|
||||
if [ -n "$tag" ]; then
|
||||
if ! delete_tag "$tag"; then
|
||||
delete_failures=$((delete_failures + 1))
|
||||
fi
|
||||
sleep 1
|
||||
fi
|
||||
done <<< "$tags_to_delete_list"
|
||||
|
||||
if [ "$delete_failures" -gt 0 ]; then
|
||||
echo "Cleanup finished with ${delete_failures} deletion failure(s)"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Cleanup completed successfully"
|
||||
@@ -1,88 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
set -ex
|
||||
|
||||
# Generate and upload wheel indices for all vllm-omni wheels in the commit directory.
|
||||
# This script should run once after all wheels have been built and uploaded.
|
||||
# All paths are under the omni/ prefix in the vllm-wheels S3 bucket.
|
||||
|
||||
# ======== setup ========
|
||||
|
||||
BUCKET="vllm-wheels"
|
||||
INDICES_OUTPUT_DIR="indices"
|
||||
PYTHON="${PYTHON_PROG:-python3}"
|
||||
SUBPATH="omni/$BUILDKITE_COMMIT"
|
||||
S3_COMMIT_PREFIX="s3://$BUCKET/$SUBPATH/"
|
||||
|
||||
# detect if python3.12+ is available
|
||||
has_new_python=$($PYTHON -c "print(1 if __import__('sys').version_info >= (3,12) else 0)")
|
||||
if [[ "$has_new_python" -eq 0 ]]; then
|
||||
# use new python from docker
|
||||
docker pull python:3-slim
|
||||
PYTHON="docker run --rm --user $(id -u):$(id -g) -v $(pwd):/app -w /app python:3-slim python3"
|
||||
fi
|
||||
|
||||
echo "Using python interpreter: $PYTHON"
|
||||
echo "Python version: $($PYTHON --version)"
|
||||
|
||||
# ======== generate and upload indices ========
|
||||
|
||||
# list all wheels in the commit directory
|
||||
echo "Existing wheels on S3:"
|
||||
aws s3 ls "$S3_COMMIT_PREFIX" || echo "(no objects found)"
|
||||
obj_json="objects.json"
|
||||
aws s3api list-objects-v2 --bucket "$BUCKET" --prefix "$SUBPATH/" --delimiter / --output json > "$obj_json"
|
||||
mkdir -p "$INDICES_OUTPUT_DIR"
|
||||
|
||||
# HACK: we do not need regex module here, but it is required by pre-commit hook
|
||||
# To avoid any external dependency, we simply replace it back to the stdlib re module
|
||||
sed -i.bak 's/import regex as re/import re/g' .buildkite/release/scripts/generate-nightly-index.py && rm -f .buildkite/release/scripts/generate-nightly-index.py.bak
|
||||
|
||||
# Generate indices -- the version is just the commit hash (not omni/{commit})
|
||||
# because relative paths are computed between the index and wheel directories,
|
||||
# both of which live under the omni/ prefix in S3.
|
||||
$PYTHON .buildkite/release/scripts/generate-nightly-index.py \
|
||||
--version "$BUILDKITE_COMMIT" \
|
||||
--current-objects "$obj_json" \
|
||||
--output-dir "$INDICES_OUTPUT_DIR" \
|
||||
--comment "commit $BUILDKITE_COMMIT"
|
||||
|
||||
# copy indices to /omni/{commit}/ unconditionally
|
||||
echo "Uploading indices to $S3_COMMIT_PREFIX"
|
||||
aws s3 cp --recursive "$INDICES_OUTPUT_DIR/" "$S3_COMMIT_PREFIX"
|
||||
|
||||
# copy to /omni/nightly/ when NIGHTLY=1
|
||||
if [[ "${NIGHTLY:-}" == "1" ]]; then
|
||||
echo "Uploading indices to overwrite /omni/nightly/"
|
||||
aws s3 cp --recursive "$INDICES_OUTPUT_DIR/" "s3://$BUCKET/omni/nightly/"
|
||||
fi
|
||||
|
||||
# detect version from any wheel in the commit directory
|
||||
first_wheel_key=$($PYTHON -c "import json; obj=json.load(open('$obj_json')); print(next((c['Key'] for c in obj.get('Contents', []) if c['Key'].endswith('.whl')), ''))")
|
||||
if [[ -z "$first_wheel_key" ]]; then
|
||||
echo "Error: No wheels found in $S3_COMMIT_PREFIX"
|
||||
exit 1
|
||||
fi
|
||||
first_wheel=$(basename "$first_wheel_key")
|
||||
aws s3 cp "s3://$BUCKET/${first_wheel_key}" "/tmp/${first_wheel}"
|
||||
version=$(unzip -p "/tmp/${first_wheel}" '**/METADATA' | grep '^Version: ' | cut -d' ' -f2)
|
||||
rm -f "/tmp/${first_wheel}"
|
||||
echo "Version in wheel: $version"
|
||||
pure_version="${version%%+*}"
|
||||
echo "Pure version (without variant): $pure_version"
|
||||
|
||||
# re-generate and copy to /omni/{version}/ only if it does not have "dev" in the version
|
||||
if [[ "$version" != *"dev"* ]]; then
|
||||
s3_version="v$pure_version"
|
||||
echo "Re-generating indices for /omni/$s3_version/"
|
||||
rm -rf "${INDICES_OUTPUT_DIR:?}"
|
||||
mkdir -p "$INDICES_OUTPUT_DIR"
|
||||
# wheel-dir is overridden to be the commit directory, so that the indices point to the correct wheel path
|
||||
$PYTHON .buildkite/release/scripts/generate-nightly-index.py \
|
||||
--version "$s3_version" \
|
||||
--wheel-dir "$BUILDKITE_COMMIT" \
|
||||
--current-objects "$obj_json" \
|
||||
--output-dir "$INDICES_OUTPUT_DIR" \
|
||||
--comment "version $pure_version"
|
||||
aws s3 cp --recursive "$INDICES_OUTPUT_DIR/" "s3://$BUCKET/omni/$s3_version/"
|
||||
fi
|
||||
@@ -1,192 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
from dataclasses import asdict, dataclass
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
from urllib.parse import quote
|
||||
|
||||
|
||||
def normalize_package_name(name: str) -> str:
|
||||
"""Normalize package name per PEP 503."""
|
||||
return re.sub(r"[-_.]+", "-", name).lower()
|
||||
|
||||
|
||||
if not sys.version_info >= (3, 12):
|
||||
raise RuntimeError("This script requires Python 3.12 or higher.")
|
||||
|
||||
INDEX_HTML_TEMPLATE = """<!DOCTYPE html>
|
||||
<html>
|
||||
<!-- {comment} -->
|
||||
<meta name="pypi:repository-version" content="1.0">
|
||||
<body>
|
||||
{items}
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
|
||||
@dataclass
|
||||
class WheelFileInfo:
|
||||
package_name: str
|
||||
version: str
|
||||
build_tag: str | None
|
||||
python_tag: str
|
||||
abi_tag: str
|
||||
platform_tag: str
|
||||
filename: str
|
||||
|
||||
|
||||
def parse_from_filename(file: str) -> WheelFileInfo:
|
||||
"""
|
||||
Parse wheel filename per PEP 427:
|
||||
{package_name}-{version}(-{build_tag})?-{python_tag}-{abi_tag}-{platform_tag}.whl
|
||||
"""
|
||||
wheel_file_re = re.compile(
|
||||
r"^(?P<package_name>.+)-(?P<version>[^-]+?)(-(?P<build_tag>[^-]+))?-(?P<python_tag>[^-]+)-(?P<abi_tag>[^-]+)-(?P<platform_tag>[^-]+)\.whl$"
|
||||
)
|
||||
match = wheel_file_re.match(file)
|
||||
if not match:
|
||||
raise ValueError(f"Invalid wheel file name: {file}")
|
||||
|
||||
return WheelFileInfo(
|
||||
package_name=match.group("package_name"),
|
||||
version=match.group("version"),
|
||||
build_tag=match.group("build_tag"),
|
||||
python_tag=match.group("python_tag"),
|
||||
abi_tag=match.group("abi_tag"),
|
||||
platform_tag=match.group("platform_tag"),
|
||||
filename=file,
|
||||
)
|
||||
|
||||
|
||||
def generate_project_list(package_names: list[str], comment: str = "") -> str:
|
||||
"""Generate top-level PEP 503 project list HTML."""
|
||||
href_tags = []
|
||||
for name in sorted(package_names):
|
||||
href_tags.append(f' <a href="{name}/">{name}/</a><br/>')
|
||||
return INDEX_HTML_TEMPLATE.format(items="\n".join(href_tags), comment=comment)
|
||||
|
||||
|
||||
def generate_package_index(
|
||||
wheel_files: list[WheelFileInfo],
|
||||
wheel_base_dir: Path,
|
||||
index_base_dir: Path,
|
||||
comment: str = "",
|
||||
) -> tuple[str, str]:
|
||||
"""Generate package index HTML and metadata JSON linking to wheel files."""
|
||||
href_tags = []
|
||||
metadata = []
|
||||
for file in sorted(wheel_files, key=lambda x: x.filename):
|
||||
relative_path = wheel_base_dir.relative_to(index_base_dir, walk_up=True) / file.filename
|
||||
# handle '+' in URL; avoid double-encoding '/' and '%2B' (AWS S3 behavior)
|
||||
file_path_quoted = quote(relative_path.as_posix(), safe=":%/")
|
||||
href_tags.append(f' <a href="{file_path_quoted}">{file.filename}</a><br/>')
|
||||
file_meta = asdict(file)
|
||||
file_meta["path"] = file_path_quoted
|
||||
metadata.append(file_meta)
|
||||
index_str = INDEX_HTML_TEMPLATE.format(items="\n".join(href_tags), comment=comment)
|
||||
metadata_str = json.dumps(metadata, indent=2)
|
||||
return index_str, metadata_str
|
||||
|
||||
|
||||
def generate_index(
|
||||
whl_files: list[str],
|
||||
wheel_base_dir: Path,
|
||||
index_base_dir: Path,
|
||||
comment: str = "",
|
||||
):
|
||||
"""
|
||||
Generate PEP 503 index for all wheel files.
|
||||
|
||||
Output structure:
|
||||
index_base_dir/
|
||||
index.html # project list linking to vllm-omni/
|
||||
vllm-omni/
|
||||
index.html # package index linking to wheel files
|
||||
metadata.json # machine-readable metadata
|
||||
"""
|
||||
parsed_files = [parse_from_filename(f) for f in whl_files]
|
||||
|
||||
if not parsed_files:
|
||||
print("No wheel files found, skipping index generation.")
|
||||
return
|
||||
|
||||
comment_str = f" ({comment})" if comment else ""
|
||||
comment_tmpl = f"Generated on {datetime.now().isoformat()}{comment_str}"
|
||||
|
||||
# Group by normalized package name
|
||||
packages: dict[str, list[WheelFileInfo]] = {}
|
||||
for file in parsed_files:
|
||||
name = normalize_package_name(file.package_name)
|
||||
packages.setdefault(name, []).append(file)
|
||||
|
||||
print(f"Found packages: {list(packages.keys())}")
|
||||
|
||||
# Generate per-package index
|
||||
for package, files in packages.items():
|
||||
package_dir = index_base_dir / package
|
||||
package_dir.mkdir(parents=True, exist_ok=True)
|
||||
index_str, metadata_str = generate_package_index(files, wheel_base_dir, package_dir, comment)
|
||||
with open(package_dir / "index.html", "w") as f:
|
||||
f.write(index_str)
|
||||
with open(package_dir / "metadata.json", "w") as f:
|
||||
f.write(metadata_str)
|
||||
|
||||
# Generate top-level project list
|
||||
project_list_str = generate_project_list(sorted(packages.keys()), comment_tmpl)
|
||||
with open(index_base_dir / "index.html", "w") as f:
|
||||
f.write(project_list_str)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description="Generate PEP 503 wheel index from S3 object listing.")
|
||||
parser.add_argument("--version", type=str, required=True, help="Version string (e.g., commit hash)")
|
||||
parser.add_argument("--current-objects", type=str, required=True, help="Path to JSON from S3 list-objects-v2")
|
||||
parser.add_argument("--output-dir", type=str, required=True, help="Directory to write index files")
|
||||
parser.add_argument("--wheel-dir", type=str, default=None, help="Wheel directory (defaults to --version)")
|
||||
parser.add_argument("--comment", type=str, default="", help="Comment for generated HTML")
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
version = args.version
|
||||
if "\\" in version or "/" in version:
|
||||
raise ValueError("Version string must not contain slashes or backslashes.")
|
||||
|
||||
output_dir = Path(args.output_dir)
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
with open(args.current_objects) as f:
|
||||
current_objects: dict[str, list[dict[str, Any]]] = json.load(f)
|
||||
|
||||
wheel_files = [
|
||||
item["Key"].split("/")[-1] for item in current_objects.get("Contents", []) if item["Key"].endswith(".whl")
|
||||
]
|
||||
|
||||
print(f"Found {len(wheel_files)} wheel files for version {version}: {wheel_files}")
|
||||
|
||||
# For release versions, filter to only matching non-dev wheels
|
||||
PY_VERSION_RE = re.compile(r"^\d+\.\d+\.\d+([a-zA-Z0-9.+-]*)?$")
|
||||
if PY_VERSION_RE.match(version):
|
||||
wheel_files = [f for f in wheel_files if version in f and "dev" not in f]
|
||||
print(f"Non-nightly version detected, wheel files used: {wheel_files}")
|
||||
else:
|
||||
print("Nightly version detected, keeping all wheel files.")
|
||||
|
||||
wheel_dir = (args.wheel_dir or version).strip().rstrip("/")
|
||||
wheel_base_dir = Path(output_dir).parent / wheel_dir
|
||||
index_base_dir = Path(output_dir)
|
||||
|
||||
generate_index(
|
||||
whl_files=wheel_files,
|
||||
wheel_base_dir=wheel_base_dir,
|
||||
index_base_dir=index_base_dir,
|
||||
comment=args.comment.strip(),
|
||||
)
|
||||
print(f"Successfully generated index in {output_dir}")
|
||||
@@ -1,87 +0,0 @@
|
||||
#!/bin/bash
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
#
|
||||
# Publish Docker images from ECR staging to DockerHub.
|
||||
#
|
||||
# Usage:
|
||||
# publish-release-images.sh # release mode (default)
|
||||
# publish-release-images.sh release
|
||||
# publish-release-images.sh nightly
|
||||
#
|
||||
# Release tags: latest, v<version>
|
||||
# Nightly tags: nightly, nightly-<commit>
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
MODE="${1:-release}"
|
||||
|
||||
DOCKERHUB_REPO="vllm/vllm-omni"
|
||||
ECR_REPO="public.ecr.aws/q9t5s3a7/vllm-omni-release-repo"
|
||||
COMMIT="$BUILDKITE_COMMIT"
|
||||
PRIMARY_TAGS=()
|
||||
EXTRA_MANIFEST_TAGS=()
|
||||
|
||||
case "${MODE}" in
|
||||
release)
|
||||
RELEASE_VERSION=$(buildkite-agent meta-data get release-version --default "" | sed 's/^v//')
|
||||
if [ -z "${RELEASE_VERSION}" ]; then
|
||||
echo "ERROR: release-version metadata not set"
|
||||
exit 1
|
||||
fi
|
||||
PRIMARY_TAGS=("latest" "v${RELEASE_VERSION}")
|
||||
;;
|
||||
nightly)
|
||||
# Commit-pinned manifest reuses nightly-{x86_64,aarch64} arch images.
|
||||
PRIMARY_TAGS=("nightly")
|
||||
EXTRA_MANIFEST_TAGS=("nightly-${COMMIT}")
|
||||
;;
|
||||
*)
|
||||
echo "Usage: $0 {release|nightly}"
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
|
||||
echo "========================================"
|
||||
echo "Publishing ${MODE} images to ${DOCKERHUB_REPO}"
|
||||
echo " Commit: ${COMMIT}"
|
||||
echo " Primary tags: ${PRIMARY_TAGS[*]}"
|
||||
if [ ${#EXTRA_MANIFEST_TAGS[@]} -gt 0 ]; then
|
||||
echo " Extra manifests: ${EXTRA_MANIFEST_TAGS[*]}"
|
||||
fi
|
||||
echo "========================================"
|
||||
|
||||
aws ecr-public get-login-password --region us-east-1 | \
|
||||
docker login --username AWS --password-stdin public.ecr.aws/q9t5s3a7
|
||||
|
||||
SRC_X86="${ECR_REPO}:${COMMIT}-x86_64"
|
||||
SRC_ARM="${ECR_REPO}:${COMMIT}-aarch64"
|
||||
|
||||
docker pull "${SRC_X86}"
|
||||
docker pull "${SRC_ARM}"
|
||||
|
||||
publish_manifest() {
|
||||
local tag="$1"
|
||||
local arch_tag_base="$2"
|
||||
docker manifest rm "${DOCKERHUB_REPO}:${tag}" || true
|
||||
docker manifest create \
|
||||
"${DOCKERHUB_REPO}:${tag}" \
|
||||
"${DOCKERHUB_REPO}:${arch_tag_base}-x86_64" \
|
||||
"${DOCKERHUB_REPO}:${arch_tag_base}-aarch64"
|
||||
docker manifest push "${DOCKERHUB_REPO}:${tag}"
|
||||
}
|
||||
|
||||
for tag in "${PRIMARY_TAGS[@]}"; do
|
||||
docker tag "${SRC_X86}" "${DOCKERHUB_REPO}:${tag}-x86_64"
|
||||
docker tag "${SRC_ARM}" "${DOCKERHUB_REPO}:${tag}-aarch64"
|
||||
docker push "${DOCKERHUB_REPO}:${tag}-x86_64"
|
||||
docker push "${DOCKERHUB_REPO}:${tag}-aarch64"
|
||||
publish_manifest "${tag}" "${tag}"
|
||||
done
|
||||
|
||||
ARCH_BASE="${PRIMARY_TAGS[0]}"
|
||||
for tag in "${EXTRA_MANIFEST_TAGS[@]}"; do
|
||||
publish_manifest "${tag}" "${ARCH_BASE}"
|
||||
done
|
||||
|
||||
echo ""
|
||||
echo "Successfully published ${MODE} images"
|
||||
@@ -1,52 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
set -e
|
||||
|
||||
BUCKET="vllm-wheels"
|
||||
SUBPATH="omni/$BUILDKITE_COMMIT"
|
||||
S3_COMMIT_PREFIX="s3://$BUCKET/$SUBPATH/"
|
||||
|
||||
RELEASE_VERSION=$(buildkite-agent meta-data get release-version | sed 's/^v//')
|
||||
if [[ -z "$RELEASE_VERSION" ]]; then
|
||||
echo "[FATAL] release-version metadata not set."
|
||||
exit 1
|
||||
fi
|
||||
echo "Release version: $RELEASE_VERSION"
|
||||
|
||||
if [[ -z "$PYPI_TOKEN" ]]; then
|
||||
echo "[FATAL] PYPI_TOKEN is not set."
|
||||
exit 1
|
||||
else
|
||||
export TWINE_USERNAME="__token__"
|
||||
export TWINE_PASSWORD="$PYPI_TOKEN"
|
||||
fi
|
||||
|
||||
set -x
|
||||
|
||||
if ! command -v uv &> /dev/null; then
|
||||
curl -LsSf https://astral.sh/uv/install.sh | sh
|
||||
export PATH="$HOME/.local/bin:$PATH"
|
||||
fi
|
||||
|
||||
uv venv --python 3.12 /tmp/vllm-omni-release-env
|
||||
source /tmp/vllm-omni-release-env/bin/activate
|
||||
uv pip install twine
|
||||
|
||||
DIST_DIR=/tmp/vllm-omni-release-dist
|
||||
mkdir -p "$DIST_DIR"
|
||||
|
||||
echo "Downloading wheels from S3:"
|
||||
aws s3 ls "$S3_COMMIT_PREFIX"
|
||||
aws s3 cp --recursive --exclude "*" --include "vllm_omni-${RELEASE_VERSION}*.whl" --exclude "*dev*" "$S3_COMMIT_PREFIX" "$DIST_DIR"
|
||||
ls -la "$DIST_DIR"
|
||||
|
||||
PYPI_WHEEL_FILES=$(find "$DIST_DIR" -name "vllm_omni-${RELEASE_VERSION}*.whl" -not -name "*+*")
|
||||
if [[ -z "$PYPI_WHEEL_FILES" ]]; then
|
||||
echo "[FATAL] No wheels found for version ${RELEASE_VERSION}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
python3 -m twine check $PYPI_WHEEL_FILES
|
||||
python3 -m twine upload --non-interactive --verbose $PYPI_WHEEL_FILES
|
||||
echo "Wheels uploaded to PyPI"
|
||||
@@ -1,33 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
set -ex
|
||||
|
||||
# Upload a single wheel to S3 under the omni/ prefix.
|
||||
# Index generation is handled separately by generate-and-upload-nightly-index.sh.
|
||||
|
||||
BUCKET="vllm-wheels"
|
||||
SUBPATH="omni/$BUILDKITE_COMMIT"
|
||||
S3_COMMIT_PREFIX="s3://$BUCKET/$SUBPATH/"
|
||||
|
||||
# ========= collect & upload the wheel ==========
|
||||
|
||||
# python3 -m build outputs to dist/ by default
|
||||
wheel_files=(dist/*.whl)
|
||||
|
||||
# Check that exactly one wheel is found
|
||||
if [[ ${#wheel_files[@]} -ne 1 ]]; then
|
||||
echo "Error: Expected exactly one wheel file in dist/, but found ${#wheel_files[@]}"
|
||||
exit 1
|
||||
fi
|
||||
wheel="${wheel_files[0]}"
|
||||
|
||||
echo "Uploading wheel: $wheel"
|
||||
|
||||
# Extract the version from the wheel
|
||||
version=$(unzip -p "$wheel" '**/METADATA' | grep '^Version: ' | cut -d' ' -f2)
|
||||
echo "Version in wheel: $version"
|
||||
|
||||
# Upload wheel to S3
|
||||
aws s3 cp "$wheel" "$S3_COMMIT_PREFIX"
|
||||
|
||||
echo "Wheel uploaded to $S3_COMMIT_PREFIX. Index generation is handled by a separate step."
|
||||
+73
@@ -0,0 +1,73 @@
|
||||
#!/bin/bash
|
||||
# Helper function to safely login to ECR Public with per-job config isolation
|
||||
# Uses DOCKER_CONFIG environment variable to prevent race conditions
|
||||
#
|
||||
# This script prevents the "device or resource busy" error by giving each
|
||||
# Buildkite job its own isolated Docker config directory.
|
||||
#
|
||||
# Usage:
|
||||
# source docker_login_ecr_public.sh && safe_docker_login_ecr_public
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# Configuration
|
||||
ECR_REGISTRY="public.ecr.aws"
|
||||
|
||||
setup_isolated_docker_config() {
|
||||
# Use BUILDKITE_JOB_ID for job-specific isolation
|
||||
# Fallback to PID if running outside Buildkite
|
||||
local job_id="${BUILDKITE_JOB_ID:-$$}"
|
||||
|
||||
# Set Docker config to job-specific directory
|
||||
export DOCKER_CONFIG="/tmp/docker-config-${job_id}"
|
||||
|
||||
# Create directory if it doesn't exist
|
||||
mkdir -p "$DOCKER_CONFIG"
|
||||
|
||||
echo "[docker-config] Using isolated Docker config: $DOCKER_CONFIG"
|
||||
}
|
||||
|
||||
check_docker_auth() {
|
||||
# Check if already authenticated to the given registry
|
||||
# Returns 0 if authenticated, 1 if not
|
||||
local registry="$1"
|
||||
|
||||
# Check if credentials exist in the isolated config
|
||||
if [[ -f "$DOCKER_CONFIG/config.json" ]]; then
|
||||
# Check if registry is present in config
|
||||
if grep -q "$registry" "$DOCKER_CONFIG/config.json" 2>/dev/null; then
|
||||
return 0
|
||||
fi
|
||||
fi
|
||||
|
||||
return 1
|
||||
}
|
||||
|
||||
safe_docker_login_ecr_public() {
|
||||
# Setup isolated config first
|
||||
setup_isolated_docker_config
|
||||
|
||||
local registry="$ECR_REGISTRY"
|
||||
|
||||
# Check if already authenticated (within this job)
|
||||
if check_docker_auth "$registry"; then
|
||||
echo "[docker-login] Already authenticated to $registry in this job"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# Perform login to isolated config directory
|
||||
echo "[docker-login] Logging in to $ECR_REGISTRY (isolated config)..."
|
||||
if aws ecr-public get-login-password --region us-east-1 | docker login --username AWS --password-stdin "$ECR_REGISTRY"; then
|
||||
echo "[docker-login] Login successful (config: $DOCKER_CONFIG)"
|
||||
return 0
|
||||
else
|
||||
local exit_code=$?
|
||||
echo "[docker-login] ERROR: Login failed with exit code $exit_code" >&2
|
||||
return $exit_code
|
||||
fi
|
||||
}
|
||||
|
||||
# Execute if run as script (not sourced)
|
||||
if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
|
||||
safe_docker_login_ecr_public
|
||||
fi
|
||||
+174
@@ -0,0 +1,174 @@
|
||||
#!/bin/bash
|
||||
# vllm-omni customized version
|
||||
# Based on: vllm/.buildkite/scripts/hardware_ci/run-amd-test.sh
|
||||
# Last synced: 2025-12-15
|
||||
# Modifications: docker image name for vllm-omni
|
||||
|
||||
# This script runs test inside the corresponding ROCm docker container.
|
||||
set -o pipefail
|
||||
|
||||
# Export Python path
|
||||
export PYTHONPATH=".."
|
||||
|
||||
# Print ROCm version
|
||||
echo "--- Confirming Clean Initial State"
|
||||
while true; do
|
||||
sleep 3
|
||||
if grep -q clean /opt/amdgpu/etc/gpu_state; then
|
||||
echo "GPUs state is \"clean\""
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
echo "--- ROCm info"
|
||||
rocminfo
|
||||
|
||||
# cleanup older docker images
|
||||
cleanup_docker() {
|
||||
# Get Docker's root directory
|
||||
docker_root=$(docker info -f '{{.DockerRootDir}}')
|
||||
if [ -z "$docker_root" ]; then
|
||||
echo "Failed to determine Docker root directory."
|
||||
exit 1
|
||||
fi
|
||||
echo "Docker root directory: $docker_root"
|
||||
# Check disk usage of the filesystem where Docker's root directory is located
|
||||
disk_usage=$(df "$docker_root" | tail -1 | awk '{print $5}' | sed 's/%//')
|
||||
# Define the threshold
|
||||
threshold=70
|
||||
if [ "$disk_usage" -gt "$threshold" ]; then
|
||||
echo "Disk usage is above $threshold%. Cleaning up Docker images and volumes..."
|
||||
# Remove dangling images (those that are not tagged and not used by any container)
|
||||
docker image prune -f
|
||||
# Remove unused volumes / force the system prune for old images as well.
|
||||
docker volume prune -f && docker system prune --force --filter "until=72h" --all
|
||||
echo "Docker images and volumes cleanup completed."
|
||||
else
|
||||
echo "Disk usage is below $threshold%. No cleanup needed."
|
||||
fi
|
||||
}
|
||||
|
||||
# Call the cleanup docker function
|
||||
cleanup_docker
|
||||
|
||||
echo "--- Resetting GPUs"
|
||||
|
||||
echo "reset" > /opt/amdgpu/etc/gpu_state
|
||||
|
||||
while true; do
|
||||
sleep 3
|
||||
if grep -q clean /opt/amdgpu/etc/gpu_state; then
|
||||
echo "GPUs state is \"clean\""
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
echo "--- Pulling container"
|
||||
## Temporary change to use AMD Docker Hub to store the vllm-ci image
|
||||
# to bypass the rate limit issue with ECR Public Gallery.
|
||||
# TODO: @tjtanaa point back to ECR Public Gallery
|
||||
# once the amd agents are configured to use ECR Public Gallery.
|
||||
# image_name="public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:${BUILDKITE_COMMIT}-rocm-omni"
|
||||
image_name="rocm/vllm-ci:${BUILDKITE_COMMIT}-rocm-omni"
|
||||
container_name="rocm_${BUILDKITE_COMMIT}_$(tr -dc A-Za-z0-9 < /dev/urandom | head -c 10; echo)"
|
||||
|
||||
# TODO: @tjtanaa uncomment this once the amd agents are configured to use ECR Public Gallery.
|
||||
# # Install AWS CLI to authenticate to ECR Public Gallery to get higher rate limit for pulling images
|
||||
# sudo apt-get update && sudo apt-get install -y awscli
|
||||
## Use safe docker login helper to prevent race conditions
|
||||
# source "$(dirname "${BASH_SOURCE[0]}")/../docker_login_ecr_public.sh"
|
||||
# safe_docker_login_ecr_public
|
||||
|
||||
## Pull the container from AMD Docker Hub
|
||||
|
||||
docker pull "${image_name}"
|
||||
|
||||
remove_docker_container() {
|
||||
docker rm -f "${container_name}" || docker image rm -f "${image_name}" || true
|
||||
}
|
||||
trap remove_docker_container EXIT
|
||||
|
||||
echo "--- Running container"
|
||||
|
||||
HF_CACHE="$(realpath ~)/huggingface"
|
||||
mkdir -p "${HF_CACHE}"
|
||||
HF_MOUNT="/root/.cache/huggingface"
|
||||
|
||||
if [[ -n "${TEST_COMMAND:-}" ]]; then
|
||||
commands="$TEST_COMMAND"
|
||||
else
|
||||
commands="$@"
|
||||
fi
|
||||
echo "Commands:$commands"
|
||||
|
||||
PARALLEL_JOB_COUNT=8
|
||||
MYPYTHONPATH=".."
|
||||
|
||||
# Test that we're launching on the machine that has
|
||||
# proper access to GPUs
|
||||
render_gid=$(getent group render | cut -d: -f3)
|
||||
if [[ -z "$render_gid" ]]; then
|
||||
echo "Error: 'render' group not found. This is required for GPU access." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# check if the command contains shard flag, we will run all shards in parallel because the host have 8 GPUs.
|
||||
# TODO: @tjtanaa reenable to run VLLM_ROCM_USE_AITER=1 when AITER is shipped with prebuilt kernels.
|
||||
if [[ $commands == *"--shard-id="* ]]; then
|
||||
# assign job count as the number of shards used
|
||||
commands=$(echo "$commands" | sed -E "s/--num-shards[[:blank:]]*=[[:blank:]]*[0-9]*/--num-shards=${PARALLEL_JOB_COUNT} /g" | sed 's/ \\ / /g')
|
||||
for GPU in $(seq 0 $(($PARALLEL_JOB_COUNT-1))); do
|
||||
# assign shard-id for each shard
|
||||
commands_gpu=$(echo "$commands" | sed -E "s/--shard-id[[:blank:]]*=[[:blank:]]*[0-9]*/--shard-id=${GPU} /g" | sed 's/ \\ / /g')
|
||||
echo "Shard ${GPU} commands:$commands_gpu"
|
||||
echo "Render devices: $BUILDKITE_AGENT_META_DATA_RENDER_DEVICES"
|
||||
docker run \
|
||||
--device /dev/kfd $BUILDKITE_AGENT_META_DATA_RENDER_DEVICES \
|
||||
--network=host \
|
||||
--shm-size=16gb \
|
||||
--group-add "$render_gid" \
|
||||
--rm \
|
||||
-e MIOPEN_DEBUG_CONV_DIRECT=0 \
|
||||
-e MIOPEN_DEBUG_CONV_GEMM=0 \
|
||||
-e VLLM_ROCM_USE_AITER=0 \
|
||||
-e HIP_VISIBLE_DEVICES="${GPU}" \
|
||||
-e HF_TOKEN \
|
||||
-v "${HF_CACHE}:${HF_MOUNT}" \
|
||||
-e "HF_HOME=${HF_MOUNT}" \
|
||||
-e "PYTHONPATH=${MYPYTHONPATH}" \
|
||||
--name "${container_name}_${GPU}" \
|
||||
"${image_name}" \
|
||||
/bin/bash -c "${commands_gpu}" \
|
||||
|& while read -r line; do echo ">>Shard $GPU: $line"; done &
|
||||
PIDS+=($!)
|
||||
done
|
||||
#wait for all processes to finish and collect exit codes
|
||||
for pid in "${PIDS[@]}"; do
|
||||
wait "${pid}"
|
||||
STATUS+=($?)
|
||||
done
|
||||
for st in "${STATUS[@]}"; do
|
||||
if [[ ${st} -ne 0 ]]; then
|
||||
echo "One of the processes failed with $st"
|
||||
exit "${st}"
|
||||
fi
|
||||
done
|
||||
else
|
||||
echo "Render devices: $BUILDKITE_AGENT_META_DATA_RENDER_DEVICES"
|
||||
docker run \
|
||||
--device /dev/kfd $BUILDKITE_AGENT_META_DATA_RENDER_DEVICES \
|
||||
--network=host \
|
||||
--shm-size=16gb \
|
||||
--group-add "$render_gid" \
|
||||
--rm \
|
||||
-e MIOPEN_DEBUG_CONV_DIRECT=0 \
|
||||
-e MIOPEN_DEBUG_CONV_GEMM=0 \
|
||||
-e VLLM_ROCM_USE_AITER=0 \
|
||||
-e HF_TOKEN \
|
||||
-v "${HF_CACHE}:${HF_MOUNT}" \
|
||||
-e "HF_HOME=${HF_MOUNT}" \
|
||||
-e "PYTHONPATH=${MYPYTHONPATH}" \
|
||||
--name "${container_name}" \
|
||||
"${image_name}" \
|
||||
/bin/bash -c "${commands}"
|
||||
fi
|
||||
+56
@@ -0,0 +1,56 @@
|
||||
#!/bin/bash
|
||||
|
||||
# This script build the XPU docker image and run the offline inference inside the container.
|
||||
set -ex
|
||||
|
||||
omni_source_dir=$(git rev-parse --show-toplevel)
|
||||
|
||||
base_image_name="xpu/vllm-omni-ci-base:${VLLM_VERSION:?VLLM_VERSION must be set}"
|
||||
image_name="xpu/vllm-omni-ci:${BUILDKITE_COMMIT:?BUILDKITE_COMMIT must be set}"
|
||||
container_name="xpu_${BUILDKITE_COMMIT}_$(
|
||||
tr -dc A-Za-z0-9 </dev/urandom | head -c 10
|
||||
echo
|
||||
)"
|
||||
|
||||
cd "${omni_source_dir}"
|
||||
if [ -z "$(docker images -q "${base_image_name}")" ]; then
|
||||
docker build --target vllm-base -t "${base_image_name}" --build-arg "VLLM_VERSION=${VLLM_VERSION}" -f docker/Dockerfile.xpu .
|
||||
fi
|
||||
|
||||
# Try building the docker image
|
||||
docker build --build-arg "VLLM_BASE=${base_image_name}" --build-arg "VLLM_VERSION=${VLLM_VERSION}" -t "${image_name}" -f docker/Dockerfile.xpu .
|
||||
|
||||
# Setup cleanup
|
||||
remove_docker_container() {
|
||||
docker rm -f "${container_name}" || true
|
||||
docker image rm -f "${image_name}" || true
|
||||
docker system prune -f || true
|
||||
}
|
||||
trap remove_docker_container EXIT
|
||||
|
||||
HF_CACHE="${HF_CACHE:-$(realpath ~)/.cache/huggingface}"
|
||||
mkdir -p "${HF_CACHE}"
|
||||
HF_MOUNT="/root/.cache/huggingface"
|
||||
|
||||
time timeout -k 30 30m docker run \
|
||||
--device /dev/dri:/dev/dri \
|
||||
--net=host \
|
||||
--ipc=host \
|
||||
-v /dev/dri/by-path:/dev/dri/by-path \
|
||||
-v "${HF_CACHE}:${HF_MOUNT}" \
|
||||
--security-opt seccomp=unconfined \
|
||||
--entrypoint="" \
|
||||
-e VLLM_LOGGING_LEVEL \
|
||||
-e VLLM_OMNI_LOGGING_LEVEL \
|
||||
-e HF_TOKEN \
|
||||
-e ZE_AFFINITY_MASK \
|
||||
--name "${container_name}" \
|
||||
"${image_name}" \
|
||||
bash -c '
|
||||
set -e
|
||||
echo $ZE_AFFINITY_MASK
|
||||
pip install tblib==3.1.0
|
||||
cd /workspace/vllm-omni
|
||||
pytest -v -s -m "core_model and xpu and B60"
|
||||
pytest -v -s -m "advanced_model and xpu and B60"
|
||||
'
|
||||
@@ -0,0 +1,145 @@
|
||||
#!/bin/bash
|
||||
|
||||
# This script build the Ascend NPU docker image and run the offline inference inside the container.
|
||||
# It serves a sanity check for compilation and basic model usage.
|
||||
set -ex
|
||||
|
||||
# Base ubuntu image with basic ascend development libraries and python installed
|
||||
VLLM_OMNI_REPO="https://github.com/vllm-project/vllm-omni.git"
|
||||
BASE_IMAGE_NAME="quay.nju.edu.cn/ascend/vllm-ascend:v0.11.0rc2"
|
||||
image_name="npu/vllm-omni-ci:${BUILDKITE_COMMIT}_${EPOCHSECONDS}"
|
||||
# image_name="npu/vllm-ci:${BUILDKITE_COMMIT}_${EPOCHSECONDS}"
|
||||
container_name="npu_${BUILDKITE_COMMIT}_$(tr -dc A-Za-z0-9 < /dev/urandom | head -c 10; echo)"
|
||||
|
||||
# BUILDKITE_AGENT_NAME format is {hostname}-{agent_idx}-{npu_card_num}cards
|
||||
agent_idx=$(echo "${BUILDKITE_AGENT_NAME}" | awk -F'-' '{print $(NF-1)}')
|
||||
echo "agent_idx: ${agent_idx}"
|
||||
builder_name="cachebuilder${agent_idx}"
|
||||
builder_cache_dir="/mnt/docker-cache${agent_idx}"
|
||||
mkdir -p ${builder_cache_dir}
|
||||
|
||||
# Try building the docker image
|
||||
cat <<EOF | DOCKER_BUILDKIT=1 docker build \
|
||||
--add-host pypi-cache:${PYPI_CACHE_HOST} \
|
||||
--builder ${builder_name} --cache-from type=local,src=${builder_cache_dir} \
|
||||
--cache-to type=local,dest=${builder_cache_dir},mode=max \
|
||||
--build-arg BUILDKITE_PULL_REQUEST="${BUILDKITE_PULL_REQUEST}" \
|
||||
--build-arg BUILDKITE_PULL_REQUEST_REPO="${BUILDKITE_PULL_REQUEST_REPO}" \
|
||||
--progress=plain --load -t ${image_name} -f - .
|
||||
FROM ${BASE_IMAGE_NAME}
|
||||
|
||||
# Define environments
|
||||
ENV DEBIAN_FRONTEND=noninteractive
|
||||
|
||||
RUN pip config set global.index-url http://pypi-cache:${PYPI_CACHE_PORT}/pypi/simple && \
|
||||
pip config set global.trusted-host pypi-cache && \
|
||||
apt-get update -y && \
|
||||
apt-get install -y python3-pip git vim wget net-tools gcc g++ cmake libnuma-dev && \
|
||||
rm -rf /var/cache/apt/* && \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Install for pytest to make the docker build cache layer always valid
|
||||
RUN --mount=type=cache,target=/root/.cache/pip \
|
||||
pip install pytest>=6.0 pytest-cov modelscope
|
||||
|
||||
COPY . .
|
||||
|
||||
# Install vllm-omni
|
||||
WORKDIR /workspace
|
||||
ARG VLLM_OMNI_REPO=https://github.com/vllm-project/vllm-omni.git
|
||||
ARG VLLM_OMNI_TAG=main
|
||||
ARG BUILDKITE_PULL_REQUEST
|
||||
ARG BUILDKITE_PULL_REQUEST_REPO
|
||||
RUN git config --global url."https://gh-proxy.test.osinfra.cn/https://github.com/".insteadOf "https://github.com/" && \
|
||||
if [ "\$BUILDKITE_PULL_REQUEST" != "false" ] && [ -n "\$BUILDKITE_PULL_REQUEST" ]; then \
|
||||
echo "Cloning and checking out PR #\$BUILDKITE_PULL_REQUEST..." && \
|
||||
git clone \$VLLM_OMNI_REPO /workspace/vllm-omni && \
|
||||
cd /workspace/vllm-omni && \
|
||||
git fetch origin pull/\$BUILDKITE_PULL_REQUEST/head:pr-\$BUILDKITE_PULL_REQUEST && \
|
||||
git checkout pr-\$BUILDKITE_PULL_REQUEST; \
|
||||
else \
|
||||
echo "Not a PR build, using main branch" && \
|
||||
git clone --depth 1 \$VLLM_OMNI_REPO /workspace/vllm-omni; \
|
||||
fi
|
||||
|
||||
RUN --mount=type=cache,target=/root/.cache/pip \
|
||||
export PIP_EXTRA_INDEX_URL=https://mirrors.huaweicloud.com/ascend/repos/pypi && \
|
||||
source /usr/local/Ascend/ascend-toolkit/set_env.sh && \
|
||||
source /usr/local/Ascend/nnal/atb/set_env.sh && \
|
||||
export LD_LIBRARY_PATH=\$LD_LIBRARY_PATH:/usr/local/Ascend/ascend-toolkit/latest/`uname -i`-linux/devlib && \
|
||||
python3 -m pip install -v -e /workspace/vllm-omni/
|
||||
|
||||
ENV VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
|
||||
WORKDIR /workspace/vllm-omni
|
||||
CMD ["/bin/bash"]
|
||||
|
||||
EOF
|
||||
|
||||
# Setup cleanup
|
||||
remove_docker_container() {
|
||||
docker rm -f "${container_name}" || true;
|
||||
docker image rm -f "${image_name}" || true;
|
||||
docker system prune -f || true;
|
||||
}
|
||||
trap remove_docker_container EXIT
|
||||
|
||||
# Generate corresponding --device args based on BUILDKITE_AGENT_NAME
|
||||
# Ascend NPU BUILDKITE_AGENT_NAME format is {hostname}-{agent_idx}-{npu_card_num}cards, and agent_idx starts from 1.
|
||||
# e.g. atlas-a2-001-1-2cards means this is the 1-th agent on atlas-a2-001 host, and it has 2 NPU cards.
|
||||
# returns --device /dev/davinci0 --device /dev/davinci1
|
||||
parse_and_gen_devices() {
|
||||
local input="$1"
|
||||
local index cards_num
|
||||
if [[ "$input" =~ ([0-9]+)-([0-9]+)cards$ ]]; then
|
||||
index="${BASH_REMATCH[1]}"
|
||||
cards_num="${BASH_REMATCH[2]}"
|
||||
else
|
||||
echo "parse error" >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
local devices=""
|
||||
local i=0
|
||||
while (( i < cards_num )); do
|
||||
local dev_idx=$(((index - 1)*cards_num + i ))
|
||||
devices="$devices --device /dev/davinci${dev_idx}"
|
||||
((i++))
|
||||
done
|
||||
|
||||
# trim leading space
|
||||
devices="${devices#"${devices%%[![:space:]]*}"}"
|
||||
# Output devices: assigned to the caller variable
|
||||
printf '%s' "$devices"
|
||||
}
|
||||
|
||||
devices=$(parse_and_gen_devices "${BUILDKITE_AGENT_NAME}") || exit 1
|
||||
|
||||
# Run the image and execute the Out-Of-Tree (OOT) platform interface test case on Ascend NPU hardware.
|
||||
# This test checks whether the OOT platform interface is functioning properly in conjunction with
|
||||
# the hardware plugin vllm-ascend.
|
||||
hf_model_cache_dir=/mnt/hf_cache${agent_idx}
|
||||
ms_model_cache_dir=/mnt/modelscope${agent_idx}
|
||||
mkdir -p ${hf_model_cache_dir}
|
||||
mkdir -p ${ms_model_cache_dir}
|
||||
docker run \
|
||||
--init \
|
||||
${devices} \
|
||||
--device /dev/davinci_manager \
|
||||
--device /dev/devmm_svm \
|
||||
--device /dev/hisi_hdc \
|
||||
-v /usr/local/dcmi:/usr/local/dcmi \
|
||||
-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
|
||||
-v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
|
||||
-v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
|
||||
-v /etc/ascend_install.info:/etc/ascend_install.info \
|
||||
-v ${hf_model_cache_dir}:/root/.cache/huggingface \
|
||||
-v ${ms_model_cache_dir}:/root/.cache/modelscope \
|
||||
--network host \
|
||||
--entrypoint="" \
|
||||
--name "${container_name}" \
|
||||
"${image_name}" \
|
||||
bash -c '
|
||||
set -e
|
||||
VLLM_USE_MODELSCOPE=True pytest -s -v tests/e2e/offline_inference/test_qwen2_5_omni.py
|
||||
'
|
||||
@@ -0,0 +1,215 @@
|
||||
steps:
|
||||
|
||||
- label: "Simple Unit Test"
|
||||
agent_pool: mi325_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export VLLM_ROCM_USE_AITER=0
|
||||
- "timeout 20m pytest -v -s -m 'core_model and cpu' --cov=vllm_omni --cov-branch --cov-report=term-missing --cov-report=html --cov-report=xml"
|
||||
|
||||
- label: "Diffusion Model Test"
|
||||
agent_pool: mi325_2
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export GPU_ARCHS=gfx942
|
||||
- timeout 20m pytest -s -v tests/e2e/offline_inference/test_t2i_model.py -m "advanced_model and diffusion" --run-level "advanced_model"
|
||||
|
||||
- label: "Diffusion Images API LoRA E2E"
|
||||
agent_pool: mi325_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- timeout 20m pytest -s -v tests/e2e/online_serving/test_images_generations_lora.py
|
||||
|
||||
- label: "Diffusion Model CPU offloading Test"
|
||||
agent_pool: mi325_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export GPU_ARCHS=gfx942
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- |
|
||||
timeout 20m bash -c '
|
||||
set +e
|
||||
pytest -s -v tests/e2e/offline_inference/test_diffusion_cpu_offload.py
|
||||
EXIT1=\$?
|
||||
pytest -s -v tests/e2e/offline_inference/test_diffusion_layerwise_offload.py
|
||||
EXIT2=\$?
|
||||
exit \$((EXIT1 | EXIT2))
|
||||
'
|
||||
|
||||
## ISSUE depends on `diffusers` package: https://github.com/huggingface/diffusers/issues/13274
|
||||
# - label: "Audio Generation Model Test"
|
||||
# agent_pool: mi325_1
|
||||
# depends_on: amd-build
|
||||
# mirror_hardwares: [amdproduction]
|
||||
# grade: Blocking
|
||||
# commands:
|
||||
# - export GPU_ARCHS=gfx942
|
||||
# - export VLLM_LOGGING_LEVEL=DEBUG
|
||||
# - export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
# - timeout 20m pytest -s -v tests/e2e/offline_inference/test_stable_audio_model.py
|
||||
|
||||
- label: "Diffusion Cache Backend Test"
|
||||
agent_pool: mi325_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export GPU_ARCHS=gfx942
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- timeout 15m pytest -s -v -m "core_model and cache and diffusion and not distributed_cuda and L4"
|
||||
|
||||
- label: "Diffusion Sequence Parallelism Test"
|
||||
agent_pool: mi325_2
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- timeout 20m pytest -s -v tests/e2e/offline_inference/test_sequence_parallel.py
|
||||
|
||||
# merge-only tests
|
||||
- label: "Diffusion Tensor Parallelism Test"
|
||||
agent_pool: mi325_2
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/offline_inference/test_zimage_parallelism.py
|
||||
|
||||
- label: "Diffusion GPU Worker Test"
|
||||
agent_pool: mi325_2
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- timeout 20m pytest -s -v tests/diffusion/test_diffusion_worker.py
|
||||
|
||||
- label: "Benchmark & Engine Test"
|
||||
agent_pool: mi325_2
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- |
|
||||
timeout 20m bash -c '
|
||||
set +e
|
||||
pytest -s -v tests/benchmarks/test_serve_cli.py
|
||||
EXIT1=\$?
|
||||
pytest -s -v tests/engine/test_async_omni_engine_abort.py
|
||||
EXIT2=\$?
|
||||
exit \$((EXIT1 | EXIT2))
|
||||
'
|
||||
|
||||
- label: "Omni Model Test Qwen2-5-Omni"
|
||||
agent_pool: mi325_2
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- timeout 20m pytest -s -v tests/e2e/offline_inference/test_qwen2_5_omni.py
|
||||
|
||||
- label: "Omni Model Test Qwen3-Omni"
|
||||
agent_pool: mi325_2
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- export VLLM_TEST_CLEAN_GPU_MEMORY=1
|
||||
- timeout 10m pytest -s -v tests/e2e/offline_inference/test_qwen3_omni.py
|
||||
- timeout 20m pytest -s -v tests/e2e/online_serving/test_qwen3_omni.py -m "advanced_model" --run-level "advanced_model"
|
||||
|
||||
- label: "Qwen3-TTS CustomVoice E2E Test"
|
||||
agent_pool: mi325_2
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- |
|
||||
timeout 20m bash -c '
|
||||
export VLLM_LOGGING_LEVEL=DEBUG
|
||||
export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
|
||||
pytest -s -v tests/e2e/online_serving/test_qwen3_tts_customvoice.py -m "advanced_model" --run-level "advanced_model" && pytest -s -v tests/e2e/offline_inference/test_qwen3_tts_customvoice.py
|
||||
'
|
||||
|
||||
- label: "Qwen3-TTS Base E2E Test"
|
||||
agent_pool: mi325_2
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- |
|
||||
timeout 20m bash -c '
|
||||
export VLLM_LOGGING_LEVEL=DEBUG
|
||||
export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
|
||||
pytest -s -v tests/e2e/online_serving/test_qwen3_tts_base.py -m "advanced_model" --run-level "advanced_model" && pytest -s -v tests/e2e/offline_inference/test_qwen3_tts_base.py
|
||||
'
|
||||
|
||||
- label: "Diffusion Image Edit Test"
|
||||
agent_pool: mi325_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export GPU_ARCHS=gfx942
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- timeout 20m pytest -s -v tests/e2e/online_serving/test_image_gen_edit.py
|
||||
|
||||
# split Bagel Model Test with H100 (Real Weights) into three tests
|
||||
- label: "Bagel Text2Img Model Test"
|
||||
agent_pool: mi325_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export GPU_ARCHS=gfx942
|
||||
- export VLLM_TEST_CLEAN_GPU_MEMORY=1
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- export VLLM_ROCM_USE_AITER_RMSNORM=0
|
||||
- timeout 30m pytest -s -v tests/e2e/offline_inference/test_bagel_text2img.py -m "advanced_model" --run-level "advanced_model" -k "shared_memory" -k "rocm"
|
||||
|
||||
- label: "Bagel Img2Img Model Test"
|
||||
agent_pool: mi325_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export GPU_ARCHS=gfx942
|
||||
- export VLLM_TEST_CLEAN_GPU_MEMORY=1
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- export VLLM_ROCM_USE_AITER_RMSNORM=0
|
||||
- timeout 30m pytest -s -v tests/e2e/offline_inference/test_bagel_img2img.py -m "advanced_model" --run-level "advanced_model" -k "rocm"
|
||||
|
||||
- label: "Bagel Online Serving Test"
|
||||
agent_pool: mi325_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export GPU_ARCHS=gfx942
|
||||
- export VLLM_TEST_CLEAN_GPU_MEMORY=1
|
||||
- export VLLM_IMAGE_FETCH_TIMEOUT=60
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- export VLLM_ROCM_USE_AITER_RMSNORM=0
|
||||
- timeout 40m pytest -s -v tests/e2e/online_serving/test_bagel_online.py -m "advanced_model" --run-level "advanced_model" -k "rocm"
|
||||
@@ -0,0 +1,179 @@
|
||||
steps:
|
||||
|
||||
- label: "Simple Unit Test"
|
||||
agent_pool: mi325_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export VLLM_ROCM_USE_AITER=0
|
||||
- "timeout 20m pytest -v -s -m 'core_model and cpu' --cov=vllm_omni --cov-branch --cov-report=term-missing --cov-report=html --cov-report=xml"
|
||||
|
||||
- label: "Diffusion Model Test"
|
||||
agent_pool: mi325_2
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- timeout 20m pytest -s -v tests/e2e/offline_inference/test_t2i_model.py -m "core_model and diffusion" --run-level "core_model"
|
||||
|
||||
- label: "Diffusion Model CPU offloading Test"
|
||||
agent_pool: mi325_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export GPU_ARCHS=gfx942
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- |
|
||||
timeout 20m bash -c '
|
||||
set +e
|
||||
pytest -s -v tests/e2e/offline_inference/test_diffusion_cpu_offload.py
|
||||
EXIT1=\$?
|
||||
pytest -s -v tests/e2e/offline_inference/test_diffusion_layerwise_offload.py
|
||||
EXIT2=\$?
|
||||
exit \$((EXIT1 | EXIT2))
|
||||
'
|
||||
|
||||
## ISSUE depends on `diffusers` package: https://github.com/huggingface/diffusers/issues/13274
|
||||
# - label: "Audio Generation Model Test"
|
||||
# agent_pool: mi325_1
|
||||
# depends_on: amd-build
|
||||
# mirror_hardwares: [amdproduction]
|
||||
# grade: Blocking
|
||||
# commands:
|
||||
# - export GPU_ARCHS=gfx942
|
||||
# - export VLLM_LOGGING_LEVEL=DEBUG
|
||||
# - export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
# - timeout 20m pytest -s -v tests/e2e/offline_inference/test_stable_audio_model.py
|
||||
|
||||
- label: "Diffusion Cache Backend Test"
|
||||
agent_pool: mi325_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export GPU_ARCHS=gfx942
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- timeout 15m pytest -s -v -m "core_model and cache and diffusion and not distributed_cuda and L4"
|
||||
|
||||
- label: "Diffusion Sequence Parallelism Test"
|
||||
agent_pool: mi325_2
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- timeout 20m pytest -s -v tests/e2e/offline_inference/test_sequence_parallel.py -m core_model
|
||||
|
||||
- label: "Diffusion GPU Worker Test"
|
||||
agent_pool: mi325_2
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- timeout 20m pytest -s -v tests/diffusion/test_diffusion_worker.py
|
||||
|
||||
- label: "Benchmark & Engine Test"
|
||||
agent_pool: mi325_2
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- |
|
||||
timeout 30m bash -c '
|
||||
set +e
|
||||
pytest -s -v tests/benchmarks/test_serve_cli.py
|
||||
EXIT1=\$?
|
||||
pytest -s -v tests/engine/test_async_omni_engine_abort.py
|
||||
EXIT2=\$?
|
||||
exit \$((EXIT1 | EXIT2))
|
||||
'
|
||||
|
||||
- label: "Omni Model Test Qwen2-5-Omni"
|
||||
agent_pool: mi325_2
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- timeout 17m pytest -s -v tests/e2e/offline_inference/test_qwen2_5_omni.py
|
||||
|
||||
- label: "Omni Model Test Qwen3-Omni"
|
||||
agent_pool: mi325_2
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- export VLLM_TEST_CLEAN_GPU_MEMORY=1
|
||||
- timeout 10m pytest -s -v tests/e2e/offline_inference/test_qwen3_omni.py
|
||||
- timeout 10m pytest -s -v tests/e2e/online_serving/test_qwen3_omni.py -m "core_model" --run-level "core_model"
|
||||
|
||||
- label: "Qwen3-TTS E2E Test"
|
||||
agent_pool: mi325_2
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
|
||||
- timeout 20m pytest -s -v tests/e2e/online_serving/test_qwen3_tts_customvoice.py -m "core_model" --run-level "core_model"
|
||||
|
||||
- label: "Diffusion Image Edit Test"
|
||||
agent_pool: mi325_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export GPU_ARCHS=gfx942
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- timeout 20m pytest -s -v tests/e2e/online_serving/test_image_gen_edit.py
|
||||
|
||||
- label: "Bagel Text2Img Model Test"
|
||||
agent_pool: mi325_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export GPU_ARCHS=gfx942
|
||||
- export VLLM_TEST_CLEAN_GPU_MEMORY=1
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- export VLLM_ROCM_USE_AITER_RMSNORM=0
|
||||
- timeout 30m pytest -s -v tests/e2e/offline_inference/test_bagel_text2img.py -m "core_model" --run-level "core_model" -k "rocm"
|
||||
|
||||
- label: "Bagel Img2Img Model Test"
|
||||
agent_pool: mi325_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export GPU_ARCHS=gfx942
|
||||
- export VLLM_TEST_CLEAN_GPU_MEMORY=1
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- export VLLM_ROCM_USE_AITER_RMSNORM=0
|
||||
- timeout 30m pytest -s -v tests/e2e/offline_inference/test_bagel_img2img.py -m "core_model" --run-level "core_model" -k "rocm"
|
||||
|
||||
- label: "Bagel Online Serving Test"
|
||||
agent_pool: mi325_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export GPU_ARCHS=gfx942
|
||||
- export VLLM_TEST_CLEAN_GPU_MEMORY=1
|
||||
- export VLLM_IMAGE_FETCH_TIMEOUT=60
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- export VLLM_ROCM_USE_AITER_RMSNORM=0
|
||||
- timeout 40m pytest -s -v tests/e2e/online_serving/test_bagel_online.py -m "core_model" --run-level "core_model" -k "rocm"
|
||||
@@ -0,0 +1,119 @@
|
||||
steps:
|
||||
|
||||
- label: "Diffusion Model Test"
|
||||
timeout_in_minutes: 30
|
||||
agent_pool: mi325_2
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export GPU_ARCHS=gfx942
|
||||
- pytest -s -v tests/e2e/offline_inference/test_t2i_model.py
|
||||
|
||||
- label: "Diffusion Images API LoRA E2E"
|
||||
timeout_in_minutes: 30
|
||||
agent_pool: mi325_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export GPU_ARCHS=gfx942
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- pytest -s -v tests/e2e/online_serving/test_images_generations_lora.py
|
||||
|
||||
- label: "Diffusion Model CPU offloading Test"
|
||||
timeout_in_minutes: 20
|
||||
agent_pool: mi325_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export GPU_ARCHS=gfx942
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- pytest -s -v tests/e2e/offline_inference/test_diffusion_cpu_offload.py
|
||||
- pytest -s -v tests/e2e/offline_inference/test_diffusion_layerwise_offload.py
|
||||
|
||||
- label: "Diffusion Cache Backend Test"
|
||||
timeout_in_minutes: 15
|
||||
agent_pool: mi325_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export GPU_ARCHS=gfx942
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- pytest -s -v -m 'core_model and cache and diffusion and not distributed_rocm and MI325'
|
||||
|
||||
- label: "Diffusion Sequence Parallelism Test"
|
||||
timeout_in_minutes: 20
|
||||
agent_pool: mi325_2
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export GPU_ARCHS=gfx942
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- pytest -s -v tests/e2e/offline_inference/test_sequence_parallel.py -m core_model
|
||||
|
||||
- label: "Diffusion Tensor Parallelism Test"
|
||||
timeout_in_minutes: 20
|
||||
agent_pool: mi325_2
|
||||
depends_on: amd-build
|
||||
commands:
|
||||
- export GPU_ARCHS=gfx942
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- pytest -s -v tests/e2e/offline_inference/test_zimage_parallelism.py
|
||||
|
||||
- label: "Diffusion GPU Worker Test"
|
||||
timeout_in_minutes: 20
|
||||
agent_pool: mi325_2
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- pytest -s -v tests/diffusion/test_diffusion_worker.py
|
||||
|
||||
- label: "Omni Model Test Qwen2-5-Omni"
|
||||
timeout_in_minutes: 15
|
||||
agent_pool: mi325_2
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export GPU_ARCHS=gfx942
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- pytest -s -v tests/e2e/offline_inference/test_qwen2_5_omni.py
|
||||
- pytest -s -v tests/engine/test_async_omni_engine_abort.py
|
||||
|
||||
- label: "Omni Model Test Qwen3-Omni"
|
||||
timeout_in_minutes: 15
|
||||
agent_pool: mi325_2
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- export VLLM_TEST_CLEAN_GPU_MEMORY="1"
|
||||
- export VLLM_ROCM_USE_AITER=0
|
||||
- pytest -s -v tests/e2e/offline_inference/test_qwen3_omni.py
|
||||
- pytest -s -v tests/e2e/online_serving/test_qwen3_omni.py
|
||||
|
||||
|
||||
- label: "Diffusion Image Edit Test"
|
||||
timeout_in_minutes: 15
|
||||
agent_pool: mi325_1
|
||||
depends_on: amd-build
|
||||
mirror_hardwares: [amdproduction]
|
||||
grade: Blocking
|
||||
commands:
|
||||
- export GPU_ARCHS=gfx942
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- pytest -s -v tests/e2e/online_serving/test_image_gen_edit.py
|
||||
@@ -0,0 +1,426 @@
|
||||
steps:
|
||||
- label: "Simple Unit Test"
|
||||
depends_on: upload-merge-pipeline
|
||||
commands:
|
||||
- "pytest -v -s -m 'core_model and cpu' --cov=vllm_omni --cov-branch --cov-report=term-missing --cov-report=html --cov-report=xml"
|
||||
agents:
|
||||
queue: "gpu_1_queue"
|
||||
plugins:
|
||||
- docker#v5.2.0:
|
||||
image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
always-pull: true
|
||||
propagate-environment: true
|
||||
environment:
|
||||
- "HF_HOME=/fsx/hf_cache"
|
||||
- "HF_TOKEN"
|
||||
volumes:
|
||||
- "/fsx/hf_cache:/fsx/hf_cache"
|
||||
|
||||
- label: "Diffusion Model Test"
|
||||
timeout_in_minutes: 30
|
||||
depends_on: upload-merge-pipeline
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/offline_inference/test_t2i_model.py -m "advanced_model and diffusion" --run-level "advanced_model"
|
||||
agents:
|
||||
queue: "gpu_1_queue" # g6.4xlarge instance on AWS, has 1 L4 GPU
|
||||
plugins:
|
||||
- docker#v5.2.0:
|
||||
image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
always-pull: true
|
||||
propagate-environment: true
|
||||
environment:
|
||||
- "HF_HOME=/fsx/hf_cache"
|
||||
- "HF_TOKEN"
|
||||
volumes:
|
||||
- "/fsx/hf_cache:/fsx/hf_cache"
|
||||
|
||||
- label: "Diffusion Images API LoRA E2E"
|
||||
timeout_in_minutes: 30
|
||||
depends_on: upload-merge-pipeline
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/online_serving/test_images_generations_lora.py
|
||||
agents:
|
||||
queue: "gpu_1_queue" # g6.4xlarge instance on AWS, has 1 L4 GPU
|
||||
plugins:
|
||||
- docker#v5.2.0:
|
||||
image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
always-pull: true
|
||||
propagate-environment: true
|
||||
environment:
|
||||
- "HF_HOME=/fsx/hf_cache"
|
||||
- "HF_TOKEN"
|
||||
volumes:
|
||||
- "/fsx/hf_cache:/fsx/hf_cache"
|
||||
|
||||
- label: "Diffusion Model CPU offloading Test"
|
||||
timeout_in_minutes: 20
|
||||
depends_on: upload-merge-pipeline
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/offline_inference/test_diffusion_cpu_offload.py
|
||||
- pytest -s -v tests/e2e/offline_inference/test_diffusion_layerwise_offload.py
|
||||
agents:
|
||||
queue: "gpu_1_queue" # g6.4xlarge instance on AWS, has 1 L4 GPU
|
||||
plugins:
|
||||
- docker#v5.2.0:
|
||||
image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
always-pull: true
|
||||
propagate-environment: true
|
||||
environment:
|
||||
- "HF_HOME=/fsx/hf_cache"
|
||||
- "HF_TOKEN"
|
||||
volumes:
|
||||
- "/fsx/hf_cache:/fsx/hf_cache"
|
||||
|
||||
- label: "Audio Generation Model Test"
|
||||
timeout_in_minutes: 20
|
||||
depends_on: upload-merge-pipeline
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/offline_inference/test_stable_audio_model.py
|
||||
agents:
|
||||
queue: "gpu_1_queue" # g6.4xlarge instance on AWS, has 1 L4 GPU
|
||||
plugins:
|
||||
- docker#v5.2.0:
|
||||
image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
always-pull: true
|
||||
propagate-environment: true
|
||||
environment:
|
||||
- "HF_HOME=/fsx/hf_cache"
|
||||
- "HF_TOKEN"
|
||||
volumes:
|
||||
- "/fsx/hf_cache:/fsx/hf_cache"
|
||||
|
||||
- label: "Diffusion Cache Backend Test"
|
||||
timeout_in_minutes: 15
|
||||
depends_on: upload-merge-pipeline
|
||||
commands:
|
||||
- pytest -s -v -m 'core_model and cache and diffusion and not distributed_cuda and L4'
|
||||
agents:
|
||||
queue: "gpu_1_queue" # g6.4xlarge instance on AWS, has 1 L4 GPU
|
||||
plugins:
|
||||
- docker#v5.2.0:
|
||||
image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
always-pull: true
|
||||
propagate-environment: true
|
||||
environment:
|
||||
- "HF_HOME=/fsx/hf_cache"
|
||||
- "HF_TOKEN"
|
||||
volumes:
|
||||
- "/fsx/hf_cache:/fsx/hf_cache"
|
||||
|
||||
- label: "Diffusion Sequence Parallelism Test"
|
||||
timeout_in_minutes: 20
|
||||
depends_on: upload-merge-pipeline
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/offline_inference/test_sequence_parallel.py
|
||||
- pytest -s -v tests/diffusion/distributed/test_ulysses_uaa_perf.py
|
||||
agents:
|
||||
queue: "gpu_4_queue" # g6.12xlarge instance on AWS, has 4 L4 GPU
|
||||
plugins:
|
||||
- docker#v5.2.0:
|
||||
image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
always-pull: true
|
||||
propagate-environment: true
|
||||
shm-size: "8gb"
|
||||
environment:
|
||||
- "HF_HOME=/fsx/hf_cache"
|
||||
- "HF_TOKEN"
|
||||
volumes:
|
||||
- "/fsx/hf_cache:/fsx/hf_cache"
|
||||
|
||||
- label: "Diffusion Tensor Parallelism Test"
|
||||
timeout_in_minutes: 20
|
||||
depends_on: upload-merge-pipeline
|
||||
commands:
|
||||
- pytest -s -v tests/e2e/offline_inference/test_zimage_parallelism.py
|
||||
agents:
|
||||
queue: "gpu_4_queue" # g6.12xlarge instance on AWS, has 4 L4 GPU
|
||||
plugins:
|
||||
- docker#v5.2.0:
|
||||
image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
always-pull: true
|
||||
propagate-environment: true
|
||||
shm-size: "8gb"
|
||||
environment:
|
||||
- "HF_HOME=/fsx/hf_cache"
|
||||
- "HF_TOKEN"
|
||||
volumes:
|
||||
- "/fsx/hf_cache:/fsx/hf_cache"
|
||||
|
||||
- label: "Diffusion GPU Worker Test"
|
||||
timeout_in_minutes: 20
|
||||
depends_on: upload-merge-pipeline
|
||||
commands:
|
||||
- pytest -s -v tests/diffusion/test_diffusion_worker.py
|
||||
agents:
|
||||
queue: "gpu_4_queue" # g6.12xlarge instance on AWS, has 4 L4 GPU
|
||||
plugins:
|
||||
- docker#v5.2.0:
|
||||
image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
always-pull: true
|
||||
propagate-environment: true
|
||||
shm-size: "8gb"
|
||||
environment:
|
||||
- "HF_HOME=/fsx/hf_cache"
|
||||
- "HF_TOKEN"
|
||||
volumes:
|
||||
- "/fsx/hf_cache:/fsx/hf_cache"
|
||||
|
||||
- label: "Engine Test"
|
||||
depends_on: upload-merge-pipeline
|
||||
commands:
|
||||
- |
|
||||
timeout 15m bash -c '
|
||||
export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
pytest -s -v tests/engine/test_async_omni_engine_abort.py
|
||||
'
|
||||
agents:
|
||||
queue: "gpu_4_queue" # g6.12xlarge instance on AWS, has 4 L4 GPU
|
||||
plugins:
|
||||
- docker#v5.2.0:
|
||||
image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
always-pull: true
|
||||
propagate-environment: true
|
||||
shm-size: "8gb"
|
||||
environment:
|
||||
- "HF_HOME=/fsx/hf_cache"
|
||||
- "HF_TOKEN"
|
||||
volumes:
|
||||
- "/fsx/hf_cache:/fsx/hf_cache"
|
||||
|
||||
- label: "Omni Model Test"
|
||||
timeout_in_minutes: 30
|
||||
depends_on: upload-merge-pipeline
|
||||
commands:
|
||||
- export VLLM_LOGGING_LEVEL=DEBUG
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- pytest -s -v tests/e2e/offline_inference/test_qwen2_5_omni.py
|
||||
- pytest -s -v tests/e2e/online_serving/test_qwen2_5_omni.py -m "advanced_model" --run-level "advanced_model"
|
||||
agents:
|
||||
queue: "gpu_4_queue" # g6.12xlarge instance on AWS, has 4 L4 GPU
|
||||
plugins:
|
||||
- docker#v5.2.0:
|
||||
image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
always-pull: true
|
||||
propagate-environment: true
|
||||
environment:
|
||||
- "HF_HOME=/fsx/hf_cache"
|
||||
- "HF_TOKEN"
|
||||
volumes:
|
||||
- "/fsx/hf_cache:/fsx/hf_cache"
|
||||
|
||||
- label: "Qwen3-TTS CustomVoice E2E Test"
|
||||
depends_on: upload-merge-pipeline
|
||||
commands:
|
||||
- |
|
||||
timeout 20m bash -c '
|
||||
export VLLM_LOGGING_LEVEL=DEBUG
|
||||
export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
|
||||
pytest -s -v tests/e2e/online_serving/test_qwen3_tts_customvoice.py -m "advanced_model" --run-level "advanced_model" && pytest -s -v tests/e2e/offline_inference/test_qwen3_tts_customvoice.py
|
||||
'
|
||||
agents:
|
||||
queue: "gpu_1_queue"
|
||||
plugins:
|
||||
- docker#v5.2.0:
|
||||
image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
always-pull: true
|
||||
propagate-environment: true
|
||||
shm-size: "8gb"
|
||||
environment:
|
||||
- "HF_HOME=/fsx/hf_cache"
|
||||
volumes:
|
||||
- "/fsx/hf_cache:/fsx/hf_cache"
|
||||
|
||||
- label: "Qwen3-TTS Base E2E Test"
|
||||
depends_on: upload-merge-pipeline
|
||||
commands:
|
||||
- |
|
||||
timeout 20m bash -c '
|
||||
export VLLM_LOGGING_LEVEL=DEBUG
|
||||
export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
|
||||
pytest -s -v tests/e2e/online_serving/test_qwen3_tts_base.py -m "advanced_model" --run-level "advanced_model" && pytest -s -v tests/e2e/offline_inference/test_qwen3_tts_base.py
|
||||
'
|
||||
agents:
|
||||
queue: "gpu_1_queue"
|
||||
plugins:
|
||||
- docker#v5.2.0:
|
||||
image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
always-pull: true
|
||||
propagate-environment: true
|
||||
shm-size: "8gb"
|
||||
environment:
|
||||
- "HF_HOME=/fsx/hf_cache"
|
||||
volumes:
|
||||
- "/fsx/hf_cache:/fsx/hf_cache"
|
||||
|
||||
- label: "Omni Model Test with H100"
|
||||
timeout_in_minutes: 30
|
||||
depends_on: upload-merge-pipeline
|
||||
commands:
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- export VLLM_TEST_CLEAN_GPU_MEMORY="1"
|
||||
- pytest -s -v tests/e2e/offline_inference/test_qwen3_omni.py
|
||||
- pytest -s -v tests/e2e/online_serving/test_qwen3_omni.py -m "advanced_model" --run-level "advanced_model"
|
||||
- pytest -s -v tests/e2e/online_serving/test_mimo_audio.py -m "advanced_model" --run-level "advanced_model"
|
||||
agents:
|
||||
queue: "mithril-h100-pool"
|
||||
plugins:
|
||||
- kubernetes:
|
||||
podSpec:
|
||||
containers:
|
||||
- image: 936637512419.dkr.ecr.us-west-2.amazonaws.com/vllm-ci-pull-through-cache/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 2
|
||||
volumeMounts:
|
||||
- name: devshm
|
||||
mountPath: /dev/shm
|
||||
- name: hf-cache
|
||||
mountPath: /root/.cache/huggingface
|
||||
env:
|
||||
- name: HF_HOME
|
||||
value: /root/.cache/huggingface
|
||||
- name: HF_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: hf-token-secret
|
||||
key: token
|
||||
nodeSelector:
|
||||
node.kubernetes.io/instance-type: gpu-h100-sxm
|
||||
volumes:
|
||||
- name: devshm
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
- name: hf-cache
|
||||
hostPath:
|
||||
path: /mnt/hf-cache
|
||||
type: DirectoryOrCreate
|
||||
|
||||
- label: "Diffusion Image Edit Test with H100 (1 GPU)"
|
||||
timeout_in_minutes: 20
|
||||
depends_on: upload-merge-pipeline
|
||||
commands:
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- pytest -s -v tests/e2e/online_serving/test_image_gen_edit.py
|
||||
agents:
|
||||
queue: "mithril-h100-pool"
|
||||
plugins:
|
||||
- kubernetes:
|
||||
podSpec:
|
||||
containers:
|
||||
- image: 936637512419.dkr.ecr.us-west-2.amazonaws.com/vllm-ci-pull-through-cache/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 1
|
||||
volumeMounts:
|
||||
- name: devshm
|
||||
mountPath: /dev/shm
|
||||
- name: hf-cache
|
||||
mountPath: /root/.cache/huggingface
|
||||
env:
|
||||
- name: HF_HOME
|
||||
value: /root/.cache/huggingface
|
||||
- name: HF_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: hf-token-secret
|
||||
key: token
|
||||
nodeSelector:
|
||||
node.kubernetes.io/instance-type: gpu-h100-sxm
|
||||
volumes:
|
||||
- name: devshm
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
- name: hf-cache
|
||||
hostPath:
|
||||
path: /mnt/hf-cache
|
||||
type: DirectoryOrCreate
|
||||
|
||||
- label: "Bagel Model Test with H100 (Real Weights)"
|
||||
timeout_in_minutes: 60
|
||||
depends_on: upload-merge-pipeline
|
||||
commands:
|
||||
- |
|
||||
timeout 55m bash -c '
|
||||
set -e
|
||||
export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
export VLLM_TEST_CLEAN_GPU_MEMORY=1
|
||||
export VLLM_IMAGE_FETCH_TIMEOUT=60
|
||||
pytest -s -v tests/e2e/offline_inference/test_bagel_text2img.py -m "advanced_model" --run-level "advanced_model" -k "shared_memory"
|
||||
pytest -s -v tests/e2e/offline_inference/test_bagel_img2img.py -m "advanced_model" --run-level "advanced_model"
|
||||
pytest -s -v tests/e2e/online_serving/test_bagel_online.py -m "advanced_model" --run-level "advanced_model"
|
||||
'
|
||||
agents:
|
||||
queue: "mithril-h100-pool"
|
||||
plugins:
|
||||
- kubernetes:
|
||||
podSpec:
|
||||
containers:
|
||||
- image: 936637512419.dkr.ecr.us-west-2.amazonaws.com/vllm-ci-pull-through-cache/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 1
|
||||
volumeMounts:
|
||||
- name: devshm
|
||||
mountPath: /dev/shm
|
||||
- name: hf-cache
|
||||
mountPath: /root/.cache/huggingface
|
||||
env:
|
||||
- name: HF_HOME
|
||||
value: /root/.cache/huggingface
|
||||
- name: HF_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: hf-token-secret
|
||||
key: token
|
||||
nodeSelector:
|
||||
node.kubernetes.io/instance-type: gpu-h100-sxm
|
||||
volumes:
|
||||
- name: devshm
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
- name: hf-cache
|
||||
hostPath:
|
||||
path: /mnt/hf-cache
|
||||
type: DirectoryOrCreate
|
||||
|
||||
- label: "Voxtral-TTS E2E Test"
|
||||
timeout_in_minutes: 20
|
||||
depends_on: upload-merge-pipeline
|
||||
commands:
|
||||
- |
|
||||
timeout 20m bash -c '
|
||||
export VLLM_LOGGING_LEVEL=DEBUG
|
||||
export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
pytest -s -v tests/e2e/online_serving/test_voxtral_tts.py -m "advanced_model" --run-level "advanced_model"
|
||||
pytest -s -v tests/e2e/offline_inference/test_voxtral_tts.py -m "advanced_model" --run-level "advanced_model"
|
||||
'
|
||||
agents:
|
||||
queue: "mithril-h100-pool"
|
||||
plugins:
|
||||
- kubernetes:
|
||||
podSpec:
|
||||
containers:
|
||||
- image: 936637512419.dkr.ecr.us-west-2.amazonaws.com/vllm-ci-pull-through-cache/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 1
|
||||
volumeMounts:
|
||||
- name: devshm
|
||||
mountPath: /dev/shm
|
||||
- name: hf-cache
|
||||
mountPath: /root/.cache/huggingface
|
||||
env:
|
||||
- name: HF_HOME
|
||||
value: /root/.cache/huggingface
|
||||
nodeSelector:
|
||||
node.kubernetes.io/instance-type: gpu-h100-sxm
|
||||
volumes:
|
||||
- name: devshm
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
- name: hf-cache
|
||||
hostPath:
|
||||
path: /mnt/hf-cache
|
||||
type: DirectoryOrCreate
|
||||
@@ -0,0 +1,466 @@
|
||||
steps:
|
||||
- label: ":full_moon: Omni Model Test with H100"
|
||||
timeout_in_minutes: 90
|
||||
depends_on: upload-nightly-pipeline
|
||||
if: build.env("NIGHTLY") == "1" || build.pull_request.labels includes "nightly-test"
|
||||
commands:
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- |
|
||||
set +e
|
||||
pytest -s -v tests/e2e/online_serving/test_qwen3_omni_expansion.py -m "advanced_model" --run-level "advanced_model"
|
||||
EXIT1=$$?
|
||||
pytest -s -v tests/examples/online_serving/test_qwen3_omni.py -m "advanced_model" --run-level "advanced_model"
|
||||
EXIT2=$$?
|
||||
pytest -s -v tests/e2e/online_serving/test_voxtral_tts.py -m "advanced_model" --run-level "advanced_model"
|
||||
EXIT3=$$?
|
||||
pytest -s -v tests/e2e/offline_inference/test_voxtral_tts.py -m "advanced_model" --run-level "advanced_model"
|
||||
EXIT4=$$?
|
||||
exit $$((EXIT1 | EXIT2 | EXIT3 | EXIT4))
|
||||
agents:
|
||||
queue: "mithril-h100-pool"
|
||||
plugins:
|
||||
- kubernetes:
|
||||
podSpec:
|
||||
containers:
|
||||
- image: 936637512419.dkr.ecr.us-west-2.amazonaws.com/vllm-ci-pull-through-cache/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 2
|
||||
volumeMounts:
|
||||
- name: devshm
|
||||
mountPath: /dev/shm
|
||||
- name: hf-cache
|
||||
mountPath: /root/.cache/huggingface
|
||||
env:
|
||||
- name: HF_HOME
|
||||
value: /root/.cache/huggingface
|
||||
- name: HF_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: hf-token-secret
|
||||
key: token
|
||||
nodeSelector:
|
||||
node.kubernetes.io/instance-type: gpu-h100-sxm
|
||||
volumes:
|
||||
- name: devshm
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
- name: hf-cache
|
||||
hostPath:
|
||||
path: /mnt/hf-cache
|
||||
type: DirectoryOrCreate
|
||||
|
||||
- label: ":full_moon: Omni Model Test with L4"
|
||||
timeout_in_minutes: 90
|
||||
depends_on: upload-nightly-pipeline
|
||||
if: build.env("NIGHTLY") == "1" || build.pull_request.labels includes "nightly-test"
|
||||
commands:
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
|
||||
- pytest -s -v tests/examples/ -m "advanced_model and L4 and omni" --run-level "advanced_model"
|
||||
- pytest -s -v tests/e2e/online_serving/test_*_expansion.py -m "advanced_model and L4 and omni" --run-level "advanced_model"
|
||||
agents:
|
||||
queue: "gpu_4_queue" # g6.12xlarge instance on AWS, has 4 L4 GPU
|
||||
plugins:
|
||||
- docker#v5.2.0:
|
||||
image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
always-pull: true
|
||||
propagate-environment: true
|
||||
shm-size: "8gb"
|
||||
environment:
|
||||
- "HF_HOME=/fsx/hf_cache"
|
||||
- "HF_TOKEN"
|
||||
volumes:
|
||||
- "/fsx/hf_cache:/fsx/hf_cache"
|
||||
|
||||
- label: ":full_moon: Diffusion Model Test with H100"
|
||||
timeout_in_minutes: 120
|
||||
depends_on: upload-nightly-pipeline
|
||||
if: build.env("NIGHTLY") == "1" || build.pull_request.labels includes "nightly-test"
|
||||
commands:
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- pytest -s -v tests/e2e/online_serving/test_*_expansion.py -k "not test_wan22_expansion" -m "advanced_model and diffusion and H100" --run-level "advanced_model"
|
||||
agents:
|
||||
queue: "mithril-h100-pool"
|
||||
plugins:
|
||||
- kubernetes:
|
||||
podSpec:
|
||||
containers:
|
||||
- image: 936637512419.dkr.ecr.us-west-2.amazonaws.com/vllm-ci-pull-through-cache/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 2
|
||||
volumeMounts:
|
||||
- name: devshm
|
||||
mountPath: /dev/shm
|
||||
- name: hf-cache
|
||||
mountPath: /root/.cache/huggingface
|
||||
env:
|
||||
- name: HF_HOME
|
||||
value: /root/.cache/huggingface
|
||||
- name: HF_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: hf-token-secret
|
||||
key: token
|
||||
nodeSelector:
|
||||
node.kubernetes.io/instance-type: gpu-h100-sxm
|
||||
volumes:
|
||||
- name: devshm
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
- name: hf-cache
|
||||
hostPath:
|
||||
path: /mnt/hf-cache
|
||||
type: DirectoryOrCreate
|
||||
|
||||
- label: ":full_moon: Diffusion Model Wan22 completed Test with H100"
|
||||
timeout_in_minutes: 90
|
||||
depends_on: upload-nightly-pipeline
|
||||
if: build.env("NIGHTLY") == "1" || build.pull_request.labels includes "nightly-test"
|
||||
commands:
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- pytest -s -v tests/e2e/online_serving/test_wan22_expansion.py -m "advanced_model" --run-level "advanced_model"
|
||||
agents:
|
||||
queue: "mithril-h100-pool"
|
||||
plugins:
|
||||
- kubernetes:
|
||||
podSpec:
|
||||
containers:
|
||||
- image: 936637512419.dkr.ecr.us-west-2.amazonaws.com/vllm-ci-pull-through-cache/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 2
|
||||
volumeMounts:
|
||||
- name: devshm
|
||||
mountPath: /dev/shm
|
||||
- name: hf-cache
|
||||
mountPath: /root/.cache/huggingface
|
||||
env:
|
||||
- name: HF_HOME
|
||||
value: /root/.cache/huggingface
|
||||
- name: HF_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: hf-token-secret
|
||||
key: token
|
||||
nodeSelector:
|
||||
node.kubernetes.io/instance-type: gpu-h100-sxm
|
||||
volumes:
|
||||
- name: devshm
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
- name: hf-cache
|
||||
hostPath:
|
||||
path: /mnt/hf-cache
|
||||
type: DirectoryOrCreate
|
||||
|
||||
- label: ":full_moon: Diffusion Model Test with L4"
|
||||
timeout_in_minutes: 60
|
||||
depends_on: upload-nightly-pipeline
|
||||
if: build.env("NIGHTLY") == "1" || build.pull_request.labels includes "nightly-test"
|
||||
commands:
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- pytest -s -v tests/e2e/online_serving/test_*_expansion.py -m "advanced_model and diffusion and L4" --run-level "advanced_model"
|
||||
agents:
|
||||
queue: "gpu_4_queue" # g6.12xlarge instance on AWS, has 4 L4 GPU
|
||||
plugins:
|
||||
- docker#v5.2.0:
|
||||
image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
always-pull: true
|
||||
propagate-environment: true
|
||||
shm-size: "8gb"
|
||||
environment:
|
||||
- "HF_HOME=/fsx/hf_cache"
|
||||
- "HF_TOKEN"
|
||||
volumes:
|
||||
- "/fsx/hf_cache:/fsx/hf_cache"
|
||||
|
||||
|
||||
- label: ":full_moon: Documentation Example Code Test with H100"
|
||||
timeout_in_minutes: 60
|
||||
depends_on: upload-nightly-pipeline
|
||||
if: build.env("NIGHTLY") == "1" || build.pull_request.labels includes "nightly-test"
|
||||
commands:
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- export VLLM_TEST_CLEAN_GPU_MEMORY="1"
|
||||
- pytest -s -v tests/examples/online_serving/test_text_to_image.py tests/examples/offline_inference/test_text_to_image.py -m "advanced_model and example and H100" --run-level "advanced_model"
|
||||
agents:
|
||||
queue: "mithril-h100-pool"
|
||||
plugins:
|
||||
- kubernetes:
|
||||
podSpec:
|
||||
containers:
|
||||
- image: 936637512419.dkr.ecr.us-west-2.amazonaws.com/vllm-ci-pull-through-cache/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 2
|
||||
volumeMounts:
|
||||
- name: devshm
|
||||
mountPath: /dev/shm
|
||||
- name: hf-cache
|
||||
mountPath: /root/.cache/huggingface
|
||||
env:
|
||||
- name: HF_HOME
|
||||
value: /root/.cache/huggingface
|
||||
- name: HF_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: hf-token-secret
|
||||
key: token
|
||||
nodeSelector:
|
||||
node.kubernetes.io/instance-type: gpu-h100-sxm
|
||||
volumes:
|
||||
- name: devshm
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
- name: hf-cache
|
||||
hostPath:
|
||||
path: /mnt/hf-cache
|
||||
type: DirectoryOrCreate
|
||||
|
||||
- label: ":full_moon: Omni Model Perf Test & Test Case Statistics"
|
||||
key: nightly-omni-performance
|
||||
timeout_in_minutes: 180
|
||||
depends_on: upload-nightly-pipeline
|
||||
if: build.env("NIGHTLY") == "1" || build.pull_request.labels includes "nightly-test"
|
||||
commands:
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- export BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
|
||||
- pytest -s -v tests/dfx/perf/scripts/run_benchmark.py
|
||||
- buildkite-agent artifact upload "tests/dfx/perf/results/*.json"
|
||||
- python tools/nightly/buildkite_testcase_statistics.py -o tests/dfx/perf/results/buildkite_testcase_statistics.html
|
||||
- buildkite-agent artifact upload "tests/dfx/perf/results/*.html"
|
||||
agents:
|
||||
queue: "mithril-h100-pool"
|
||||
plugins:
|
||||
- kubernetes:
|
||||
podSpec:
|
||||
containers:
|
||||
- image: 936637512419.dkr.ecr.us-west-2.amazonaws.com/vllm-ci-pull-through-cache/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 2
|
||||
volumeMounts:
|
||||
- name: devshm
|
||||
mountPath: /dev/shm
|
||||
- name: hf-cache
|
||||
mountPath: /root/.cache/huggingface
|
||||
env:
|
||||
- name: HF_HOME
|
||||
value: /root/.cache/huggingface
|
||||
- name: HF_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: hf-token-secret
|
||||
key: token
|
||||
nodeSelector:
|
||||
node.kubernetes.io/instance-type: gpu-h100-sxm
|
||||
volumes:
|
||||
- name: devshm
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
- name: hf-cache
|
||||
hostPath:
|
||||
path: /mnt/hf-cache
|
||||
type: DirectoryOrCreate
|
||||
|
||||
- label: ":full_moon: GEBench Accuracy Test with H100"
|
||||
key: nightly-gebench-accuracy
|
||||
timeout_in_minutes: 60
|
||||
depends_on: upload-nightly-pipeline
|
||||
if: build.env("NIGHTLY") == "1" || build.pull_request.labels includes "nightly-test"
|
||||
commands:
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- pytest -s -v tests/e2e/accuracy/test_gebench_h100_smoke.py --run-level advanced_model --gebench-model Qwen/Qwen-Image-2512 --accuracy-judge-model QuantTrio/Qwen3-VL-30B-A3B-Instruct-AWQ --accuracy-gpu 0 --gebench-port 8093 --accuracy-workers 1
|
||||
- buildkite-agent artifact upload "tests/e2e/accuracy/artifacts/gebench_qwen-image-2512/summary*.json"
|
||||
agents:
|
||||
queue: "mithril-h100-pool"
|
||||
plugins:
|
||||
- kubernetes:
|
||||
podSpec:
|
||||
containers:
|
||||
- image: 936637512419.dkr.ecr.us-west-2.amazonaws.com/vllm-ci-pull-through-cache/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 1
|
||||
volumeMounts:
|
||||
- name: devshm
|
||||
mountPath: /dev/shm
|
||||
- name: hf-cache
|
||||
mountPath: /root/.cache/huggingface
|
||||
env:
|
||||
- name: HF_HOME
|
||||
value: /root/.cache/huggingface
|
||||
- name: HF_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: hf-token-secret
|
||||
key: token
|
||||
nodeSelector:
|
||||
node.kubernetes.io/instance-type: gpu-h100-sxm
|
||||
volumes:
|
||||
- name: devshm
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
- name: hf-cache
|
||||
hostPath:
|
||||
path: /mnt/hf-cache
|
||||
type: DirectoryOrCreate
|
||||
|
||||
- label: ":full_moon: GEdit-Bench Accuracy Test with H100"
|
||||
key: nightly-gedit-bench-accuracy
|
||||
timeout_in_minutes: 60
|
||||
depends_on: upload-nightly-pipeline
|
||||
if: build.env("NIGHTLY") == "1" || build.pull_request.labels includes "nightly-test"
|
||||
commands:
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- pytest -s -v tests/e2e/accuracy/test_gedit_bench_h100_smoke.py --run-level advanced_model --gedit-model Qwen/Qwen-Image-Edit --accuracy-judge-model QuantTrio/Qwen3-VL-30B-A3B-Instruct-AWQ --accuracy-gpu 0 --gedit-port 8093 --gedit-samples-per-group 20 --accuracy-workers 1
|
||||
- buildkite-agent artifact upload "tests/e2e/accuracy/artifacts/gedit_scores_qwen-image-edit/qwen-image-edit_all_all_vie_score_*.csv"
|
||||
- buildkite-agent artifact upload "tests/e2e/accuracy/artifacts/gedit_scores_qwen-image-edit/qwen-image-edit_all_all_summary_*.json"
|
||||
agents:
|
||||
queue: "mithril-h100-pool"
|
||||
plugins:
|
||||
- kubernetes:
|
||||
podSpec:
|
||||
containers:
|
||||
- image: 936637512419.dkr.ecr.us-west-2.amazonaws.com/vllm-ci-pull-through-cache/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 1
|
||||
volumeMounts:
|
||||
- name: devshm
|
||||
mountPath: /dev/shm
|
||||
- name: hf-cache
|
||||
mountPath: /root/.cache/huggingface
|
||||
env:
|
||||
- name: HF_HOME
|
||||
value: /root/.cache/huggingface
|
||||
- name: VLLM_HTTP_TIMEOUT_KEEP_ALIVE
|
||||
value: "120"
|
||||
- name: HF_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: hf-token-secret
|
||||
key: token
|
||||
nodeSelector:
|
||||
node.kubernetes.io/instance-type: gpu-h100-sxm
|
||||
volumes:
|
||||
- name: devshm
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
- name: hf-cache
|
||||
hostPath:
|
||||
path: /mnt/hf-cache
|
||||
type: DirectoryOrCreate
|
||||
|
||||
- label: ":full_moon: Wan22 I2V Accuracy Test with H100"
|
||||
key: nightly-wan22-i2v-accuracy
|
||||
timeout_in_minutes: 180
|
||||
depends_on: upload-nightly-pipeline
|
||||
if: build.env("NIGHTLY") == "1" || build.pull_request.labels includes "nightly-test"
|
||||
commands:
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- pytest -s -v tests/e2e/accuracy/wan22_i2v/test_wan22_i2v_video_similarity.py --run-level advanced_model
|
||||
agents:
|
||||
queue: "mithril-h100-pool"
|
||||
plugins:
|
||||
- kubernetes:
|
||||
podSpec:
|
||||
containers:
|
||||
- image: 936637512419.dkr.ecr.us-west-2.amazonaws.com/vllm-ci-pull-through-cache/q9t5s3a7/vllm-ci-test-repo:c392ce21e9cf9ea65c52b866447793db10e0261c
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 2
|
||||
volumeMounts:
|
||||
- name: devshm
|
||||
mountPath: /dev/shm
|
||||
- name: hf-cache
|
||||
mountPath: /root/.cache/huggingface
|
||||
env:
|
||||
- name: HF_HOME
|
||||
value: /root/.cache/huggingface
|
||||
- name: HF_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: hf-token-secret
|
||||
key: token
|
||||
nodeSelector:
|
||||
node.kubernetes.io/instance-type: gpu-h100-sxm
|
||||
volumes:
|
||||
- name: devshm
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
- name: hf-cache
|
||||
hostPath:
|
||||
path: /mnt/hf-cache
|
||||
type: DirectoryOrCreate
|
||||
|
||||
- label: ":full_moon: Qwen-Image Diffusion Perf Test with H100"
|
||||
key: nightly-qwen-image-performance
|
||||
timeout_in_minutes: 300
|
||||
depends_on: upload-nightly-pipeline
|
||||
if: build.env("NIGHTLY") == "1" || build.pull_request.labels includes "nightly-test"
|
||||
commands:
|
||||
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
- export DIFFUSION_BENCHMARK_DIR=tests/dfx/perf/results
|
||||
- export CACHE_DIT_VERSION=1.3.0
|
||||
- pytest -s -v tests/dfx/perf/scripts/run_diffusion_benchmark.py --config-file tests/dfx/perf/tests/test_qwen_image_vllm_omni.json
|
||||
- buildkite-agent artifact upload "tests/dfx/perf/results/benchmark_results_*.json"
|
||||
- buildkite-agent artifact upload "tests/dfx/perf/results/logs/*.log"
|
||||
agents:
|
||||
queue: "mithril-h100-pool"
|
||||
plugins:
|
||||
- kubernetes:
|
||||
podSpec:
|
||||
containers:
|
||||
- image: 936637512419.dkr.ecr.us-west-2.amazonaws.com/vllm-ci-pull-through-cache/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 4
|
||||
volumeMounts:
|
||||
- name: devshm
|
||||
mountPath: /dev/shm
|
||||
- name: hf-cache
|
||||
mountPath: /root/.cache/huggingface
|
||||
env:
|
||||
- name: HF_HOME
|
||||
value: /root/.cache/huggingface
|
||||
- name: HF_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: hf-token-secret
|
||||
key: token
|
||||
nodeSelector:
|
||||
node.kubernetes.io/instance-type: gpu-h100-sxm
|
||||
volumes:
|
||||
- name: devshm
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
- name: hf-cache
|
||||
hostPath:
|
||||
path: /mnt/hf-cache
|
||||
type: DirectoryOrCreate
|
||||
|
||||
# No need to run this step for PRs with label nightly-test
|
||||
- label: ":email: Nightly Collection & Email"
|
||||
key: nightly-perf-distribution
|
||||
depends_on:
|
||||
- nightly-omni-performance
|
||||
- nightly-qwen-image-performance
|
||||
if: build.env("NIGHTLY") == "1"
|
||||
commands:
|
||||
- pip install openpyxl
|
||||
- export DEFAULT_INPUT_DIR=tests/dfx/perf/results
|
||||
- export DEFAULT_OUTPUT_DIR=tests/dfx/perf/results
|
||||
- buildkite-agent artifact download "tests/dfx/perf/results/*.json" . --step nightly-omni-performance
|
||||
- buildkite-agent artifact download "tests/dfx/perf/results/*.json" . --step nightly-qwen-image-performance
|
||||
- buildkite-agent artifact download "tests/dfx/perf/results/*.html" . --step nightly-omni-performance
|
||||
- python tools/nightly/generate_nightly_perf_excel.py
|
||||
- python tools/nightly/generate_nightly_perf_html.py
|
||||
- python tools/nightly/send_nightly_email.py --report-file "tests/dfx/perf/results/*.xlsx, tests/dfx/perf/results/*.html"
|
||||
- buildkite-agent artifact upload "tests/dfx/perf/results/*.xlsx"
|
||||
- buildkite-agent artifact upload "tests/dfx/perf/results/*.html"
|
||||
agents:
|
||||
queue: "cpu_queue_premerge"
|
||||
@@ -0,0 +1,528 @@
|
||||
steps:
|
||||
- label: "Simple Unit Test"
|
||||
depends_on: upload-ready-pipeline
|
||||
commands:
|
||||
- "timeout 20m pytest -v -s -m 'core_model and cpu' --cov=vllm_omni --cov-branch --cov-report=term-missing --cov-report=html --cov-report=xml"
|
||||
agents:
|
||||
queue: "gpu_1_queue"
|
||||
plugins:
|
||||
- docker#v5.2.0:
|
||||
image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
always-pull: true
|
||||
propagate-environment: true
|
||||
environment:
|
||||
- "HF_HOME=/fsx/hf_cache"
|
||||
- "HF_TOKEN"
|
||||
volumes:
|
||||
- "/fsx/hf_cache:/fsx/hf_cache"
|
||||
|
||||
- label: "Voxtral TTS CUDA Unit Test"
|
||||
timeout_in_minutes: 10
|
||||
depends_on: upload-ready-pipeline
|
||||
commands:
|
||||
- "timeout 10m pytest -s -v tests/model_executor/models/voxtral_tts/test_cuda_graph_acoustic_transformer.py"
|
||||
agents:
|
||||
queue: "gpu_1_queue"
|
||||
plugins:
|
||||
- docker#v5.2.0:
|
||||
image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
always-pull: true
|
||||
propagate-environment: true
|
||||
environment:
|
||||
- "HF_HOME=/fsx/hf_cache"
|
||||
volumes:
|
||||
- "/fsx/hf_cache:/fsx/hf_cache"
|
||||
|
||||
- label: "Diffusion Model Test"
|
||||
depends_on: upload-ready-pipeline
|
||||
commands:
|
||||
- timeout 30m pytest -s -v tests/e2e/offline_inference/test_t2i_model.py -m "core_model and diffusion" --run-level "core_model"
|
||||
agents:
|
||||
queue: "gpu_1_queue"
|
||||
plugins:
|
||||
- docker#v5.2.0:
|
||||
image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
always-pull: true
|
||||
propagate-environment: true
|
||||
environment:
|
||||
- "HF_HOME=/fsx/hf_cache"
|
||||
- "HF_TOKEN"
|
||||
volumes:
|
||||
- "/fsx/hf_cache:/fsx/hf_cache"
|
||||
|
||||
- label: "Diffusion Batching Test"
|
||||
depends_on: upload-ready-pipeline
|
||||
commands:
|
||||
- timeout 20m pytest -s -v tests/e2e/offline_inference/test_qwen_image_diffusion_batching.py -m "core_model and diffusion" --run-level "core_model"
|
||||
agents:
|
||||
queue: "gpu_1_queue" # g6.4xlarge instance on AWS, has 1 L4 GPU
|
||||
plugins:
|
||||
- docker#v5.2.0:
|
||||
image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
always-pull: true
|
||||
propagate-environment: true
|
||||
environment:
|
||||
- "HF_HOME=/fsx/hf_cache"
|
||||
- "HF_TOKEN"
|
||||
volumes:
|
||||
- "/fsx/hf_cache:/fsx/hf_cache"
|
||||
|
||||
- label: "Custom Pipeline Test"
|
||||
depends_on: upload-ready-pipeline
|
||||
commands:
|
||||
- timeout 20m pytest -s -v tests/e2e/offline_inference/custom_pipeline/ -m "core_model"
|
||||
agents:
|
||||
queue: "gpu_1_queue"
|
||||
plugins:
|
||||
- docker#v5.2.0:
|
||||
image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
always-pull: true
|
||||
propagate-environment: true
|
||||
environment:
|
||||
- "HF_HOME=/fsx/hf_cache"
|
||||
volumes:
|
||||
- "/fsx/hf_cache:/fsx/hf_cache"
|
||||
|
||||
- label: "Diffusion Model CPU offloading Test"
|
||||
depends_on: upload-ready-pipeline
|
||||
commands:
|
||||
- timeout 10m pytest -s -v tests/e2e/offline_inference/test_diffusion_cpu_offload.py
|
||||
agents:
|
||||
queue: "gpu_1_queue" # g6.4xlarge instance on AWS, has 1 L4 GPU
|
||||
plugins:
|
||||
- docker#v5.2.0:
|
||||
image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
always-pull: true
|
||||
propagate-environment: true
|
||||
environment:
|
||||
- "HF_HOME=/fsx/hf_cache"
|
||||
- "HF_TOKEN"
|
||||
volumes:
|
||||
- "/fsx/hf_cache:/fsx/hf_cache"
|
||||
|
||||
- label: "Audio Generation Model Test"
|
||||
depends_on: upload-ready-pipeline
|
||||
commands:
|
||||
- timeout 20m pytest -s -v tests/e2e/offline_inference/test_stable_audio_model.py
|
||||
agents:
|
||||
queue: "gpu_1_queue" # g6.4xlarge instance on AWS, has 1 L4 GPU
|
||||
plugins:
|
||||
- docker#v5.2.0:
|
||||
image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
always-pull: true
|
||||
propagate-environment: true
|
||||
environment:
|
||||
- "HF_HOME=/fsx/hf_cache"
|
||||
- "HF_TOKEN"
|
||||
volumes:
|
||||
- "/fsx/hf_cache:/fsx/hf_cache"
|
||||
|
||||
- label: "Diffusion Cache Backend Test"
|
||||
depends_on: upload-ready-pipeline
|
||||
commands:
|
||||
- timeout 15m pytest -s -v -m 'core_model and cache and diffusion and not distributed_cuda and L4'
|
||||
agents:
|
||||
queue: "gpu_1_queue" # g6.4xlarge instance on AWS, has 1 L4 GPU
|
||||
plugins:
|
||||
- docker#v5.2.0:
|
||||
image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
always-pull: true
|
||||
propagate-environment: true
|
||||
environment:
|
||||
- "HF_HOME=/fsx/hf_cache"
|
||||
- "HF_TOKEN"
|
||||
volumes:
|
||||
- "/fsx/hf_cache:/fsx/hf_cache"
|
||||
|
||||
- label: "Diffusion Sequence Parallelism Test"
|
||||
depends_on: upload-ready-pipeline
|
||||
commands:
|
||||
- timeout 20m pytest -s -v tests/e2e/offline_inference/test_sequence_parallel.py -m core_model
|
||||
agents:
|
||||
queue: "gpu_4_queue" # g6.12xlarge instance on AWS, has 4 L4 GPU
|
||||
plugins:
|
||||
- docker#v5.2.0:
|
||||
image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
always-pull: true
|
||||
propagate-environment: true
|
||||
shm-size: "8gb"
|
||||
environment:
|
||||
- "HF_HOME=/fsx/hf_cache"
|
||||
- "HF_TOKEN"
|
||||
volumes:
|
||||
- "/fsx/hf_cache:/fsx/hf_cache"
|
||||
|
||||
- label: "Diffusion GPU Worker Test"
|
||||
depends_on: upload-ready-pipeline
|
||||
commands:
|
||||
- timeout 20m pytest -s -v tests/diffusion/test_diffusion_worker.py
|
||||
agents:
|
||||
queue: "gpu_4_queue" # g6.12xlarge instance on AWS, has 4 L4 GPU
|
||||
plugins:
|
||||
- docker#v5.2.0:
|
||||
image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
always-pull: true
|
||||
propagate-environment: true
|
||||
shm-size: "8gb"
|
||||
environment:
|
||||
- "HF_HOME=/fsx/hf_cache"
|
||||
- "HF_TOKEN"
|
||||
volumes:
|
||||
- "/fsx/hf_cache:/fsx/hf_cache"
|
||||
|
||||
|
||||
- label: "Engine Test"
|
||||
depends_on: upload-ready-pipeline
|
||||
commands:
|
||||
- |
|
||||
timeout 15m bash -c '
|
||||
export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
pytest -s -v tests/engine/test_async_omni_engine_abort.py
|
||||
'
|
||||
agents:
|
||||
queue: "gpu_4_queue" # g6.12xlarge instance on AWS, has 4 L4 GPU
|
||||
plugins:
|
||||
- docker#v5.2.0:
|
||||
image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
always-pull: true
|
||||
propagate-environment: true
|
||||
shm-size: "8gb"
|
||||
environment:
|
||||
- "HF_HOME=/fsx/hf_cache"
|
||||
- "HF_TOKEN"
|
||||
volumes:
|
||||
- "/fsx/hf_cache:/fsx/hf_cache"
|
||||
|
||||
|
||||
- label: "Omni Model Test"
|
||||
depends_on: upload-ready-pipeline
|
||||
commands:
|
||||
- |
|
||||
timeout 17m bash -c '
|
||||
export VLLM_LOGGING_LEVEL=DEBUG
|
||||
export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
pytest -s -v tests/e2e/online_serving/test_qwen2_5_omni.py -m "core_model" --run-level "core_model"
|
||||
'
|
||||
agents:
|
||||
queue: "gpu_4_queue" # g6.12xlarge instance on AWS, has 4 L4 GPU
|
||||
plugins:
|
||||
- docker#v5.2.0:
|
||||
image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
always-pull: true
|
||||
propagate-environment: true
|
||||
environment:
|
||||
- "HF_HOME=/fsx/hf_cache"
|
||||
- "HF_TOKEN"
|
||||
volumes:
|
||||
- "/fsx/hf_cache:/fsx/hf_cache"
|
||||
|
||||
- label: "Omni Model Test with H100"
|
||||
depends_on: upload-ready-pipeline
|
||||
commands:
|
||||
- |
|
||||
timeout 20m bash -c '
|
||||
export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
pytest -s -v tests/e2e/online_serving/test_qwen3_omni.py -m "core_model" --run-level "core_model"
|
||||
'
|
||||
agents:
|
||||
queue: "mithril-h100-pool"
|
||||
plugins:
|
||||
- kubernetes:
|
||||
podSpec:
|
||||
containers:
|
||||
- image: 936637512419.dkr.ecr.us-west-2.amazonaws.com/vllm-ci-pull-through-cache/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 2
|
||||
volumeMounts:
|
||||
- name: devshm
|
||||
mountPath: /dev/shm
|
||||
- name: hf-cache
|
||||
mountPath: /root/.cache/huggingface
|
||||
env:
|
||||
- name: HF_HOME
|
||||
value: /root/.cache/huggingface
|
||||
nodeSelector:
|
||||
node.kubernetes.io/instance-type: gpu-h100-sxm
|
||||
volumes:
|
||||
- name: devshm
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
- name: hf-cache
|
||||
hostPath:
|
||||
path: /mnt/hf-cache
|
||||
type: DirectoryOrCreate
|
||||
|
||||
- label: "MiMo-Audio E2E Test with H100"
|
||||
depends_on: upload-ready-pipeline
|
||||
commands:
|
||||
- |
|
||||
timeout 30m bash -c '
|
||||
export VLLM_LOGGING_LEVEL=DEBUG
|
||||
export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
pytest -s -v tests/e2e/online_serving/test_mimo_audio.py -m "core_model" --run-level "core_model"
|
||||
'
|
||||
agents:
|
||||
queue: "mithril-h100-pool"
|
||||
plugins:
|
||||
- kubernetes:
|
||||
podSpec:
|
||||
containers:
|
||||
- image: 936637512419.dkr.ecr.us-west-2.amazonaws.com/vllm-ci-pull-through-cache/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 2
|
||||
volumeMounts:
|
||||
- name: devshm
|
||||
mountPath: /dev/shm
|
||||
- name: hf-cache
|
||||
mountPath: /root/.cache/huggingface
|
||||
env:
|
||||
- name: HF_HOME
|
||||
value: /root/.cache/huggingface
|
||||
- name: HF_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: hf-token-secret
|
||||
key: token
|
||||
nodeSelector:
|
||||
node.kubernetes.io/instance-type: gpu-h100-sxm
|
||||
volumes:
|
||||
- name: devshm
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
- name: hf-cache
|
||||
hostPath:
|
||||
path: /mnt/hf-cache
|
||||
type: DirectoryOrCreate
|
||||
|
||||
- label: "Qwen3-TTS E2E Test"
|
||||
depends_on: upload-ready-pipeline
|
||||
commands:
|
||||
- |
|
||||
timeout 20m bash -c '
|
||||
export VLLM_LOGGING_LEVEL=DEBUG
|
||||
export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
export VLLM_ALLOW_LONG_MAX_MODEL_LEN="1"
|
||||
pytest -s -v tests/e2e/online_serving/test_qwen3_tts_customvoice.py -m "core_model" --run-level "core_model"
|
||||
'
|
||||
agents:
|
||||
queue: "gpu_1_queue"
|
||||
plugins:
|
||||
- docker#v5.2.0:
|
||||
image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
always-pull: true
|
||||
propagate-environment: true
|
||||
shm-size: "8gb"
|
||||
environment:
|
||||
- "HF_HOME=/fsx/hf_cache"
|
||||
- "HF_TOKEN"
|
||||
volumes:
|
||||
- "/fsx/hf_cache:/fsx/hf_cache"
|
||||
|
||||
- label: "Voxtral-TTS E2E Test"
|
||||
timeout_in_minutes: 20
|
||||
depends_on: upload-ready-pipeline
|
||||
commands:
|
||||
- |
|
||||
timeout 20m bash -c '
|
||||
export VLLM_LOGGING_LEVEL=DEBUG
|
||||
export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
pytest -s -v tests/e2e/online_serving/test_voxtral_tts.py -m "advanced_model" --run-level "advanced_model"
|
||||
pytest -s -v tests/e2e/offline_inference/test_voxtral_tts.py -m "advanced_model" --run-level "advanced_model"
|
||||
'
|
||||
agents:
|
||||
queue: "mithril-h100-pool"
|
||||
plugins:
|
||||
- kubernetes:
|
||||
podSpec:
|
||||
containers:
|
||||
- image: 936637512419.dkr.ecr.us-west-2.amazonaws.com/vllm-ci-pull-through-cache/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 1
|
||||
volumeMounts:
|
||||
- name: devshm
|
||||
mountPath: /dev/shm
|
||||
- name: hf-cache
|
||||
mountPath: /root/.cache/huggingface
|
||||
env:
|
||||
- name: HF_HOME
|
||||
value: /root/.cache/huggingface
|
||||
nodeSelector:
|
||||
node.kubernetes.io/instance-type: gpu-h100-sxm
|
||||
volumes:
|
||||
- name: devshm
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
- name: hf-cache
|
||||
hostPath:
|
||||
path: /mnt/hf-cache
|
||||
type: DirectoryOrCreate
|
||||
|
||||
# - label: "Diffusion Image Edit Test with H100 (1 GPU)"
|
||||
# depends_on: upload-ready-pipeline
|
||||
# commands:
|
||||
# - |
|
||||
# timeout 20m bash -c '
|
||||
# export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
# pytest -s -v tests/e2e/online_serving/test_image_gen_edit.py
|
||||
# '
|
||||
# agents:
|
||||
# queue: "mithril-h100-pool"
|
||||
# plugins:
|
||||
# - kubernetes:
|
||||
# podSpec:
|
||||
# containers:
|
||||
# - image: 936637512419.dkr.ecr.us-west-2.amazonaws.com/vllm-ci-pull-through-cache/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
# resources:
|
||||
# limits:
|
||||
# nvidia.com/gpu: 1
|
||||
# volumeMounts:
|
||||
# - name: devshm
|
||||
# mountPath: /dev/shm
|
||||
# - name: hf-cache
|
||||
# mountPath: /root/.cache/huggingface
|
||||
# env:
|
||||
# - name: HF_HOME
|
||||
# value: /root/.cache/huggingface
|
||||
# nodeSelector:
|
||||
# node.kubernetes.io/instance-type: gpu-h100-sxm
|
||||
# volumes:
|
||||
# - name: devshm
|
||||
# emptyDir:
|
||||
# medium: Memory
|
||||
# - name: hf-cache
|
||||
# hostPath:
|
||||
# path: /mnt/hf-cache
|
||||
# type: DirectoryOrCreate
|
||||
|
||||
- label: "Bagel Text2Img Model Test with H100"
|
||||
depends_on: upload-ready-pipeline
|
||||
commands:
|
||||
- |
|
||||
timeout 30m bash -c '
|
||||
export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
export VLLM_TEST_CLEAN_GPU_MEMORY=1
|
||||
pytest -s -v tests/e2e/offline_inference/test_bagel_text2img.py -m "core_model" --run-level "core_model"
|
||||
'
|
||||
agents:
|
||||
queue: "mithril-h100-pool"
|
||||
plugins:
|
||||
- kubernetes:
|
||||
podSpec:
|
||||
containers:
|
||||
- image: 936637512419.dkr.ecr.us-west-2.amazonaws.com/vllm-ci-pull-through-cache/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 1
|
||||
volumeMounts:
|
||||
- name: devshm
|
||||
mountPath: /dev/shm
|
||||
- name: hf-cache
|
||||
mountPath: /root/.cache/huggingface
|
||||
env:
|
||||
- name: HF_HOME
|
||||
value: /root/.cache/huggingface
|
||||
- name: HF_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: hf-token-secret
|
||||
key: token
|
||||
nodeSelector:
|
||||
node.kubernetes.io/instance-type: gpu-h100-sxm
|
||||
volumes:
|
||||
- name: devshm
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
- name: hf-cache
|
||||
hostPath:
|
||||
path: /mnt/hf-cache
|
||||
type: DirectoryOrCreate
|
||||
|
||||
- label: "Bagel Img2Img Model Test with H100"
|
||||
depends_on: upload-ready-pipeline
|
||||
commands:
|
||||
- |
|
||||
timeout 30m bash -c '
|
||||
export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
export VLLM_TEST_CLEAN_GPU_MEMORY=1
|
||||
pytest -s -v tests/e2e/offline_inference/test_bagel_img2img.py -m "core_model" --run-level "core_model"
|
||||
'
|
||||
agents:
|
||||
queue: "mithril-h100-pool"
|
||||
plugins:
|
||||
- kubernetes:
|
||||
podSpec:
|
||||
containers:
|
||||
- image: 936637512419.dkr.ecr.us-west-2.amazonaws.com/vllm-ci-pull-through-cache/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 1
|
||||
volumeMounts:
|
||||
- name: devshm
|
||||
mountPath: /dev/shm
|
||||
- name: hf-cache
|
||||
mountPath: /root/.cache/huggingface
|
||||
env:
|
||||
- name: HF_HOME
|
||||
value: /root/.cache/huggingface
|
||||
- name: HF_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: hf-token-secret
|
||||
key: token
|
||||
nodeSelector:
|
||||
node.kubernetes.io/instance-type: gpu-h100-sxm
|
||||
volumes:
|
||||
- name: devshm
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
- name: hf-cache
|
||||
hostPath:
|
||||
path: /mnt/hf-cache
|
||||
type: DirectoryOrCreate
|
||||
|
||||
- label: "Bagel Online Serving Test with H100"
|
||||
depends_on: upload-ready-pipeline
|
||||
commands:
|
||||
- |
|
||||
timeout 40m bash -c '
|
||||
export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
export VLLM_TEST_CLEAN_GPU_MEMORY=1
|
||||
export VLLM_IMAGE_FETCH_TIMEOUT=60
|
||||
pytest -s -v tests/e2e/online_serving/test_bagel_online.py -m "core_model" --run-level "core_model"
|
||||
'
|
||||
agents:
|
||||
queue: "mithril-h100-pool"
|
||||
plugins:
|
||||
- kubernetes:
|
||||
podSpec:
|
||||
containers:
|
||||
- image: 936637512419.dkr.ecr.us-west-2.amazonaws.com/vllm-ci-pull-through-cache/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
|
||||
resources:
|
||||
limits:
|
||||
nvidia.com/gpu: 1
|
||||
volumeMounts:
|
||||
- name: devshm
|
||||
mountPath: /dev/shm
|
||||
- name: hf-cache
|
||||
mountPath: /root/.cache/huggingface
|
||||
env:
|
||||
- name: HF_HOME
|
||||
value: /root/.cache/huggingface
|
||||
- name: HF_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: hf-token-secret
|
||||
key: token
|
||||
nodeSelector:
|
||||
node.kubernetes.io/instance-type: gpu-h100-sxm
|
||||
volumes:
|
||||
- name: devshm
|
||||
emptyDir:
|
||||
medium: Memory
|
||||
- name: hf-cache
|
||||
hostPath:
|
||||
path: /mnt/hf-cache
|
||||
type: DirectoryOrCreate
|
||||
@@ -0,0 +1,59 @@
|
||||
{# vllm-omni customized version
|
||||
Based on: https://github.com/vllm-project/ci-infra/blob/main/buildkite/test-template-amd.j2
|
||||
Last synced: 2025-12-15
|
||||
Modifications: Removed unused CUDA/NVIDIA logic, keeping only AMD tests
|
||||
#}
|
||||
{% set docker_image_amd = "rocm/vllm-ci:$BUILDKITE_COMMIT-rocm-omni" %}
|
||||
{% set default_working_dir = "/app/vllm-omni" %}
|
||||
|
||||
- group: "AMD Tests"
|
||||
depends_on: ~
|
||||
steps:
|
||||
- label: "AMD: :docker: build image"
|
||||
depends_on: ~
|
||||
soft_fail: false
|
||||
commands:
|
||||
- "docker build -f docker/Dockerfile.rocm -t {{ docker_image_amd }} --target test --build-arg ARG_PYTORCH_ROCM_ARCH=gfx942 --progress plain ."
|
||||
- "docker push {{ docker_image_amd }}"
|
||||
key: "amd-build"
|
||||
env:
|
||||
DOCKER_BUILDKIT: "1"
|
||||
retry:
|
||||
automatic:
|
||||
- exit_status: -1 # Agent was lost
|
||||
limit: 2
|
||||
- exit_status: -10 # Agent was lost
|
||||
limit: 2
|
||||
- exit_status: 128 # Git connectivity issues
|
||||
limit: 2
|
||||
- exit_status: 1 # Machine occasionally fail
|
||||
limit: 1
|
||||
agents:
|
||||
queue: amd-cpu
|
||||
|
||||
{% for step in steps %}
|
||||
{% if step.mirror_hardwares and mirror_hw in step.mirror_hardwares %}
|
||||
- label: "{{ step.agent_pool }}: {{ step.label }}"
|
||||
depends_on: amd-build
|
||||
agents:
|
||||
{% if step.agent_pool %}
|
||||
queue: amd_{{ step.agent_pool }}
|
||||
{% else %}
|
||||
queue: amd_mi325_1
|
||||
{% endif %}
|
||||
{% set cmd_body = (step.command or (step.commands | join("\n"))) | trim %}
|
||||
{% set indented_cmd = cmd_body | replace("\n", "\n ") %}
|
||||
command: bash .buildkite/scripts/hardware_ci/run-amd-test.sh
|
||||
env:
|
||||
DOCKER_BUILDKIT: "1"
|
||||
TEST_COMMAND: |-
|
||||
(command rocm-smi || true) && cd {{ (step.working_dir or default_working_dir) | safe }}
|
||||
{{ indented_cmd | safe }}
|
||||
priority: 100
|
||||
{% if step.grade and step.grade == "Blocking" %}
|
||||
soft_fail: false
|
||||
{% else %}
|
||||
soft_fail: true
|
||||
{% endif%}
|
||||
{% endif %}
|
||||
{% endfor %}
|
||||
@@ -1 +0,0 @@
|
||||
../../.agents/skills/3d-animation-short-generator
|
||||
@@ -1,532 +0,0 @@
|
||||
---
|
||||
name: add-diffusion-model
|
||||
description: Add a new diffusion model (text-to-image, text-to-video, image-to-video, text-to-audio, image editing) to vLLM-Omni, including native non-Diffusers ports, reference-parity validation, Cache-DiT, offload, and parallelism support (TP, SP/USP, CFG-Parallel, HSDP). Use when integrating or reviewing a new diffusion model, porting a Diffusers pipeline or custom model repository, creating a DiT adapter, reusing shared examples, or qualifying multi-GPU and memory optimizations.
|
||||
---
|
||||
|
||||
# Adding a Diffusion Model to vLLM-Omni
|
||||
|
||||
## Overview
|
||||
|
||||
This skill guides you through adding a new diffusion model to vLLM-Omni. The model may come from HuggingFace Diffusers (structured pipeline) or from a private/custom repo. The workflow differs significantly depending on the source.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Before starting, determine:
|
||||
|
||||
1. **Model category**: Text-to-Image, Text-to-Video, Image-to-Video, Image Editing, Text-to-Audio, or Omni
|
||||
2. **Reference source**: Diffusers pipeline, custom repo, or a combination
|
||||
3. **Model HuggingFace ID** or local checkpoint path
|
||||
4. **Architecture**: Scheduler, text encoder, VAE, transformer/backbone
|
||||
|
||||
## Step 0: Classify the Migration Path
|
||||
|
||||
Check the model's HF repo for `model_index.json`. This determines your path:
|
||||
|
||||
| Scenario | How to identify | Migration path |
|
||||
|----------|----------------|----------------|
|
||||
| **Already supported** | `_class_name` in `model_index.json` matches a key in `_DIFFUSION_MODELS` in `registry.py` | Skip implementation, then validate model-specific examples, tests, and docs as needed |
|
||||
| **Diffusers-based** | Has standard `model_index.json` with `_diffusers_version`, subfolders for `transformer/`, `vae/`, etc. | Follow **Path A** below |
|
||||
| **Native non-Diffusers model** | No Diffusers index, non-standard checkpoint hierarchy, or custom architecture in a separate repo | Follow **Path B** below; port the runtime natively unless an external adapter was explicitly requested |
|
||||
| **Hybrid** | Has some diffusers components (VAE) but custom transformer/fusion | Mix of Path A and Path B |
|
||||
|
||||
Before coding, write a short integration contract covering runtime ownership,
|
||||
checkpoint discovery, reference revision, I/O geometry, attention semantics,
|
||||
CFG behavior, auxiliary components, target hardware, and the default deployment.
|
||||
For Path B, hybrid models, or any optimization work, read
|
||||
[references/native-model-integration-checklist.md](references/native-model-integration-checklist.md)
|
||||
and use its phase gates.
|
||||
|
||||
## Path A: Diffusers-Based Model
|
||||
|
||||
For models with a standard diffusers layout. See [references/transformer-adaptation.md](references/transformer-adaptation.md) for detailed code patterns.
|
||||
|
||||
### A1. Analyze `model_index.json`
|
||||
|
||||
Identify components: `transformer`, `scheduler`, `vae`, `text_encoder`, `tokenizer`.
|
||||
|
||||
### A2. Create model directory
|
||||
|
||||
```
|
||||
vllm_omni/diffusion/models/your_model_name/
|
||||
├── __init__.py
|
||||
├── pipeline_your_model.py
|
||||
└── your_model_transformer.py
|
||||
```
|
||||
|
||||
### A3. Adapt transformer
|
||||
|
||||
1. Copy from diffusers source. Remove mixins (`ModelMixin`, `ConfigMixin`, `AttentionModuleMixin`).
|
||||
2. Replace attention with `vllm_omni.diffusion.attention.layer.Attention` (QKV shape: `[B, seq, heads, head_dim]`).
|
||||
3. Add `od_config: OmniDiffusionConfig | None = None` to `__init__`.
|
||||
4. Add `load_weights()` method mapping diffusers weight names to vllm-omni names.
|
||||
5. Add class attributes for acceleration features such as `_repeated_blocks` and `_layerwise_offload_blocks_attrs` (see [references/transformer-adaptation.md](references/transformer-adaptation.md) for examples).
|
||||
|
||||
### A4. Adapt pipeline
|
||||
|
||||
Inherit from `nn.Module`. The key contract:
|
||||
|
||||
```python
|
||||
class YourPipeline(nn.Module):
|
||||
def __init__(self, *, od_config: OmniDiffusionConfig, prefix: str = ""):
|
||||
# Load VAE, text encoder, tokenizer via from_pretrained()
|
||||
# Instantiate transformer (weights loaded later via weights_sources)
|
||||
self.weights_sources = [
|
||||
DiffusersPipelineLoader.ComponentSource(
|
||||
model_or_path=od_config.model, subfolder="transformer",
|
||||
prefix="transformer.", fall_back_to_pt=True)]
|
||||
|
||||
def forward(self, req: OmniDiffusionRequest) -> DiffusionOutput:
|
||||
# Encode prompt → prepare latents → denoise loop → VAE decode
|
||||
return DiffusionOutput(output=output)
|
||||
|
||||
def load_weights(self, weights):
|
||||
return AutoWeightsLoader(self).load_weights(weights)
|
||||
```
|
||||
|
||||
Add post/pre-process functions in the same pipeline file. Register them in `registry.py`.
|
||||
|
||||
### A4.1 Add progress bar support (recommended)
|
||||
|
||||
For pipelines with a standard denoising loop, prefer the existing progress bar pattern instead of hand-rolled logging.
|
||||
|
||||
```python
|
||||
from vllm_omni.diffusion.models.progress_bar import ProgressBarMixin
|
||||
|
||||
class YourPipeline(nn.Module, ProgressBarMixin):
|
||||
def forward(self, req: OmniDiffusionRequest) -> DiffusionOutput:
|
||||
# ... prepare timesteps / latents ...
|
||||
with self.progress_bar(total=len(timesteps)) as progress_bar:
|
||||
for i, t in enumerate(timesteps):
|
||||
# predict noise / scheduler step
|
||||
latents = ...
|
||||
progress_bar.update()
|
||||
|
||||
return DiffusionOutput(output=output)
|
||||
```
|
||||
|
||||
For custom loop structures, follow `vllm_omni/diffusion/models/progress_bar.py` and existing pipelines using `ProgressBarMixin`.
|
||||
|
||||
### A5. Register, test, docs → continue at Step 4 below.
|
||||
|
||||
---
|
||||
|
||||
## Path B: Native / Non-Diffusers Model
|
||||
|
||||
For models without a Diffusers pipeline—weights in custom formats and model
|
||||
code in another public or private repository. Treat that repository as a pinned
|
||||
correctness oracle. A request for native support means the vLLM-Omni runtime
|
||||
must not import the reference implementation; an external adapter is appropriate
|
||||
only when the requested scope explicitly permits that dependency. See
|
||||
[references/custom-model-patterns.md](references/custom-model-patterns.md) for
|
||||
concrete integration patterns.
|
||||
|
||||
### B1. Understand the reference repo
|
||||
|
||||
Study the original model's code to identify:
|
||||
- **Model architecture files** (transformers, fusion modules, embeddings)
|
||||
- **Weight format** (safetensors, `.pth`, custom checkpoint structure)
|
||||
- **Weight loading helpers** (custom init functions, checkpoint loaders)
|
||||
- **Pre/post-processing** (image/audio transforms, tokenization, VAE encode/decode)
|
||||
- **External dependencies** (packages not on PyPI)
|
||||
- **Config format** (JSON config files, hardcoded dicts)
|
||||
|
||||
### B2. Decide what lives WHERE
|
||||
|
||||
This is the key design decision for custom models. Follow these placement rules:
|
||||
|
||||
| Code type | Where to place | Example |
|
||||
|-----------|---------------|---------|
|
||||
| **Pipeline orchestration** (init, forward, denoise loop) | `vllm_omni/diffusion/models/<name>/pipeline_<name>.py` | Always required |
|
||||
| **Custom transformer/backbone** (ported and adapted to vllm-omni) | `vllm_omni/diffusion/models/<name>/<name>_transformer.py` or similar | `wan2_2.py`, `fusion.py`, `bagel_transformer.py` |
|
||||
| **Custom sub-models** (VAE, fusion, autoencoder) | `vllm_omni/diffusion/models/<name>/` as separate files | `autoencoder.py`, `fusion.py` |
|
||||
| **Reference-only code** | Keep outside the runtime; use a pinned revision for golden outputs and architecture analysis | Reference inference script |
|
||||
| **Explicit external adapter dependency** | External package, only when the requested scope and maintainer direction allow it | Compatibility adapter, not native support |
|
||||
| **Hardcoded model configs** | Module-level dicts in pipeline file | `VIDEO_CONFIG`, `AUDIO_CONFIG` dicts |
|
||||
| **Download/setup script** | `examples/offline_inference/<name>/download_<name>.py` | `download_<name>.py` |
|
||||
| **Custom `model_index.json`** | Generated by download script, placed at model root | Minimal: `{"_class_name": "YourPipeline", ...}` |
|
||||
|
||||
### B3. Handle external dependencies
|
||||
|
||||
If the model's code lives in a separate git repo, first decide whether the
|
||||
requested deliverable is native support or an external adapter. Do not silently
|
||||
choose the adapter path.
|
||||
|
||||
**Option 1: Port the code directly** (default for native support)
|
||||
|
||||
Copy the essential model files into `vllm_omni/diffusion/models/<name>/` and
|
||||
adapt them to shared vLLM-Omni contracts. Keep checkpoint loading strict and
|
||||
use the pinned reference only to generate parity evidence.
|
||||
|
||||
**Option 2: Import with graceful fallback** (adapter scope only)
|
||||
|
||||
```python
|
||||
try:
|
||||
from external_model.utils import init_vae, load_checkpoint
|
||||
except ImportError:
|
||||
raise ImportError(
|
||||
"Failed to import from dependency 'external_model'. "
|
||||
"Please run the download script first."
|
||||
)
|
||||
```
|
||||
|
||||
Use an external runtime dependency only when the user explicitly requested an
|
||||
adapter or a maintainer approved the exception. Document the dependency,
|
||||
pinned revision, installation path, and unsupported native features.
|
||||
|
||||
### B4. Handle custom weight loading
|
||||
|
||||
Custom models have two common patterns for weight loading:
|
||||
|
||||
**Pattern 1: Bypass standard loader** (eager custom init)
|
||||
|
||||
When the original model has complex custom init functions that load weights in `__init__`:
|
||||
|
||||
```python
|
||||
class CustomPipeline(nn.Module):
|
||||
def __init__(self, *, od_config, prefix=""):
|
||||
super().__init__()
|
||||
model = od_config.model
|
||||
# Load everything eagerly in __init__ using custom helpers
|
||||
self.vae = custom_init_vae(model, device=self.device)
|
||||
self.text_encoder = custom_init_text_encoder(model, device=self.device)
|
||||
self.transformer = CustomFusionModel(CONFIG)
|
||||
load_custom_checkpoint(
|
||||
self.transformer,
|
||||
checkpoint_path=os.path.join(model, "model.safetensors"),
|
||||
)
|
||||
# NO weights_sources defined — bypasses standard loader
|
||||
|
||||
def load_weights(self, weights):
|
||||
pass # No-op — all weights loaded in __init__
|
||||
```
|
||||
|
||||
**Pattern 2: Use standard loader with custom `load_weights`** (BAGEL style)
|
||||
|
||||
When weights are in safetensors format but need name remapping:
|
||||
|
||||
```python
|
||||
from vllm.model_executor.model_loader.weight_utils import default_weight_loader
|
||||
|
||||
class CustomPipeline(nn.Module):
|
||||
def __init__(self, *, od_config, prefix=""):
|
||||
super().__init__()
|
||||
# Instantiate model architecture without weights
|
||||
self.bagel = BagelModel(config)
|
||||
self.vae = AutoEncoder(ae_params)
|
||||
|
||||
# Point loader at the safetensors in the model root
|
||||
self.weights_sources = [
|
||||
DiffusersPipelineLoader.ComponentSource(
|
||||
model_or_path=od_config.model,
|
||||
subfolder=None, # weights at root, not in subfolder
|
||||
prefix="",
|
||||
fall_back_to_pt=False,
|
||||
)
|
||||
]
|
||||
|
||||
def load_weights(self, weights):
|
||||
# Custom name remapping for non-diffusers weight names
|
||||
params = dict(self.named_parameters())
|
||||
loaded = set()
|
||||
for name, tensor in weights:
|
||||
# Remap original weight names to vllm-omni module names
|
||||
name = self._remap_weight_name(name)
|
||||
if name in params:
|
||||
param = params[name]
|
||||
weight_loader = getattr(param, "weight_loader", default_weight_loader)
|
||||
weight_loader(param, tensor)
|
||||
loaded.add(name)
|
||||
return loaded
|
||||
```
|
||||
|
||||
### B5. Create the `model_index.json`
|
||||
|
||||
Prefer a `model_index.json` at the model root when vLLM-Omni owns an assembled
|
||||
checkpoint directory. For custom models, this is minimal:
|
||||
|
||||
```json
|
||||
{
|
||||
"_class_name": "YourModelPipeline",
|
||||
"custom_key": "path/to/custom_weights.safetensors"
|
||||
}
|
||||
```
|
||||
|
||||
The `_class_name` must match a key in `_DIFFUSION_MODELS` in `registry.py`.
|
||||
Additional keys are model-specific (accessed via `od_config.model_config`).
|
||||
|
||||
If the released repository is immutable and has neither a root `config.json`
|
||||
nor a Diffusers index, add it to a generic native-checkpoint signature resolver:
|
||||
match the exact Hub ID or a distinctive set of local files and return the
|
||||
pipeline class. Do not use model-name substrings or add parallel one-off
|
||||
predicates in CLI, config, and serving consumers.
|
||||
|
||||
If the model's weights come from multiple HF repos, write a **download script** that:
|
||||
1. Downloads from each repo
|
||||
2. Assembles into a single directory
|
||||
3. Generates `model_index.json`
|
||||
4. Installs any external dependencies (git clone + `.pth` file)
|
||||
|
||||
Place at: `examples/offline_inference/<name>/download_<name>.py`
|
||||
|
||||
### B6. Handle multi-modal inputs
|
||||
|
||||
If the model accepts images, audio, or other multi-modal inputs, implement the protocol classes from `vllm_omni/diffusion/models/interface.py`:
|
||||
|
||||
```python
|
||||
from vllm_omni.diffusion.models.interface import SupportImageInput, SupportAudioInput
|
||||
|
||||
class MyPipeline(nn.Module, SupportImageInput, SupportAudioInput):
|
||||
# Protocol markers — the engine uses these to enable proper input routing
|
||||
pass
|
||||
```
|
||||
|
||||
Preprocessing for custom models is typically done **inside `forward()`** rather than via registered pre-process functions, since the logic is often tightly coupled to the model.
|
||||
|
||||
### B7. Continue at Step 4 below.
|
||||
|
||||
---
|
||||
|
||||
## Common Steps (Both Paths)
|
||||
|
||||
### Step 4: Register Model in registry.py
|
||||
|
||||
Edit `vllm_omni/diffusion/registry.py`:
|
||||
|
||||
```python
|
||||
_DIFFUSION_MODELS = {
|
||||
"YourModelPipeline": ("your_model_name", "pipeline_your_model", "YourModelPipeline"),
|
||||
}
|
||||
_DIFFUSION_POST_PROCESS_FUNCS = {
|
||||
"YourModelPipeline": "get_your_model_post_process_func", # if applicable
|
||||
}
|
||||
_DIFFUSION_PRE_PROCESS_FUNCS = {
|
||||
"YourModelPipeline": "get_your_model_pre_process_func", # if applicable
|
||||
}
|
||||
```
|
||||
|
||||
The registry key is the `_class_name` from `model_index.json`. The tuple is `(folder_name, module_file, class_name)`.
|
||||
|
||||
Create `__init__.py` exporting the pipeline class and any factory functions.
|
||||
|
||||
### Step 5: Run, Test, Debug
|
||||
|
||||
Use the appropriate existing example script:
|
||||
|
||||
| Category | Script |
|
||||
|----------|--------|
|
||||
| Text-to-Image | `examples/offline_inference/text_to_image/text_to_image.py` |
|
||||
| Text-to-Video | `examples/offline_inference/text_to_video/text_to_video.py` |
|
||||
| Image-to-Video | `examples/offline_inference/image_to_video/image_to_video.py` |
|
||||
| Image-to-Image | `examples/offline_inference/image_to_image/image_edit.py` |
|
||||
| Text-to-Audio | `examples/offline_inference/text_to_audio/text_to_audio.py` |
|
||||
|
||||
Reuse these shared scripts even for custom models when their request and output
|
||||
contracts fit. Create a dedicated model script only when the shared category
|
||||
cannot represent the protocol, and document that gap in the PR.
|
||||
|
||||
**Validation**: No errors, output is meaningful, quality matches reference implementation.
|
||||
|
||||
See [references/troubleshooting.md](references/troubleshooting.md) for common errors.
|
||||
|
||||
### Step 6: Add Example Scripts
|
||||
|
||||
Only when the shared category scripts cannot represent the model, create:
|
||||
- `examples/offline_inference/your_model_name/` — offline script + README
|
||||
- `examples/online_serving/your_model_name/` — server script + client
|
||||
- Download script if weights require assembly from multiple sources
|
||||
|
||||
### Step 7: Update Documentation
|
||||
|
||||
Follow the [`add-recipe` skill](../add-recipe/SKILL.md) to add or update the
|
||||
model-family recipe and its `recipes/README.md` row with verified specifications,
|
||||
hardware, commands, feature links, and qualification evidence.
|
||||
|
||||
Required updates:
|
||||
1. `docs/user_guide/diffusion/parallelism/overview.md` — parallelism support overview/table
|
||||
2. `docs/user_guide/diffusion/cpu_offload.md` — if CPU offload supported (add to supported models table)
|
||||
3. `docs/user_guide/diffusion/cache_acceleration/teacache.md` — if TeaCache supported
|
||||
4. `docs/user_guide/diffusion/cache_acceleration/cache_dit.md` — if Cache-DiT supported
|
||||
5. Offline example docs under `examples/offline_inference/<name>/` (`README.md` or category-specific `.md`)
|
||||
6. `examples/online_serving/<name>/README.md` — online serving docs
|
||||
|
||||
### Step 8: Add E2E Tests
|
||||
|
||||
**Follow the [vllm-omni-test skill](../vllm-omni-test/SKILL.md)** for markers, file naming, Buildkite wiring, and run commands. Also read [l4_functionality_tests.inc.md](https://github.com/vllm-project/vllm-omni/blob/main/docs/contributing/ci/test_examples/l4_functionality_tests.inc.md), [test_system_overview.md](https://github.com/vllm-project/vllm-omni/blob/main/docs/contributing/ci/test_system_overview.md), and [test_writing_guide.md](https://github.com/vllm-project/vllm-omni/blob/main/docs/contributing/ci/test_writing_guide.md).
|
||||
|
||||
Classify the model's **CI priority** first:
|
||||
|
||||
| Priority | Required test levels | Files & markers |
|
||||
|----------|---------------------|-----------------|
|
||||
| **High** (listed in [#1832](https://github.com/vllm-project/vllm-omni/issues/1832) or on the diffusion hot path) | **L1** · **L2** online · **L3** online + offline · **L4** feature + performance | See table below |
|
||||
| **Medium** (*normal priority* in L4 docs) | **L3** online + offline · **L4** feature only | Fewer L4 parametrized rows |
|
||||
| **Low** | **L4** feature only | One or two `*_expansion.py` cases |
|
||||
|
||||
**Per-level deliverables (diffusion / `pytest.mark.diffusion`):**
|
||||
|
||||
| Level | Location | Marker | CI pipeline | Notes |
|
||||
|-------|----------|--------|-------------|-------|
|
||||
| **L1** | `tests/diffusion/models/{slug}/`, `tests/diffusion/cache/`, transformer unit tests | `core_model` + `cpu` | `test-ready.yml` | Weight remap, `_sp_plan`, cache enabler registration, shape contracts |
|
||||
| **L2** | `tests/e2e/online_serving/test_{slug}.py` (and offline if the category is offline-first) | **`core_model` + `advanced_model`** (both on baseline smoke) + `diffusion` + `@hardware_test` / `hardware_marks` | `test-ready.yml` | Default deploy smoke — minimal `num_inference_steps`, single prompt |
|
||||
| **L3** | `tests/e2e/online_serving/test_{slug}.py` **and** `tests/e2e/offline_inference/test_{slug}.py` when offline matters | Baseline smoke: **`core_model` + `advanced_model`**; heavier cases: `advanced_model` only (+ `diffusion`) | `test-merge.yml` **or** merged into nightly diffusion function job | Real weights, streaming/API paths, LoRA/offload smoke |
|
||||
| **L4** | `tests/e2e/online_serving/test_{slug}_expansion.py` (+ offline expansion if needed) | `full_model` + `diffusion` | `test-nightly.yml` (X2I / X2V / X2A function groups) | Feature combos per [#1832](https://github.com/vllm-project/vllm-omni/issues/1832); perf → `tests/dfx/perf/tests/test_{model}_vllm_omni.json` with per-case **`mark`** (`hardware_marks` + `full_model` + `diffusion`) |
|
||||
|
||||
**L2 & L3 online — same file, dual marks on the baseline smoke:** The **first / simplest** case in `test_{slug}.py` (default deploy, minimal steps, single prompt) should carry **both** `@pytest.mark.core_model` and `@pytest.mark.advanced_model` on the **same** function so L2 (`test-ready.yml`) and L3 (`test-merge.yml`) share one smoke test. Heavier deploy variants or API paths in the same file use **`advanced_model` only**. When L3 moves to nightly, migrate those heavier cases into `test_{slug}_expansion.py` with `full_model` and remove the dedicated `test-merge.yml` job (see `test_longcat_image_expansion.py`, `test_qwen_image_expansion.py`).
|
||||
|
||||
**L4 design (high priority):** Combine multiple supported features (Cache-DiT, TP, USP, CFG, HSDP, CPU offload, quantization) into **few parametrized** `OmniServerParams` rows so each feature appears in at least one case without exploding GPU jobs. Shard single-GPU vs multi-GPU cases across the nightly X2I/X2V function steps (`cards_1` vs `not cards_1`).
|
||||
|
||||
**L4 design (medium / low):** One or two parametrized rows covering the best quality/perf trade-off; skip perf JSON unless the model is high priority.
|
||||
|
||||
**Reference implementations:** `tests/e2e/online_serving/test_qwen_image_edit_expansion.py`, `tests/e2e/online_serving/test_longcat_image_expansion.py`, `tests/e2e/online_serving/test_hunyuan_video_15_expansion.py`.
|
||||
|
||||
**Keep model-specific code inside test modules — not `tests/helpers/{slug}.py`:** deploy constants, prompts, sampling dicts, and inline `request_config` / `form_data` belong in each `test_{slug}.py` and `test_{slug}_expansion.py`. Do not add per-model files under `tests/helpers/`; reuse only repo-wide harness (`mark`, `media`, `runtime`, `stage_config`, `assertions`). **L2+ online/offline e2e:** reuse or add `send_*_request` in `tests/helpers/runtime.py` — tests call the handler, not raw `omni.generate` / HTTP. See [vllm-omni-test skill](../vllm-omni-test/SKILL.md) § **Runtime send helpers**.
|
||||
|
||||
Keep the model suite proportional using the six distinct failure owners in the
|
||||
native integration checklist. Combine supported features into a few
|
||||
parametrized E2E rows instead of creating one test per optimization.
|
||||
|
||||
### Step 9: Add Cache-DiT Acceleration
|
||||
|
||||
Add caching only after uncached single-device correctness. Read
|
||||
[references/cache-dit-patterns.md](references/cache-dit-patterns.md), use the
|
||||
automatic single-block-list path when possible, and add a registered
|
||||
`BlockAdapter` only for genuinely custom block topology.
|
||||
|
||||
Verify a real cache hit and compare quality with the uncached baseline. Make a
|
||||
speed claim only at a realistic step count where warmup permits hits; an
|
||||
all-warmup smoke proves integration, not acceleration.
|
||||
|
||||
---
|
||||
|
||||
### Step 10: Add Parallelism Support
|
||||
|
||||
After the model works on a single GPU, add multi-GPU parallelism. Add each type incrementally, testing after each addition.
|
||||
|
||||
See [references/parallelism-patterns.md](references/parallelism-patterns.md) for detailed code patterns and API reference.
|
||||
|
||||
**Recommended order**: TP → SP/USP → CFG Parallel → HSDP
|
||||
|
||||
#### 10a. Tensor Parallelism (TP)
|
||||
|
||||
Replace compatible projections with vLLM parallel linears, preserve checkpoint
|
||||
fusion/loading, and use local head counts. Require query/KV head divisibility
|
||||
and compare a multi-rank forward with the one-rank oracle.
|
||||
|
||||
#### 10b. Sequence Parallelism (SP / USP)
|
||||
|
||||
Prefer the declarative `_sp_plan`. For packed variable-length attention or
|
||||
learned-sink LSE correction, keep model math explicit and reuse shared exchange
|
||||
utilities. Validate uneven sequence splits, RoPE coordinates, and outputs
|
||||
against the one-rank oracle.
|
||||
|
||||
#### 10c. CFG Parallel
|
||||
|
||||
Confirm the model uses CFG. Then reuse `CFGParallelMixin`, overriding prediction
|
||||
or recombination only for non-standard or multi-output pipelines. Distinguish a
|
||||
packed positive/negative implementation from two independent branches and
|
||||
validate the two-rank result against the packed one-rank oracle.
|
||||
|
||||
#### 10d. HSDP (Hybrid Sharded Data Parallel)
|
||||
|
||||
Declare layer shard conditions and ignored rank-local modules; preserve mixed
|
||||
checkpoint dtypes. HSDP cannot combine with TP. Measure parameter loading,
|
||||
FSDP materialization, warm HBM, and host PSS rather than assuming sharding saves
|
||||
peak memory. Keep the resident layout as default if HSDP is worse.
|
||||
|
||||
#### 10e. Update parallelism documentation
|
||||
|
||||
After adding parallelism support, update:
|
||||
1. `docs/user_guide/diffusion/parallelism/overview.md` — add your model to the support overview/table
|
||||
2. Record which parallelism methods are supported (USP, Ring, CFG, TP, HSDP, VAE-Patch)
|
||||
|
||||
### Step 11: Add CPU Offload Support
|
||||
|
||||
Implement `SupportsComponentDiscovery` on your pipeline class to enable
|
||||
`--enable-cpu-offload` and `--enable-layerwise-offload`. The protocol
|
||||
declares which submodules the offloader should manage:
|
||||
|
||||
```python
|
||||
from typing import ClassVar
|
||||
from vllm_omni.diffusion.models.interface import SupportsComponentDiscovery
|
||||
|
||||
class YourPipeline(nn.Module, SupportsComponentDiscovery):
|
||||
_dit_modules: ClassVar[list[str]] = ["transformer"]
|
||||
_encoder_modules: ClassVar[list[str]] = ["text_encoder"]
|
||||
_vae_modules: ClassVar[list[str]] = ["vae"]
|
||||
_resident_modules: ClassVar[list[str]] = [] # optional
|
||||
```
|
||||
|
||||
- `_dit_modules`: denoising submodules (kept on GPU during diffusion loop)
|
||||
- `_encoder_modules`: encoder/vision submodules (offloaded to CPU during diffusion loop)
|
||||
- `_vae_modules`: VAE(s) (handled by both sequential and layerwise backends)
|
||||
- `_resident_modules`: additional modules to pin on GPU during layerwise
|
||||
offloading (e.g. embedders, connectors). Only used by the layerwise
|
||||
backend. Optional — defaults to `[]`.
|
||||
|
||||
All attribute names support dotted paths for nested submodules
|
||||
(e.g. `"pipe.transformer"`, `"bagel.time_embedder"`).
|
||||
|
||||
Pipelines without `SupportsComponentDiscovery` fall back to scanning
|
||||
well-known attribute names (`transformer`, `text_encoder`, `vae`,
|
||||
etc.), which fails for non-standard names.
|
||||
|
||||
Keep model-specific checkpoint paths, nested block aliases, layout transforms,
|
||||
and component lifecycles in the model package. Change a shared offloader only
|
||||
for a general contract, demonstrate another consumer or a framework-level bug,
|
||||
and add one focused shared regression. Avoid `if ModelName` branches in shared
|
||||
backends.
|
||||
|
||||
### Step 12: Performance Profiling
|
||||
|
||||
After verifying correctness and implementing parallelism/caching, profile the model's performance to identify bottlenecks and ensure optimal execution.
|
||||
|
||||
See the [Profiling Single-Stage Diffusion](../../../docs/contributing/profiling.md#3-profiling-single-stage-diffusion) guide for detailed instructions on:
|
||||
1. Using the PyTorch profiler (`profiler: "torch"`) to capture detailed CPU/CUDA traces.
|
||||
2. Using Nsight Systems (`nsys`) with `profiler: "cuda"` for low-overhead CUDA traces.
|
||||
3. Controlling profiling via `omni.start_profile()` and `omni.stop_profile()`.
|
||||
|
||||
Report cold E2E, warm user latency, steady-state wave time, peak allocated and
|
||||
reserved HBM, host PSS for the full process tree, and stage boundaries. State
|
||||
whether prompt encoding and VAE/audio decoding are included. For multi-device
|
||||
layouts, plot user latency against throughput per device and keep different
|
||||
denoising-step counts on separate Pareto frontiers.
|
||||
|
||||
---
|
||||
|
||||
## Pre-commit conventions
|
||||
|
||||
New library files must pass the local gates in
|
||||
[docs/contributing/README.md](../../../docs/contributing/README.md#linting).
|
||||
That page is the full hook list (SPDX, forbidden imports including Hugging Face
|
||||
Hub / Triton / pickle, `torch.cuda`, mypy, test marks, markdownlint, Buildkite,
|
||||
shellcheck). In particular:
|
||||
|
||||
- SPDX copyright is `vLLM-Omni project` (stale `vLLM project` is rewritten).
|
||||
- Use `import regex as re` and `pybase64` in `vllm_omni/`; do not import stdlib
|
||||
`re` or `base64`. Hugging Face Hub downloads go through
|
||||
`vllm.transformers_utils.repo_utils`.
|
||||
- Do not add `torch.cuda.*` call sites; use `current_omni_platform`.
|
||||
- New `tests/**/test_*.py` files need a CI level mark and a hardware mark.
|
||||
- Do not expand `CHECK_IMPORTS[*].allowed_files` or `ALLOWED_FILES` without review.
|
||||
- GitHub Actions skips SPDX/shellcheck/mypy-3.10/test-marks/markdownlint; run
|
||||
`pre-commit` locally.
|
||||
|
||||
## Iterative Development Tips
|
||||
|
||||
1. **Start minimal**: Basic generation first, no parallelism/caching
|
||||
2. **Use `--enforce-eager`**: Disable torch.compile during debugging
|
||||
3. **Use small models**: Test with smaller variants first
|
||||
4. **Check tensor shapes**: Most errors are reshape mismatches in attention
|
||||
5. **Add features incrementally**: Single GPU → TP → SP → CFG → HSDP → Cache-DiT
|
||||
6. **For custom models**: Run the pinned reference separately, then port the runtime natively; do not ship temporary imports from the reference implementation
|
||||
7. **Cache-DiT before parallelism tuning**: Cache-DiT is lossy — verify quality at baseline before combining with parallelism
|
||||
8. **Combine lossless + lossy**: e.g., TP + SP + Cache-DiT for maximum throughput
|
||||
|
||||
## Reference Files
|
||||
|
||||
- [vllm-omni-test skill](../vllm-omni-test/SKILL.md) — L1–L4 markers, naming, Buildkite wiring, run commands
|
||||
- [Transformer Adaptation](references/transformer-adaptation.md) — porting transformers from diffusers
|
||||
- [Custom Model Patterns](references/custom-model-patterns.md) — patterns for non-diffusers models
|
||||
- [Native Model Integration Checklist](references/native-model-integration-checklist.md) — ownership boundaries, phase gates, qualification matrix, and review evidence
|
||||
- [Parallelism Patterns](references/parallelism-patterns.md) — TP, SP/USP, CFG parallel, HSDP implementation details
|
||||
- [Cache-DiT Patterns](references/cache-dit-patterns.md) — cache-dit acceleration for standard and custom architectures
|
||||
- [Troubleshooting](references/troubleshooting.md) — common errors and fixes
|
||||
@@ -1,264 +0,0 @@
|
||||
# Cache-DiT Patterns Reference
|
||||
|
||||
## Overview
|
||||
|
||||
Cache-DiT accelerates Diffusion Transformers by caching intermediate computation results across denoising steps. Adjacent steps produce similar features, so redundant computations can be skipped.
|
||||
|
||||
Three caching strategies:
|
||||
- **DBCache**: Dynamic block-level caching — selectively computes or caches transformer blocks based on residual differences
|
||||
- **TaylorSeer**: Calibration-based prediction using Taylor expansion to estimate block outputs
|
||||
- **SCM** (Step Computation Masking): Dynamic step skipping based on configurable policies
|
||||
|
||||
Speedup is model-, step-, and policy-dependent. Treat a cache-enabled run as
|
||||
integration evidence until logs or a focused test show real hits and repeated
|
||||
timings show an improvement over the uncached baseline.
|
||||
|
||||
**Official docs**: https://docs.vllm.ai/projects/vllm-omni/en/latest/design/feature/cache_dit
|
||||
|
||||
## Architecture
|
||||
|
||||
vLLM-Omni integrates cache-dit through `CacheDiTBackend`:
|
||||
|
||||
| Component | Purpose |
|
||||
|-----------|---------|
|
||||
| `CacheDiTBackend` | Unified backend — auto-selects enabler (standard or custom) |
|
||||
| `enable_cache_for_dit()` | Default enabler for standard single-transformer models |
|
||||
| `CUSTOM_DIT_ENABLERS` dict | Registry of custom enablers keyed by pipeline class name |
|
||||
| `BlockAdapter` | Wraps complex architectures (multi-block-list or multi-transformer) |
|
||||
| `ForwardPattern` | Specifies block forward signature: `Pattern_0`, `Pattern_1`, `Pattern_2` |
|
||||
| `ParamsModifier` | Per-transformer or per-block-list config customization |
|
||||
| `DBCacheConfig` | Configuration for DBCache parameters |
|
||||
| `cache_dit.refresh_context()` | Updates cache context when `num_inference_steps` changes |
|
||||
|
||||
**Source files**:
|
||||
- `vllm_omni/diffusion/cache/cachedit/__init__.py` — public package API
|
||||
- `vllm_omni/diffusion/cache/cachedit/backend.py` — generic backend lifecycle and integration
|
||||
- `vllm_omni/diffusion/cache/cachedit/config.py` — Cache-DiT configuration types
|
||||
- `vllm_omni/diffusion/cache/cachedit/model_specific.py` — model-specific adapters, enablers, and registry
|
||||
|
||||
Outside `vllm_omni/diffusion/cache`, import Cache-DiT symbols only from
|
||||
`vllm_omni.diffusion.cache.cachedit`.
|
||||
|
||||
## Standard Models: Automatic Support
|
||||
|
||||
Most DiT models follow this pattern:
|
||||
- Single transformer with one `nn.ModuleList` of blocks
|
||||
- Standard forward signature
|
||||
- Compatible with cache-dit's automatic detection
|
||||
|
||||
**Examples**: Qwen-Image, Z-Image, FLUX
|
||||
|
||||
No code changes needed. `CacheDiTBackend` automatically uses `enable_cache_for_dit()`:
|
||||
|
||||
```python
|
||||
from vllm_omni import Omni
|
||||
|
||||
omni = Omni(
|
||||
model="Qwen/Qwen-Image",
|
||||
cache_backend="cache_dit",
|
||||
cache_config={
|
||||
"Fn_compute_blocks": 1,
|
||||
"Bn_compute_blocks": 0,
|
||||
"max_warmup_steps": 4,
|
||||
}
|
||||
)
|
||||
```
|
||||
|
||||
What happens automatically:
|
||||
|
||||
```python
|
||||
def enable_cache_for_dit(pipeline, cache_config):
|
||||
db_cache_config = DBCacheConfig(
|
||||
num_inference_steps=None,
|
||||
Fn_compute_blocks=cache_config.Fn_compute_blocks,
|
||||
Bn_compute_blocks=cache_config.Bn_compute_blocks,
|
||||
max_warmup_steps=cache_config.max_warmup_steps,
|
||||
max_cached_steps=cache_config.max_cached_steps,
|
||||
max_continuous_cached_steps=cache_config.max_continuous_cached_steps,
|
||||
residual_diff_threshold=cache_config.residual_diff_threshold,
|
||||
)
|
||||
|
||||
cache_dit.enable_cache(pipeline.transformer, cache_config=db_cache_config)
|
||||
|
||||
def refresh_cache_context(pipeline, num_inference_steps, verbose=True):
|
||||
cache_dit.refresh_context(
|
||||
pipeline.transformer, num_inference_steps=num_inference_steps, verbose=verbose
|
||||
)
|
||||
return refresh_cache_context
|
||||
```
|
||||
|
||||
## Custom Architectures: Writing Custom Enablers
|
||||
|
||||
### When you need a custom enabler
|
||||
|
||||
- Model has multiple block lists in one transformer (e.g., `transformer_blocks` + `single_transformer_blocks`)
|
||||
- Model has two transformers (e.g., high-noise + low-noise like Wan2.2)
|
||||
- Model uses non-standard block forward signature
|
||||
|
||||
### Pattern 1: Multi-Block-List (LongCat-Image style)
|
||||
|
||||
Single transformer with two block lists:
|
||||
|
||||
```python
|
||||
import cache_dit
|
||||
from cache_dit import BlockAdapter, ForwardPattern, ParamsModifier, DBCacheConfig
|
||||
|
||||
def enable_cache_for_your_model(pipeline, cache_config):
|
||||
db_cache_config = DBCacheConfig(
|
||||
num_inference_steps=None,
|
||||
Fn_compute_blocks=cache_config.Fn_compute_blocks,
|
||||
Bn_compute_blocks=cache_config.Bn_compute_blocks,
|
||||
max_warmup_steps=cache_config.max_warmup_steps,
|
||||
max_cached_steps=cache_config.max_cached_steps,
|
||||
max_continuous_cached_steps=cache_config.max_continuous_cached_steps,
|
||||
residual_diff_threshold=cache_config.residual_diff_threshold,
|
||||
)
|
||||
|
||||
cache_dit.enable_cache(
|
||||
BlockAdapter(
|
||||
transformer=pipeline.transformer,
|
||||
blocks=[
|
||||
pipeline.transformer.transformer_blocks,
|
||||
pipeline.transformer.single_transformer_blocks,
|
||||
],
|
||||
forward_pattern=[ForwardPattern.Pattern_1, ForwardPattern.Pattern_1],
|
||||
params_modifiers=[ParamsModifier(...)],
|
||||
),
|
||||
cache_config=db_cache_config,
|
||||
)
|
||||
|
||||
def refresh_cache_context(pipeline, num_inference_steps, verbose=True):
|
||||
cache_dit.refresh_context(
|
||||
pipeline.transformer, num_inference_steps=num_inference_steps, verbose=verbose
|
||||
)
|
||||
return refresh_cache_context
|
||||
```
|
||||
|
||||
For single transformer with multiple block lists, `refresh_context` works the same as standard models — call it once on the transformer.
|
||||
|
||||
### Pattern 2: Dual-Transformer (Wan2.2 style)
|
||||
|
||||
Two transformers with separate configs:
|
||||
|
||||
```python
|
||||
def enable_cache_for_dual_transformer(pipeline, cache_config):
|
||||
db_cache_config = DBCacheConfig(...)
|
||||
|
||||
cache_dit.enable_cache(
|
||||
BlockAdapter(
|
||||
transformer=[pipeline.transformer, pipeline.transformer_2],
|
||||
blocks=[pipeline.transformer.blocks, pipeline.transformer_2.blocks],
|
||||
forward_pattern=[ForwardPattern.Pattern_2, ForwardPattern.Pattern_2],
|
||||
params_modifiers=[
|
||||
ParamsModifier(...), # Config for transformer 1
|
||||
ParamsModifier(...), # Config for transformer 2
|
||||
],
|
||||
),
|
||||
cache_config=db_cache_config,
|
||||
)
|
||||
|
||||
def refresh_cache_context(pipeline, num_inference_steps, verbose=True):
|
||||
high_steps, low_steps = _split_inference_steps(num_inference_steps)
|
||||
cache_dit.refresh_context(
|
||||
pipeline.transformer, num_inference_steps=high_steps, verbose=verbose
|
||||
)
|
||||
cache_dit.refresh_context(
|
||||
pipeline.transformer_2, num_inference_steps=low_steps, verbose=verbose
|
||||
)
|
||||
return refresh_cache_context
|
||||
```
|
||||
|
||||
Key difference: `refresh_context` must be called on **each transformer separately** with its own step count.
|
||||
|
||||
### Choosing the ForwardPattern
|
||||
|
||||
| Pattern | Block forward signature | Example models |
|
||||
|---------|------------------------|----------------|
|
||||
| `Pattern_0` | `block(hidden_states, **kwargs)` → residual added inside block | Default |
|
||||
| `Pattern_1` | `block(hidden_states, **kwargs)` → returns `(hidden_states, ...)` tuple | FLUX-style single blocks |
|
||||
| `Pattern_2` | `block(hidden_states, **kwargs)` → `(hidden_states, ...)` with different residual pattern | Wan2.2 blocks |
|
||||
|
||||
Inspect your block's `forward()` return type and residual connection pattern to choose the right one. See [Cache-DiT API Reference](https://cache-dit.readthedocs.io/en/latest/user_guide/CACHE_API/) for details.
|
||||
|
||||
## Registering Custom Enablers
|
||||
|
||||
Add your enabler to `CUSTOM_DIT_ENABLERS` in
|
||||
`vllm_omni/diffusion/cache/cachedit/model_specific.py`:
|
||||
|
||||
```python
|
||||
CUSTOM_DIT_ENABLERS = {
|
||||
"Wan22Pipeline": enable_cache_for_wan22,
|
||||
"LongCatImagePipeline": enable_cache_for_longcat_image,
|
||||
"YourModelPipeline": enable_cache_for_your_model,
|
||||
}
|
||||
```
|
||||
|
||||
The key must match `pipeline.__class__.__name__`.
|
||||
|
||||
## Configuration Parameters
|
||||
|
||||
| Parameter | Default | Description |
|
||||
|-----------|---------|-------------|
|
||||
| `Fn_compute_blocks` | 1 | Number of blocks to always compute at the front |
|
||||
| `Bn_compute_blocks` | 0 | Number of blocks to always compute at the back |
|
||||
| `max_warmup_steps` | 4 | Steps to run without caching at the beginning |
|
||||
| `max_cached_steps` | — | Max total cached steps |
|
||||
| `max_continuous_cached_steps` | — | Max consecutive cached steps |
|
||||
| `residual_diff_threshold` | 0.24 | Threshold for deciding whether to cache a block |
|
||||
|
||||
### Tuning for quality vs speed
|
||||
|
||||
| Goal | Adjustments |
|
||||
|------|-------------|
|
||||
| **More speed, acceptable quality loss** | Higher `residual_diff_threshold` (0.24-0.4), lower `max_warmup_steps` (2-4) |
|
||||
| **Better quality, less speed** | Lower `residual_diff_threshold` (0.12-0.18), higher `max_warmup_steps` (6-8), lower `max_continuous_cached_steps` (2) |
|
||||
|
||||
## Testing
|
||||
|
||||
```python
|
||||
from vllm_omni import Omni
|
||||
from vllm_omni.inputs.data import OmniDiffusionSamplingParams
|
||||
|
||||
omni = Omni(
|
||||
model="your-model-name",
|
||||
cache_backend="cache_dit",
|
||||
cache_config={
|
||||
"Fn_compute_blocks": 1,
|
||||
"Bn_compute_blocks": 0,
|
||||
"max_warmup_steps": 4,
|
||||
"residual_diff_threshold": 0.24,
|
||||
}
|
||||
)
|
||||
images = omni.generate(
|
||||
"a beautiful landscape",
|
||||
OmniDiffusionSamplingParams(num_inference_steps=50),
|
||||
)
|
||||
```
|
||||
|
||||
CLI (online serving):
|
||||
|
||||
```bash
|
||||
vllm serve your-model --omni --port 8098 \
|
||||
--cache-backend cache_dit \
|
||||
--cache-config '{"Fn_compute_blocks": 1, "Bn_compute_blocks": 0, "max_warmup_steps": 4}'
|
||||
```
|
||||
|
||||
**Verification checklist**:
|
||||
1. Logs show "Cache-dit enabled successfully on xxx"
|
||||
2. A focused test or summary confirms at least one real cache hit
|
||||
3. Quality is compared with `cache_backend=None`
|
||||
4. Performance uses a realistic step count, repeated timings, and a policy
|
||||
whose warmup does not consume the whole run
|
||||
|
||||
## Excluded Models
|
||||
|
||||
Models listed in `_NO_CACHE_ACCELERATION` in `vllm_omni/diffusion/registry.py` do not support cache-dit (e.g., `NextStep11Pipeline`, `StableDiffusionPipeline`). Check this set before attempting to enable cache-dit.
|
||||
|
||||
## Reference Implementations
|
||||
|
||||
| Model | Path | Notes |
|
||||
|-------|------|-------|
|
||||
| Standard DiT | `cachedit.backend::enable_cache_for_dit` | Default enabler, automatic |
|
||||
| Wan2.2 | `cachedit.model_specific::enable_cache_for_wan22` | Dual-transformer, auto-detects mode |
|
||||
| LongCat | `cachedit.config::CacheDiTAdapterConfig` | Declarative multi-block-list adapter |
|
||||
| BAGEL | `cachedit.model_specific::BagelCachedAdapter` | Custom cached adapter |
|
||||
@@ -1,286 +0,0 @@
|
||||
# Custom Model Patterns Reference
|
||||
|
||||
Patterns for adding models that don't come from the standard diffusers pipeline format.
|
||||
|
||||
## Directory Structure Comparison
|
||||
|
||||
### Diffusers-based model (e.g., Wan2.2)
|
||||
|
||||
```
|
||||
vllm_omni/diffusion/models/wan2_2/
|
||||
├── __init__.py # Exports pipeline + transformer + helpers
|
||||
├── pipeline_wan2_2.py # Pipeline: loads components via from_pretrained()
|
||||
├── pipeline_wan2_2_i2v.py # Variant pipeline for image-to-video
|
||||
└── wan2_2_transformer.py # Transformer: ported from diffusers, uses Attention layer
|
||||
```
|
||||
|
||||
The transformer is loaded separately via `weights_sources` + `load_weights()`. Non-transformer components (VAE, text encoder) are loaded in `__init__` via `from_pretrained()`.
|
||||
|
||||
### External adapter (not native support)
|
||||
|
||||
```
|
||||
vllm_omni/diffusion/models/<name>/
|
||||
├── __init__.py # Exports pipeline only
|
||||
├── pipeline_<name>.py # Pipeline: loads ALL weights in __init__ via custom helpers
|
||||
└── <backbone>.py # Custom transformer / fusion architecture
|
||||
|
||||
examples/offline_inference/<task>/
|
||||
└── download_<name>.py # Downloads weights and/or clones code repo
|
||||
```
|
||||
|
||||
All weights are loaded eagerly in `__init__`. `load_weights()` is a no-op. The
|
||||
external dependency is imported with try/except. Use this pattern only when the
|
||||
requested scope explicitly allows an adapter; it does not satisfy a request for
|
||||
a native vLLM-Omni implementation.
|
||||
|
||||
### Custom model with ported code (e.g., BAGEL)
|
||||
|
||||
```
|
||||
vllm_omni/diffusion/models/bagel/
|
||||
├── __init__.py
|
||||
├── pipeline_bagel.py # Pipeline: instantiates models, uses weights_sources
|
||||
├── bagel_transformer.py # Full LLM backbone (Qwen2-MoT) ported into vllm-omni
|
||||
└── autoencoder.py # Custom VAE ported from original repo
|
||||
```
|
||||
|
||||
Model code is fully ported (no external dependency). Uses `weights_sources` and `load_weights()` with custom name remapping to handle non-diffusers safetensors format.
|
||||
|
||||
## Weight Loading Patterns
|
||||
|
||||
### Pattern 1: Standard diffusers flow (Wan2.2, Z-Image, FLUX)
|
||||
|
||||
```
|
||||
init → create transformer (empty) → set weights_sources → [loader calls load_weights()]
|
||||
```
|
||||
|
||||
- `weights_sources` points to safetensors in HF subfolder (e.g., `transformer/`)
|
||||
- `load_weights()` receives `(name, tensor)` pairs from the loader
|
||||
- Name remapping handles diffusers→vllm-omni differences (QKV fusion, Sequential index removal)
|
||||
|
||||
### Pattern 2: Custom safetensors at root (BAGEL)
|
||||
|
||||
```
|
||||
init → create all models (empty) → set weights_sources(subfolder=None) → [loader calls load_weights()]
|
||||
```
|
||||
|
||||
- `weights_sources` points to **root** of model directory, not a subfolder
|
||||
- Weights have non-diffusers names (e.g., `bagel.language_model.model.layers.0.self_attn.q_proj.weight`)
|
||||
- `load_weights()` does heavy name normalization
|
||||
|
||||
```python
|
||||
self.weights_sources = [
|
||||
DiffusersPipelineLoader.ComponentSource(
|
||||
model_or_path=od_config.model,
|
||||
subfolder=None, # root directory
|
||||
prefix="", # no prefix stripping
|
||||
fall_back_to_pt=False,
|
||||
)
|
||||
]
|
||||
```
|
||||
|
||||
### Pattern 3: Fully custom loading
|
||||
|
||||
```
|
||||
init → load ALL weights eagerly via custom helpers → load_weights() = no-op
|
||||
```
|
||||
|
||||
- No `weights_sources` attribute — standard loader finds nothing to iterate
|
||||
- Custom init functions (e.g., `init_wan_vae_2_2()`, `load_fusion_checkpoint()`) handle downloading and loading
|
||||
- `load_weights()` is `pass`
|
||||
- Weights may come from multiple HF repos in different formats (`.pth`, `.safetensors`)
|
||||
|
||||
Use this only for an explicitly approved external adapter when:
|
||||
- The original model has complex, well-tested loading code you don't want to rewrite
|
||||
- Weights span multiple HF repos
|
||||
- Weight format is non-standard (e.g., a single `.pth` file, not sharded safetensors)
|
||||
|
||||
## model_index.json for Custom Models
|
||||
|
||||
Standard diffusers `model_index.json`:
|
||||
```json
|
||||
{
|
||||
"_class_name": "WanPipeline",
|
||||
"_diffusers_version": "0.35.0.dev0",
|
||||
"scheduler": ["diffusers", "UniPCMultistepScheduler"],
|
||||
"transformer": ["diffusers", "WanTransformer3DModel"],
|
||||
"vae": ["diffusers", "AutoencoderKLWan"]
|
||||
}
|
||||
```
|
||||
|
||||
Custom model `model_index.json` (minimal):
|
||||
```json
|
||||
{
|
||||
"_class_name": "YourCustomPipeline",
|
||||
"fusion": "weights/model.safetensors"
|
||||
}
|
||||
```
|
||||
|
||||
The only **required** field is `_class_name` — it must match a key in
|
||||
`_DIFFUSION_MODELS` in `registry.py`. Other fields are model-specific and
|
||||
accessible via `od_config.model_config`.
|
||||
|
||||
When the released repository cannot be modified and contains neither this file
|
||||
nor a root `config.json`, use the generic native-checkpoint signature resolver.
|
||||
Match the exact Hub ID or a distinctive local file set; never infer support from
|
||||
a model-name substring.
|
||||
|
||||
## External Dependency Management
|
||||
|
||||
### Git clone + .pth injection
|
||||
|
||||
```python
|
||||
def download_dependency():
|
||||
CACHE_DIR.mkdir(parents=True, exist_ok=True)
|
||||
with open(LOCK_FILE, "w") as f:
|
||||
fcntl.flock(f, fcntl.LOCK_EX)
|
||||
if not DEPENDENCY_DIR.exists():
|
||||
subprocess.run([
|
||||
"git", "clone", "--depth", "1",
|
||||
REPO_URL, "--branch", BRANCH,
|
||||
str(DEPENDENCY_DIR)
|
||||
], check=True)
|
||||
fcntl.flock(f, fcntl.LOCK_UN)
|
||||
|
||||
# Add to Python path via .pth file
|
||||
site_packages = Path(site.getsitepackages()[0])
|
||||
pth_file = site_packages / "vllm_omni_dependency.pth"
|
||||
pth_file.write_text(str(DEPENDENCY_DIR))
|
||||
```
|
||||
|
||||
### Direct port (BAGEL pattern)
|
||||
|
||||
Copy essential files from the original repo into `vllm_omni/diffusion/models/<name>/`. Adapt imports to use vllm-omni utilities. Benefits: no external dependency, no git clone step. Drawback: must maintain the ported code.
|
||||
|
||||
This is the default for native model support. Keep the reference repository at
|
||||
a pinned revision for architecture analysis and golden outputs, but do not
|
||||
import it from production code.
|
||||
|
||||
## Multi-Modal Input/Output Protocols
|
||||
|
||||
Custom models that handle images, audio, or video I/O should implement protocol classes:
|
||||
|
||||
```python
|
||||
from vllm_omni.diffusion.models.interface import (
|
||||
SupportImageInput, # Model accepts image input
|
||||
SupportAudioInput, # Model accepts audio input
|
||||
SupportAudioOutput, # Model produces audio output
|
||||
)
|
||||
|
||||
class MyPipeline(nn.Module, SupportImageInput, SupportAudioInput, SupportAudioOutput):
|
||||
pass # Protocol markers enable proper engine routing
|
||||
```
|
||||
|
||||
The engine checks `isinstance(pipeline, SupportImageInput)` at startup to configure input validation and warmup behavior.
|
||||
|
||||
## Hardcoded Config vs Config Files
|
||||
|
||||
Diffusers models use `config.json` in each subfolder. Custom models often use:
|
||||
|
||||
**Module-level config dicts**:
|
||||
```python
|
||||
VIDEO_CONFIG = {
|
||||
"patch_size": [1, 2, 2], "model_type": "ti2v",
|
||||
"dim": 3072, "ffn_dim": 14336, "num_heads": 24, "num_layers": 30, ...
|
||||
}
|
||||
```
|
||||
|
||||
**Loaded from custom JSON** (BAGEL):
|
||||
```python
|
||||
cfg_path = os.path.join(model_path, "config.json")
|
||||
with open(cfg_path) as f:
|
||||
bagel_cfg = json.load(f)
|
||||
vae_cfg = bagel_cfg.get("vae_config", {})
|
||||
```
|
||||
|
||||
## Custom Architecture Patterns
|
||||
|
||||
### Split forward API
|
||||
|
||||
When a fusion model needs to interleave blocks from two backbones:
|
||||
|
||||
```python
|
||||
class WanModel(nn.Module):
|
||||
def prepare_transformer_block_kwargs(self, x, t, context, ...):
|
||||
# Patch embed, time embed, text embed, RoPE
|
||||
return x, e, kwargs
|
||||
|
||||
def post_transformer_block_out(self, x, grid_sizes, e):
|
||||
# Output projection, unpatchify
|
||||
return output
|
||||
|
||||
def forward(self, *args, **kwargs):
|
||||
raise NotImplementedError # Fusion model handles block iteration
|
||||
```
|
||||
|
||||
The `FusionModel` then iterates blocks in lock-step:
|
||||
```python
|
||||
for video_block, audio_block in zip(self.video_model.blocks, self.audio_model.blocks):
|
||||
video_out = video_block(video_hidden, ...)
|
||||
audio_out = audio_block(audio_hidden, ...)
|
||||
# Cross-attend between modalities
|
||||
video_out = cross_attention(video_out, audio_out)
|
||||
audio_out = cross_attention(audio_out, video_out)
|
||||
```
|
||||
|
||||
### LLM-as-denoiser (BAGEL)
|
||||
|
||||
When the backbone is a language model that also does diffusion:
|
||||
|
||||
```python
|
||||
class BagelModel(nn.Module):
|
||||
def __init__(self):
|
||||
self.language_model = Qwen2MoTForCausalLM(config)
|
||||
self.vit_model = SiglipVisionModel(vit_config)
|
||||
```
|
||||
|
||||
The LLM processes both text tokens and latent image tokens in a single forward pass, using KV caching for the text portion.
|
||||
|
||||
## Pre/Post Processing for Custom Models
|
||||
|
||||
Custom models typically handle pre/post processing **inside `forward()`** rather than via registered functions, because the logic is tightly coupled:
|
||||
|
||||
```python
|
||||
def forward(self, req: OmniDiffusionRequest) -> DiffusionOutput:
|
||||
# Inline preprocessing
|
||||
image = self._load_and_resize_image(req.prompts[0].get("multi_modal_data", {}).get("image"))
|
||||
image_latent = self._vae_encode(image)
|
||||
|
||||
# ... denoising loop ...
|
||||
|
||||
# Inline postprocessing
|
||||
pil_image = self._decode_to_pil(latents)
|
||||
return DiffusionOutput(output=[pil_image])
|
||||
```
|
||||
|
||||
If pre/post functions are not registered in `_DIFFUSION_PRE_PROCESS_FUNCS` / `_DIFFUSION_POST_PROCESS_FUNCS`, the engine simply skips those steps.
|
||||
|
||||
## Download Script Template
|
||||
|
||||
```python
|
||||
# examples/offline_inference/<name>/download_<name>.py
|
||||
from huggingface_hub import snapshot_download
|
||||
import json, os
|
||||
|
||||
def main(output_dir):
|
||||
# Download model weights from HF
|
||||
snapshot_download(repo_id="org/model-weights", local_dir=os.path.join(output_dir, "weights"))
|
||||
|
||||
# Download additional components if from separate repos
|
||||
snapshot_download(repo_id="org/vae-weights", local_dir=os.path.join(output_dir, "vae"),
|
||||
allow_patterns=["*.safetensors"])
|
||||
|
||||
# Generate model_index.json
|
||||
config = {"_class_name": "YourPipeline", "custom_key": "weights/model.safetensors"}
|
||||
with open(os.path.join(output_dir, "model_index.json"), "w") as f:
|
||||
json.dump(config, f, indent=2)
|
||||
|
||||
# Install external code dependency (if needed)
|
||||
download_dependency()
|
||||
|
||||
if __name__ == "__main__":
|
||||
import argparse
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--output-dir", default="./your_model")
|
||||
args = parser.parse_args()
|
||||
main(args.output_dir)
|
||||
```
|
||||
@@ -1,131 +0,0 @@
|
||||
# Native Diffusion Model Integration Checklist
|
||||
|
||||
Use this checklist for native non-Diffusers models, hybrid pipelines, and
|
||||
model-specific optimization work.
|
||||
|
||||
## 1. Freeze the contract before coding
|
||||
|
||||
Record:
|
||||
|
||||
- Reference repository and immutable revision.
|
||||
- Official checkpoint ID, local layout, license, and required auxiliary files.
|
||||
- Pipeline stages: prompt/image encoding, denoising, video/image/audio decoding,
|
||||
and muxing.
|
||||
- Token packing, RoPE coordinates, attention sinks/LSE behavior, MoE layout,
|
||||
scheduler equations, CFG formulation, dtypes, and checkpoint transforms.
|
||||
- Output geometry, frame count/FPS, audio rate/channels, supported tasks, and
|
||||
default generation steps.
|
||||
- Available local hardware versus the official qualification hardware.
|
||||
- Intended default deployment and optional optimization paths.
|
||||
|
||||
State whether the deliverable is a native implementation or an external
|
||||
adapter. For native support, use the reference as an oracle only.
|
||||
|
||||
## 2. Use phase gates
|
||||
|
||||
| Phase | Required exit evidence |
|
||||
|---|---|
|
||||
| Architecture | Component/weight inventory; unsupported reference dependencies identified; ownership decisions documented |
|
||||
| Native correctness | Strict checkpoint load; tiny forward or one-step golden vs pinned reference; output contract validated |
|
||||
| Framework integration | Registry/discovery, shared request types, existing category example, serving route, docs, and recipe work without model-name branching in shared scripts |
|
||||
| Lossless distribution | One-rank oracle vs each supported TP/SP/CFG/HSDP/VAE layout; released-checkpoint smoke for the recommended layout |
|
||||
| Memory optimization | Ordinary/layerwise/DLO behavior, peak HBM, full-tree host PSS, and stage timeline; shared backend unchanged unless a general contract is proven |
|
||||
| Lossy optimization | Cache hit demonstrated; quality delta measured; speed claim made only when warmup permits hits |
|
||||
| Full qualification | Default-resolution E2E at official step count for every supported task; media metadata and hashes recorded |
|
||||
| Review readiness | Focused tests, Ruff/format, strict docs, DCO, current-main rebase, issue/RFC for large changes, concise PR evidence and limitations |
|
||||
|
||||
Do not start performance tuning before native one-step parity. Do not claim a
|
||||
feature from topology validation alone when released weights can exercise it.
|
||||
|
||||
## 3. Keep ownership narrow
|
||||
|
||||
Default placement:
|
||||
|
||||
| Concern | Owner |
|
||||
|---|---|
|
||||
| Architecture, checkpoint remap, packed attention math, nested block alias, auxiliary lifecycle | Model package |
|
||||
| Kernel/version dispatch already useful to multiple models | Shared attention utility |
|
||||
| Request-dependent example defaults | Typed model-extra entry consumed through a registry |
|
||||
| Exact-ID/local-file detection for repositories without standard metadata | Generic native-checkpoint signature table |
|
||||
| General offload/cache/HSDP bug | Shared layer plus one focused shared regression |
|
||||
|
||||
Before changing shared code, answer:
|
||||
|
||||
1. Is this a framework contract or only this checkpoint's layout?
|
||||
2. Can the model expose an existing declarative hook instead?
|
||||
3. Is there a second consumer or a framework-level failing test?
|
||||
4. Can the shared change avoid importing or naming the model?
|
||||
|
||||
If the answers do not justify a shared change, keep it model-local.
|
||||
|
||||
## 4. Reuse examples without leaking model logic
|
||||
|
||||
- Reuse the existing `x_to_y.py` category script.
|
||||
- Resolve the pipeline class through standard metadata or native signature
|
||||
discovery, not path substrings.
|
||||
- Put dynamic model defaults and validation in `model_extras`; let shared
|
||||
examples consume only the generic typed API.
|
||||
- Do not create image/speech/default abstractions without a real consumer.
|
||||
- Add a dedicated example only when the shared request/output protocol cannot
|
||||
express the model.
|
||||
|
||||
## 5. Build a bounded evidence matrix
|
||||
|
||||
Start with the recommended resident layout. Add only supported, useful points:
|
||||
|
||||
- Single request: TP, SP, or hybrid TP/SP.
|
||||
- Throughput: DLO/DP layouts when concurrent requests are valid.
|
||||
- CFG: compare CFG-parallel output with the packed-CFG oracle only if the model
|
||||
actually uses CFG.
|
||||
- HSDP: record loading and materialization peaks; it can use more HBM than a
|
||||
resident SP layout.
|
||||
- VAE patch parallelism: compare decoded tensors or media exactly where the
|
||||
algorithm should be lossless.
|
||||
- Cache: separate cache integration from acceleration evidence.
|
||||
|
||||
With four devices, qualify four-device layouts such as DP4, DP2SP2, SP4, or
|
||||
CFG2xSP2. Do not present CFG2xSP4 or an official eight-device runtime as locally
|
||||
validated.
|
||||
|
||||
For each measured layout record:
|
||||
|
||||
- Cold E2E and warm user latency.
|
||||
- Steady-state wave time and throughput per device.
|
||||
- Prompt, denoise, video decode, audio decode, and mux times.
|
||||
- Peak allocated/reserved HBM and full-process-tree host PSS.
|
||||
- Step count, seed, output geometry, frame/FPS/audio metadata, and parity metric.
|
||||
|
||||
Plot latency versus throughput/device. Keep one-step, four-step, and full-step
|
||||
results separate because they are not quality-equivalent.
|
||||
|
||||
## 6. Keep tests proportional
|
||||
|
||||
Prefer a small set with distinct failure ownership:
|
||||
|
||||
1. Strict weight-key/dtype/load contract.
|
||||
2. Tiny native golden or one-step reference parity.
|
||||
3. One parametrized distributed-parity test covering supported layouts.
|
||||
4. Pipeline/request/output geometry and serving smoke.
|
||||
5. Shared-example/default test when the category script changes.
|
||||
6. One shared regression only for a genuine shared-layer fix.
|
||||
|
||||
Avoid duplicating the same shape or configuration assertion across
|
||||
model-specific files. Keep expensive released-checkpoint evidence in the PR or
|
||||
recipe unless the repository's CI tier explicitly requires it.
|
||||
|
||||
## 7. Write evidence, not marketing
|
||||
|
||||
In the PR description include:
|
||||
|
||||
- A compact architecture/data-flow explanation.
|
||||
- Default deployment and why it is recommended.
|
||||
- Code-change counts by component.
|
||||
- Accuracy and distributed-parity evidence tied to commands/scripts.
|
||||
- Separate Pareto figures for different step counts.
|
||||
- Supported versus validated versus planned optimizations.
|
||||
- Hardware gaps and untested official-runtime claims.
|
||||
|
||||
Call a result a bounded local qualification unless it includes repeated timing
|
||||
and variance. State whether latency includes text/image encoding and all
|
||||
decoders. Preserve the reference revision and exact commands needed to
|
||||
reproduce the evidence.
|
||||
@@ -1,572 +0,0 @@
|
||||
# Parallelism Patterns Reference
|
||||
|
||||
## Overview
|
||||
|
||||
vLLM-Omni supports multiple parallelism strategies for diffusion models. Each targets a different bottleneck:
|
||||
|
||||
| Strategy | Splits | Best For | Constraint |
|
||||
|----------|--------|----------|------------|
|
||||
| Tensor Parallel (TP) | Model layers across GPUs | Latency reduction, large models | Requires fast GPU interconnect, `num_heads % tp == 0` |
|
||||
| Sequence Parallel (SP/USP) | Sequence tokens across GPUs | Long sequences (video, high-res) | Near-linear scaling |
|
||||
| CFG Parallel | Positive/negative CFG branches | Models using classifier-free guidance | Exactly 2 GPUs |
|
||||
| HSDP | Weight shards via FSDP2 | VRAM reduction | Cannot combine with TP |
|
||||
| VAE Patch Parallel | VAE decode spatial tiles | Large VAE outputs | Auto-enables tiling |
|
||||
|
||||
**Recommended integration order**: TP → SP → CFG Parallel → HSDP
|
||||
|
||||
**Official design docs**:
|
||||
- TP: https://docs.vllm.ai/projects/vllm-omni/en/latest/design/feature/tensor_parallel
|
||||
- SP: https://docs.vllm.ai/projects/vllm-omni/en/latest/design/feature/sequence_parallel
|
||||
- CFG: https://docs.vllm.ai/projects/vllm-omni/en/latest/design/feature/cfg_parallel
|
||||
- HSDP: https://docs.vllm.ai/projects/vllm-omni/en/latest/design/feature/hsdp
|
||||
|
||||
---
|
||||
|
||||
## Tensor Parallelism (TP)
|
||||
|
||||
Replace standard `nn.Linear` with vLLM's parallel linear layers. This is the most invasive change but provides direct VRAM savings and compute speedup.
|
||||
|
||||
### Layer replacement rules
|
||||
|
||||
| Pattern | vLLM Layer | When to Use |
|
||||
|---------|-----------|-------------|
|
||||
| Fan-out (first in FFN) | `ColumnParallelLinear` | Projection that splits output across ranks |
|
||||
| Fan-in (second in FFN) | `RowParallelLinear` | Projection that gathers across ranks |
|
||||
| QKV projection | `QKVParallelLinear` | Fused Q/K/V for self-attention |
|
||||
| Single Q or K or V | `ColumnParallelLinear` | Separate projections (cross-attention) |
|
||||
| Attention output | `RowParallelLinear` | Output projection after attention |
|
||||
| Must not shard | `ReplicatedLinear` | Layers that must stay replicated |
|
||||
|
||||
### MLP Block (Up-Down Pattern)
|
||||
|
||||
```python
|
||||
from vllm.model_executor.layers.linear import (
|
||||
ColumnParallelLinear, RowParallelLinear,
|
||||
)
|
||||
|
||||
class TPFeedForward(nn.Module):
|
||||
def __init__(self, dim, ffn_dim):
|
||||
super().__init__()
|
||||
self.fc1 = ColumnParallelLinear(dim, ffn_dim, bias=False, return_bias=False)
|
||||
self.fc2 = RowParallelLinear(
|
||||
ffn_dim, dim, bias=False,
|
||||
input_is_parallel=True, # Input already sharded from fc1
|
||||
return_bias=False,
|
||||
)
|
||||
|
||||
def forward(self, x):
|
||||
x, _ = self.fc1(x)
|
||||
x = torch.nn.functional.gelu(x)
|
||||
x, _ = self.fc2(x)
|
||||
return x
|
||||
```
|
||||
|
||||
### Attention Block (QKV-Out Pattern)
|
||||
|
||||
```python
|
||||
from vllm.model_executor.layers.linear import QKVParallelLinear, RowParallelLinear
|
||||
from vllm_omni.diffusion.attention.layer import Attention
|
||||
|
||||
class TPSelfAttention(nn.Module):
|
||||
def __init__(self, dim, num_heads, num_kv_heads=None):
|
||||
super().__init__()
|
||||
num_kv_heads = num_kv_heads or num_heads
|
||||
self.head_dim = dim // num_heads
|
||||
|
||||
self.to_qkv = QKVParallelLinear(
|
||||
hidden_size=dim,
|
||||
head_size=self.head_dim,
|
||||
total_num_heads=num_heads,
|
||||
total_num_kv_heads=num_kv_heads,
|
||||
bias=False,
|
||||
return_bias=False,
|
||||
)
|
||||
self.to_out = RowParallelLinear(
|
||||
dim, dim, bias=False,
|
||||
input_is_parallel=True,
|
||||
return_bias=False,
|
||||
)
|
||||
self.attn = Attention(
|
||||
num_heads=self.to_qkv.num_heads, # Local heads per GPU
|
||||
head_size=self.head_dim,
|
||||
softmax_scale=1.0 / (self.head_dim ** 0.5),
|
||||
causal=False,
|
||||
num_kv_heads=self.to_qkv.num_kv_heads, # Local KV heads per GPU
|
||||
role="self",
|
||||
)
|
||||
|
||||
def forward(self, x):
|
||||
qkv, _ = self.to_qkv(x)
|
||||
q, k, v = qkv.split(
|
||||
[self.to_qkv.num_heads * self.head_dim,
|
||||
self.to_qkv.num_kv_heads * self.head_dim,
|
||||
self.to_qkv.num_kv_heads * self.head_dim],
|
||||
dim=-1,
|
||||
)
|
||||
B, S, _ = x.shape
|
||||
q = q.view(B, S, self.to_qkv.num_heads, self.head_dim)
|
||||
k = k.view(B, S, self.to_qkv.num_kv_heads, self.head_dim)
|
||||
v = v.view(B, S, self.to_qkv.num_kv_heads, self.head_dim)
|
||||
out = self.attn(q, k, v)
|
||||
out = out.reshape(B, S, -1)
|
||||
out, _ = self.to_out(out)
|
||||
return out
|
||||
```
|
||||
|
||||
### QKV Fusion in load_weights
|
||||
|
||||
When you fuse separate Q/K/V into `QKVParallelLinear`, map diffusers' separate weight names:
|
||||
|
||||
```python
|
||||
stacked_params_mapping = [
|
||||
("to_qkv", "to_q", "q"),
|
||||
("to_qkv", "to_k", "k"),
|
||||
("to_qkv", "to_v", "v"),
|
||||
]
|
||||
|
||||
def load_weights(self, weights):
|
||||
params = dict(self.named_parameters())
|
||||
loaded = set()
|
||||
for name, tensor in weights:
|
||||
for fused_name, orig_name, shard_id in stacked_params_mapping:
|
||||
if orig_name in name:
|
||||
name = name.replace(orig_name, fused_name)
|
||||
param = params[name]
|
||||
param.weight_loader(param, tensor, shard_id)
|
||||
loaded.add(name)
|
||||
break
|
||||
else:
|
||||
if name in params:
|
||||
param = params[name]
|
||||
if hasattr(param, "weight_loader"):
|
||||
param.weight_loader(param, tensor)
|
||||
else:
|
||||
default_weight_loader(param, tensor)
|
||||
loaded.add(name)
|
||||
return loaded
|
||||
```
|
||||
|
||||
### RMSNorm with TP
|
||||
|
||||
When RMSNorm sits between TP-sharded dimensions, use `DistributedRMSNorm` — it computes global RMS via all-reduce across TP ranks. See the Wan2.2 implementation for the pattern.
|
||||
|
||||
### TP Constraints
|
||||
|
||||
- `num_heads % tp_size == 0`
|
||||
- `num_kv_heads % tp_size == 0`
|
||||
- Use `self.to_qkv.num_heads` (local per-GPU count), not total heads, for split sizes
|
||||
|
||||
### Testing TP
|
||||
|
||||
```bash
|
||||
python text_to_image.py --model Your-org/your-model \
|
||||
--tensor-parallel-size 2 --output "tp_test.png"
|
||||
```
|
||||
|
||||
**Verify**: speedup, memory reduction proportional to TP size, quality matches single-GPU.
|
||||
|
||||
### Reference implementations
|
||||
|
||||
| Model | Path |
|
||||
|-------|------|
|
||||
| Z-Image | `vllm_omni/diffusion/models/z_image/z_image_transformer.py` |
|
||||
| FLUX | `vllm_omni/diffusion/models/flux/flux_transformer.py` |
|
||||
| Qwen-Image | `vllm_omni/diffusion/models/qwen_image/qwen_image_transformer.py` |
|
||||
|
||||
---
|
||||
|
||||
## Sequence Parallelism (SP / USP)
|
||||
|
||||
SP splits sequence tokens across GPUs using Ulysses (all-to-all) or Ring (P2P) communication. It is applied non-intrusively via the `_sp_plan` dict — no changes to `forward()` logic.
|
||||
|
||||
### Approach 1: Non-Intrusive `_sp_plan` (Recommended)
|
||||
|
||||
The framework automatically registers hooks to shard inputs and gather outputs at `nn.Module` boundaries.
|
||||
|
||||
#### Step 1: Identify module boundaries
|
||||
|
||||
Find where tensors need sharding/gathering:
|
||||
|
||||
```python
|
||||
class MyTransformer(nn.Module):
|
||||
def __init__(self):
|
||||
self.patch_embed = PatchEmbed() # Before blocks
|
||||
self.pos_embed = RoPE() # RoPE may need splitting
|
||||
self.blocks = nn.ModuleList([...]) # Blocks process sharded x
|
||||
self.norm_out = LayerNorm()
|
||||
self.proj_out = Linear() # Gather after this
|
||||
|
||||
def forward(self, x):
|
||||
x = self.patch_embed(x)
|
||||
pos = self.pos_embed(x)
|
||||
for block in self.blocks:
|
||||
x = block(x, pos)
|
||||
x = self.norm_out(x)
|
||||
return self.proj_out(x)
|
||||
```
|
||||
|
||||
#### Step 2: Handle inline operations
|
||||
|
||||
`_sp_plan` hooks only work at `nn.Module` boundaries. Inline ops like `torch.cat()` must be extracted into submodules:
|
||||
|
||||
```python
|
||||
# BAD: Inline — hooks can't intercept
|
||||
unified = torch.cat([x, cap_feats], dim=1)
|
||||
|
||||
# GOOD: Extract into submodule
|
||||
class UnifiedPrepare(nn.Module):
|
||||
def forward(self, x, cap_feats):
|
||||
return torch.cat([x, cap_feats], dim=1)
|
||||
|
||||
self.unified_prepare = UnifiedPrepare()
|
||||
unified = self.unified_prepare(x, cap_feats)
|
||||
```
|
||||
|
||||
Common cases: `torch.cat()`, `pad_sequence()`, `tensor.reshape()`, complex preprocessing.
|
||||
|
||||
#### Step 3: Write `_sp_plan`
|
||||
|
||||
**Pattern 1: Shard at first block, gather at output** (most common)
|
||||
|
||||
```python
|
||||
from vllm_omni.diffusion.distributed.sp_plan import (
|
||||
SequenceParallelInput, SequenceParallelOutput,
|
||||
)
|
||||
|
||||
class StandardTransformer(nn.Module):
|
||||
_sp_plan = {
|
||||
"blocks.0": {
|
||||
"hidden_states": SequenceParallelInput(split_dim=1, expected_dims=3),
|
||||
},
|
||||
"proj_out": SequenceParallelOutput(gather_dim=1, expected_dims=3),
|
||||
}
|
||||
```
|
||||
|
||||
**Pattern 2: Shard RoPE outputs separately**
|
||||
|
||||
```python
|
||||
class TransformerWithRoPE(nn.Module):
|
||||
_sp_plan = {
|
||||
"rope": {
|
||||
0: SequenceParallelInput(split_dim=1, expected_dims=4, split_output=True),
|
||||
1: SequenceParallelInput(split_dim=1, expected_dims=4, split_output=True),
|
||||
},
|
||||
"blocks.0": {
|
||||
"hidden_states": SequenceParallelInput(split_dim=1, expected_dims=3),
|
||||
},
|
||||
"proj_out": SequenceParallelOutput(gather_dim=1, expected_dims=3),
|
||||
}
|
||||
```
|
||||
|
||||
**Pattern 3: Dual-stream (shard image, replicate text)**
|
||||
|
||||
```python
|
||||
class DualStreamTransformer(nn.Module):
|
||||
_sp_plan = {
|
||||
"rope_preparer": {
|
||||
2: SequenceParallelInput(split_dim=0, expected_dims=2, split_output=True),
|
||||
3: SequenceParallelInput(split_dim=0, expected_dims=2, split_output=True),
|
||||
},
|
||||
"transformer_blocks.0": {
|
||||
"hidden_states": SequenceParallelInput(split_dim=1, expected_dims=3),
|
||||
},
|
||||
"proj_out": SequenceParallelOutput(gather_dim=1, expected_dims=3),
|
||||
}
|
||||
```
|
||||
|
||||
### API Reference
|
||||
|
||||
**SequenceParallelInput**:
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `split_dim` | int | Dimension to split (usually 1 for sequence) |
|
||||
| `expected_dims` | int/None | Expected tensor rank for validation |
|
||||
| `split_output` | bool | `False`: shard input params; `True`: shard output tensors |
|
||||
| `auto_pad` | bool | Auto-pad if sequence not divisible by world_size |
|
||||
|
||||
**SequenceParallelOutput**:
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `gather_dim` | int | Dimension to gather (usually 1 for sequence) |
|
||||
| `expected_dims` | int/None | Expected tensor rank for validation |
|
||||
|
||||
**Module naming**:
|
||||
|
||||
| Key | Meaning |
|
||||
|-----|---------|
|
||||
| `"blocks.0"` | First element of ModuleList |
|
||||
| `"blocks.*"` | All elements of ModuleList |
|
||||
| `"rope"` | Named submodule |
|
||||
|
||||
**Dictionary value types**:
|
||||
|
||||
| Key type | split_output | Description |
|
||||
|----------|-------------|-------------|
|
||||
| `"param_name"` (str) | False | Shard input parameter by name |
|
||||
| `0, 1, ...` (int) | True | Shard output tuple by index |
|
||||
|
||||
### Approach 2: Intrusive Modification (Complex Cases)
|
||||
|
||||
For dynamic sharding logic that can't be expressed via `_sp_plan`:
|
||||
|
||||
```python
|
||||
from vllm_omni.diffusion.distributed.sp_sharding import sp_shard, sp_gather
|
||||
|
||||
def forward(self, hidden_states, ...):
|
||||
if self.parallel_config.sequence_parallel_size > 1:
|
||||
hidden_states = sp_shard(hidden_states, dim=1)
|
||||
for block in self.blocks:
|
||||
hidden_states = block(hidden_states)
|
||||
if self.parallel_config.sequence_parallel_size > 1:
|
||||
hidden_states = sp_gather(hidden_states, dim=1)
|
||||
return hidden_states
|
||||
```
|
||||
|
||||
Use intrusive modification as a last resort — `_sp_plan` is preferred for maintainability.
|
||||
|
||||
### UAA Mode (Experimental)
|
||||
|
||||
`ulysses_mode="advanced_uaa"` handles arbitrary sequence lengths and head counts that aren't divisible by `ulysses_degree`. Uses variable all-to-all split sizes and temporary head padding.
|
||||
|
||||
### Combining SP methods
|
||||
|
||||
Ulysses and Ring can be combined: `ulysses_degree × ring_degree = total SP GPUs`.
|
||||
|
||||
```python
|
||||
DiffusionParallelConfig(ulysses_degree=2, ring_degree=2) # 4 GPUs total
|
||||
```
|
||||
|
||||
### Testing SP
|
||||
|
||||
```bash
|
||||
# Offline
|
||||
python text_to_image.py --model Your-model --ulysses-degree 2
|
||||
|
||||
# Online serving
|
||||
vllm serve Your-model --omni --usp 2
|
||||
```
|
||||
|
||||
### Reference implementations
|
||||
|
||||
| Model | Path |
|
||||
|-------|------|
|
||||
| Qwen-Image | `vllm_omni/diffusion/models/qwen_image/qwen_image_transformer.py` |
|
||||
| Wan2.2 | `vllm_omni/diffusion/models/wan2_2/wan2_2_transformer.py` |
|
||||
| Z-Image | `vllm_omni/diffusion/models/z_image/z_image_transformer.py` |
|
||||
|
||||
---
|
||||
|
||||
## CFG Parallelism
|
||||
|
||||
Distributes positive/negative Classifier-Free Guidance branches across 2 GPUs.
|
||||
|
||||
### Implementation
|
||||
|
||||
Inherit `CFGParallelMixin` and implement `diffuse()`:
|
||||
|
||||
```python
|
||||
from vllm_omni.diffusion.distributed.cfg_parallel import CFGParallelMixin
|
||||
|
||||
class YourPipeline(nn.Module, CFGParallelMixin):
|
||||
def diffuse(self, latents, timesteps, prompt_embeds, negative_embeds,
|
||||
do_true_cfg, true_cfg_scale, **kwargs):
|
||||
for i, t in enumerate(timesteps):
|
||||
positive_kwargs = {
|
||||
"hidden_states": latents,
|
||||
"encoder_hidden_states": prompt_embeds,
|
||||
"timestep": t,
|
||||
}
|
||||
negative_kwargs = {
|
||||
"hidden_states": latents,
|
||||
"encoder_hidden_states": negative_embeds,
|
||||
"timestep": t,
|
||||
} if do_true_cfg else None
|
||||
|
||||
noise_pred = self.predict_noise_maybe_with_cfg(
|
||||
do_true_cfg=do_true_cfg,
|
||||
true_cfg_scale=true_cfg_scale,
|
||||
positive_kwargs=positive_kwargs,
|
||||
negative_kwargs=negative_kwargs,
|
||||
)
|
||||
latents = self.scheduler_step_maybe_with_cfg(
|
||||
noise_pred, t, latents, do_true_cfg
|
||||
)
|
||||
return latents
|
||||
```
|
||||
|
||||
### Customization hooks
|
||||
|
||||
| Method | Override when |
|
||||
|--------|-------------|
|
||||
| `predict_noise()` | Non-standard transformer call (e.g., dual-transformer like Wan2.2) |
|
||||
| `cfg_normalize_function()` | Custom normalization (e.g., LongCat with clamping) |
|
||||
| `combine_cfg_noise()` | Multi-output models (e.g., video + audio: CFG on video, positive-only on audio) |
|
||||
|
||||
**Custom predict_noise** (Wan2.2 — selects active transformer):
|
||||
|
||||
```python
|
||||
def predict_noise(self, current_model=None, **kwargs):
|
||||
if current_model is None:
|
||||
current_model = self.transformer
|
||||
return current_model(**kwargs)[0]
|
||||
```
|
||||
|
||||
**Custom combine_cfg_noise** (multi-output):
|
||||
|
||||
```python
|
||||
def combine_cfg_noise(self, positive_pred, negative_pred, scale, normalize):
|
||||
video_pos, audio_pos = positive_pred
|
||||
video_neg, audio_neg = negative_pred
|
||||
video_combined = super().combine_cfg_noise(video_pos, video_neg, scale, normalize)
|
||||
return (video_combined, audio_pos)
|
||||
```
|
||||
|
||||
### Composite scheduler for multi-output
|
||||
|
||||
When each output has its own schedule:
|
||||
|
||||
```python
|
||||
class VideoAudioScheduler:
|
||||
def __init__(self, video_scheduler, audio_scheduler):
|
||||
self.video_scheduler = video_scheduler
|
||||
self.audio_scheduler = audio_scheduler
|
||||
|
||||
def step(self, noise_pred, t, latents, return_dict=False, generator=None):
|
||||
video_out = self.video_scheduler.step(
|
||||
noise_pred[0], t[0], latents[0], return_dict=False, generator=generator
|
||||
)[0]
|
||||
audio_out = self.audio_scheduler.step(
|
||||
noise_pred[1], t[1], latents[1], return_dict=False, generator=generator
|
||||
)[0]
|
||||
return ((video_out, audio_out),)
|
||||
```
|
||||
|
||||
### Testing CFG Parallel
|
||||
|
||||
```bash
|
||||
python text_to_image.py --model Your-model \
|
||||
--cfg-parallel-size 2 --cfg-scale 4.0 \
|
||||
--negative-prompt "ugly, unclear"
|
||||
```
|
||||
|
||||
**Constraint**: `guidance_scale > 1.0` and negative prompt must be provided.
|
||||
|
||||
### Reference implementations
|
||||
|
||||
| Model | Path |
|
||||
|-------|------|
|
||||
| Qwen-Image | `vllm_omni/diffusion/models/qwen_image/cfg_parallel.py` |
|
||||
| Wan2.2 | `vllm_omni/diffusion/models/wan2_2/pipeline_wan2_2.py` |
|
||||
| Mixin base | `vllm_omni/diffusion/distributed/cfg_parallel.py` |
|
||||
|
||||
---
|
||||
|
||||
## HSDP (Hybrid Sharded Data Parallel)
|
||||
|
||||
Shards model weights across GPUs using PyTorch FSDP2. Reduces per-GPU VRAM without changing computation.
|
||||
|
||||
### Implementation
|
||||
|
||||
Add `_hsdp_shard_conditions` to the transformer class:
|
||||
|
||||
```python
|
||||
class YourTransformer(nn.Module):
|
||||
@staticmethod
|
||||
def _is_transformer_block(name: str, module) -> bool:
|
||||
return "blocks" in name and name.split(".")[-1].isdigit()
|
||||
|
||||
_hsdp_shard_conditions = [_is_transformer_block]
|
||||
```
|
||||
|
||||
For MoE models, add additional conditions:
|
||||
|
||||
```python
|
||||
class MoETransformer(nn.Module):
|
||||
@staticmethod
|
||||
def _is_transformer_block(name, module):
|
||||
return "blocks" in name and name.split(".")[-1].isdigit()
|
||||
|
||||
@staticmethod
|
||||
def _is_moe_expert(name, module):
|
||||
return "experts" in name and name.split(".")[-1].isdigit()
|
||||
|
||||
_hsdp_shard_conditions = [_is_transformer_block, _is_moe_expert]
|
||||
```
|
||||
|
||||
A module is sharded if **any** condition returns `True`.
|
||||
|
||||
### Constraints
|
||||
|
||||
- Cannot combine with Tensor Parallelism
|
||||
- For standalone HSDP (no other parallelism), `hsdp_shard_size` must be specified explicitly
|
||||
- Can combine with SP: HSDP reduces memory while SP distributes sequence
|
||||
|
||||
### Testing HSDP
|
||||
|
||||
```python
|
||||
from vllm_omni.diffusion.data import DiffusionParallelConfig
|
||||
|
||||
parallel_config = DiffusionParallelConfig(use_hsdp=True, hsdp_shard_size=8)
|
||||
omni = Omni(model="your-model", parallel_config=parallel_config)
|
||||
```
|
||||
|
||||
Or CLI:
|
||||
|
||||
```bash
|
||||
vllm serve Your-model --omni --use-hsdp
|
||||
```
|
||||
|
||||
**Verify**: logs show "HSDP Inference: replicate_size=..., shard_size=..." and "Sharded N modules + root". Check VRAM reduction.
|
||||
|
||||
### Reference implementations
|
||||
|
||||
| Model | Path |
|
||||
|-------|------|
|
||||
| Wan2.2 | `vllm_omni/diffusion/models/wan2_2/wan2_2_transformer.py` |
|
||||
| HSDP Core | `vllm_omni/diffusion/distributed/hsdp.py` |
|
||||
|
||||
---
|
||||
|
||||
## VAE Patch Parallelism
|
||||
|
||||
Shards VAE decode spatially across ranks using tiling:
|
||||
|
||||
```bash
|
||||
python text_to_image.py --model Your-model --vae-patch-parallel-size 4
|
||||
```
|
||||
|
||||
Auto-enables `--vae-use-tiling`. Uses `DistributedAutoencoderKLWan` or similar distributed VAE. Set `vae_patch_parallel_size` in `DiffusionParallelConfig`.
|
||||
|
||||
---
|
||||
|
||||
## Combining Parallelism Methods
|
||||
|
||||
Common multi-GPU recipes:
|
||||
|
||||
```bash
|
||||
# 4 GPUs: CFG (2) × Ulysses (2)
|
||||
python text_to_image.py --model Qwen/Qwen-Image \
|
||||
--cfg-parallel-size 2 --ulysses-degree 2
|
||||
|
||||
# 8 GPUs: Ulysses (4) × Ring (2) + VAE patch (8)
|
||||
python text_to_video.py --model Wan-AI/Wan2.2-T2V-A14B-Diffusers \
|
||||
--ulysses-degree 4 --ring-degree 2 --vae-patch-parallel-size 8
|
||||
|
||||
# 2 GPUs: HSDP + Ulysses (cannot combine HSDP with TP)
|
||||
vllm serve Your-model --omni --use-hsdp --usp 2
|
||||
```
|
||||
|
||||
## Discovering Parallelism Support
|
||||
|
||||
Check which parallelism methods a model supports:
|
||||
|
||||
| Check | How |
|
||||
|-------|-----|
|
||||
| **Ulysses / Ring SP** | Transformer defines `_sp_plan`. Search: `grep -r '_sp_plan' vllm_omni/diffusion/models/` |
|
||||
| **CFG Parallel** | Pipeline inherits `CFGParallelMixin`. Search: `grep -r 'CFGParallelMixin' vllm_omni/diffusion/models/` |
|
||||
| **TP** | Uses `ColumnParallelLinear` / `QKVParallelLinear`. Search: `grep -r 'ParallelLinear\|QKVParallel' vllm_omni/diffusion/models/<model>/` |
|
||||
| **HSDP** | Transformer defines `_hsdp_shard_conditions`. Search: `grep -r '_hsdp_shard_conditions' vllm_omni/diffusion/models/` |
|
||||
|
||||
The canonical per-model support overview/table is in `docs/user_guide/diffusion/parallelism/overview.md`.
|
||||
@@ -1,239 +0,0 @@
|
||||
# Transformer Adaptation Reference
|
||||
|
||||
## Adapting a Diffusers Transformer to vLLM-Omni
|
||||
|
||||
### Step-by-step Checklist
|
||||
|
||||
1. Copy the transformer class from diffusers source
|
||||
2. Remove all mixin classes — inherit only from `nn.Module`
|
||||
3. Replace attention dispatch with `vllm_omni.diffusion.attention.layer.Attention`
|
||||
4. Replace logger with `vllm.logger.init_logger`
|
||||
5. Add `od_config: OmniDiffusionConfig | None = None` to `__init__`
|
||||
6. Remove training-only code (gradient checkpointing, dropout)
|
||||
7. Add `load_weights()` method for weight loading from safetensors
|
||||
8. Add class-level attributes for acceleration features
|
||||
|
||||
### Mixin Removal
|
||||
|
||||
Remove these diffusers mixins (and their imports):
|
||||
|
||||
```python
|
||||
# Remove all of these:
|
||||
from diffusers.models.modeling_utils import ModelMixin
|
||||
from diffusers.configuration_utils import ConfigMixin, register_to_config
|
||||
from diffusers.models.attention_processor import AttentionModuleMixin
|
||||
from diffusers.loaders import PeftAdapterMixin, FromOriginalModelMixin
|
||||
|
||||
# Replace:
|
||||
class MyTransformer(ModelMixin, ConfigMixin, AttentionModuleMixin):
|
||||
# With:
|
||||
class MyTransformer(nn.Module):
|
||||
```
|
||||
|
||||
Also remove `@register_to_config` decorators from `__init__`.
|
||||
|
||||
### Attention Replacement
|
||||
|
||||
The vLLM-Omni `Attention` layer wraps backend selection (FlashAttention, SDPA, SageAttn, etc.) and supports sequence parallelism hooks.
|
||||
|
||||
**QKV tensor shape must be `[batch, seq_len, num_heads, head_dim]`.**
|
||||
|
||||
#### Self-Attention Pattern
|
||||
|
||||
```python
|
||||
from vllm_omni.diffusion.attention.layer import Attention
|
||||
from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata
|
||||
|
||||
class SelfAttentionBlock(nn.Module):
|
||||
def __init__(self, dim, num_heads):
|
||||
super().__init__()
|
||||
self.num_heads = num_heads
|
||||
self.head_dim = dim // num_heads
|
||||
|
||||
self.to_q = nn.Linear(dim, dim)
|
||||
self.to_k = nn.Linear(dim, dim)
|
||||
self.to_v = nn.Linear(dim, dim)
|
||||
self.to_out = nn.Linear(dim, dim)
|
||||
|
||||
self.attn = Attention(
|
||||
num_heads=num_heads,
|
||||
head_size=self.head_dim,
|
||||
softmax_scale=1.0 / (self.head_dim ** 0.5),
|
||||
causal=False,
|
||||
num_kv_heads=num_heads,
|
||||
role="self",
|
||||
)
|
||||
|
||||
def forward(self, x, attn_mask=None):
|
||||
B, S, _ = x.shape
|
||||
q = self.to_q(x).view(B, S, self.num_heads, self.head_dim)
|
||||
k = self.to_k(x).view(B, S, self.num_heads, self.head_dim)
|
||||
v = self.to_v(x).view(B, S, self.num_heads, self.head_dim)
|
||||
|
||||
attn_metadata = AttentionMetadata(attn_mask=attn_mask)
|
||||
out = self.attn(q, k, v, attn_metadata=attn_metadata)
|
||||
out = out.reshape(B, S, -1)
|
||||
return self.to_out(out)
|
||||
```
|
||||
|
||||
**Declaring the attention role.** Always pass a `role` string to `Attention(...)`. It is the lookup key users target with `--diffusion-attention-config.per_role.<role>.backend` to swap kernels per site without touching model code.
|
||||
|
||||
| Convention | When to use |
|
||||
|---|---|
|
||||
| `role="self"` | Q/K/V come from the same hidden state |
|
||||
| `role="cross"` | K/V come from a separate `encoder_hidden_states`; pair with `skip_sequence_parallel=True` if K/V is replicated across SP ranks |
|
||||
|
||||
For multi-modal sites that don't fit `self` / `cross`, use a dot-namespaced role and provide `role_category` so user config can fall back to the generic category:
|
||||
|
||||
```python
|
||||
self.audio_to_video_attn = Attention(
|
||||
num_heads=num_heads, head_size=self.head_dim,
|
||||
softmax_scale=1.0 / (self.head_dim ** 0.5), causal=False,
|
||||
role="mymodel.audio_to_video",
|
||||
role_category="cross",
|
||||
)
|
||||
```
|
||||
|
||||
#### Fused QKV with TP (Advanced)
|
||||
|
||||
For tensor parallelism, use vLLM's parallel linear layers:
|
||||
|
||||
```python
|
||||
from vllm.model_executor.layers.linear import (
|
||||
QKVParallelLinear, RowParallelLinear
|
||||
)
|
||||
|
||||
class TPSelfAttention(nn.Module):
|
||||
def __init__(self, dim, num_heads):
|
||||
super().__init__()
|
||||
self.num_heads = num_heads
|
||||
self.head_dim = dim // num_heads
|
||||
|
||||
self.to_qkv = QKVParallelLinear(
|
||||
hidden_size=dim,
|
||||
head_size=self.head_dim,
|
||||
total_num_heads=num_heads,
|
||||
total_num_kv_heads=num_heads,
|
||||
)
|
||||
self.to_out = RowParallelLinear(dim, dim)
|
||||
|
||||
self.attn = Attention(
|
||||
num_heads=num_heads,
|
||||
head_size=self.head_dim,
|
||||
softmax_scale=1.0 / (self.head_dim ** 0.5),
|
||||
causal=False,
|
||||
num_kv_heads=num_heads,
|
||||
role="self",
|
||||
)
|
||||
```
|
||||
|
||||
### Logger Replacement
|
||||
|
||||
```python
|
||||
# Replace:
|
||||
from diffusers.utils import logging
|
||||
logger = logging.get_logger(__name__)
|
||||
|
||||
# With:
|
||||
from vllm.logger import init_logger
|
||||
logger = init_logger(__name__)
|
||||
```
|
||||
|
||||
### Custom Layers from vLLM-Omni
|
||||
|
||||
Available utility layers:
|
||||
|
||||
```python
|
||||
from vllm.model_executor.layers.layernorm import RMSNorm
|
||||
from vllm_omni.diffusion.layers.rope import RotaryEmbedding
|
||||
from vllm_omni.diffusion.layers.adalayernorm import AdaLayerNorm
|
||||
```
|
||||
|
||||
### Config Support
|
||||
|
||||
```python
|
||||
from vllm_omni.diffusion.data import OmniDiffusionConfig
|
||||
|
||||
class MyTransformer(nn.Module):
|
||||
def __init__(self, *, od_config=None, num_layers=28, hidden_size=3072, **kwargs):
|
||||
super().__init__()
|
||||
self.od_config = od_config
|
||||
self.parallel_config = od_config.parallel_config if od_config else None
|
||||
# ... build layers
|
||||
```
|
||||
|
||||
The transformer config values come from `model_index.json` → `config.json` in the transformer subfolder. The pipeline uses `get_transformer_config_kwargs(od_config.tf_model_config, TransformerClass)` to filter config keys to match the `__init__` signature.
|
||||
|
||||
### Weight Loading
|
||||
|
||||
The `load_weights` method receives an iterable of `(name, tensor)` from safetensors files, with the prefix (e.g., `"transformer."`) already stripped by the loader.
|
||||
|
||||
```python
|
||||
from vllm.model_executor.model_loader.weight_utils import default_weight_loader
|
||||
|
||||
class MyTransformer(nn.Module):
|
||||
def load_weights(self, weights):
|
||||
params = dict(self.named_parameters())
|
||||
loaded = set()
|
||||
for name, tensor in weights:
|
||||
# Optional: remap names from diffusers to vllm-omni naming
|
||||
# e.g., "ff.net.0.proj" -> "ff.net_0.proj"
|
||||
|
||||
if name in params:
|
||||
param = params[name]
|
||||
if hasattr(param, "weight_loader"):
|
||||
param.weight_loader(param, tensor)
|
||||
else:
|
||||
default_weight_loader(param, tensor)
|
||||
loaded.add(name)
|
||||
return loaded
|
||||
```
|
||||
|
||||
#### QKV Fusion in load_weights
|
||||
|
||||
If you fused separate Q/K/V into a `QKVParallelLinear`, you need to map diffusers' separate weight names:
|
||||
|
||||
```python
|
||||
stacked_params_mapping = [
|
||||
("to_qkv", "to_q", "q"),
|
||||
("to_qkv", "to_k", "k"),
|
||||
("to_qkv", "to_v", "v"),
|
||||
]
|
||||
|
||||
def load_weights(self, weights):
|
||||
params = dict(self.named_parameters())
|
||||
loaded = set()
|
||||
for name, tensor in weights:
|
||||
for fused_name, orig_name, shard_id in stacked_params_mapping:
|
||||
if orig_name in name:
|
||||
name = name.replace(orig_name, fused_name)
|
||||
param = params[name]
|
||||
param.weight_loader(param, tensor, shard_id)
|
||||
loaded.add(name)
|
||||
break
|
||||
else:
|
||||
# Normal loading
|
||||
...
|
||||
return loaded
|
||||
```
|
||||
|
||||
### Class-Level Attributes for Features
|
||||
|
||||
|
||||
```python
|
||||
class MyTransformer(nn.Module):
|
||||
# torch.compile: list block class names that repeat and can be compiled
|
||||
_repeated_blocks = ["MyTransformerBlock"]
|
||||
|
||||
# CPU offload: attribute name of the nn.ModuleList containing blocks
|
||||
_layerwise_offload_blocks_attrs = ["blocks"]
|
||||
|
||||
# LoRA: mapping of fused param names to original param names
|
||||
packed_modules_mapping = {"to_qkv": ["to_q", "to_k", "to_v"]}
|
||||
|
||||
# Sequence parallelism plan (advanced — add after basic impl works)
|
||||
_sp_plan = {
|
||||
"blocks.0": SequenceParallelInput(split_dim=1),
|
||||
"proj_out": SequenceParallelOutput(gather_dim=1),
|
||||
}
|
||||
```
|
||||
@@ -1,178 +0,0 @@
|
||||
# Troubleshooting Reference
|
||||
|
||||
## Common Errors When Adding a Diffusion Model
|
||||
|
||||
### ImportError / ModuleNotFoundError
|
||||
|
||||
**Cause**: Missing or incorrect registration.
|
||||
|
||||
**Fix checklist**:
|
||||
1. Model registered in `vllm_omni/diffusion/registry.py` `_DIFFUSION_MODELS` dict
|
||||
2. `__init__.py` exports the pipeline class
|
||||
3. Pipeline file exists at the correct path: `vllm_omni/diffusion/models/{folder}/{file}.py`
|
||||
4. Class name in registry matches the actual class name in the file
|
||||
|
||||
### Shape Mismatch in Attention
|
||||
|
||||
**Symptom**: `RuntimeError: shape mismatch` or `expected 4D tensor`
|
||||
|
||||
**Cause**: QKV tensors not reshaped to `[batch, seq_len, num_heads, head_dim]`.
|
||||
|
||||
**Fix**: Before calling `self.attn(q, k, v, ...)`, ensure:
|
||||
```python
|
||||
q = q.view(batch, seq_len, self.num_heads, self.head_dim)
|
||||
k = k.view(batch, kv_seq_len, self.num_kv_heads, self.head_dim)
|
||||
v = v.view(batch, kv_seq_len, self.num_kv_heads, self.head_dim)
|
||||
```
|
||||
|
||||
After attention, reshape back:
|
||||
```python
|
||||
out = out.reshape(batch, seq_len, -1)
|
||||
```
|
||||
|
||||
### Weight Loading Failures
|
||||
|
||||
**Symptom**: `RuntimeError: size mismatch for parameter ...` or missing keys
|
||||
|
||||
**Debugging**:
|
||||
1. Print diffusers weight names: `safetensors.safe_open(path, "pt").keys()`
|
||||
2. Print model parameter names: `dict(model.named_parameters()).keys()`
|
||||
3. Compare and add name remappings in `load_weights()`
|
||||
|
||||
**Common remappings needed**:
|
||||
- `ff.net.0.proj` → `ff.net_0.proj` (PyTorch Sequential indexing)
|
||||
- `.to_out.0.` → `.to_out.` (Sequential unwrapping)
|
||||
- `scale_shift_table` → moved to a wrapper module
|
||||
|
||||
### Black/Blank/Noisy Output
|
||||
|
||||
**Possible causes**:
|
||||
1. **Wrong latent normalization**: Check VAE expects latents scaled by `vae.config.scaling_factor`
|
||||
2. **Wrong scheduler**: Using the wrong scheduler class or wrong `flow_shift`
|
||||
3. **Missing CFG**: Some models require `guidance_scale > 1.0` with negative prompt
|
||||
4. **Wrong timestep format**: Some schedulers expect float, others expect int/long
|
||||
5. **Missing post-processing**: Raw VAE output may need denormalization
|
||||
|
||||
**Quick test**: Run with diffusers directly using the same seed and compare latents at each step.
|
||||
|
||||
### OOM (Out of Memory)
|
||||
|
||||
**Solutions** (in order of preference):
|
||||
1. `--enforce-eager` to disable torch.compile (saves compile memory)
|
||||
2. `--enable-cpu-offload` for model-level offload
|
||||
3. `--enable-layerwise-offload` for block-level offload (better for large models)
|
||||
4. `--vae-use-slicing --vae-use-tiling` for VAE memory reduction
|
||||
5. Reduce resolution: `--height 480 --width 832`
|
||||
6. Use TP: `--tensor-parallel-size 2`
|
||||
|
||||
### Different Output vs Diffusers Reference
|
||||
|
||||
**Common causes**:
|
||||
1. **Attention backend difference**: FlashAttention vs SDPA may produce slightly different results. Force SDPA via `--diffusion-attention-backend TORCH_SDPA` (or `DIFFUSION_ATTENTION_BACKEND=TORCH_SDPA`) to match diffusers; for finer-grained per-role overrides see `--diffusion-attention-config.per_role.<role>.backend`
|
||||
2. **Float precision**: vLLM-Omni may use bfloat16 where diffusers uses float32 for some operations
|
||||
3. **Missing normalization**: Check all LayerNorm/RMSNorm are preserved
|
||||
4. **Scheduler rounding**: Some schedulers have numerical sensitivity
|
||||
|
||||
### Tensor Parallel Errors
|
||||
|
||||
**Symptom**: `AssertionError: not divisible` or incorrect output with TP>1
|
||||
|
||||
**Fix**:
|
||||
1. Verify `num_heads % tp_size == 0` and `num_kv_heads % tp_size == 0`
|
||||
2. Ensure `ColumnParallelLinear` / `RowParallelLinear` are used correctly
|
||||
3. Check that norms between parallel layers use distributed norm if needed
|
||||
4. Verify `load_weights` handles TP sharding for norm weights
|
||||
5. Use `self.to_qkv.num_heads` (local heads per GPU) for QKV split sizes, not total heads
|
||||
|
||||
**Missing `input_is_parallel=True`**:
|
||||
|
||||
`RowParallelLinear` expects sharded input from `ColumnParallelLinear`:
|
||||
```python
|
||||
self.w1 = ColumnParallelLinear(dim, hidden_dim, return_bias=False)
|
||||
self.w2 = RowParallelLinear(hidden_dim, dim, input_is_parallel=True, return_bias=False)
|
||||
```
|
||||
|
||||
### Sequence Parallel Errors
|
||||
|
||||
**Symptom**: Incorrect output or crashes with `--ulysses-degree N` or `--usp N`
|
||||
|
||||
**Possible causes**:
|
||||
1. **Inline operations between shard/gather points**: `torch.cat()`, `pad_sequence()` etc. not at `nn.Module` boundaries. Fix: extract into submodule.
|
||||
2. **Wrong `split_dim`**: Check the tensor shape at the shard point. Sequence dimension is typically `dim=1` for `[B, S, D]` tensors.
|
||||
3. **RoPE not sharded**: If RoPE is computed separately, add it to `_sp_plan` with `split_output=True`.
|
||||
4. **Sequence not divisible by SP degree**: Use `auto_pad=True` in `SequenceParallelInput` or switch to `ulysses_mode="advanced_uaa"`.
|
||||
|
||||
**Debugging**: Add `expected_dims=N` to `SequenceParallelInput`/`Output` for shape validation at runtime.
|
||||
|
||||
### CFG Parallel Errors
|
||||
|
||||
**Symptom**: CFG parallel not activating, no speedup
|
||||
|
||||
**Fix checklist**:
|
||||
1. Pipeline inherits `CFGParallelMixin`
|
||||
2. `guidance_scale > 1.0`
|
||||
3. Negative prompt provided (even if empty string)
|
||||
4. `--cfg-parallel-size 2` specified
|
||||
5. `diffuse()` method calls `predict_noise_maybe_with_cfg()` and `scheduler_step_maybe_with_cfg()`
|
||||
|
||||
**Symptom**: Different output with CFG parallel vs sequential
|
||||
|
||||
**Possible cause**: Non-deterministic scheduler. Fix: pass `generator=torch.Generator(device).manual_seed(seed)` to `scheduler_step_maybe_with_cfg()`.
|
||||
|
||||
### HSDP Errors
|
||||
|
||||
**Symptom**: HSDP not activating or errors during weight loading
|
||||
|
||||
**Fix checklist**:
|
||||
1. Transformer defines `_hsdp_shard_conditions` class attribute
|
||||
2. Shard condition functions return `True` for correct modules (test with `model.named_modules()`)
|
||||
3. Not combining with TP (HSDP and TP are incompatible)
|
||||
4. For standalone HSDP, `hsdp_shard_size` is specified explicitly
|
||||
|
||||
**Verify**: Check logs for "HSDP Inference: replicate_size=..., shard_size=..." and "Sharded N modules + root".
|
||||
|
||||
### Cache-DiT Not Applied
|
||||
|
||||
**Symptom**: No speedup, no cache-related log messages
|
||||
|
||||
**Fix checklist**:
|
||||
1. Model not in `_NO_CACHE_ACCELERATION` in `registry.py`
|
||||
2. Pipeline class name matches `CUSTOM_DIT_ENABLERS` key (if using custom enabler)
|
||||
3. `cache_backend="cache_dit"` specified
|
||||
4. Check logs for "Cache-dit enabled successfully on xxx"
|
||||
|
||||
**Verify pipeline name**: `print(pipeline.__class__.__name__)` — must match registry key.
|
||||
|
||||
### Cache-DiT Quality Degradation
|
||||
|
||||
**Symptom**: Artifacts or lower quality with cache-dit
|
||||
|
||||
**Fix**: Reduce aggressiveness:
|
||||
```python
|
||||
cache_config={
|
||||
"residual_diff_threshold": 0.12, # Lower from 0.24
|
||||
"max_warmup_steps": 6, # Increase from 4
|
||||
"max_continuous_cached_steps": 2, # Reduce if higher
|
||||
}
|
||||
```
|
||||
|
||||
If quality is still poor, the model may need a custom enabler with per-block-list `ParamsModifier` tuning.
|
||||
|
||||
### Model Not Detected / Wrong Pipeline Class
|
||||
|
||||
**Symptom**: `ValueError: Model class ... not found in diffusion model registry`
|
||||
|
||||
**Cause**: The model's `model_index.json` has a `_class_name` for the pipeline that doesn't match registry keys.
|
||||
|
||||
**Fix**: The registry key must match the diffusers pipeline class name from `model_index.json`. If using a different name, map it in the registry:
|
||||
```python
|
||||
"DiffusersPipelineClassName": ("your_folder", "your_file", "YourVllmClassName"),
|
||||
```
|
||||
|
||||
## Debugging Workflow
|
||||
|
||||
1. **Add verbose logging**: Use `logger.info()` to print tensor shapes at each stage
|
||||
2. **Compare step-by-step**: Run diffusers and vllm-omni side by side, comparing tensors after each major operation
|
||||
3. **Use small configs**: Reduce `num_inference_steps=2`, small resolution for fast iteration
|
||||
4. **Test transformer isolation**: Feed the same input to both diffusers and vllm-omni transformers, compare outputs
|
||||
5. **Binary search for bugs**: Comment out blocks/layers to isolate where divergence starts
|
||||
@@ -1,126 +0,0 @@
|
||||
---
|
||||
name: add-recipe
|
||||
description: Add or update an in-repository vLLM-Omni model recipe with verified task, input, output, hardware, command, feature, and validation contracts. Use when creating files under recipes/, restructuring a recipe after review, documenting a newly supported model, or synchronizing recipe claims with support tables and shared feature guides.
|
||||
---
|
||||
|
||||
# Add a vLLM-Omni Recipe
|
||||
|
||||
## Source the contract
|
||||
|
||||
Read [`recipes/TEMPLATE.md`](../../../recipes/TEMPLATE.md) and
|
||||
[`recipes/README.md`](../../../recipes/README.md) before editing. Inspect the
|
||||
canonical model card/repository, implementation, shared examples, tests, and
|
||||
available local qualification evidence.
|
||||
|
||||
Do not infer support from a model name, sibling recipe, registry entry, or
|
||||
topology validator. Mark unexecuted configurations as configuration-only.
|
||||
|
||||
## Route by model family
|
||||
|
||||
Read only the contributor and user documentation relevant to the model:
|
||||
|
||||
| Family | Contributor guide | User-facing documentation to synchronize |
|
||||
|---|---|---|
|
||||
| Diffusion | [`adding a diffusion model`](../add-diffusion-model/SKILL.md) | `docs/user_guide/diffusion_features.md`, applicable diffusion feature guides, and shared image/video/audio examples |
|
||||
| TTS | [`adding a TTS model`](../add-tts-model/SKILL.md) | `examples/offline_inference/text_to_speech/README.md`, `examples/online_serving/text_to_speech/README.md`, and `docs/serving/speech_api.md` |
|
||||
| Omni | [`adding an omni model`](../../../docs/contributing/model/adding_omni_model.md) | model-family offline/online example docs, `docs/serving/chat_completions_api.md`, and `docs/user_guide/feature_compatibility.md` |
|
||||
|
||||
All families must update `docs/models/supported_models.md` and the matching row
|
||||
in `recipes/README.md`. Do not add a modality-specific support document when
|
||||
the repository has no such table; update the closest shared user contract.
|
||||
|
||||
## Structure the recipe
|
||||
|
||||
Keep the recipe task-oriented and use one hardware-specific file named
|
||||
`recipes/<vendor>/<model>-<hardware>.md`. A recipe may contain multiple
|
||||
deployment topologies on that hardware, but each additional accelerator model
|
||||
or platform requires its own suffixed file.
|
||||
|
||||
1. **Summary:** vendor, exact model ID, runtime, modes, named hardware,
|
||||
recommended deployment, and maintainer.
|
||||
2. **Supported model contract:** put task, input, output, and provided-profile
|
||||
tables before setup or commands.
|
||||
3. **References:** link the canonical model source, shared examples, supported
|
||||
model table, and feature matrix.
|
||||
4. **Checkpoint/setup:** pin revisions when possible and state required assets.
|
||||
5. **Hardware:** document the single accelerator named by the file suffix,
|
||||
including every validated device-count/topology profile on it.
|
||||
6. **Commands:** reuse shared `examples/` entrypoints unless the model contract
|
||||
truly requires a dedicated script. Include offline and online paths that
|
||||
were validated.
|
||||
7. **Supported features:** use a compact model-specific topology/status table
|
||||
with links to shared feature guides.
|
||||
8. **Verification:** provide a quick command and exact expected output
|
||||
contract.
|
||||
9. **Qualification evidence:** report correctness, memory, and timing evidence
|
||||
with its measurement scope and caveats.
|
||||
|
||||
## Separate hardware from software
|
||||
|
||||
For every locally validated hardware profile, record:
|
||||
|
||||
- accelerator vendor/model and per-device memory;
|
||||
- number of devices;
|
||||
- interconnect (`NVLink`, `PCIe`, or the platform equivalent);
|
||||
- host memory when CPU staging/offload is material;
|
||||
- whether the profile is runtime-qualified or configuration-only.
|
||||
|
||||
Record OS, Python, driver/runtime, framework versions, and the vLLM-Omni
|
||||
revision in a separate software-environment table. Keep precision, worker
|
||||
count, and DP/TP/SP/PP sizes with the deployment profile or exact command.
|
||||
|
||||
Never generalize a result from one accelerator family to another or combine
|
||||
NVIDIA, AMD, NPU, or distinct accelerator models in one recipe. Create a
|
||||
separate hardware-suffixed recipe and `recipes/README.md` row. Distinguish
|
||||
upstream requirements from hardware exercised by the PR.
|
||||
|
||||
## Keep shared documentation concise
|
||||
|
||||
Link the applicable diffusion, TTS, omni, serving, or design guide for generic
|
||||
feature semantics and launch instructions. The recipe should contain only the
|
||||
model-specific status, valid topology, required flag difference, and evidence
|
||||
boundary.
|
||||
|
||||
Put unsupported combinations in the same feature table. Avoid repeating the
|
||||
same model-specific prose below the global feature matrix; the recipe is the
|
||||
detailed source of truth.
|
||||
|
||||
## Report evidence precisely
|
||||
|
||||
For every performance or memory value, state:
|
||||
|
||||
- checkpoint/revision and workload;
|
||||
- accelerator and device count;
|
||||
- precision and topology;
|
||||
- cold/warm scope, step count, and concurrency;
|
||||
- peak HBM and host-memory metric when measured;
|
||||
- output/quality guard and sample count;
|
||||
- monitoring overhead or other known variance.
|
||||
|
||||
Call a single bounded run qualification evidence, not a benchmark. Do not add
|
||||
a benchmark script unless the contribution explicitly requires one.
|
||||
|
||||
## Synchronize repository documentation
|
||||
|
||||
Update all applicable locations:
|
||||
|
||||
- the row in `recipes/README.md`;
|
||||
- `docs/models/supported_models.md`;
|
||||
- the family-specific documentation selected in **Route by model family**;
|
||||
- shared example documentation when commands or flags change.
|
||||
|
||||
Keep global support tables compact. Link to the recipe instead of duplicating
|
||||
its deployment explanation.
|
||||
|
||||
## Validate
|
||||
|
||||
Run, at minimum:
|
||||
|
||||
```bash
|
||||
pre-commit run --files <changed recipe/docs/skill files>
|
||||
mkdocs build --strict
|
||||
git diff --check
|
||||
```
|
||||
|
||||
Remove generated documentation artifacts after validation. Confirm every
|
||||
recipe link resolves and every claimed profile has matching code or evidence.
|
||||
@@ -1,592 +0,0 @@
|
||||
---
|
||||
name: add-tts-model
|
||||
description: "Integrate a new text-to-speech model into vLLM-Omni from HuggingFace reference implementation through production-ready serving with streaming and CUDA graph acceleration. Use when adding a new TTS model, wiring stage separation for speech synthesis, enabling online voice generation serving, debugging TTS integration behavior, or building audio output pipelines."
|
||||
---
|
||||
|
||||
# TTS Model Integration Workflow
|
||||
|
||||
## Overview
|
||||
|
||||
```
|
||||
HF Reference -> Stage Separation -> Online Serving -> Async Chunk -> CUDA Graph -> Pre-commit/DCO
|
||||
(Phase 1) (Phase 2) (Phase 3) (Phase 4) (Phase 5) (Phase 6)
|
||||
```
|
||||
|
||||
Three architecture patterns are supported:
|
||||
|
||||
- **Two-stage pipeline** (e.g. Qwen3-TTS, Fish Speech, CosyVoice3): AR
|
||||
code-predictor → audio decoder, connected via async_chunk for low-latency
|
||||
streaming. Use this for maximum performance.
|
||||
- **Single-stage AR via generator** (e.g. MOSS-TTS-Nano): entire model runs
|
||||
inside one AR worker, streaming audio chunks through a per-request
|
||||
`inference_stream()` generator. Use this when the upstream model bundles AR
|
||||
+ codec inseparably. See [references/single-stage-ar.md](references/single-stage-ar.md).
|
||||
- **Single-stage, vLLM-native base LM + side computation** (e.g. VoxCPM2):
|
||||
the base language model runs under vLLM's PagedAttention as a normal AR
|
||||
model; diffusion / VAE / side computations run outside vLLM and are
|
||||
attached via the runner post-processing hook. This is a distinct pattern
|
||||
from the generator approach above — do not confuse the two.
|
||||
|
||||
The single-stage variants skip Phase 4 (async_chunk) but Phase 5 (CUDA graph)
|
||||
is still encouraged for the inner AR loop.
|
||||
|
||||
## Cross-Cutting Invariants
|
||||
|
||||
These rules apply to every TTS model regardless of architecture (AR vs AR+diffusion, single-stage vs two-stage, codec-based vs VAE-based). They surface repeatedly across PRs — check them at the end of every phase.
|
||||
|
||||
### I1. Streaming output contract
|
||||
|
||||
Pick exactly one per-step semantics for `forward()` and document it in the docstring:
|
||||
|
||||
- **Delta**: yield only new audio samples produced this step. Preferred — linear cost, low memory.
|
||||
- **Cumulative**: re-decode from step 0 every call. O(N²); only acceptable if the codec has no streaming decode path.
|
||||
|
||||
If you choose **delta**, verify the full emit→consolidate→consume chain:
|
||||
|
||||
1. `forward()` returns `{"model_outputs": <new_chunk_only>, ...}`
|
||||
2. `_consolidate_multimodal_tensors()` in `vllm_omni/engine/output_processor.py` concatenates the audio key into one tensor at finish. If it skips the key (`continue`), offline consumers receive only the final chunk. See `output_processor.py` for the concrete list of handled modality keys.
|
||||
3. Streaming consumers (SSE, Gradio) receive per-step deltas; offline consumers (`engine.generate()`) receive a single concatenated tensor.
|
||||
|
||||
Cumulative-vs-delta mismatch is the most common silent bug — offline RTF benchmarks pass, but users hear replays or truncation.
|
||||
|
||||
### I2. Multimodal output consumer hygiene
|
||||
|
||||
`outputs[0].outputs[0].multimodal_output[<key>]` can be any of `Tensor`, `list[Tensor]` (pre-consolidation snapshot), `np.ndarray`, or scalar. When writing tests, examples, and benchmarks:
|
||||
|
||||
- **Never** use `dict.get("a") or dict.get("b")` on tensor values — Python evaluates the tensor's boolean, raising `RuntimeError: Boolean value of Tensor with more than one value is ambiguous`. Use explicit `if x is None` chains.
|
||||
- Always defensively handle the list form: `if isinstance(x, list): x = torch.cat([t.reshape(-1) for t in x], dim=0)`.
|
||||
- Assert `shape` / `dtype` / `duration` explicitly; do not rely on truthiness for presence checks.
|
||||
|
||||
### I3. Hot-loop GPU discipline
|
||||
|
||||
Inside any per-step model loop (AR decode, diffusion solver, CFM Euler, vocoder block loop):
|
||||
|
||||
- No `tensor.item()`, `.cpu()`, or `.tolist()` — each triggers a GPU→CPU sync; at 10 steps × 60 frames × 4 ops that is 2400 syncs per request.
|
||||
- Prefer `dst.copy_(src)` over `dst.fill_(src.item())` when writing a scalar tensor into a buffer.
|
||||
- Prefer `torch.compile(Model.forward, fullgraph=False)` on the whole forward over per-submodule compile — fewer dispatch boundaries, larger fusion regions. Measure before choosing granularity.
|
||||
- No Python-side control flow that depends on tensor values; use `torch.where` / masking instead.
|
||||
|
||||
Profile first, optimize second. See the profiling docs / project memory for the trace-analysis workflow.
|
||||
|
||||
### I4. Validation pyramid
|
||||
|
||||
Offline RTF alone is necessary but not sufficient. Every new TTS model must pass all three:
|
||||
|
||||
| Layer | Catches | Tool |
|
||||
|-------|---------|------|
|
||||
| Offline RTF / duration check | Throughput regressions, missing audio, wrong sample rate | `end2end.py`, pytest e2e |
|
||||
| Browser streaming playback | Delta/cumulative bugs, chunk boundary glitches, TTFP regressions | Gradio demo over `/v1/audio/speech?stream=true` |
|
||||
| Concurrent requests | Per-request state leaks, codec window round-robin gaps | `max_num_seqs>1` smoke test with 4+ parallel prompts |
|
||||
|
||||
Declaring a model "done" without all three has shipped regressions more than once.
|
||||
|
||||
### I5. Per-request state is owned by the request, not the model
|
||||
|
||||
If the model caches *anything* across `forward()` calls (streaming generators, codec buffers, sliding-window pads, CUDA graph state), key it by request ID:
|
||||
|
||||
```python
|
||||
self._state: dict[str, YourState] = {} # request_key → state
|
||||
# fetch: request_key = str(info.get("_omni_req_id", "0"))
|
||||
# free on finish: del self._state[request_key]
|
||||
```
|
||||
|
||||
A shared buffer silently corrupts audio across concurrent requests — the symptom is crosstalk or truncation only under load.
|
||||
|
||||
## Phase 1: HuggingFace Reference
|
||||
|
||||
**Goal**: Understand the reference implementation and verify it produces correct audio.
|
||||
|
||||
### Steps
|
||||
|
||||
1. **Run the reference model** end-to-end using the official HuggingFace / GitHub code
|
||||
2. **Document the architecture**:
|
||||
- What are the sub-models? (AR decoder, codec decoder, vocoder, etc.)
|
||||
- What is the token vocabulary? (semantic codes, RVQ codebooks, special tokens)
|
||||
- What is the output format? (sample rate, channels, codec type)
|
||||
3. **Capture reference outputs** for comparison during integration
|
||||
4. **Identify the config structure**: `config.json` fields, `model_type`, sub-model configs
|
||||
|
||||
### Key Questions
|
||||
|
||||
- How many codebooks? What are the codebook sizes?
|
||||
- What special tokens exist? (`<|voice|>`, `<|audio_start|>`, `<|im_end|>`, etc.)
|
||||
- What is the token-to-ID mapping for codec codes?
|
||||
- What is the hop length / frame rate of the codec?
|
||||
- Does the model support voice cloning? How? (reference audio encoding, speaker embeddings, etc.)
|
||||
|
||||
### Deliverables
|
||||
|
||||
- Working reference script that produces audio
|
||||
- Architecture diagram / notes
|
||||
- Token vocabulary mapping
|
||||
- Reference audio samples for regression testing
|
||||
|
||||
## Phase 2: Stage Separation (Offline Inference)
|
||||
|
||||
**Goal**: Split the model into vLLM-Omni stages and get offline inference working.
|
||||
|
||||
### Steps
|
||||
|
||||
1. **Register the model** in `vllm_omni/model_executor/models/registry.py`
|
||||
2. **Create config classes** (`configuration_<model>.py`) with `model_type` registration
|
||||
3. **Implement Stage 0** (AR model):
|
||||
- Subclass appropriate base (e.g., wrap Qwen3 decoder layers)
|
||||
- Implement `forward()` for autoregressive token generation
|
||||
- Handle special token logic (start/stop tokens, codec token mapping)
|
||||
- If dual-AR (like Fish Speech), implement Fast AR as a nested module
|
||||
4. **Implement Stage 1** (Decoder):
|
||||
- Load codec weights (may need lazy loading from separate checkpoint)
|
||||
- Implement `forward()`: codec codes -> audio waveform
|
||||
- Return `OmniOutput` with `multimodal_outputs`
|
||||
5. **Define and register the pipeline topology** in
|
||||
`vllm_omni/model_executor/models/<model>/pipeline.py`
|
||||
6. **Create the deploy YAML** under `vllm_omni/deploy/` for placement,
|
||||
memory sizing, connectors, and runtime overrides
|
||||
7. **Create stage input processor** for prompt building and inter-stage handoff
|
||||
8. **Write end2end.py** test script
|
||||
|
||||
### Critical Parameters to Get Right
|
||||
|
||||
| Parameter | Impact if Wrong |
|
||||
|-----------|----------------|
|
||||
| Hop length | Audio duration wrong, streaming noise |
|
||||
| Token ID mapping | Garbage codes -> noise output |
|
||||
| Codebook count/size | Shape mismatch crashes |
|
||||
| Stop token | Generation never stops or stops too early |
|
||||
| dtype / autocast | Numerical issues, silent quality degradation |
|
||||
| Repetition penalty | Must match reference (often 1.0 for TTS) |
|
||||
|
||||
### Debugging Priority (from experience)
|
||||
|
||||
When audio output is wrong, check in this order:
|
||||
|
||||
1. **RoPE / attention**: Are position encodings correct? Is the attention mask right?
|
||||
2. **Normalization**: RMSNorm epsilon, layer norm placement (pre vs post)
|
||||
3. **Hop length**: Product of all upsample rates in the codec decoder
|
||||
4. **Token mapping**: Are codec IDs correctly offset from the vocabulary base?
|
||||
5. **Sampling parameters**: Temperature, top_k, top_p, repetition_penalty
|
||||
6. **Tensor layout**: Codebook-major vs frame-major ordering
|
||||
7. **dtype**: Float32 for codec decoders (autocast can corrupt audio)
|
||||
|
||||
### Streaming Correctness Rules (single-stage and two-stage)
|
||||
|
||||
These bugs appear in almost every new TTS PR. Check all before the first push. See also the cross-cutting invariants I1 (output contract) and I5 (per-request state) above — the rules below are the Phase 2-specific instances of those invariants:
|
||||
|
||||
- **Accumulate codes across AR steps** — each `forward()` appends new codes; do not reset between steps or audio will be truncated (fish speech: `fix: accumulate audio_codes across steps`)
|
||||
- **Emit delta audio, not full waveform** — in streaming mode yield only the new chunk per step, not the re-decoded full waveform from step 0 (fish speech: `fix: emit delta audio not full waveform`)
|
||||
- **All return paths must emit `model_outputs`** — if any early-return branch skips setting `model_outputs`, the serving layer silently drops that step's audio (fish speech: `fix: ensure ALL return paths emit model_outputs`)
|
||||
- **Per-request state isolation** — for batched concurrent requests, key all state by request ID; a shared buffer corrupts audio across requests (fish speech: `fix: per-request vocode + delta emission`)
|
||||
- **Codec tensor device** — move codec codes to the codec decoder's device before calling decode; mismatches cause silent CPU fallback or crashes (fish speech: `fix: use model device for CUDA stream`)
|
||||
- **AR stage `max_num_seqs`** — set to at least 4 in production deploy configs; for single-stage models this is the only stage. For two-stage models, Stage 0 (AR) needs `max_num_seqs ≥ 4` to pipeline concurrent requests; Stage 1 (codec decoder) is model-specific and may intentionally use `max_num_seqs: 1`. Defaulting the AR stage to 1 causes audio gaps under concurrency because the codec window round-robins across requests (RFC #2568)
|
||||
|
||||
### Optional Dependency Handling
|
||||
|
||||
Patch optional dependencies (`torchaudio` / `torchcodec` / `soundfile`) at
|
||||
the top of `load_weights()`, not at module import. Failures to do so cause
|
||||
cryptic errors only on environments missing the optional package — after
|
||||
the model is already deployed. See
|
||||
[references/optional-deps.md](references/optional-deps.md) for the full
|
||||
pattern, signature constraints, and MOSS-TTS-Nano reference.
|
||||
|
||||
### Single-Stage AR Pattern (alternative to two-stage)
|
||||
|
||||
When the upstream model cannot be cleanly split into an AR stage and a
|
||||
separate decoder, run the full pipeline inside a single AR worker and
|
||||
stream audio through a per-request `inference_stream()` generator keyed by
|
||||
`_omni_req_id`. Define one `StagePipelineConfig` with
|
||||
`execution_type=StageExecutionType.LLM_AR`, `engine_output_type="audio"`,
|
||||
`final_output=True`, and `owns_tokenizer=True`. Set `async_chunk: false` in
|
||||
the deploy YAML. Only extract params from
|
||||
`additional_information` that you actually forward, or pre-commit fails
|
||||
`ruff F841`.
|
||||
|
||||
Full walkthrough with the complete `forward()` / `_create_stream_gen()`
|
||||
skeleton plus pipeline/deploy definitions:
|
||||
[references/single-stage-ar.md](references/single-stage-ar.md). For an
|
||||
in-tree reference, look for any single-stage AR model under
|
||||
`vllm_omni/model_executor/models/`, such as MOSS-TTS-Nano.
|
||||
|
||||
**VoxCPM2 is a different pattern** and should not reuse this skeleton — it
|
||||
runs the base LM under vLLM PagedAttention with external side-computation.
|
||||
See `plan/voxcpm2_native_ar_design.md`.
|
||||
|
||||
### Deliverables
|
||||
|
||||
- Model files in `vllm_omni/model_executor/models/<model_name>/`
|
||||
- Registered `pipeline.py` topology
|
||||
- Deploy YAML under `vllm_omni/deploy/`
|
||||
- Working `end2end.py` at `examples/offline_inference/text_to_speech/<model>/end2end.py`
|
||||
- New section in `examples/offline_inference/text_to_speech/README.md` (table row + per-model section). Do **not** create a top-level `examples/offline_inference/<model>/` dir or a per-model `README.md` inside `text_to_speech/<model>/` — the hub README is the documented surface and the mkdocs `generate_examples` hook only descends one level into `examples/<category>/`.
|
||||
|
||||
## Phase 3: Online Serving
|
||||
|
||||
**Goal**: Expose the model via `/v1/audio/speech` API endpoint.
|
||||
|
||||
### Steps
|
||||
|
||||
1. **Write one adapter** under `vllm_omni/entrypoints/openai/tts_adapters/`.
|
||||
`serving_speech.py` should not need an edit — detection, stage discovery and
|
||||
dispatch are all derived from what the adapter declares.
|
||||
|
||||
Create `vllm_omni/entrypoints/openai/tts_adapters/your_model.py`:
|
||||
```python
|
||||
@register_tts_adapter
|
||||
class YourModelAdapter(ARTTSAdapter):
|
||||
name = "your_model" # registry key + log label
|
||||
stage_keys = frozenset({"your_stage_key"}) # the deploy yaml's model_stage
|
||||
|
||||
def validate(self, request) -> str | None:
|
||||
if not request.input or not request.input.strip():
|
||||
return "Input text cannot be empty"
|
||||
return None
|
||||
|
||||
async def build(self, request, sampling_params_list, has_inline_ref_audio):
|
||||
params = {"text": [request.input]}
|
||||
if request.voice is not None:
|
||||
params["voice"] = [request.voice]
|
||||
return PreparedRequest(
|
||||
prompt={"prompt": request.input},
|
||||
tts_params=params,
|
||||
model_type=self.name,
|
||||
)
|
||||
```
|
||||
|
||||
Then add the module to the import block at the bottom of
|
||||
`tts_adapters/__init__.py` so it registers. That import line is the only shared
|
||||
file a new model touches — which is also why the old rebase-conflict hotspot is
|
||||
gone.
|
||||
|
||||
> **Pure-diffusion TTS does not go through adapters yet.** Under
|
||||
> `for_diffusion()`, `create_speech()` routes straight to
|
||||
> `_create_diffusion_speech()` and never calls `validate()`/`build()`.
|
||||
> `DiffusionTTSAdapter` is scaffolding with no production subclass, so logic
|
||||
> placed in one would silently never run. Diffusion-engine models follow the
|
||||
> existing diffusion path; wiring it through adapters is open work (#4855).
|
||||
|
||||
**If a stage key alone cannot identify the model**, declare
|
||||
`model_archs = frozenset({"YourModelForConditionalGeneration"})`; add
|
||||
`arch_identifies_entry_stage = True` when the model owns no stage key at all
|
||||
(Ming dense). For a rule that is not set membership, override `matches()` (see
|
||||
`covo_audio.py`). For a genuine overlap with another adapter, give one an
|
||||
explicit `detect_priority` — `test_tts_detection.py` fails on an unordered
|
||||
overlap. Models that only serve speech in some topologies override
|
||||
`stage_serves_speech()` (see `audex.py`).
|
||||
|
||||
**Reuse shared helpers via `self.ctx.server`** rather than reimplementing them:
|
||||
`_resolve_ref_audio`, `_apply_uploaded_speaker`, `_validate_ref_audio_format`,
|
||||
`_max_instructions_length`. Read a comparable adapter first — `fish_speech.py`
|
||||
(voice cloning), `higgs_audio_v3.py` (parameter-heavy), `moss_tts.py` (family
|
||||
sharing a base class).
|
||||
|
||||
> **Do not add `self._tts_model_type == ...` branches to `serving_speech.py`.**
|
||||
> Older models predate the adapter framework and still have them; they are being
|
||||
> migrated out (RFC #4327, #4855). `tools/pre_commit/check_tts_adapter.py` is a
|
||||
> ratchet on the remaining count and fails the commit if it grows. Behaviour that
|
||||
> no adapter hook can express is a missing hook — propose it on the RFC.
|
||||
|
||||
> **Unused variable rule**: only extract fields in `build()` that are actually
|
||||
> forwarded to the model. Unused extractions fail `ruff F841`. For voice-cloning
|
||||
> fields (`ref_audio` -> `prompt_audio_path`, `ref_text` -> `prompt_text`), add
|
||||
> them to the params and verify they reach the model call.
|
||||
|
||||
2. **Handle model-specific parameters**:
|
||||
- Voice cloning: `ref_audio` encoding and prompt injection
|
||||
- `max_new_tokens` override in sampling params
|
||||
- Model-specific default values
|
||||
3. **Create client scripts**: `speech_client.py`, `run_server.sh`
|
||||
4. **Test all response formats**: wav, mp3, flac, pcm
|
||||
5. **Add Gradio demo**: Interactive web UI with streaming support
|
||||
|
||||
### Voice Cloning Pattern
|
||||
|
||||
```python
|
||||
import base64
|
||||
from pathlib import Path
|
||||
|
||||
def build_voice_clone_prompt(ref_audio_path: str, text: str, codec) -> list:
|
||||
"""Build prompt with reference audio for voice cloning, called from the adapter."""
|
||||
audio_bytes = Path(ref_audio_path).read_bytes()
|
||||
codes = codec.encode(audio_bytes) # Encode on CPU using model's codec (e.g., DAC)
|
||||
token_ids = [code + codec.vocab_offset for code in codes.flatten().tolist()]
|
||||
return [
|
||||
{"role": "system", "content": f"<|voice|>{''.join(chr(t) for t in token_ids)}"},
|
||||
{"role": "user", "content": text},
|
||||
]
|
||||
```
|
||||
|
||||
### Test Case Writing (CI Levels)
|
||||
|
||||
**Follow the [vllm-omni-test skill](../vllm-omni-test/SKILL.md)** for markers, file naming (`test_{slug}.py` / `test_{slug}_expansion.py`), Buildkite wiring, and copy-paste run commands. Also read [test_system_overview.md](https://github.com/vllm-project/vllm-omni/blob/main/docs/contributing/ci/test_system_overview.md) and [test_writing_guide.md](https://github.com/vllm-project/vllm-omni/blob/main/docs/contributing/ci/test_writing_guide.md).
|
||||
|
||||
Classify the model's **CI priority** first (high / medium / low). High-priority TTS models are typically those on the integration hot path or listed in tracking issues such as [#1832](https://github.com/vllm-project/vllm-omni/issues/1832); medium and low tiers cover the long tail. When unsure, ask the reviewer which tier applies.
|
||||
|
||||
| Priority | Required test levels | Files & markers |
|
||||
|----------|---------------------|-----------------|
|
||||
| **High** | **L1** unit/logic · **L2** online smoke · **L3** online + offline integration · **L4** feature + performance | See table below |
|
||||
| **Medium** | **L3** online + offline · **L4** feature only | Skip dedicated L1/L2 unless fixing a logic bug |
|
||||
| **Low** | **L4** feature only | One or two `*_expansion.py` parametrized cases |
|
||||
|
||||
**Per-level deliverables (TTS / `pytest.mark.tts`):**
|
||||
|
||||
| Level | Location | Marker | CI pipeline | Notes |
|
||||
|-------|----------|--------|-------------|-------|
|
||||
| **L1** | `tests/model_executor/…`, `tests/entrypoints/openai_api/…`, stage-processor tests | `core_model` + `cpu` | `test-ready.yml` | Prompt assembly, async_chunk helpers, adapter validation — no GPU |
|
||||
| **L2** | `tests/e2e/online_serving/test_{slug}.py` | **`core_model` + `advanced_model`** (both on baseline smoke) + `tts` + `@hardware_test(...)` | `test-ready.yml` (`ready` label) | Default deploy smoke: single `/v1/audio/speech` or offline `OmniRunner` path |
|
||||
| **L3** | `tests/e2e/online_serving/test_{slug}.py` **and** `tests/e2e/offline_inference/test_{slug}.py` | Baseline smoke: **`core_model` + `advanced_model`**; heavier cases: `advanced_model` only (+ `tts`) | `test-merge.yml` **or** merged into nightly TTS function job | Streaming, voice clone, batch/queue, async_chunk |
|
||||
| **L4** | `tests/e2e/online_serving/test_{slug}_expansion.py`, optional offline expansion | `full_model` + `tts` | `test-nightly.yml` (`:full_moon: TTS · Function Test with L4`) | Feature matrix; perf → `tests/dfx/perf/tests/test_tts.json` |
|
||||
|
||||
**L2 & L3 online — same file, dual marks on the baseline smoke:** The **first / simplest** case in `test_{slug}.py` (default deploy, single non-streaming `/v1/audio/speech` or equivalent offline path) should carry **both** `@pytest.mark.core_model` **and** `@pytest.mark.advanced_model` on the **same** function so it runs in L2 (`test-ready.yml`, `--run-level core_model`, basic validation) and L3 (`test-merge.yml`, `--run-level advanced_model`, deeper validation) without duplicating the test. In-tree examples: `test_voxcpm2_tts.py::test_text_to_audio_001`, `test_qwen3_tts_customvoice.py::test_text_to_audio_001`.
|
||||
|
||||
Heavier scenarios in the same file use **`advanced_model` only** (streaming, extra languages, concurrency, async_chunk, batch). Example: `test_voice_clone_en_streaming_001` → `advanced_model` only. When migrating L3 to nightly, move those heavier cases into `test_{slug}_expansion.py` with `full_model` and drop the dedicated merge job (see `test_ming_tts_expansion.py`, `test_glm_tts_expansion.py`).
|
||||
|
||||
```python
|
||||
@pytest.mark.core_model
|
||||
@pytest.mark.advanced_model
|
||||
@pytest.mark.tts
|
||||
@hardware_test(res={"cuda": "L4"}, num_cards=1)
|
||||
@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
|
||||
def test_voice_clone_en_non_streaming_001(omni_server, online_client) -> None:
|
||||
online_client.send_audio_speech_request({...})
|
||||
```
|
||||
|
||||
**L4 consolidation:** Prefer parametrized `OmniServerParams` rows (`default`, `async_chunk`, feature flags) in one expansion module rather than many merge-only files ([#1832](https://github.com/vllm-project/vllm-omni/issues/1832)).
|
||||
|
||||
**L4 performance (high-priority models):** Add latency / throughput / stress rows in `tests/dfx/perf/tests/test_tts.json`, or a dedicated `tests/dfx/perf/tests/test_{slug}.json` when the model must not join the shared nightly server matrix before integration lands (see VoxCPM2 / Coqui XTTS pattern). Register the model in `benchmarks/tts/model_configs.yaml` for local `bench_tts.py`. Wire a **separate** `test-nightly.yml` Perf Test step when the JSON is not merged into `test_tts.json` yet.
|
||||
|
||||
**Keep model-specific code inside test modules — not `tests/helpers/{slug}.py`:**
|
||||
|
||||
- Put `MODEL`, deploy path, vendored `REF_AUDIO_URL`, `get_prompt()`, and inline `request_config` dicts **in each** `test_{slug}.py`, `test_{slug}_expansion.py`, offline `test_{slug}.py`, and L1 `test_{slug}_*.py` as needed.
|
||||
- **Do not** add `tests/helpers/{slug}.py` (or `tests/helpers/{model_name}.py`) to deduplicate constants or request builders across those files. A little duplication is intentional; follow in-tree references such as `tests/e2e/online_serving/test_glm_tts.py` and `tests/e2e/online_serving/test_cosyvoice3_tts_expansion.py`.
|
||||
- `tests/helpers/` is for **repo-wide** harness code only (`mark.py`, `media.py`, `runtime.py`, `stage_config.py`, `assertions.py`, `fixtures/`). Import those; do not extend the tree with per-model modules.
|
||||
|
||||
**Runtime send helpers (`tests/helpers/runtime.py`) — online and offline e2e:**
|
||||
|
||||
| Path | Fixture | Call |
|
||||
|------|---------|------|
|
||||
| Online `/v1/*` | `online_client` | `online_client.send_*_request(request_config)` |
|
||||
| Offline inference | `offline_client` | `offline_client.send_*_request(request_config)` |
|
||||
|
||||
1. **Grep `runtime.py` first** — reuse `send_omni_request`, `send_diffusion_request`, `send_audio_speech_request` (online + offline Qwen-style TTS), `send_single_stage_tts_request` (Coqui XTTS / MOSS-TTS-Nano offline), etc.
|
||||
2. **No matching helper** → add `send_<feature>_request` (or `send_<route>_http_request` for negative/dfx) in **`runtime.py`** with general `assert_*` bundled inside, **then** call it from the test.
|
||||
3. **Test file** holds `request_config` dicts only — not `omni.generate`, not `_collect_audio()`, not raw HTTP/SDK.
|
||||
4. Wire `tests/helpers/runtime.py` into Buildkite `source_file_dependencies` when you add helpers.
|
||||
|
||||
See [vllm-omni-test skill](../vllm-omni-test/SKILL.md) § **Runtime send helpers** for full tables and exceptions.
|
||||
|
||||
### Deliverables
|
||||
|
||||
- One adapter file under `tts_adapters/` plus its line in the package import block
|
||||
- Client scripts and server launcher under `examples/online_serving/text_to_speech/<model>/`
|
||||
- Gradio demo with streaming and voice cloning UI in the same dir
|
||||
- E2E tests per **Test Case Writing (CI Levels)** above (priority tier determines L1–L4 scope)
|
||||
- Buildkite wired per level: `test-ready.yml` (L1/L2), `test-merge.yml` or nightly function job (L3), `test-nightly.yml` (L4) — see [vllm-omni-test skill](../vllm-omni-test/SKILL.md)
|
||||
- New section in `examples/online_serving/text_to_speech/README.md` (table row + per-model section). Do **not** create a top-level `examples/online_serving/<model>/` dir or a per-model `README.md` inside `text_to_speech/<model>/`.
|
||||
|
||||
### E2E test pitfalls to avoid
|
||||
|
||||
- **One `OmniServerParams` set per file.** `omni_server` is module-scoped; a second
|
||||
id in the same file forces mid-module teardown/restart and exposes startup
|
||||
races (`APIConnectionError` on the first request post-restart). Split variants
|
||||
into separate files instead.
|
||||
- **No external URL fetches from the server.** CI and some dev hosts can't
|
||||
reach `raw.githubusercontent.com` over TLS. Inline ref audio as
|
||||
`data:audio/wav;base64,...`; the serving layer accepts both URL and data URL.
|
||||
- **Use the harness readiness gate.** The fixture waits for HTTP 200 on
|
||||
`/health`; don't add `time.sleep` in tests. If warmup is incomplete, make
|
||||
`/health` return non-200 until you're actually ready.
|
||||
- **Mark tests per the CI Levels table** — baseline smoke: `core_model` + `advanced_model`; heavier cases: `advanced_model` only; L4 expansion: `full_model`
|
||||
- **No per-model helper modules** — do not create `tests/helpers/{slug}.py`; keep constants and `request_config` payloads in the test file
|
||||
- **Online and offline e2e go through `runtime.py`** — `online_client.send_audio_speech_request` (online); `offline_client.send_audio_speech_request` (Qwen-style offline) or `send_single_stage_tts_request` (single-stage offline). Add a new `send_*_request` in `runtime.py` when none fits; do not embed `omni.generate` or HTTP in tests
|
||||
|
||||
## Phase 4: Async Chunk (Streaming)
|
||||
|
||||
**Goal**: Enable inter-stage streaming so audio chunks are produced while AR generation continues.
|
||||
|
||||
### Steps
|
||||
|
||||
1. **Update the pipeline topology** so the producing stage declares its async
|
||||
handoff processor with `async_chunk_process_next_stage_input_func`.
|
||||
2. **Update the deploy YAML** to enable async chunk and configure the connector:
|
||||
```yaml
|
||||
async_chunk: true
|
||||
|
||||
connectors:
|
||||
connector_of_shared_memory:
|
||||
name: SharedMemoryConnector
|
||||
extra:
|
||||
codec_streaming: true
|
||||
codec_chunk_frames: 25
|
||||
codec_left_context_frames: 25
|
||||
```
|
||||
3. **Implement chunk handling in Stage 1**:
|
||||
- Accept partial input (chunk of codec codes)
|
||||
- Handle left context for smooth audio boundaries
|
||||
- Return partial audio in `OmniOutput`
|
||||
4. **Test streaming**:
|
||||
- Verify audio quality matches non-streaming output
|
||||
- Check for artifacts at chunk boundaries
|
||||
- Measure TTFA (time to first audio)
|
||||
5. **Update online serving** to support `stream=true` with PCM output
|
||||
|
||||
### Streaming Architecture
|
||||
|
||||
```
|
||||
Stage 0 (AR) Stage 1 (Decoder)
|
||||
| |
|
||||
|-- chunk 0 (25 frames) ------> decode -> audio chunk 0 -> client
|
||||
|-- chunk 1 (25 frames) ------> decode -> audio chunk 1 -> client
|
||||
|-- chunk 2 (25 frames) ------> decode -> audio chunk 2 -> client
|
||||
...
|
||||
```
|
||||
|
||||
### Key Considerations
|
||||
|
||||
- **Left context overlap**: Prevents audible artifacts at chunk boundaries
|
||||
- **Hop length matters**: `context_audio_samples = context_frames * hop_length`
|
||||
- **First chunk latency**: Can use larger initial chunk for better quality, then smaller chunks
|
||||
|
||||
### Deliverables
|
||||
|
||||
- Updated pipeline async handoff processor and deploy config with
|
||||
`async_chunk: true`
|
||||
- Smooth streaming audio without boundary artifacts
|
||||
- TTFA metrics
|
||||
|
||||
## Phase 5: CUDA Graph Acceleration
|
||||
|
||||
**Goal**: Capture the AR loop as a CUDA graph for significant speedup.
|
||||
|
||||
### Steps
|
||||
|
||||
1. **Identify the hot loop**: The AR decoding loop that runs N steps per token
|
||||
2. **Create static buffers**:
|
||||
- KV caches with fixed max sequence length
|
||||
- Pre-built causal masks and position tensors per step
|
||||
- Static input/output tensors
|
||||
3. **Implement graph capture**:
|
||||
- Warm up with real data
|
||||
- Capture the forward pass
|
||||
- Replay with updated inputs
|
||||
4. **Handle constraints**:
|
||||
- Use `torch.argmax` instead of `torch.multinomial` (graph-safe)
|
||||
- Fixed batch size (fall back to eager for other sizes)
|
||||
- No dynamic control flow inside the graph
|
||||
|
||||
See [references/cuda-graph-example.md](references/cuda-graph-example.md) for
|
||||
a worked skeleton (Qwen3-TTS code predictor, 16-step AR loop), performance
|
||||
expectations (3–5× on the graphed component for fixed batch_size=1), and the
|
||||
graph-safety constraints you must honor inside the captured region.
|
||||
|
||||
### Deliverables
|
||||
|
||||
- CUDA graph implementation for the AR hot loop
|
||||
- Benchmark script comparing eager vs graph performance
|
||||
- Documentation of constraints and fallback behavior
|
||||
|
||||
## Phase 6: Pre-commit and DCO
|
||||
|
||||
**Goal**: Every commit passes `pre-commit` lint and carries a DCO
|
||||
`Signed-off-by` line that matches the author email.
|
||||
|
||||
- Install hooks once: `pre-commit install`.
|
||||
- Run `pre-commit run --files <changed-files>` before every push; accept any
|
||||
auto-fixes, stage, re-commit.
|
||||
- Sign every commit with `git commit -s`. DCO checks that author email and
|
||||
`Signed-off-by` email match — `git config user.email` must match your
|
||||
GitHub account email.
|
||||
|
||||
Common pre-commit failures, recovery commands for missing sign-off, and the
|
||||
full `pre-commit run` invocation for a TTS model:
|
||||
[references/precommit-dco.md](references/precommit-dco.md).
|
||||
|
||||
## Documentation and recipe
|
||||
|
||||
Follow the [`add-recipe` skill](../add-recipe/SKILL.md) to add or update the
|
||||
in-repository model-family recipe, the `recipes/README.md` index, the supported
|
||||
models table, TTS example documentation, and the speech API contract using
|
||||
validated evidence.
|
||||
|
||||
## Integration Checklist
|
||||
|
||||
Use this checklist when integrating a new TTS model:
|
||||
|
||||
### Cross-Cutting Invariants (verify at end of every phase)
|
||||
- [ ] I1: `forward()` docstring states cumulative vs delta; consolidation path audited end-to-end
|
||||
- [ ] I2: Tests / examples / benchmarks never use `dict.get(a) or dict.get(b)` on tensor values; list form handled
|
||||
- [ ] I3: No `.item()` / `.cpu()` / Python branch on tensor values inside per-step loops
|
||||
- [ ] I4: Offline RTF, browser streaming playback, and concurrent-request smoke test all pass
|
||||
- [ ] I5: Any cross-step cache keyed by `_omni_req_id`; entries freed when the request finishes
|
||||
|
||||
### Phase 1: HF Reference
|
||||
- [ ] Reference model runs and produces correct audio
|
||||
- [ ] Architecture documented (stages, codebooks, tokens, sample rate)
|
||||
- [ ] Reference audio samples saved for comparison
|
||||
|
||||
### Phase 2: Stage Separation
|
||||
- [ ] Model registered in `registry.py`
|
||||
- [ ] Config classes created with `model_type` registration
|
||||
- [ ] Stage 0 (AR) implemented and generates correct tokens
|
||||
- [ ] Stage 1 (Decoder) produces correct audio from tokens — dtype float32 for codec decoder
|
||||
- [ ] AR stage `max_num_seqs` ≥ 4 in the production deploy config unless the model has a tested lower limit
|
||||
- [ ] Optional dependency fallbacks handled at `load_weights()` time (torchaudio/soundfile/etc.)
|
||||
- [ ] Streaming: codec codes accumulated across AR steps (not reset per step)
|
||||
- [ ] Streaming: delta audio emitted per chunk, not full re-decoded waveform
|
||||
- [ ] Streaming: all `forward()` return paths emit `model_outputs`
|
||||
- [ ] Streaming: per-request state keyed by request ID (not shared across requests)
|
||||
- [ ] Streaming: codec tensors moved to codec decoder device before decode
|
||||
- [ ] Pipeline topology defined in `pipeline.py` and registered in `OMNI_PIPELINES`
|
||||
- [ ] Deploy YAML created under `vllm_omni/deploy/`
|
||||
- [ ] `end2end.py` produces audio matching reference quality
|
||||
- [ ] README.md written
|
||||
|
||||
### Phase 3: Online Serving
|
||||
- [ ] Adapter written under `tts_adapters/` and registered in the import block
|
||||
- [ ] Only extract params in `build()` that are forwarded to the model call (ruff F841)
|
||||
- [ ] Prompt builder handles text input correctly
|
||||
- [ ] Voice cloning works (if supported)
|
||||
- [ ] All response formats work (wav, mp3, flac, pcm)
|
||||
- [ ] Client scripts and server launcher created
|
||||
- [ ] E2E tests added per model priority tier (see **Test Case Writing (CI Levels)**)
|
||||
- [ ] Buildkite entries match level: `test-ready.yml` / `test-merge.yml` or nightly TTS job / `test-nightly.yml`
|
||||
- [ ] Gradio demo working
|
||||
- [ ] Documentation added (offline + online docs, nav, supported models)
|
||||
- [ ] In-repository recipe and `recipes/README.md` row added via the `add-recipe` skill
|
||||
|
||||
### Phase 4: Async Chunk
|
||||
- [ ] Pipeline declares the async handoff processor
|
||||
- [ ] Deploy config sets `async_chunk: true` and connector chunk parameters
|
||||
- [ ] Stage 1 handles partial chunks correctly
|
||||
- [ ] No audio artifacts at chunk boundaries
|
||||
- [ ] Streaming via API (`stream=true`) works
|
||||
- [ ] TTFA measured and acceptable
|
||||
|
||||
### Phase 5: CUDA Graph
|
||||
- [ ] Hot loop identified and profiled
|
||||
- [ ] Static buffers allocated
|
||||
- [ ] Graph captured and replays correctly
|
||||
- [ ] Benchmark shows meaningful speedup
|
||||
- [ ] Fallback to eager works for unsupported configs
|
||||
|
||||
### Phase 6: Pre-commit and DCO
|
||||
- [ ] `pre-commit run --files <changed>` passes before every push
|
||||
- [ ] Every commit has `Signed-off-by` matching the author email (`git commit -s`)
|
||||
- [ ] `git config user.email` matches the email registered on your GitHub account
|
||||
- [ ] Hook list, failure table, and DCO recovery: [references/precommit-dco.md](references/precommit-dco.md)
|
||||
|
||||
## References
|
||||
|
||||
In-skill references (details split out of the main body):
|
||||
|
||||
- [references/single-stage-ar.md](references/single-stage-ar.md) — full `forward()` / generator skeleton for the MOSS-TTS-Nano-style pattern
|
||||
- [references/optional-deps.md](references/optional-deps.md) — torchaudio / torchcodec fallback pattern
|
||||
- [references/cuda-graph-example.md](references/cuda-graph-example.md) — Qwen3-TTS code-predictor CUDA graph skeleton
|
||||
- [references/precommit-dco.md](references/precommit-dco.md) — full pre-commit invocation, failure table, DCO recovery
|
||||
|
||||
Project docs and adjacent skills:
|
||||
|
||||
- [TTS audio skill](../vllm-omni-audio-tts/SKILL.md) — supported models and usage
|
||||
- [Fish Speech integration](../vllm-omni-audio-tts/references/fish-speech.md) — complete example of Phases 1–3
|
||||
- [Qwen3-TTS reference](../vllm-omni-audio-tts/references/qwen-tts.md) — complete example of all 5 phases
|
||||
- [vllm-omni-test skill](../vllm-omni-test/SKILL.md) — L1–L4 markers, naming, Buildkite wiring, run commands
|
||||
- [Adding a TTS model (developer guide)](https://github.com/vllm-project/vllm-omni/blob/main/docs/contributing/model/adding_tts_model.md)
|
||||
- `plan/voxcpm2_native_ar_design.md` — VoxCPM2's vLLM-native AR + side-computation pattern (distinct from the generator-based single-stage described above)
|
||||
@@ -1,42 +0,0 @@
|
||||
# CUDA Graph Example: Qwen3-TTS Code Predictor
|
||||
|
||||
Reference sketch for capturing the 16-step code-predictor AR loop as a single
|
||||
CUDA graph. Adapt the shapes, number of steps, and KV-head layout to your
|
||||
model.
|
||||
|
||||
```python
|
||||
import torch
|
||||
|
||||
class CodePredictorGraph:
|
||||
"""Captures the 16-step code predictor AR loop as a single CUDA graph."""
|
||||
|
||||
def setup_graph(self, device: torch.device, kv_heads: int = 4, head_dim: int = 64):
|
||||
self.num_steps = 16
|
||||
self.kv_cache = torch.zeros(1, kv_heads, self.num_steps, head_dim, device=device)
|
||||
self.positions = torch.arange(self.num_steps, device=device)
|
||||
self.causal_mask = torch.tril(torch.ones(self.num_steps, self.num_steps, device=device))
|
||||
self.input_buf = torch.zeros(1, 1, kv_heads * head_dim, device=device)
|
||||
self.output_buf = torch.zeros(1, self.num_steps, device=device, dtype=torch.long)
|
||||
# Warm up, then: self.graph = torch.cuda.CUDAGraph(); self.graph.capture(...)
|
||||
|
||||
def run_graph(self, initial_input: torch.Tensor) -> torch.Tensor:
|
||||
self.input_buf.copy_(initial_input)
|
||||
self.graph.replay()
|
||||
return self.output_buf.clone()
|
||||
```
|
||||
|
||||
## Performance expectations (Qwen3-TTS code predictor)
|
||||
|
||||
- **3–5× speedup** on the graphed component.
|
||||
- Effective only for fixed batch sizes (typically `batch_size=1`).
|
||||
- Fall back to eager for any shape/config that wasn't captured — do not try
|
||||
to recapture per request.
|
||||
|
||||
## Graph-safety constraints
|
||||
|
||||
- `torch.argmax` instead of `torch.multinomial`.
|
||||
- Fixed batch size.
|
||||
- No Python control flow that branches on tensor values inside the captured
|
||||
region (use `torch.where` / masks).
|
||||
- No `.item()`, `.cpu()`, `.tolist()` — each would break the capture or
|
||||
cause a GPU→CPU sync during replay.
|
||||
@@ -1,47 +0,0 @@
|
||||
# Optional Dependency Handling
|
||||
|
||||
Models that rely on `torchaudio`, `torchcodec`, `soundfile`, or other optional
|
||||
packages must handle the missing-package case at import time, not at call
|
||||
time. Failing to do this causes cryptic errors only on environments without
|
||||
the optional package — after the model is already deployed.
|
||||
|
||||
## Pattern (used in MOSS-TTS-Nano)
|
||||
|
||||
```python
|
||||
def _patch_torchaudio_load() -> None:
|
||||
"""Fallback torchaudio.load/save to soundfile if torchcodec is unavailable."""
|
||||
try:
|
||||
import torchcodec # noqa: F401
|
||||
return # torchcodec present, torchaudio works as-is
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
import soundfile as sf
|
||||
import torchaudio
|
||||
|
||||
def _sf_load(path, **kwargs):
|
||||
data, sr = sf.read(str(path), dtype="float32", always_2d=True)
|
||||
return torch.from_numpy(data).T, sr
|
||||
|
||||
torchaudio.load = _sf_load
|
||||
# patch .save similarly if needed
|
||||
```
|
||||
|
||||
## Rules
|
||||
|
||||
- Mirror the full signature of the replaced function. `torchaudio.load`
|
||||
accepts `frame_offset`, `num_frames`, `normalize`, `channels_first`,
|
||||
`format` — missing any of them causes `TypeError` from calling code.
|
||||
- Catch `except Exception`, not just `ImportError`. `import torchaudio`
|
||||
itself can fail with non-`ImportError` errors on broken installs. Log the
|
||||
exception type and message (`logger.warning("torchaudio probe failed: %s: %s",
|
||||
type(exc).__name__, exc)`) before falling back, so unrelated errors are not
|
||||
silently swallowed.
|
||||
- Call the patch function at the top of `load_weights()` before loading any
|
||||
audio assets. Do not call it at module import time.
|
||||
|
||||
## Reference implementation
|
||||
|
||||
Any in-tree model that patches `torchaudio.load` in its `load_weights()` —
|
||||
e.g. MOSS-TTS-Nano's `modeling_moss_tts_nano.py` once that integration
|
||||
lands.
|
||||
Loaded 100 of 4510 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user