anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
2,405 Commits 235 Branches 31 Tags
62aeffd15c73a17e8134bf1e60716ff388cf752d
Commit Graph
2405 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Gao Han 62aeffd15c Merge branch 'main' into fix/tp-local-kv-recv-consensus 2026-08-01 23:23:52 +08:00
SYLARandlishunyang12 4b1547b4d8 [Bugfix] Fix Wan2.2 omitted guidance scale resolution (#5615) 
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Signed-off-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
 2026-08-01 15:17:33 +00:00
SYLARandlishunyang12 6e8c279a68 [Bugfix] Match image file metadata to output format (#5619) 
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
 2026-08-01 15:09:27 +00:00
SYLARandlishunyang12 6f98c807d2 [Bugfix] Restore BAGEL CFG KV transfers (#5620) 
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
 2026-08-01 15:06:05 +00:00
tzhouamandClaude Fable 5 bac3eafc18 review: scope KV receive consensus to the TP group, compare full KV state 
Address both P1 review comments:

- The exchange now runs on the stage's tensor-model-parallel group
  (vLLM parallel state) instead of pt.world. World also holds DP
  replicas and PP stages working on different requests, so the old
  group could mix verdicts across requests and — worse — deadlock if
  one replica skipped the exchange (dummy request) while another
  entered it. Within the TP group every skip guard is symmetric.

- Ranks now exchange (primary_received, sorted applied-KV roles)
  instead of a bare flag, using the same field enumeration as
  apply/broadcast, so partially-received CFG companion KV
  (cfg_text/cfg_img) diverging between ranks is caught too. Verdict is
  identical-state-everywhere: any divergence clears KV on every rank;
  symmetric absence is kept because it is consistent (Bagel's empty
  unconditional branch).

Local e2e rerun of test_bagel[tp_vae_patch_parallel_2] passed and the
new path fired on a real divergence (one rank primary+metadata, the
other only cfg_kv_request_ids) — both ranks fell back together.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-08-01 16:29:29 +08:00
Librium 185e3d6df6 [Bugfix] Preserve channel-last video tensor layout (#5418) 
Signed-off-by: pxljs <1621352782@qq.com>
 2026-08-01 07:05:57 +00:00
amy-why-3459andCursor 67aa6c55a4 [Bugfix][MiniCPM-o] Align Daily-Omni bench with MiniCPM interleaved A… (#5625) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-07-31 22:41:00 -07:00
Shenglei Fu 9d67f8e02f [CI/Build] Let audio assertions pin the transcript language (#5646) 
Signed-off-by: Shenglei Fu <sfu@confluent.io>
 2026-07-31 21:44:28 -07:00
akshatvishu cd5daaea6a [BugFix][Metrics] Fix per-stage time and token stats in metrics API (#4974) 
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
 2026-08-01 03:54:17 +00:00
Harenome 安仁 Ranaivoarivony-Razanajato 8cb79308c6 diffusion: Fix silent override of explicit guidance_scale=0 (#4999) 
Signed-off-by: Harenome Ranaivoarivony-Razanajato <harenome.razanajato1@huawei.com>
 2026-08-01 03:19:21 +00:00
NATURE 0a03359224 [Bugfix][MiniCPM-o] Stabilize single-GPU audio generation (#5637) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
 2026-08-01 03:01:06 +00:00
Weiming Liao fe02e8b81e [CI][NPU] Add MiniCPM-o 4.5 dependencies (#5660) 
Signed-off-by: Weiming Liao <liaowm5@gmail.com>
 2026-08-01 02:57:12 +00:00
Gao Han 0eda607077 Merge branch 'main' into fix/tp-local-kv-recv-consensus 2026-08-01 10:50:23 +08:00
SYLAR 896f05e7e6 [Kernel] Add typed SAGE for TRTLLM_ATTN (#5509) 
Signed-off-by: lishunyang12 <lishunyang12@163.com>
 2026-07-31 23:11:39 +00:00
林鑫 d33c905d80 [Perf] Cache RoPE cos/sin tables in code predictor (#5503) 
Signed-off-by: I-wave <2083236231@qq.com>
 2026-07-31 22:03:18 +08:00
Nathan PriceandClaude Opus 5 4715570acc [Bugfix][Full Duplex] Make runtime_control payloads JSON serializable (#5613) 
Signed-off-by: thecodewrangler <nathan@abridge.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-07-31 03:17:25 -07:00
wuli666 4956a41f12 [Bugfix] Default missing legacy stage input sources (#5498) 
Signed-off-by: wuli666 <421774554@qq.com>
 2026-07-31 03:00:14 -07:00
ruirui(rein) yang 2739f0776b [Bugfix][MiniCPM-o]Fix implicit deploy connector resolution for registered pipelines (#5533) 
Signed-off-by: R2-Y <ruiruyang2@gmail.com>
 2026-07-31 02:38:26 -07:00
maithilijoshi20andmaithilijoshi20 2d8ef4754c [Bugfix] Restore Gradio demo dependency (#5632) 
Signed-off-by: maithilijoshi20 <maithilijoshi2003@gmail.com>
Co-authored-by: maithilijoshi20 <maithilijoshi2003@gmail.com>
 2026-07-31 02:24:45 -07:00
SYLARandlishunyang12 f8a8fbf72e [Bugfix][CI] Reuse servers across paired perf cases (#5624) 
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
 2026-07-31 08:23:17 +00:00
boatman 226cb4a38c [Bugfix][VoxCPM2] Fix Chunked Prefill Length Mismatch (#5416) 
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com>
 2026-07-31 05:44:53 +00:00
SYLARlishunyang12Copilot Autofix powered by AI
 		0a51a18570 [Bugfix] Leave MiniCPM-o vocoder memory headroom (#5621) 
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Signed-off-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
 2026-07-31 03:35:13 +00:00
SYLARandlishunyang12 499d930fe5 [Bugfix] Use MiniCPM-o speech template in long audio test (#5623) 
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
 2026-07-31 03:16:17 +00:00
tzhouamandClaude Fable 5 6d767a9a5b [BugFix] Make pure-TP KV receive fallback all-or-nothing across ranks 
Fixes the recurring Bagel tp_vae_patch_parallel_2 NCCL ALLREDUCE
timeout (#5627, previously #5487): in pure TP the transfer topology is
LOCAL on every rank — each rank fetches its own per-rank KV key with an
independent recv_timeout and distribute_kv_cache is a no-op. A receive
timeout returns silently, so a single rank that missed its shard fell
back alone into the pipeline's no-KV prefill (Bagel
forward_cache_update_text through the TP-sharded LM) and entered TP
allreduces its peers never joined. The hung [~20, 3584] allreduce
(NumelIn=71680) blocked 600s until the NCCL watchdog killed the worker,
after which every request failed with "DiffusionExecutor is closed".
PR #5482 fixed the orchestrator-level CFG bundle race but never touched
this lower KV-transfer layer, which is why the signature recurred at
its own merge commit.

After the per-rank receive, pure-TP ranks now exchange their receive
flags (rank 0 collects and broadcasts the verdict over the stage world
group's CPU object plane) and act on it together: if any rank missed,
ranks that did receive drop their applied KV — cleared symmetrically
with what apply/broadcast set — so every rank takes the identical no-KV
prefill path and the TP collective schedule can never diverge. The skip
guards (topology, need_recv_cache, dummy warmup id) depend only on
values identical across ranks, so ranks always agree on whether the
exchange happens; a rank whose connector failed to initialize still
participates and reports a miss instead of deadlocking the exchange.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-07-31 11:08:13 +08:00
Nick CaoClaudeHongsheng Liu
 		273fc8eb1d [Tests] Further extend the tiny model testing framework and enable multiple models (#5353) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-07-31 07:47:26 +08:00
Zhou TaichangtzhouamClaude Fable 5
 		a951dd0cb4 [BugFix] Never dispatch diffusion with an incomplete CFG companion bundle (#5482) 
Signed-off-by: tzhouam <tzhouam@gmail.com>
Co-authored-by: tzhouam <tzhouam@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-30 08:17:24 -07:00
akshatvishu 4563999ec0 [BugFix][Diffusion] Add CPU LAPACK fallback for FlowUniPC (#5329) 
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
 2026-07-30 15:09:48 +00:00
Weiming Liaoand汪志鹏 86d7864588 [Bugfix][Diffusion] Honor HF_HUB_OFFLINE in OmniDiffusionConfig (align with AR stage) (#5403) 
Signed-off-by: Weiming Liao <liaowm5@gmail.com>
Signed-off-by: Weiming Liao <fayespica@outlook.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-07-30 15:07:46 +00:00
Shenglei FuandClaude 3b0718f22e [Bugfix] Fix active_stream_window silently stalling audio generation (#5349) (#5373) 
Signed-off-by: Shenglei Fu <sfu@confluent.io>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-07-30 14:56:18 +00:00
Yuanheng Zhao ce3d8e4bac [Bufix] Ming-omni-tts-16.8B Skip rejected and unused weights loading (#5607) 
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
 2026-07-30 22:38:40 +08:00
Henry JandClaude Fable 5 81cd7964d4 [Bugfix] Qwen3-TTS full-payload: emit one placeholder frame on a degenerate take instead of an empty payload (#5472) 
Signed-off-by: henryj <349043+henryj@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-30 14:27:39 +00:00
Sy03 09c6ad4204 [CI/Build][MiniCPM-o] Allow auto-response before input commit (#5567) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-07-30 14:14:19 +00:00
Yash JainandClaude Fable 5 f7e87e7cfd [Bugfix][Diffusion] Apply MagCache residual once per skipped step (#5561) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-30 07:09:25 -07:00
evanchuenganddavid6666666 744c65b730 [Feature] Distributed layerwise offload with DP multi-concurrency + mmap weight… (#5397) 
Signed-off-by: evanchueng <evanchueng@outlook.com>
Signed-off-by: david6666666 <530634352@qq.com>
Co-authored-by: david6666666 <530634352@qq.com>
 2026-07-30 13:35:38 +00:00
Yukim1 b3f4fbf9e2 [Refactor] Migrate Step-Audio2 offline test to deploy config (#5319) 
Signed-off-by: zwhzzz0821 <2831474076@qq.com>
 2026-07-30 09:59:05 +00:00
Yukim1 1752e8ee5b [Refactor] Migrate Qwen3 Omni thinker-only deploy config (#5285) 
Signed-off-by: zwhzzz0821 <2831474076@qq.com>
 2026-07-30 09:57:33 +00:00
Alex Brooks 089bb88706 [Model] Remove Text Encoder Output Layer Hardcoding in Flux2 (#5589) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-07-30 09:55:42 +00:00
SYLARandlishunyang12 04b408ffd2 [Bugfix] Add Cosmos3-Edge preset (fix wrong Nano/Super defaults in T2V/I2V examples) (#5596) 
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
 2026-07-30 08:41:30 +00:00
TaffyOfficialandTaffyOfficial 0c88f8a0b6 [Refactor] Normalize diffusion request extra arguments in Chat serving (#5171) 
Signed-off-by: TaffyOfficial <2324465096@qq.com>
Co-authored-by: TaffyOfficial <2324465096@qq.com>
 2026-07-30 07:08:12 +00:00
wangyu 1d6c0816a4 [CI/Build] Fix nested group breaking nightly pipeline upload (#5572) 
Signed-off-by: wangyu <410167048@qq.com>
 2026-07-30 11:43:08 +08:00
TJian d50ea9fccc [ROCm] [CI] Fix ROCm CI for v0.26.0 (#5528) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-07-30 11:27:32 +08:00
Yash JainandClaude Fable 5 daec3a6a49 [Bugfix] Trim AR sampled logprob rows with new_token_ids on mid-step stop (#5564) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-30 03:00:53 +00:00
ruirui(rein) yangandGao Han 0461da345a [bugfix][MiniCPM-o]delete duplicate files (#5535) 
Signed-off-by: R2-Y <ruiruyang2@gmail.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-07-30 09:59:44 +08:00
y-nullamy-why-3459Hongsheng LiuGao Han
 		aa5364f705 [refactor][MiniCPM-o] Consolidate MiniCPM-o 4.5 2-GPU deploy configs and E2E tests (#5458) 
Signed-off-by: y_null <y_null@qq.com>
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-07-30 09:56:19 +08:00
Mu GuanLin 1b718f7b15 [Feat] [CI] [bugfix] Generalize LTX CFG parallelism to the complete guidance plan (#5547) 
Signed-off-by: mglyn <1203789601@qq.com>
 2026-07-29 17:40:40 +00:00
zzhanddengyunyang dba05c3517 Enable Diffusion Image models asynchronous output (#4981) 
Signed-off-by: zzh <943967662@qq.com>
Co-authored-by: dengyunyang <584797741@qq.com>
 2026-07-29 17:21:55 +00:00
whisper f2d9eb1630 [Bugfix][CI/Build] Avoid refetching HunyuanVideo I2V input during sim… (#5552) 
Signed-off-by: whisper-la <131673492+whisper-la@users.noreply.github.com>
 2026-07-29 09:49:07 -07:00
Weiming Liao fa0658eabb [Doc][NPU] Fix a missed version reference: 0.25.0 → v0.26.0 (#5538) 
Signed-off-by: Weiming Liao <liaowm5@gmail.com>
 2026-07-29 23:53:20 +08:00
wangyu 7c17111ff5 [Refactor][CI] Reorganize test tree, and move slow e2e to weekly (#4951) 
Signed-off-by: wangyu <410167048@qq.com>
 2026-07-29 23:53:03 +08:00
zzh 925347332f Add vae-patch-parallel-size and vae-use-tiling for HunyuanImage Benchmark (#5502) 
Signed-off-by: zzh <943967662@qq.com>
 2026-07-29 02:43:45 -07:00