anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
2,394 Commits 235 Branches 31 Tags
bac3eafc18d2e5389f9db0f50a788e94969f73ea
Commit Graph
2394 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
tzhouamandClaude Fable 5 bac3eafc18 review: scope KV receive consensus to the TP group, compare full KV state 
Address both P1 review comments:

- The exchange now runs on the stage's tensor-model-parallel group
  (vLLM parallel state) instead of pt.world. World also holds DP
  replicas and PP stages working on different requests, so the old
  group could mix verdicts across requests and — worse — deadlock if
  one replica skipped the exchange (dummy request) while another
  entered it. Within the TP group every skip guard is symmetric.

- Ranks now exchange (primary_received, sorted applied-KV roles)
  instead of a bare flag, using the same field enumeration as
  apply/broadcast, so partially-received CFG companion KV
  (cfg_text/cfg_img) diverging between ranks is caught too. Verdict is
  identical-state-everywhere: any divergence clears KV on every rank;
  symmetric absence is kept because it is consistent (Bagel's empty
  unconditional branch).

Local e2e rerun of test_bagel[tp_vae_patch_parallel_2] passed and the
new path fired on a real divergence (one rank primary+metadata, the
other only cfg_kv_request_ids) — both ranks fell back together.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-08-01 16:29:29 +08:00
Gao Han 0eda607077 Merge branch 'main' into fix/tp-local-kv-recv-consensus 2026-08-01 10:50:23 +08:00
SYLAR 896f05e7e6 [Kernel] Add typed SAGE for TRTLLM_ATTN (#5509) 
Signed-off-by: lishunyang12 <lishunyang12@163.com>
 2026-07-31 23:11:39 +00:00
林鑫 d33c905d80 [Perf] Cache RoPE cos/sin tables in code predictor (#5503) 
Signed-off-by: I-wave <2083236231@qq.com>
 2026-07-31 22:03:18 +08:00
Nathan PriceandClaude Opus 5 4715570acc [Bugfix][Full Duplex] Make runtime_control payloads JSON serializable (#5613) 
Signed-off-by: thecodewrangler <nathan@abridge.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-07-31 03:17:25 -07:00
wuli666 4956a41f12 [Bugfix] Default missing legacy stage input sources (#5498) 
Signed-off-by: wuli666 <421774554@qq.com>
 2026-07-31 03:00:14 -07:00
ruirui(rein) yang 2739f0776b [Bugfix][MiniCPM-o]Fix implicit deploy connector resolution for registered pipelines (#5533) 
Signed-off-by: R2-Y <ruiruyang2@gmail.com>
 2026-07-31 02:38:26 -07:00
maithilijoshi20andmaithilijoshi20 2d8ef4754c [Bugfix] Restore Gradio demo dependency (#5632) 
Signed-off-by: maithilijoshi20 <maithilijoshi2003@gmail.com>
Co-authored-by: maithilijoshi20 <maithilijoshi2003@gmail.com>
 2026-07-31 02:24:45 -07:00
SYLARandlishunyang12 f8a8fbf72e [Bugfix][CI] Reuse servers across paired perf cases (#5624) 
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
 2026-07-31 08:23:17 +00:00
boatman 226cb4a38c [Bugfix][VoxCPM2] Fix Chunked Prefill Length Mismatch (#5416) 
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com>
 2026-07-31 05:44:53 +00:00
SYLARlishunyang12Copilot Autofix powered by AI
 		0a51a18570 [Bugfix] Leave MiniCPM-o vocoder memory headroom (#5621) 
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Signed-off-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
 2026-07-31 03:35:13 +00:00
SYLARandlishunyang12 499d930fe5 [Bugfix] Use MiniCPM-o speech template in long audio test (#5623) 
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
 2026-07-31 03:16:17 +00:00
tzhouamandClaude Fable 5 6d767a9a5b [BugFix] Make pure-TP KV receive fallback all-or-nothing across ranks 
Fixes the recurring Bagel tp_vae_patch_parallel_2 NCCL ALLREDUCE
timeout (#5627, previously #5487): in pure TP the transfer topology is
LOCAL on every rank — each rank fetches its own per-rank KV key with an
independent recv_timeout and distribute_kv_cache is a no-op. A receive
timeout returns silently, so a single rank that missed its shard fell
back alone into the pipeline's no-KV prefill (Bagel
forward_cache_update_text through the TP-sharded LM) and entered TP
allreduces its peers never joined. The hung [~20, 3584] allreduce
(NumelIn=71680) blocked 600s until the NCCL watchdog killed the worker,
after which every request failed with "DiffusionExecutor is closed".
PR #5482 fixed the orchestrator-level CFG bundle race but never touched
this lower KV-transfer layer, which is why the signature recurred at
its own merge commit.

After the per-rank receive, pure-TP ranks now exchange their receive
flags (rank 0 collects and broadcasts the verdict over the stage world
group's CPU object plane) and act on it together: if any rank missed,
ranks that did receive drop their applied KV — cleared symmetrically
with what apply/broadcast set — so every rank takes the identical no-KV
prefill path and the TP collective schedule can never diverge. The skip
guards (topology, need_recv_cache, dummy warmup id) depend only on
values identical across ranks, so ranks always agree on whether the
exchange happens; a rank whose connector failed to initialize still
participates and reports a miss instead of deadlocking the exchange.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-07-31 11:08:13 +08:00
Nick CaoClaudeHongsheng Liu
 		273fc8eb1d [Tests] Further extend the tiny model testing framework and enable multiple models (#5353) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-07-31 07:47:26 +08:00
Zhou TaichangtzhouamClaude Fable 5
 		a951dd0cb4 [BugFix] Never dispatch diffusion with an incomplete CFG companion bundle (#5482) 
Signed-off-by: tzhouam <tzhouam@gmail.com>
Co-authored-by: tzhouam <tzhouam@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-30 08:17:24 -07:00
akshatvishu 4563999ec0 [BugFix][Diffusion] Add CPU LAPACK fallback for FlowUniPC (#5329) 
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
 2026-07-30 15:09:48 +00:00
Weiming Liaoand汪志鹏 86d7864588 [Bugfix][Diffusion] Honor HF_HUB_OFFLINE in OmniDiffusionConfig (align with AR stage) (#5403) 
Signed-off-by: Weiming Liao <liaowm5@gmail.com>
Signed-off-by: Weiming Liao <fayespica@outlook.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-07-30 15:07:46 +00:00
Shenglei FuandClaude 3b0718f22e [Bugfix] Fix active_stream_window silently stalling audio generation (#5349) (#5373) 
Signed-off-by: Shenglei Fu <sfu@confluent.io>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-07-30 14:56:18 +00:00
Yuanheng Zhao ce3d8e4bac [Bufix] Ming-omni-tts-16.8B Skip rejected and unused weights loading (#5607) 
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
 2026-07-30 22:38:40 +08:00
Henry JandClaude Fable 5 81cd7964d4 [Bugfix] Qwen3-TTS full-payload: emit one placeholder frame on a degenerate take instead of an empty payload (#5472) 
Signed-off-by: henryj <349043+henryj@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-30 14:27:39 +00:00
Sy03 09c6ad4204 [CI/Build][MiniCPM-o] Allow auto-response before input commit (#5567) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-07-30 14:14:19 +00:00
Yash JainandClaude Fable 5 f7e87e7cfd [Bugfix][Diffusion] Apply MagCache residual once per skipped step (#5561) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-30 07:09:25 -07:00
evanchuenganddavid6666666 744c65b730 [Feature] Distributed layerwise offload with DP multi-concurrency + mmap weight… (#5397) 
Signed-off-by: evanchueng <evanchueng@outlook.com>
Signed-off-by: david6666666 <530634352@qq.com>
Co-authored-by: david6666666 <530634352@qq.com>
 2026-07-30 13:35:38 +00:00
Yukim1 b3f4fbf9e2 [Refactor] Migrate Step-Audio2 offline test to deploy config (#5319) 
Signed-off-by: zwhzzz0821 <2831474076@qq.com>
 2026-07-30 09:59:05 +00:00
Yukim1 1752e8ee5b [Refactor] Migrate Qwen3 Omni thinker-only deploy config (#5285) 
Signed-off-by: zwhzzz0821 <2831474076@qq.com>
 2026-07-30 09:57:33 +00:00
Alex Brooks 089bb88706 [Model] Remove Text Encoder Output Layer Hardcoding in Flux2 (#5589) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-07-30 09:55:42 +00:00
SYLARandlishunyang12 04b408ffd2 [Bugfix] Add Cosmos3-Edge preset (fix wrong Nano/Super defaults in T2V/I2V examples) (#5596) 
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
 2026-07-30 08:41:30 +00:00
TaffyOfficialandTaffyOfficial 0c88f8a0b6 [Refactor] Normalize diffusion request extra arguments in Chat serving (#5171) 
Signed-off-by: TaffyOfficial <2324465096@qq.com>
Co-authored-by: TaffyOfficial <2324465096@qq.com>
 2026-07-30 07:08:12 +00:00
wangyu 1d6c0816a4 [CI/Build] Fix nested group breaking nightly pipeline upload (#5572) 
Signed-off-by: wangyu <410167048@qq.com>
 2026-07-30 11:43:08 +08:00
TJian d50ea9fccc [ROCm] [CI] Fix ROCm CI for v0.26.0 (#5528) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-07-30 11:27:32 +08:00
Yash JainandClaude Fable 5 daec3a6a49 [Bugfix] Trim AR sampled logprob rows with new_token_ids on mid-step stop (#5564) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-30 03:00:53 +00:00
ruirui(rein) yangandGao Han 0461da345a [bugfix][MiniCPM-o]delete duplicate files (#5535) 
Signed-off-by: R2-Y <ruiruyang2@gmail.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-07-30 09:59:44 +08:00
y-nullamy-why-3459Hongsheng LiuGao Han
 		aa5364f705 [refactor][MiniCPM-o] Consolidate MiniCPM-o 4.5 2-GPU deploy configs and E2E tests (#5458) 
Signed-off-by: y_null <y_null@qq.com>
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-07-30 09:56:19 +08:00
Mu GuanLin 1b718f7b15 [Feat] [CI] [bugfix] Generalize LTX CFG parallelism to the complete guidance plan (#5547) 
Signed-off-by: mglyn <1203789601@qq.com>
 2026-07-29 17:40:40 +00:00
zzhanddengyunyang dba05c3517 Enable Diffusion Image models asynchronous output (#4981) 
Signed-off-by: zzh <943967662@qq.com>
Co-authored-by: dengyunyang <584797741@qq.com>
 2026-07-29 17:21:55 +00:00
whisper f2d9eb1630 [Bugfix][CI/Build] Avoid refetching HunyuanVideo I2V input during sim… (#5552) 
Signed-off-by: whisper-la <131673492+whisper-la@users.noreply.github.com>
 2026-07-29 09:49:07 -07:00
Weiming Liao fa0658eabb [Doc][NPU] Fix a missed version reference: 0.25.0 → v0.26.0 (#5538) 
Signed-off-by: Weiming Liao <liaowm5@gmail.com>
 2026-07-29 23:53:20 +08:00
wangyu 7c17111ff5 [Refactor][CI] Reorganize test tree, and move slow e2e to weekly (#4951) 
Signed-off-by: wangyu <410167048@qq.com>
 2026-07-29 23:53:03 +08:00
zzh 925347332f Add vae-patch-parallel-size and vae-use-tiling for HunyuanImage Benchmark (#5502) 
Signed-off-by: zzh <943967662@qq.com>
 2026-07-29 02:43:45 -07:00
R0CKSTAR 7be7e0d249 Support Python 3.10 runtime compatibility (#5366) 
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
 2026-07-29 17:11:04 +08:00
Canlin Guo 18be2ba3df [Perf] Enable compile for MOSS-TTS-Local v1.5 talker and codec (#5530) 
Signed-off-by: Canlin Guo <canlinguosdu@gmail.com>
 2026-07-29 14:09:06 +08:00
BruceLoveDecimalandGao Han 1c6e731339 fix(diffusion): convert caller sampling params instead of discarding … (#4115) 
Signed-off-by: BruceLoveDecimal <39156883+BruceLoveDecimal@users.noreply.github.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-07-29 13:24:51 +08:00
Zeyu Huang | 黃澤宇 b8e7b21188 [skip ci][Misc] remove stray md introduced in #4652 (#5473) 
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
 2026-07-29 12:42:25 +08:00
Weiming Liao 39663d5287 [NPU] upgrade to v0.26.0 (#5490) 
Signed-off-by: Weiming Liao <liaowm5@gmail.com>
 2026-07-29 11:21:04 +08:00
R0CKSTAR ab0da662c0 [Hardware] Enable CosyVoice3 on Moore Threads MUSA (#5164) 
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
 2026-07-29 02:46:27 +00:00
Zhang Jian fa41fdf252 [recipe][NPU] add cosmos3-super recipe on 8xA3 (#5097) 
Signed-off-by: Zhang Jian <jianmusings@gmail.com>
 2026-07-29 08:29:33 +08:00
Joshna-MedisettyandChendi.Xue 7fa40681ff Xpu/vllm 0.26 fixes (#5476) 
Signed-off-by: Joshna Medisetty <joshna.medisetty@intel.com>
Co-authored-by: Chendi.Xue <chendi.xue@intel.com>
 2026-07-29 08:15:36 +08:00
iancarrasco-b10 23bb4395bb [Feature] Flush the speech WebSocket on input.done without closing it (#5517) 
Signed-off-by: iancarrasco-b10 <ian.carrasco@baseten.co>
 2026-07-28 23:46:19 +00:00
Sy03andHongsheng Liu 0feb92bdfa [Bugfix][Minicpm] Support full-duplex three-stage serving (#5380) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-07-29 01:58:34 +08:00
张芷瑜（Zhang Zhiyu）andZhiyu0603 f6e59c329b [Bugfix] Return diffusion metrics from /v1/images/generations (#5278) 
Signed-off-by: Zhiyu0603 <Zhiyu0603@users.noreply.github.com>
Co-authored-by: Zhiyu0603 <Zhiyu0603@users.noreply.github.com>
 2026-07-28 16:38:37 +00:00