anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
3,428 Commits 235 Branches 31 Tags
codex/shared-stage-payload-transport
Commit Graph
3428 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Sy03 f3d0dbe5a8 [Bugfix] Preserve host transport fallback when discovery cleanup fails 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-10-05 22:07:14 +08:00
Sy03 06f562cb3d [Core] Cache host ring discovery without blocking empty polls 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-10-05 21:56:29 +08:00
Sy03 d98feb3c65 [Bugfix] Claim fallback stage segments before tracking readers 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-10-05 21:56:29 +08:00
Sy03 3322a78856 [Core] Optimize mixed stage payloads and scalar restoration 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-10-05 21:56:29 +08:00
Sy03 bb3f5b5cb7 [Core] Preserve native scalar encoding in stage tensor frames 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-10-05 21:56:29 +08:00
Sy03 4f99090212 [Doc] Specify the host ring process namespace requirement 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-10-05 21:56:29 +08:00
Sy03 06202d8f2a [Core] Reduce local stage payload overhead with shared host rings 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-10-05 21:56:29 +08:00
Sy03 023716ecc4 [Bugfix] Do not assume a stream decoder in MRv2 eager-MTP preprocess (#8484) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-10-05 09:29:38 -04:00
JohnnyJohn Nunezamy-why-3459
 		5351cd6645 [Bugfix] Qwen3-Omni thinker: create deepstack buffers on the real device (#8461) 
Signed-off-by: John Nunez <johnnynunez@users.noreply.github.com>
Co-authored-by: John Nunez <johnnynunez@users.noreply.github.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-10-05 21:00:04 +08:00
ztyan 091b256674 [Model][Perf] IndexTTS2: drop per-step hidden D2H in latent mode (#8478) 
Signed-off-by: ztyan <191593482+vivivi-111@users.noreply.github.com>
 2026-10-05 20:20:25 +08:00
Yueqian Lin 6dd0d1f931 [Bugfix] Update routed-expert tests for native auxiliary output (#8500) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
v0.31.0rc1 		2026-10-05 17:40:07 +08:00
JungleandCodex 67d21b5fff [Core] Add token_ids output type and enforce canonical names (#7843) 
Signed-off-by: Jungle430 <junglece430@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-05 16:49:03 +08:00
Zhou Taichang 0b0d2d7009 [Rebase] Align vLLM-Omni with vLLM v0.31.0 (#8459) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-10-05 14:11:01 +08:00
Scott Ye 1f47fa2f29 [Model][Core] Avoid unused CosyVoice3 payloads and redundant AR output work (#8473) 
Signed-off-by: Sorbett <sorbet111@163.com>
 2026-10-05 04:52:19 +00:00
汪志鹏 5ca23106fd [Model] Add SheetSage2 preprocessing and YuE2 score handoff (#8371) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-10-05 11:09:37 +08:00
MaciejBalaNVBartosz Stefaniak汪志鹏
 		bbd539d2b6 [feature] Enable multi-node inference through Ray backend (#7938) 
Signed-off-by: Maciej Bala <mbala@nvidia.com>
Co-authored-by: Bartosz Stefaniak <bstefaniak@nvidia.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-10-05 02:53:36 +00:00
Yueqian Lin c1e84ce946 [Bugfix] Wire output buffers into duplex model test harnesses (#8489) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-10-04 23:55:19 +00:00
bcsdhjew f781f7bb08 [Core][Frontend] Bound duplex output delivery and reject cancelled audio (#7644) 
Signed-off-by: Nolen Liang <nliang@nvidia.com>
Signed-off-by: bcsdhjew <nliang@nvidia.com>
 2026-10-04 14:06:04 -07:00
SunandOpenAI Codex 3f20252151 [Model] Fuse MammothModa2 QK norm and RoPE (#7969) 
Signed-off-by: levius <2114377220@qq.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-10-05 02:44:45 +08:00
TonybotNi 855254d4af [Bugfix] Validate MammothModa2 text-to-image dimensions (#7482) 
Signed-off-by: mudrobot <mudrobot@foxmail.com>
 2026-10-05 02:43:35 +08:00
quan-yi-aiandHongsheng Liu 65f5a1d5b3 [Bugfix][Examples][MammothModa2] Serve T2I via LLM-typed DiT stage (#7199) + benchmark harness and profiler wiring (#7293) 
Signed-off-by: quan-yi-ai <82917002+quan-yi-ai@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-10-05 02:42:58 +08:00
hurukawa b9cf74352a [Perf] Support continuous batching for MammothModa2 (#7954) 
Signed-off-by: nagisa-kun <1434936049@qq.com>
Signed-off-by: nagisa <1434936049@qq.com>
 2026-10-05 02:22:37 +08:00
hurukawa c9c478dd07 [Perf] Support TeaCache for MammothModa2 (#5357) 
Signed-off-by: nagisa-kun <1434936049@qq.com>
Signed-off-by: nagisa <1434936049@qq.com>
 2026-10-05 01:25:56 +08:00
andyluo7 c9bee166fb ci: route AR paged attention tests to GPU lanes (#8346) 
Signed-off-by: andyluo7 <andy.luo@amd.com>
 2026-10-04 23:13:21 +13:00
Rahul SteigerandRahul Steiger 4934741df9 [Bugfix] Add TRTLLM attention custom op and execution contract (#7214) 
Signed-off-by: Rahul Steiger <rsteiger@aws-cmh-slurm-1-vscode-04.cm.cluster>
Signed-off-by: Rahul Steiger <rsteiger@nvidia.com>
Co-authored-by: Rahul Steiger <rsteiger@aws-cmh-slurm-1-vscode-04.cm.cluster>
 2026-10-04 09:33:51 +00:00
nodeeeeee b65f97bc48 [Feature] Add MiniMax H3 res_multistep sampler (#8378) 
Signed-off-by: nodeeeeee <zhangkai.nodeee@gmail.com>
 2026-10-04 07:26:19 +00:00
YAN YIXINandClaude Opus 5.5 148cb943ce [Bugfix][CI/Build] Fix layerwise offload shutdown and enable it for the LTX-2 distilled T2V accuracy test (#8453) 
Signed-off-by: nikiwang-125 <yixinyan79@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-04 07:25:51 +00:00
andyluo7 3ab726c126 test: bound CPU vocoder reference threads (#8347) 
Signed-off-by: andyluo7 <andy.luo@amd.com>
 2026-10-04 19:19:22 +13:00
Sy03 1c1b831f58 [Model] Batch Qwen3-TTS Talker prompt projection and settled-row preprocessing (#8477) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-10-04 05:08:24 +00:00
Sy03 dd765fbf64 [Model] Select CosyVoice3 streaming defaults by device capability (#8433) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-10-03 21:59:36 -07:00
andyluo7 b171731c81 fix: preserve CosyVoice full payload and live conditioning (#8343) 
Signed-off-by: andyluo7 <andy.luo@amd.com>
 2026-10-04 16:03:27 +13:00
andyluo7 09d704907f fix: preserve Torch seeded sampling on ROCm (#8341) 
Signed-off-by: andyluo7 <andy.luo@amd.com>
 2026-10-04 16:03:21 +13:00
andyluo7 c150600896 [Bugfix] Route MOSS reference attention through ROCm FlashAttention (#8463) 
Signed-off-by: andyluo7 <andy.luo@amd.com>
 2026-10-04 16:02:22 +13:00
andyluo7 ee8fdab1dc [CI/Build] Fix MOSS CPU tests on ROCm workers (#8447) 
Signed-off-by: andyluo7 <andy.luo@amd.com>
 2026-10-03 09:13:51 +08:00
Sy03andYueqian Lin 3bc3f1a7d0 [Model] Optimize MOSS Local 1.5 MRV2 serving and progressive chunks (#8213) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
 2026-10-02 18:03:14 +00:00
Zheng Wengangandamy-why-3459 527982d886 [Bugfix] Skip async-chunk SHM put for stage-0-final requests (#7245) 
Signed-off-by: ZhengWG <zwg0606@gmail.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-10-02 09:14:39 -07:00
Sy03 2ba5074ca0 [Model] Optimize CosyVoice3 packed streaming with bounded HiFT graphs (#8420) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-10-02 08:49:50 -07:00
Yueqian Lin 4025bb30f6 [Perf][AuK] Graph the encoder prefill, cache reference latents and fuse the codec activation (#8305) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-10-02 15:43:55 +00:00
Yueqian Lin 13c66da752 [Perf][AuK] Precompute adaLN modulations per request and run the conv position embedding as one GEMM (#8328) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-10-02 08:12:29 -07:00
Nick CaoandCodex 0e6791e9c4 [Bugfix] Initialize eager state in Higgs test fixture (#8426) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-02 10:22:52 -04:00
Zheng Wengangandamy-why-3459 eb1699bfca [FEAT][Fronted] Support incremental prefill for realtime video streaming (#5894) 
Signed-off-by: ZhengWG <zwg0606@gmail.com>
Signed-off-by: Zheng Wengang <zwg0606@gmail.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-10-02 22:18:51 +08:00
Sy03 e372aa3a41 [Bugfix] Restore Fish Speech and CosyVoice3 audio quality at high concurrency (#8422) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-10-02 07:11:04 -07:00
Sy03princeprideClaude Opus 5.5
 		ae58c2d8b9 [Model] YuE2: async single-stage synthesis and CUDA graph acceleration (#8407) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: Wang Zhipeng <wangzhipeng628@gmail.com>
Co-authored-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 05:17:06 -07:00
linzhenpl07Claude Opus 5Zhou Taichang
 		b10270a01c [AR-Diffusion] Page in kernel-legal blocks so the default resolution can run (#6481) 
Signed-off-by: linzhenpl07 <linzhenpl07@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2026-10-02 20:05:04 +08:00
Sy03 e84014dffb [Model] Add single-stage Qwen3-TTS with in-Talker streaming codec decode (#8259) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-10-02 21:34:30 +10:00
Rahul Steiger 596487fe62 [Core] Prototype diffusion attention execution contracts with FA4 (#7379) 2026-10-02 17:58:47 +08:00
MaciejBalaNVand汪志鹏 97d6c693a6 [feature] Add HSDP pre-sharded option (#7948) 
Signed-off-by: Maciej Bala <mbala@nvidia.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-10-02 09:35:09 +00:00
汪志鹏 b742f86136 [CI/Build] Restore streaming executor fixture contract (#8418) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-10-02 01:45:56 -07:00
andyluo7 53c7340668 test: measure offload memory inside diffusion workers (#8189) 
Signed-off-by: andyluo7 <andy.luo@amd.com>
 2026-10-02 14:07:20 +08:00
Sy03 d93e1a8e09 [Model] Optimize Qwen3-Omni MRv2 handoff, code prediction and audio decoding (#8223) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-10-01 22:22:54 -07:00