anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
3,412 Commits 235 Branches 31 Tags
757dd3b2c4eb8a5efaa28503f0e91e8cb19d03ce
Commit Graph
102 Commits
Author SHA1 Message Date
tzhouam b256e1ec4e Adapt Moss sampling and runner fixtures to vLLM 0.31 interfaces 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-10-03 22:22:39 +08:00
tzhouam 16f73d0da2 fix: match vLLM 0.31 generation profiling API 
Accept the upstream randomize_inputs profiling flag while preserving structured codec inputs during generation-stage dummy runs.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-10-03 21:23:18 +08:00
tzhouam 008fa25263 Merge main into dev/vllm-align for vLLM 0.31 release 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-10-03 21:05:17 +08:00
Sy03andYueqian Lin 3bc3f1a7d0 [Model] Optimize MOSS Local 1.5 MRV2 serving and progressive chunks (#8213) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
 2026-10-02 18:03:14 +00:00
Sy03princeprideClaude Opus 5.5
 		ae58c2d8b9 [Model] YuE2: async single-stage synthesis and CUDA graph acceleration (#8407) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: Wang Zhipeng <wangzhipeng628@gmail.com>
Co-authored-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 05:17:06 -07:00
tzhouam 2721f16ca8 rebase: align vllm-omni with vLLM ac9126e58aa7 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-10-02 10:37:50 +08:00
Youyuan Liandprincepride 423f34326e [Model] Add YuE2-3B text-to-music (model + /v1/audio/speech adapter) (#7886) 
Signed-off-by: Youyuan Li <105433145+Darcy-Lee@users.noreply.github.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: princepride <wangzhipeng628@gmail.com>
 2026-10-01 18:04:31 +08:00
Sy03 56763cd6c3 [Model] Enable Higgs Audio v3 MRV2 streaming and fix mixed-prefill capture bounds (#8226) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-30 11:32:27 +00:00
boatmanHongsheng LiuYueqian Lin
 		35cf6ac355 [Feat][Perf][AuK]Support Online Serving and DiT CUDAGraph (#7469) 
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
 2026-09-30 00:18:38 +00:00
Sy03 001f8a80b9 [Core] Extend the shared multi-stage execution framework for mixed V1/MRv2 pipelines (#8184) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-29 07:40:57 -07:00
Gao Han a8576ccb72 [Bugfix] Fix MiniCPM-o 4.5 seeded turns and window E2E inputs (#8160) 
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-09-26 01:42:53 +08:00
bcsdhjew fe9e22df3d [Bugfix] Restore MiniCPM duplex Seed-TTS startup (#8152) 
Signed-off-by: Nolen Liang <nliang@nvidia.com>
Signed-off-by: bcsdhjew <nliang@nvidia.com>
 2026-09-25 14:52:46 +00:00
NATUREandliuqihao 7a8d956ece [Bugfix] Fix MiniCPM-o 4.5 native duplex speech alignment (#7974) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
Co-authored-by: liuqihao <liuqihao970610@gmail.com>
 2026-09-24 10:04:28 +00:00
Joshna-Medisetty 5e6b9047c9 [Bugfix] Drain the omni connector output on the main thread in the AR runner (#7750) 
Signed-off-by: Joshna Medisetty <joshna.medisetty@intel.com>
 2026-09-23 11:36:56 -07:00
Yuan WuandHongsheng Liu cb5f508212 [Feature] Transfer AR-to-diffusion payloads over NIXL (#6264) 
Signed-off-by: yuanwu <yuan.wu@intel.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-23 18:26:53 +08:00
Yueqian Lin 3f473a358b [Model] Accelerate Breeze-TTS-2 non-streaming inference (#7752) 2026-09-22 17:10:15 -07:00
0z5aand0z5a 8bc0c38b90 [Bugfix] Add opt-in MiniCPM-o 4.5 Stage-0 sliding window (#7631) 
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@uni-tuebingen.de>
Co-authored-by: 0z5a <dezhen.lu@uni-tuebingen.de>
 2026-09-21 18:34:22 -07:00
Sy03 473da2cf82 [Perf][Qwen3-TTS] Enable event-driven orchestration by default (#7088) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-21 12:09:19 -07:00
Sy03 a9e2117f5e [Core][Model] Enable Qwen3-TTS MRV2 and optimize the TTS pipeline (#7781) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-20 23:03:08 -07:00
DanaerLeeandzijianc2 16734bbdf3 [Bugfix] Route text-only chat as per-request comprehension in HunyuanImage3 AR sampler (#6111) 
Signed-off-by: MrlixiangWE <mrdanaer@gmail.com>
Co-authored-by: zijianc2 <157244773+zijianc2@users.noreply.github.com>
 2026-09-20 15:23:53 +00:00
Zheng Wengangamy-why-3459Hongsheng Liu
 		470ec60848 [Core] Refactor omni prefix cache into Manager/Controller (#6654) 
Signed-off-by: ZhengWG <zwg0606@gmail.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-19 21:23:59 +08:00
Sunamy-why-3459Yueqian Lin
 		7b4fc7ce2e [Model] Migrate MammothModa2 DiT to the shared diffusion runtime (#7134) 
Signed-off-by: levius <2114377220@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
 2026-09-18 13:29:18 -07:00
chickeytonandClaude Opus 5 99ff4f3070 [Core][Frontend] Unified Full-duplex Framework (#7413) 
Signed-off-by: chickeyton <ngton2014@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 15:35:36 +08:00
Mengjie ZhaoandCursor c17126c219 [Model] Add Gepard-1.0 OpenAI /v1/audio/speech serving (#7499) 
Signed-off-by: Mengjie Zhao <zmj0129@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-15 14:56:51 -07:00
Deep Shah 5761ad5516 [Bugfix] Restore the async PP sampled-token broadcast in the AR runner (#7393) 
Signed-off-by: Deep Shah <deep@socratic.co>
 2026-09-14 20:41:03 -07:00
Deep Shah 2f1a18e69f [Bugfix][Qwen3-Omni] Fix synchronous Thinker pipeline parallelism (#7345) 
Signed-off-by: Deep Shah <deep@socratic.co>
 2026-09-14 10:02:21 -07:00
Yuan Wu 6753bbd18c [Core] Add NIXL omni connector (#6093) 
Signed-off-by: yuanwu <yuan.wu@intel.com>
Signed-off-by: Yuan Wu <yuan.wu@intel.com>
 2026-09-13 16:35:02 +08:00
Sy03 b02862beee [Model] Optimize MOSS-TTS Local batched execution and streaming codec (#7202) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-09 13:02:12 -07:00
NATURE 5b002727b0 [Core] Split Omni connector model runner mixin (#6903) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
 2026-09-09 13:49:43 +08:00
NancyFyongandNancyFyong e74fe3c6e8 [Model] Add Audio8 TTS Preview 0.6B (DualAR, 44.1 kHz codec) (#6157) 
Signed-off-by: NancyFyong <NancyFyong@users.noreply.github.com>
Co-authored-by: NancyFyong <NancyFyong@users.noreply.github.com>
 2026-09-08 19:00:25 -07:00
Xiaobaixia 615f97b9c7 [Feature] Define the runner preprocess phase contract and regression coverage (#7136) 
Signed-off-by: xiaobaixia222 <289982752@qq.com>
 2026-09-06 17:20:04 +00:00
Zhou TaichangandClaude Fable 5 142151fef5 Parallel stage initialization (admission + SH/EX device locks) (Worker refactor 1/N) (#5224) 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-09-05 22:22:41 +08:00
chickeytonClaude Fable 5Gao Han
 		5d7fdf9350 [Core] Graduate MiniCPM-o 4.5 and PersonaPlex full-duplex serving out of experimental (#6196) 
Signed-off-by: chickeyton <ngton2014@gmail.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-09-05 08:20:21 +00:00
Inesh Reddy ChappidiandGao Han 66d938100b [Bugfix] Give exact-shape codec models an unpadded input_ids view (#6775) 
Signed-off-by: Inesh Reddy <ineshreddy249@gmail.com>
Signed-off-by: IneshReddy249 <ineshreddy249@gmail.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-09-04 15:28:49 -07:00
akshatvishu c41925689d [BugFix][Higgs-Audio-V3] Fix transcript and concurrent sampling tests (#6422) 
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
 2026-08-30 08:52:15 +00:00
Nguyen Kha Nhat LongCursorSamit
 		bb5bf8fd0a [Bugfix] Fix colocate-async sleep admission race and deliver AR abort tokens (#6367) 
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Signed-off-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-08-29 09:11:39 +00:00
amy-why-3459 da94c3ebe1 [Bugfix][MiniCPM-o] Bind omni-duplex video frames to the unit they close (#6404) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-08-28 16:58:20 +08:00
Hongsheng Liuandhsliuustc0106 4245fd6955 [Fix] Restore MammothModa2 support (#6694) 
Signed-off-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
Co-authored-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
 2026-08-28 08:32:32 +08:00
Zhou TaichangandClaude Opus 5 651b03225b [Rebase] Rebase to vllm 0.28.0 (#6606) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-27 06:12:55 +00:00
Sy03andsy03 432b2c6ca3 [Model] Add native full-duplex Nemotron VoiceChat serving (#6089) 
Signed-off-by: Sysy <1370724210@qq.com>
Signed-off-by: sy03 <137072421@qq.com>
Co-authored-by: sy03 <137072421@qq.com>
 2026-08-26 13:19:20 -07:00
wangyu 1e2a6446f7 [CI] Remove AudioX and MammothModa2 support (#6353) 
Signed-off-by: wangyu <410167048@qq.com>
 2026-08-23 04:22:05 +00:00
NATUREandamy-why-3459 4f7c08c0eb [Bugfix][MiniCPM-o] Fix async-chunk snapshot replacement and prompt cleanup (#6406) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-08-21 16:37:21 +08:00
Zhou TaichangandClaude Fable 5 10906e3a6d [Refactor] Worker/ModelRunner runner correctness fixes (G2/N) (#5452) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-20 16:47:50 +08:00
amy-why-3459andCursor 9c61d5b67b [Bugfix][MiniCPM-o] Stop Talker mid-utterance truncation and load cod… (#6346) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-20 11:50:07 +08:00
boatman 65b3d41b3a [Migrate] Move sampling parameter overrides from legacy dispatches to TTS adapters (#5272) 
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com>
 2026-08-16 21:14:26 -07:00
Sy03andYueqian Lin 58007c0084 [New Model] Support IndexTTS 2.5 (#5957) 
Signed-off-by: Sysy <1370724210@qq.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-08-11 23:00:35 +00:00
Zhou TaichangandClaude Opus 5 2ce15ab7a0 [Rebase] Rebase to vllm 0.27.0 (#5976) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-11 15:23:46 +00:00
R0CKSTARHongsheng Liuamy-why-3459
 		e04210d690 [Hardware][MUSA] Enable Qwen3-Omni ModelOpt FP8 inference (#5671) 
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-08-11 17:19:09 +08:00
Yueqian LinandSy0307 e788ef6e76 [Feature] PersonaPlex (Moshi-based full-duplex S2S): native vLLM port + duplex serving (#4771) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Signed-off-by: Sy0307 <24841857+Sy0307@users.noreply.github.com>
Co-authored-by: Sy0307 <24841857+Sy0307@users.noreply.github.com>
 2026-08-04 20:10:33 -07:00
Shenglei FuandClaude f24a6165ff [Test] Prefix-cache passthrough test coverage prerequisite for W2 (#4855) (#5310) 
Signed-off-by: Shenglei Fu <sfu@confluent.io>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-04 20:03:57 -07:00