anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
3,412 Commits 235 Branches 31 Tags
757dd3b2c4eb8a5efaa28503f0e91e8cb19d03ce
Commit Graph
226 Commits
Author SHA1 Message Date
tzhouam 16f73d0da2 fix: match vLLM 0.31 generation profiling API 
Accept the upstream randomize_inputs profiling flag while preserving structured codec inputs during generation-stage dummy runs.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-10-03 21:23:18 +08:00
tzhouam 008fa25263 Merge main into dev/vllm-align for vLLM 0.31 release 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-10-03 21:05:17 +08:00
Sy03princeprideClaude Opus 5.5
 		ae58c2d8b9 [Model] YuE2: async single-stage synthesis and CUDA graph acceleration (#8407) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: Wang Zhipeng <wangzhipeng628@gmail.com>
Co-authored-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 05:17:06 -07:00
tzhouam 2721f16ca8 rebase: align vllm-omni with vLLM ac9126e58aa7 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-10-02 10:37:50 +08:00
Sy03 56763cd6c3 [Model] Enable Higgs Audio v3 MRV2 streaming and fix mixed-prefill capture bounds (#8226) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-30 11:32:27 +00:00
Sy03 001f8a80b9 [Core] Extend the shared multi-stage execution framework for mixed V1/MRv2 pipelines (#8184) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-29 07:40:57 -07:00
jingchengtian 12e9280010 [Perf][TTS] Run talker_mtp eagerly when a batch carries explicit seeds (#8009) 
Signed-off-by: jingchengtian <tjc1995@126.com>
 2026-09-28 13:31:48 -04:00
BeatSeatandamy-why-3459 c8a1419e02 [Performance][MiniCPM-o] Zero-copy block-aligned KV sliding window with in-place Re-RoPE for duplex Stage-0 (#7821) 
Signed-off-by: BeatSeat <wendavid552@gmail.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-09-28 10:11:54 +08:00
NATUREandliuqihao 7a8d956ece [Bugfix] Fix MiniCPM-o 4.5 native duplex speech alignment (#7974) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
Co-authored-by: liuqihao <liuqihao970610@gmail.com>
 2026-09-24 10:04:28 +00:00
Joshna-Medisetty 5e6b9047c9 [Bugfix] Drain the omni connector output on the main thread in the AR runner (#7750) 
Signed-off-by: Joshna Medisetty <joshna.medisetty@intel.com>
 2026-09-23 11:36:56 -07:00
Yueqian Lin 3f473a358b [Model] Accelerate Breeze-TTS-2 non-streaming inference (#7752) 2026-09-22 17:10:15 -07:00
Zhou TaichangClaude Opus 5Gao Han
 		59db6a4285 [Rebase] Rebase to vLLM 0.30.0 (#7820) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-09-22 15:52:01 +00:00
Sy03 473da2cf82 [Perf][Qwen3-TTS] Enable event-driven orchestration by default (#7088) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-21 12:09:19 -07:00
Sy03 a9e2117f5e [Core][Model] Enable Qwen3-TTS MRV2 and optimize the TTS pipeline (#7781) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-20 23:03:08 -07:00
DanaerLeeandzijianc2 16734bbdf3 [Bugfix] Route text-only chat as per-request comprehension in HunyuanImage3 AR sampler (#6111) 
Signed-off-by: MrlixiangWE <mrdanaer@gmail.com>
Co-authored-by: zijianc2 <157244773+zijianc2@users.noreply.github.com>
 2026-09-20 15:23:53 +00:00
Wu JIAZHENAcerak01-fyamy-why-3459
 		102ba716bf [Feat] Add native Mooncake KV transfer from AR to DiT (#7166) 
Signed-off-by: asukaqaq-s <1311722138@qq.com>
Signed-off-by: Acerak01-fy <wfy2003324@163.com>
Co-authored-by: Acerak01-fy <wfy2003324@163.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-09-20 14:44:55 +00:00
Zheng Wengangamy-why-3459Hongsheng Liu
 		470ec60848 [Core] Refactor omni prefix cache into Manager/Controller (#6654) 
Signed-off-by: ZhengWG <zwg0606@gmail.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-19 21:23:59 +08:00
Sy03 4200768f59 [Frontend] Scale Omni serving across API processes (#6923) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-16 22:29:31 -07:00
Clodagh Walsh a1a234a0b4 [Voxtral] Raise BadRequest for ref_audio (#7012) 
Signed-off-by: Clodagh Walsh <clodaghwalsh17@gmail.com>
 2026-09-15 18:26:08 -07:00
Deep Shah 5761ad5516 [Bugfix] Restore the async PP sampled-token broadcast in the AR runner (#7393) 
Signed-off-by: Deep Shah <deep@socratic.co>
 2026-09-14 20:41:03 -07:00
Deep Shah 2f1a18e69f [Bugfix][Qwen3-Omni] Fix synchronous Thinker pipeline parallelism (#7345) 
Signed-off-by: Deep Shah <deep@socratic.co>
 2026-09-14 10:02:21 -07:00
Yueqian Lin 5c69fc2a8a [Refactor] Declare model-local KV held outside the paged manager (#6171) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-09-11 08:44:32 +08:00
Zhou TaichangandClaude Opus 5 aff7d64948 [Rebase] Rebase to vLLM 0.29.0 (#7230) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-10 12:08:28 +08:00
Sy03 b02862beee [Model] Optimize MOSS-TTS Local batched execution and streaming codec (#7202) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-09 13:02:12 -07:00
NATURE 5b002727b0 [Core] Split Omni connector model runner mixin (#6903) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
 2026-09-09 13:49:43 +08:00
Zhou TaichangandClaude Fable 5 142151fef5 Parallel stage initialization (admission + SH/EX device locks) (Worker refactor 1/N) (#5224) 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-09-05 22:22:41 +08:00
chickeytonClaude Fable 5Gao Han
 		5d7fdf9350 [Core] Graduate MiniCPM-o 4.5 and PersonaPlex full-duplex serving out of experimental (#6196) 
Signed-off-by: chickeyton <ngton2014@gmail.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-09-05 08:20:21 +00:00
Inesh Reddy ChappidiandGao Han 66d938100b [Bugfix] Give exact-shape codec models an unpadded input_ids view (#6775) 
Signed-off-by: Inesh Reddy <ineshreddy249@gmail.com>
Signed-off-by: IneshReddy249 <ineshreddy249@gmail.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-09-04 15:28:49 -07:00
akshatvishu c41925689d [BugFix][Higgs-Audio-V3] Fix transcript and concurrent sampling tests (#6422) 
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
 2026-08-30 08:52:15 +00:00
Nguyen Kha Nhat LongCursorSamit
 		bb5bf8fd0a [Bugfix] Fix colocate-async sleep admission race and deliver AR abort tokens (#6367) 
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Signed-off-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-08-29 09:11:39 +00:00
Wallbreazzz 4ad455e61e [Feature][TTS] adaptive chunk ramp (Phase 2 buffer-feedback controller) (#6001) 
Signed-off-by: Wallbreazzz <110282866+Wallbreazzz@users.noreply.github.com>
 2026-08-28 02:11:05 -07:00
Hongsheng Liuandhsliuustc0106 4245fd6955 [Fix] Restore MammothModa2 support (#6694) 
Signed-off-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
Co-authored-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
 2026-08-28 08:32:32 +08:00
Zhou TaichangandClaude Opus 5 651b03225b [Rebase] Rebase to vllm 0.28.0 (#6606) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-27 06:12:55 +00:00
Sy03andsy03 432b2c6ca3 [Model] Add native full-duplex Nemotron VoiceChat serving (#6089) 
Signed-off-by: Sysy <1370724210@qq.com>
Signed-off-by: sy03 <137072421@qq.com>
Co-authored-by: sy03 <137072421@qq.com>
 2026-08-26 13:19:20 -07:00
wy17003andHongsheng Liu b454e34966 [Refactor][2/N Scheduler]Replace _FULL_PAYLOAD_INPUT_STAGES with a resolved stage transport capability. (#6149) 
Signed-off-by: wy17003 <yaow17003@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-25 15:24:59 +08:00
wangyu 1e2a6446f7 [CI] Remove AudioX and MammothModa2 support (#6353) 
Signed-off-by: wangyu <410167048@qq.com>
 2026-08-23 04:22:05 +00:00
NATUREandamy-why-3459 4f7c08c0eb [Bugfix][MiniCPM-o] Fix async-chunk snapshot replacement and prompt cleanup (#6406) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-08-21 16:37:21 +08:00
Inesh Reddy Chappidi cd3867c373 [Model] Suppress silence codec tokens for the first N Qwen3-TTS decod… (#5048) 
Signed-off-by: Inesh Reddy <ineshreddy249@gmail.com>
 2026-08-20 09:22:37 -07:00
Zhou TaichangandClaude Fable 5 10906e3a6d [Refactor] Worker/ModelRunner runner correctness fixes (G2/N) (#5452) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-20 16:47:50 +08:00
amy-why-3459andCursor 9c61d5b67b [Bugfix][MiniCPM-o] Stop Talker mid-utterance truncation and load cod… (#6346) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-20 11:50:07 +08:00
Yukim1 6f55a06fab [Docs] Remove stale stage config references (#6270) 
Signed-off-by: zwhzzz0821 <2831474076@qq.com>
 2026-08-20 07:09:12 +08:00
Yueqian LinandHongsheng Liu 611c083587 [BugFix][TTS] R1.1-R1.4: turn silent async-chunk failures into visible ones (#6033) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-17 10:51:17 +08:00
mjZhaoElaineClaude Opus 5Yueqian Lin
 		35cacf020c [Model] Add Gepard-1.0 native-AR FSQ/NanoCodec TTS (offline inference) (#5666) 
Signed-off-by: Mengjie Zhao <zmj0129@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-08-14 22:06:05 -07:00
Yuekai ZhangandClaude Fable 5 5be7541d01 [Model] Support NVIDIA-NemotronLabs-VoiceChat-11B offline speech-to-speech (#5842) 
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-12 16:25:48 +08:00
王金旭xuruiClaude Opus 4.8
 		9b0df3909a [Feature] timestamps for TTS via a forced-aligner pooling (#4795) 
Signed-off-by: wjinxu <1299461899@qq.com>
Co-authored-by: xurui <xurui1@modelbest.onaliyun.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-08-12 05:33:34 +00:00
Sy03andYueqian Lin 58007c0084 [New Model] Support IndexTTS 2.5 (#5957) 
Signed-off-by: Sysy <1370724210@qq.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-08-11 23:00:35 +00:00
Zhou TaichangandClaude Opus 5 2ce15ab7a0 [Rebase] Rebase to vllm 0.27.0 (#5976) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-11 15:23:46 +00:00
R0CKSTARHongsheng Liuamy-why-3459
 		e04210d690 [Hardware][MUSA] Enable Qwen3-Omni ModelOpt FP8 inference (#5671) 
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-08-11 17:19:09 +08:00
Caspian ChenandSYLAR 593b404539 [Quantization][Qwen2.5-Omni] Support thinker-only ModelOpt NVFP4 W4A4 checkpoints (#5073) 
Signed-off-by: Caspian443 <scrisis843@gmail.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-08-09 02:45:57 +08:00
Yueqian LinandSy0307 e788ef6e76 [Feature] PersonaPlex (Moshi-based full-duplex S2S): native vLLM port + duplex serving (#4771) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Signed-off-by: Sy0307 <24841857+Sy0307@users.noreply.github.com>
Co-authored-by: Sy0307 <24841857+Sy0307@users.noreply.github.com>
 2026-08-04 20:10:33 -07:00