anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
3,465 Commits 235 Branches 31 Tags
main
Commit Graph
79 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Sy03 7be49aa0ce [Model] Default MOSS Local 1.5 to adaptive CUDA MRV2 with MPS (#8438) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-10-06 08:20:46 -07:00
Sy03 023716ecc4 [Bugfix] Do not assume a stream decoder in MRv2 eager-MTP preprocess (#8484) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-10-05 09:29:38 -04:00
Sy03 1c1b831f58 [Model] Batch Qwen3-TTS Talker prompt projection and settled-row preprocessing (#8477) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-10-04 05:08:24 +00:00
Sy03 dd765fbf64 [Model] Select CosyVoice3 streaming defaults by device capability (#8433) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-10-03 21:59:36 -07:00
Sy03andYueqian Lin 3bc3f1a7d0 [Model] Optimize MOSS Local 1.5 MRV2 serving and progressive chunks (#8213) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
 2026-10-02 18:03:14 +00:00
Sy03 2ba5074ca0 [Model] Optimize CosyVoice3 packed streaming with bounded HiFT graphs (#8420) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-10-02 08:49:50 -07:00
Sy03 e372aa3a41 [Bugfix] Restore Fish Speech and CosyVoice3 audio quality at high concurrency (#8422) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-10-02 07:11:04 -07:00
Sy03princeprideClaude Opus 5.5
 		ae58c2d8b9 [Model] YuE2: async single-stage synthesis and CUDA graph acceleration (#8407) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: Wang Zhipeng <wangzhipeng628@gmail.com>
Co-authored-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 05:17:06 -07:00
Sy03 e84014dffb [Model] Add single-stage Qwen3-TTS with in-Talker streaming codec decode (#8259) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-10-02 21:34:30 +10:00
Sy03 d93e1a8e09 [Model] Optimize Qwen3-Omni MRv2 handoff, code prediction and audio decoding (#8223) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-10-01 22:22:54 -07:00
Sy03 56763cd6c3 [Model] Enable Higgs Audio v3 MRV2 streaming and fix mixed-prefill capture bounds (#8226) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-30 11:32:27 +00:00
Sy03 5af3a16526 [Model] Add MiniCPM-o 4.5 turn-mode MRv2 and batched audio inference (#8222) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-29 11:41:39 -07:00
Sy03andzack 1ea0ee3b38 [Model] Add opt-in packed Flow and batched HiFT inference for CosyVoice3 (#8224) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: zack <huixindaddy@yahoo.com>
Co-authored-by: zack <huixindaddy@yahoo.com>
 2026-09-29 15:48:36 +00:00
Sy03 001f8a80b9 [Core] Extend the shared multi-stage execution framework for mixed V1/MRv2 pipelines (#8184) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-29 07:40:57 -07:00
Sy03 e1303e5067 [Model] Optimize Qwen3-TTS streaming throughput and first audio on CUDA (#8163) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-26 22:51:04 +08:00
Sy03 d31a2a3ba8 [Bugfix] Fix Nemotron VoiceChat duplex nightly handshake (#7884) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-25 17:25:15 +00:00
Sy03 5d0963b3dc [Bugfix] Fix MRV2 payload slicing with CUDA Graph padding (#8159) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-25 16:07:22 +00:00
Sy03 551a03102f [Bugfix][Qwen3-TTS] Decode full payload from model buffer (#8132) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-25 10:03:58 +08:00
Sy03 8953d082c0 [Bugfix] Fix Qwen3-TTS prefill probe on MRv2 runner (#8065) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-23 10:17:53 +00:00
Sy03 5634f03eb1 [Bugfix] Restore MRV2 full-payload transport for offline Qwen3-TTS (#8010) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-23 01:51:20 +00:00
Sy03 c7cd37b61e [Model] Reduce MOSS TTS reference audio preparation overhead (#7725) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-22 12:19:18 +08:00
Sy03 473da2cf82 [Perf][Qwen3-TTS] Enable event-driven orchestration by default (#7088) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-21 12:09:19 -07:00
Sy03 60a933b9ab [Model] Default Qwen3-TTS to experimental Model Runner V2 (#7930) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-21 18:56:06 +00:00
Sy03 a9e2117f5e [Core][Model] Enable Qwen3-TTS MRV2 and optimize the TTS pipeline (#7781) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-20 23:03:08 -07:00
Sy03 4200768f59 [Frontend] Scale Omni serving across API processes (#6923) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-16 22:29:31 -07:00
Sy03 333f14e97e [Bugfix][MiniCPM-o] Align structured chat content with native omni rendering (#7344) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-10 11:38:11 +08:00
Sy03 b02862beee [Model] Optimize MOSS-TTS Local batched execution and streaming codec (#7202) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-09 13:02:12 -07:00
Sy03Nathan PriceEric
 		21dd4b33ac [Bugfix][MiniCPM-o] Stabilize native duplex streaming and Stage-1 handoffs (#6529) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Nathan Price <nathan@abridge.com>
Co-authored-by: Eric(Qun) <186020953+0xtoward@users.noreply.github.com>
 2026-08-29 13:05:24 -07:00
Sy03 3adee400a0 [Bugfix][Qwen3-TTS] Handle codec generations that exhaust their token budget (#6728) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-08-29 08:59:05 -07:00
Sy03 009b992c5b [Bugfix][MiniCPM-o] Isolate duplex handoff failures and barge-in (#6170) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-08-28 15:46:41 +08:00
Sy03andsy03 432b2c6ca3 [Model] Add native full-duplex Nemotron VoiceChat serving (#6089) 
Signed-off-by: Sysy <1370724210@qq.com>
Signed-off-by: sy03 <137072421@qq.com>
Co-authored-by: sy03 <137072421@qq.com>
 2026-08-26 13:19:20 -07:00
Sy03andSy03 480e10ba9c [CI/Build] Match perf warmups to benchmark concurrency (#6356) 
Signed-off-by: Sy03 <24841857+Sy0307@users.noreply.github.com>
Co-authored-by: Sy03 <24841857+Sy0307@users.noreply.github.com>
 2026-08-20 00:25:49 -07:00
Sy03 26e003715c [Perf] Enable async schedule for MOSS-TTS (#6241) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-08-19 06:36:19 +00:00
Sy03 dbc0dd6d85 [Bugfix][Higgs-Audio-V3] Disable XQA decode (#6068) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: Sysy <1370724210@qq.com>
 2026-08-13 10:00:50 -07:00
Sy03andYueqian Lin 58007c0084 [New Model] Support IndexTTS 2.5 (#5957) 
Signed-off-by: Sysy <1370724210@qq.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-08-11 23:00:35 +00:00
Sy03 09c6ad4204 [CI/Build][MiniCPM-o] Allow auto-response before input commit (#5567) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-07-30 14:14:19 +00:00
Sy03andHongsheng Liu 0feb92bdfa [Bugfix][Minicpm] Support full-duplex three-stage serving (#5380) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-07-29 01:58:34 +08:00
Sy03 e043818abf [Bugfix][Frontend] Remove async-chunk guard from realtime API (#5383) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-07-25 09:20:47 +08:00
Sy03lishunyang12Yueqian LinStormMelodyGao HanHongsheng Liu
 		05b794f29b [Full Duplex] Feat: Support Full-Duplex realtime runtime & add MiniCPM-o 4.5 demo (#3907) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: StormMelody <22371184@zju.edu.cn>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-07-23 16:45:18 +08:00
Sy03 aa21412c01 [Fix][VoxCPM2] Follow up unified graph config gate (#4878) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-07-04 19:58:21 +00:00
Sy03andYueqian Lin 1b31f96240 [Bugfix] Fix Qwen3-TTS no-async-chunk payload splitting (#4870) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
 2026-07-04 19:57:55 +00:00
Sy03andYueqian Lin 0798b0bf03 [Bugfix][Qwen3-TTS] Seed residual MTP sampling (#4869) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
 2026-07-04 17:47:11 +00:00
Sy03andYueqian Lin 9eaca72382 [Perf][VoxCPM2] Add unified decode graph optimization (#4606) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-07-04 05:30:48 +00:00
Sy03Claude Sonnet 4.6Yueqian Lin
 		f11b32a42e [Bugfix] Qwen3-TTS talker handles multi-token replay on resume (#4559) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-07-02 04:46:05 +00:00
Sy03 c39ac9fffd [Bugfix] Stream Qwen3-TTS WebSocket input as one request (#4731) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-06-26 07:48:30 +00:00
Sy03Yueqian LinHongsheng Liu
 		e27fd46ab9 [Perf] Optimize Higgs Audio v3 serving (#4204) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-06-11 13:29:31 -07:00
Sy03 9c9d4ed4a8 [Fix][HiggsAudioV3] Fix ramp-down off-by-one crash and buffer state bugs in Stage0 talker (#4219) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-06-08 17:33:20 +00:00
Sy03Yueqian LinHongsheng LiuGao Han
 		c33ca60208 [Perf][VoxCPM2] Optimize VoxCPM2 high-concurrency decode throughput. (#3882) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-06-03 23:08:32 +08:00
Sy03Dan250124Yueqian LinYueqian Lin
 		d4c139508f [Feature] Add precomputed custom voices and Qwen3-TTS ref-context cache (#3492) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Dan250124 <416947747@qq.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-05-31 17:51:24 +00:00
Sy03andYueqian Lin d37912b071 [TTS][Perf] Optimize Fish Speech S2 Pro high-concurrency serving (#3773) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-05-27 13:25:49 -05:00
Sy03andamy-why-3459 ab59673f21 [Bugfix][Qwen3-Omni] Handle short Code2Wav chunk outputs (#3687) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-05-18 18:59:23 +00:00
Sy03Hongsheng LiuSYLAR
 		c99df1ebd9 [TTS][Perf] Optimize Qwen3-TTS high-concurrency serving (#3662) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-05-17 16:58:59 -04:00
Sy03 2c73cf3522 [Perf] Fix Qwen3-TTS latency regression (#3485) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-05-10 23:07:38 +00:00
Sy03 2b85474c23 [Bugfix] Propagate seed to Qwen3-TTS Fast AR sampler (#3350) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-05-05 14:57:55 -04:00
Sy03andHongsheng Liu 368afea2f2 [Bugfix][Fish Speech] Flatten Fast AR RoPE positions (#3299) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-05-02 20:39:32 +08:00
Sy03shimei.jmhSYLAR
 		7b876324c2 [Feature] Streaming video input with EVS frame filtering (RFC #2201 Phase 2-4) (#2342) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: shimei.jmh <shimei.jmh@taobao.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-04-24 22:54:35 +08:00
Sy03 26edc7f2fb [Bugfix][VoxCPM2]: Fix vectorized_gather OOB under concurrent prefill+decode batches (#2903) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-04-19 20:39:14 +12:00
Sy03 a683b1dd75 [Bugfix][VoxCPM2] Fix voice-clone decode loop by padding prefill prompt (#2894) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-04-18 14:04:44 +00:00
Sy03 7d64a7c996 [BugFix][VoxCPM2]: split multichar Chinese tokens to match training tokenization (#2832) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-04-16 10:07:48 -05:00
Sy03 322620fd57 [Fix][Fish Speech] Remove redundant get_vocab() in control token encoding (#2842) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-04-16 12:47:39 +00:00
Sy03 38d5f2d530 [Perf] VoxCPM2: Speedup by manual CUDA Graph capture for scaffold/residual forward (#2803) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-04-15 14:09:22 +08:00
Sy03Yueqian LinYueqian Lin
 		14f7910900 [Perf]: Speedup VoxCPM2 TTS performance and Support PagedAttention (#2690) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-04-14 09:16:47 +12:00
Sy03 cb4d13a658 [Perf][Fish Speech] Enable CUDA Graph capture for Fast AR code predictor (#2520) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-04-13 04:53:35 +00:00
Sy03 694be6f2e1 [Bugfix] Fix Fish Speech voice clone FileNotFoundError on multi-GPU (#2606) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-04-09 20:34:34 +00:00
Sy03 8b57c6205e [Perf][Fish Speech] Free unused DAC codec components to save VRAM (#2430) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-04-05 20:49:18 +00:00
Sy03andYueqian Lin f6cfacdd16 [Perf][Qwen3-TTS] Free unused decoder in Talker SpeechTokenizer to VRAM (#2429) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-04-05 19:04:37 +00:00
Sy03 025408f693 [Fix] [Qwen3-TTS] Qwen3-TTS streaming chunk-boundary artifacts (#2480) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-04-05 14:40:38 -04:00
Sy03 7fb86d51cd [BugFix][FishSpeech] Fix structured voice clone prefill conditioning (#2446) 2026-04-03 06:56:33 +00:00
Sy03 d9912eb6e9 [Bugfix] Fix Fish Speech S2 Pro prompt handling for truncated audio & emotion tag (#2268) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-03-28 07:00:00 +08:00
Sy03 d15c91d6d2 [Bugfix] Fix qwen3-omni async thinker to talker decode alignment for #1758 (#2019) 2026-03-26 11:07:11 +08:00
Sy03 fc32da7624 [Perf] [TTS] Improve Fish Speech S2 Pro voice cloning TTFP (#2145) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-03-25 11:41:10 -04:00
Sy03 44770f9f20 [Fix] Qwen3 TTS audio handling for long ref_audio (#2104) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: Sy0307 <1370724210@qq.com>
 2026-03-24 20:02:52 +00:00
Sy03andsy0307 072647eeb2 [Perf] Improve Fish Speech S2 Pro inference performance (#1859) 
Signed-off-by: sy0307 <sy0307@users.noreply.github.com>
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: sy0307 <sy0307@users.noreply.github.com>
 2026-03-21 01:15:02 -04:00
Sy03Yueqian Linlinyueqian
 		5d28b6a40c [Feat][Qwen3-TTS] Support streaming audio output for websocket (#1719) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: linyueqian <linyueqian@outlook.com>
 2026-03-11 23:48:41 -04:00
Sy03 761eff930a [Fix][Qwen3-TTS] Preserve ref_code decoder context for Base ICL (#1731) 2026-03-09 13:36:27 -04:00
Sy03Yueqian Linlinyueqian
 		d6871db6a3 [Feat]: Offline inference supports async_chunk (#1415) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: linyueqian <linyueqian@outlook.com>
 2026-03-09 11:33:38 +08:00
Sy03 90ec499bd5 [Refactor][Perf] Qwen3-TTS: re-prefill Code Predictor with torch.compile + enable Code2Wav decoder CUDA Graph (#1617) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-03-06 18:21:55 +13:00
Sy03 0cc3481acf [Feat] : Support Async chunk cleanup (#1087) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-02-27 14:39:33 +08:00
Sy03andHongsheng Liu efbe4116ff [Feature] : Support disaggregated inference pipeline for Qwen3_TTS (#1161) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-21 07:36:44 +08:00