anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
2,122 Commits 235 Branches 31 Tags
21f0986fd4c2bbf386b540a458056a7c73d8f0f9
Commit Graph
361 Commits
Author SHA1 Message Date
tzhouamandClaude Fable 5 35a423c9bd [BugFix][CI] Address review follow-ups from v0.24.0 rebase (#4709) 
Addresses Gaohan123's post-approval review comments on #4709:

- pipeline_dreamzero: restore platform-dispatched synchronize() instead of
  torch.cuda so DreamZero keeps working on non-CUDA hardware (regression
  of #4399 introduced during the rebase).
- omni_scheduler_mixin: drop the VLLM_OMNI_INPUT_WAIT_TIMEOUT_S env-var
  override; the full-payload input-wait timeout is now a plain constant.
- diffusion_worker: implement _DiffusionVllmModelConfig.is_diffusion by
  mirroring upstream ModelConfig.is_diffusion (dLLM canvas_length check)
  and document why omni DiT pipelines must not enable dLLM paths.
- qwen3_tts: make _embedding_dtype a class attribute on the talker and
  prompt-embeds builder so __new__-based test stubs inherit it; remove
  the (duplicated) manual assignments from the preprocess test.
- .buildkite: document why the rebase raised per-step timeouts (steps
  were killed by the old limits, masking real failures; e.g. CustomVoice
  passes in ~9-15 min but cold runs were killed at 20m).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-07-02 07:36:14 +00:00
Sy03Claude Sonnet 4.6Yueqian Lin
 		f11b32a42e [Bugfix] Qwen3-TTS talker handles multi-token replay on resume (#4559) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-07-02 04:46:05 +00:00
xinlongClaude Opus 4.8amy-why-3459
 		4d2ee15156 [BugFix] Fix Code2Wav async_chunk streaming tail drop (boundary clicks + ~1.2% time compression) (#4706) 
Signed-off-by: zhuxinlong <zhuxinlong959@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-07-02 09:13:42 +08:00
Zhou Taichang a560ed184d [Rebase] Rebase to vllm v0.24.0 (#4709) 2026-06-30 19:41:06 +08:00
Peiqi YinChenguang ZHENGCursorHongsheng LiuGao Han
 		504698dec8 [Core] Performance fix for orchestrator bottleneck: separates inter-stage from client outputs (#4527) 
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Co-authored-by: Chenguang ZHENG <645327136@qq.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-06-29 17:43:36 +02:00
Ayaka Mikazuki deb9bd4efc Fix Qwen2.5-Omni AutoRound loading (#4781) 
Signed-off-by: Ayaka <ayaka@mail.shn.hk>
 2026-06-29 07:34:27 +00:00
Isaac Hung e9d4931781 [Core] Remove dead custom_process_input_func hooks in stage input processors (#4531) 
Signed-off-by: Isaac Hung <manhei5001@gmail.com>
 2026-06-29 05:17:25 +02:00
LHXuuuYuanheng ZhaoYuanheng ZhaoHongsheng Liu
 		0b04f2a0d6 [Model] Add Ming-omni-tts MoE 16.8B-A3B + CFM CUDAGraph (#4341) 
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com>
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Co-authored-by: Yuanheng Zhao <54058983+yuanheng-zhao@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-06-25 22:11:35 +08:00
Canlin Guo 2e6c83c89c [Model] Support MOSS-TTS-local-v1.5 (#4664) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-06-25 15:43:03 +08:00
TaffyOfficialTaffyOfficialGao Han
 		d71cacc74c [perf] avoid HunyuanImage3 timestep scalar sync (#4363) 
Signed-off-by: TaffyOfficial <2324465096@qq.com>
Co-authored-by: TaffyOfficial <2324465096@qq.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-06-24 11:05:35 +08:00
Alex Brooks 465f1464d2 [Feat] Support Pipeline Resolution Based on HF Config & Make Pipeline Registry Eager (#3760) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-06-24 11:02:29 +08:00
Nick Cao 67d14b5ac4 [Refactor] Replace hardcoded CUDA device selection with platform-agnostic API (#4496) 
Signed-off-by: Nick Cao <ncao@redhat.com>
 2026-06-23 14:38:32 +08:00
BeatSeat 044240d8eb [Model] Add IndexTTS2 text-to-speech support (#3838) 
Signed-off-by: BeatSeat <wendavid552@gmail.com>
Signed-off-by: David <david@podcast.ai>
Signed-off-by: wendavid552 <wendavid552@gmail.com>
Signed-off-by: WenDavid <davidzhangcf@foxmail.com>
 2026-06-22 19:24:01 +00:00
TaffyOfficialandTaffyOfficial f4d922d8f6 [Bugfix][HunyuanImage3] Align AR RGB conversion with official semantics (#4502) 
Signed-off-by: TaffyOfficial <2324465096@qq.com>
Co-authored-by: TaffyOfficial <2324465096@qq.com>
 2026-06-22 16:09:03 +00:00
Chenguang Zhengandamy-why-3459 eaa16e1941 [Feature] Support Async Omni output materialization (#4476) 
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-06-22 16:07:07 +00:00
Yihong JinandClaude Opus 4.8 a60242092a [Bugfix][Qwen3-Omni] Pass Thinker lm_head prefix so NVFP4 exclude-list is honored (#4528) 
Signed-off-by: Yihong Jin <yihongjin404@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-22 12:16:44 +08:00
yangyonggit ccb5c53ea5 [Cleanup] Remove dead/stale modules with no repo references (fixes #4009) (#4044) 
Signed-off-by: leo.yang <leo.yang.engineer@gmail.com>
 2026-06-22 09:23:04 +08:00
rein yang afbcc51672 [Model] Aura support - Non async chunk path (#4257) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Signed-off-by: R2-Y <ruiruyang2@gmail.com>
 2026-06-21 22:08:26 +08:00
Yueqian LinSy03Hongsheng Liu
 		cead615288 [Bugfix][Qwen3-Omni] Fix Code2Wav CUDA-graph output length surplus (#4466) (#4525) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Sy03 <1370724210@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-06-19 23:58:45 -07:00
Zhengyuan Su (苏政渊)andClaude Opus 4.8 2d0b1d8a29 [Diffusion] add GR00T-N1.7 pipeline with OpenPI serving (#3798) 
Signed-off-by: Zhengyuan Su <su.zhengyuan@u.nus.edu>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-20 11:45:26 +08:00
Yueqian Lin 208932210f [Chore][TTS] Reference recipes.vllm.ai recipes, remove dead code, and fix stale references (#4567) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-06-19 15:05:26 +08:00
yangyonggit 63ba11b8b1 [Perf] Stacked audio ops: batched matmul and embedding gather in MOSS-TTS talker (#4230) 
Signed-off-by: leo.yang <leo.yang.engineer@gmail.com>
 2026-06-19 00:38:32 +00:00
yangyonggitandYueqian Lin 1228a109f0 [Bugfix] MOSS-TTS: fix cross-request audio corruption under batching (#4415) 
Signed-off-by: leo.yang <leo.yang.engineer@gmail.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-06-18 18:11:51 +00:00
akshatvishu d78a92ec54 [bugfix] Shim PreTrainedModel._tp_plan to fix MOSS-TTS-Nano load crash (#4398) 
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
 2026-06-17 17:17:03 +08:00
akshatvishu d8a72be206 fix(models): remove invalid sampling_metadata parameter from logits_processor (#4456) 
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
 2026-06-16 14:41:40 +08:00
Yuekai ZhangandClaude Opus 4.8 dc14380796 [BugFix] higgs-audio-v3: load codec lazily under load_format=dummy (#4368) 
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-16 14:07:16 +08:00
Zhou Taichang 7b837944a1 [Rebase] Rebase to vllm 0.23.0 (#4286) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-06-13 16:43:02 +00:00
Yueqian Lin d75a6433b4 [Refactor] TTS serving adapter framework + migrate TTS models (excl. ming_flash_omni) (#4330) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-06-13 04:22:54 +00:00
wuli666linyueqianwuli666
 		9bc8705c98 [New Model] Step-Audio2 (#464) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: wuli666 <djjpro975@gmail.com>
Co-authored-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: wuli666 <djjpro975@gmail.com>
 2026-06-13 01:00:33 +00:00
Henry JjjwClaude Fable 5Yueqian Lin
 		1733831f78 [Bugfix] Fix cross-request codes.ref leak in Qwen3-TTS make_omni_output (#4373) 
Signed-off-by: jjw <m@jjw.in>
Co-authored-by: jjw <m@jjw.in>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-06-13 00:41:49 +00:00
zyz111222andCanlin Guo 6de0ca36bb [Hardware][Ascend] Adapt Qwen3 TTS for 310P (#4283) 
Signed-off-by: zouyizhou <zouyizhou@huawei.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
 2026-06-12 09:39:21 +00:00
tanhaoan333 aa59dc50c7 [bugfix]VoxCPM2 audio encoder adapt other than CUDA (#4374) 
Signed-off-by: tanhaoan333 <tanhaoan@huawei.com>
 2026-06-12 08:54:29 +00:00
Megha AgarwalCopilotHongsheng Liumeghaagr13Boao ShiTaichang ZhouCursor
 		8f869c8ea5 [Refactor] Output Processor Phase 2: separate multimodal output channel (#1601) (#2744) 
Signed-off-by: Megha Agarwal <agarwalmegha1308@gmail.com>
Signed-off-by: meghaagr13 <divyanshsinghvi@gmail.com>
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: meghaagr13 <divyanshsinghvi@gmail.com>
Co-authored-by: Boao Shi <aoibosh@connect.hku.hk>
Co-authored-by: Taichang Zhou <tzhouam@connect.ust.hk>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-06-12 15:19:14 +08:00
yangyonggit a3834e32cf feat(moss-tts): add CUDA Graph support for codec decoder (#4157) 
Signed-off-by: leo.yang <leo.yang.engineer@gmail.com>
 2026-06-11 22:17:31 +00:00
Sy03Yueqian LinHongsheng Liu
 		e27fd46ab9 [Perf] Optimize Higgs Audio v3 serving (#4204) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-06-11 13:29:31 -07:00
tanhaoan333 e26f5cb918 [NPU] Support VoxCPM2 model (#4310) 
Signed-off-by: tanhaoan333 <tanhaoan@huawei.com>
 2026-06-11 14:12:06 +08:00
ChenWenjingandSy03 1fe267d6c9 [Bugfix][VoxCPM2] Fix VoxCPM2 concurrent speech quality (#4319) 
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
Co-authored-by: Sy03 <1370724210@qq.com>
 2026-06-11 11:12:07 +08:00
Guanzhe (Leo) Huang c2b4564a67 [BugFix] moss_tts_nano: eager-init lm + audio_tokenizer in __init__ so load_format: dummy works (#3230) 
Signed-off-by: leohuang257 <masugchds@gmail.com>
 2026-06-10 20:50:53 +08:00
akshatvishuandHongsheng Liu 2fa4c7a4a9 [Model] Add Ming-omni-tts dense 0.5B pipeline (#2906) 
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-06-10 20:11:04 +08:00
Yuekai ZhangandClaude Opus 4.8 f1bf544208 [Perf][TTS] Optimize cosyvoice TTFP and throughput using TensorRT (#4168) 
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-10 16:42:34 +08:00
Juan Pablo ZuluagaJuanPZuluagaGao Han
 		57227dc7f3 [Perf][Bugfix] qwen3-tts hot path: prefix-cache OOM guards + talker/orchestrator micro-opts (#3689) 
Signed-off-by: JuanPZuluaga <juanz9312@gmal.com>
Co-authored-by: JuanPZuluaga <juanz9312@gmal.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-06-09 10:36:57 -07:00
Sy03 9c9d4ed4a8 [Fix][HiggsAudioV3] Fix ramp-down off-by-one crash and buffer state bugs in Stage0 talker (#4219) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-06-08 17:33:20 +00:00
amy-why-3459 fc7393e485 [Refactor]Refactoring audio_in_video implementation (#3566) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-06-08 14:14:02 +08:00
zzhzengchuangCanlin Guo
 		eb1b782dd1 [BugFix] fix hunyuan image3 offline cot (#4174) 
Signed-off-by: zzh <943967662@qq.com>
Signed-off-by: zengchuang <zengchuang3@huawei.com>
Co-authored-by: zengchuang <zengchuang3@huawei.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
 2026-06-08 10:58:44 +08:00
Yueqian Lin fce0f51089 [Perf][Higgs-Audio-V3] LRU cache for voice-clone ref-audio encode (#4200) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-06-07 03:09:16 +00:00
Yuekai ZhangandYueqian Lin 82e74ce972 [TTS][New Model] support bosonai/higgs-audio-v3-tts-4b (#4169) 
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
 2026-06-06 15:59:12 +00:00
Yaegaki1Erikaandreandeng 84569280b1 [Bugfix]Fix HunyuanImage3 conditional image and prompt kwargs alignment (#4145) 
Signed-off-by: reandeng <reandeng@tencent.com>
Signed-off-by: Yaegaki1Erika <70182590+Yaegaki1Erika@users.noreply.github.com>
Co-authored-by: reandeng <reandeng@tencent.com>
 2026-06-05 12:47:25 +08:00
Yuanheng Zhaoandakshatvishu 95d56cf062 [Bugfix] Fix Compatibility of Ming-flash-omni-2.0 on transformers 5.X and vllm 0.22 (#4080) 
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
Co-authored-by: akshatvishu <akshatnayak197@gmail.com>
 2026-06-05 09:27:42 +08:00
ZacheryAU bfc7cedcf1 Benchmark data statistics for each stage of omni models (#3628) 
Signed-off-by: ZacheryAU <70869627+ZacheryAU@users.noreply.github.com>
 2026-06-04 21:43:35 +08:00
Yuekai Zhang a1cc583665 fix: cosyvoice3 batch>1 inference (#3910) 
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
 2026-06-04 08:58:53 +00:00