anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
2,199 Commits 235 Branches 31 Tags
f18bb69b7ddac5b2fe73e28c38c7fbe26a8da9cb
Commit Graph
494 Commits
Author SHA1 Message Date
tzhouamandClaude Opus 4.8 f18bb69b7d fix(entrypoints): pass ChatTemplateConfig to pooling serving classes for vLLM 0.25 
vLLM 0.25 reworked the pooling serving base: PoolingServing.__init__ now
requires a `chat_template_config: ChatTemplateConfig` keyword and no longer
accepts the old `chat_template` / `chat_template_content_format` /
`trust_request_chat_template` kwargs. Build a ChatTemplateConfig once and pass
it to the embedding / pooling / classification / score / responses serving
objects. ServingTokenization keeps the flat kwargs (unchanged upstream).

Without this, omni_init_app_state crashes at startup (missing required arg /
unexpected kwargs) whenever an embed/classify/score task is served.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-11 14:21:18 +08:00
tzhouam b73036f538 rebase: align vllm-omni with vLLM 740f379faefd 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-07-11 14:21:15 +08:00
tzhouam 7dcbe4e957 rebase: align vllm-omni with vLLM 2dfaae752b4d 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-07-11 14:21:14 +08:00
Kushanamkushanamkahmadian
 		ff6f0da271 [Core] Composable parallel: Phase 1 strategy overlay (TP/DP/stage-replica) (RFC #4084) (#4281) 
Signed-off-by: Kushan A. <kahmadian@nvidia.com>
Signed-off-by: kushanam <kushanam@users.noreply.github.com>
Signed-off-by: Kushan A <kahmadian@nvidia.com>
Signed-off-by: kahmadian <kahmadian@users.noreply.github.com>
Co-authored-by: kushanam <kushanam@users.noreply.github.com>
Co-authored-by: kahmadian <kahmadian@users.noreply.github.com>
 2026-07-10 21:29:24 -06:00
LHXuuu 5c3923509d [BugFix][Ming-TTS] Use uploaded direct speaker embeddings by voice (#5005) 
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com>
 2026-07-10 07:06:18 +00:00
Alex Brooks b7d08fe5f6 [Core / Bugfix] Add Mechanism for Endpoint Rejection (#4762) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-07-08 09:00:47 -06:00
汪志鹏 862e0a2b27 [BugFix] Preserve diffusion extra_body params for out-of-process CFG Parallel (#4900) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-07-05 17:06:30 +00:00
Vensen ddba6de2ab [Bugfix][sleep mode]: guard generation on partial wake and ensure wake idempotency (#4834) 
Signed-off-by: vensen <vensenmu@gmail.com>
 2026-07-05 14:50:08 +00:00
Sy03andYueqian Lin 1b31f96240 [Bugfix] Fix Qwen3-TTS no-async-chunk payload splitting (#4870) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
 2026-07-04 19:57:55 +00:00
Sy03andYueqian Lin 0798b0bf03 [Bugfix][Qwen3-TTS] Seed residual MTP sampling (#4869) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
 2026-07-04 17:47:11 +00:00
boatman 29dcdec66d [Cleanup]Replace legacy logging branches with PreparedRequest.model_type (#4862) 
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com>
 2026-07-04 05:39:20 +00:00
Kotaro Kikuchi 46f10478dc [Bugfix] Qwen-Image-Layered: allow layers=2 to match upstream demo (#4777) 
Signed-off-by: Kotaro Kikuchi <ktrk115@gmail.com>
 2026-07-04 03:06:24 +00:00
LHXuuu 991fb7408b [BugFix][Ming-TTS] Improve Seed-TTS WER accuracy (#4859) 
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com>
 2026-07-03 12:15:03 +02:00
Canlin Guo a8b2bf11d2 [Feature] Support MOSS-TTS-Local-v1.5 Streaming (#4804) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-07-03 16:00:30 +08:00
Alex Brooks 5c51782c6b [Bugfix] Realign Serving Tokenization Name to Fix Error Handling Routes (#4816) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-07-02 15:23:35 +08:00
ZacheryAU b0f6b9de96 [Bugfix] clean per-stage audio RTF and align audio metrics (#4794) 2026-07-02 02:55:44 +00:00
Zhou Taichang a560ed184d [Rebase] Rebase to vllm v0.24.0 (#4709) 2026-06-30 19:41:06 +08:00
faceless void 11f6a0b72e Make speech streaming default to SSE (#4679) 
Signed-off-by: syd520zy <529477025@qq.com>
 2026-06-30 02:17:24 +00:00
JiangJie ZhangandSamit c45ac74d43 [Core][Frontend] Support request-level batching for diffusion pipelines (#4079) 
Signed-off-by: jader <yjader@foxmail.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-06-29 18:09:19 +02:00
Peiqi YinChenguang ZHENGCursorHongsheng LiuGao Han
 		504698dec8 [Core] Performance fix for orchestrator bottleneck: separates inter-stage from client outputs (#4527) 
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Co-authored-by: Chenguang ZHENG <645327136@qq.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-06-29 17:43:36 +02:00
Yueqian Lin 92f7208559 [Bugfix] CosyVoice3: wrap ref_text in instruction template (#4644) (#4756) 2026-06-27 14:38:33 -07:00
Rahul SteigerandHongsheng Liu 327e9dcaf2 [Feature] Spatially-sharded (SP) decode for the Wan VAE (#4620) 
Signed-off-by: Rahul Steiger <rsteiger@nvidia.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-06-27 09:01:25 +08:00
Yueqian LinandSy03 b66cd024b9 [Bugfix] Fix /v1/audio/speech usage token accounting for Qwen3-TTS (#4646) (#4673) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Sy03 <1370724210@qq.com>
 2026-06-27 06:13:31 +08:00
Gao Han c582805e7a Revert "[Bugfix] Use MediaConnector for image/video URL fetching to prevent SSRF" (#4751) 2026-06-27 01:18:48 +08:00
Sy03 c39ac9fffd [Bugfix] Stream Qwen3-TTS WebSocket input as one request (#4731) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-06-26 07:48:30 +00:00
LHXuuuYuanheng ZhaoYuanheng ZhaoHongsheng Liu
 		0b04f2a0d6 [Model] Add Ming-omni-tts MoE 16.8B-A3B + CFM CUDAGraph (#4341) 
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com>
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Co-authored-by: Yuanheng Zhao <54058983+yuanheng-zhao@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-06-25 22:11:35 +08:00
Honghan ZhuClaude Opus 4.7zhuhh97
 		5db89b41c4 [Core] hunyuan-image prefetch kv (#4448) 
Signed-off-by: z00806815 <zhuhonghan@huawei.com>
Signed-off-by: zhuhh97 <zhuhonghan@huawei.com>
Signed-off-by: dph97 <nenbaying@163.com>
Signed-off-by: zhh <zhh@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: zhuhh97 <zhuhh97@users.noreply.github.com>
 2026-06-25 21:31:49 +08:00
Canlin Guo 2e6c83c89c [Model] Support MOSS-TTS-local-v1.5 (#4664) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-06-25 15:43:03 +08:00
WeiQing ChenandIanniu d94108693a [BugFix] Fix Wan2.2-VACE-Fun cache and serving lifecycle (#4667) 
Signed-off-by: Ianniu <985683179yzy@gmail.com>
Signed-off-by: david6666666 <530634352@qq.com>
Co-authored-by: Ianniu <985683179yzy@gmail.com>
 2026-06-25 11:20:58 +08:00
Nick CaoandHongsheng Liu d8edf0c9d5 [Bugfix] Use MediaConnector for image/video URL fetching to prevent SSRF (#2565) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-06-25 06:28:26 +08:00
Zeng ChuangandHongsheng Liu 67bd02f767 [Feat] Enable streaming CoT display for HunyuanImage-3.0 AR generation (#4148) 
Signed-off-by: zengchuang <zengchuang3@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-06-24 22:08:30 +08:00
Yuanheng Zhao fb3abb38fb [refactor] Flatten Ming-flash-omni-2.0 imagegen args (#4587) 
Signed-off-by: Yuanheng <jonathan.zhaoyh@gmail.com>
 2026-06-24 17:01:58 +08:00
Alex Brooks 465f1464d2 [Feat] Support Pipeline Resolution Based on HF Config & Make Pipeline Registry Eager (#3760) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-06-24 11:02:29 +08:00
rongfu.lengandGao Han a2d1dafc65 [Feature] add a new response_format is file type when image generate (#1673) 
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-06-24 09:04:13 +08:00
Zeyu Huang | 黃澤宇Copilot Autofix powered by AICursor
 		3901ad8165 [feat] Streaming diffusion video generation output (#3737) 
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
Signed-off-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-06-23 09:57:53 +00:00
Yuanheng Zhao 9177b37e2f [NFC] Update misleading seed-tts-eval optional dependency guidance (#4619) 
Signed-off-by: Yuanheng <jonathan.zhaoyh@gmail.com>
 2026-06-22 16:35:42 -07:00
Ian EavesYueqian LinYueqian Lin
 		50bc51ffdf fix: return proper HTTP status codes for audio voice endpoints (#3969) 
Signed-off-by: Ian Eaves <ian.k.eaves@gmail.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-06-22 22:57:07 +00:00
BeatSeat 044240d8eb [Model] Add IndexTTS2 text-to-speech support (#3838) 
Signed-off-by: BeatSeat <wendavid552@gmail.com>
Signed-off-by: David <david@podcast.ai>
Signed-off-by: wendavid552 <wendavid552@gmail.com>
Signed-off-by: WenDavid <davidzhangcf@foxmail.com>
 2026-06-22 19:24:01 +00:00
+2
faceless voidNick CaoChendi.XueHongsheng LiuKristofferJunhong LiulishunyangCopilot Autofix powered by AIHyoseop SongZhou TaichangAliciaamy-why-3459
 		ec4ec0f604 Add SSE stream_format support for audio speech streaming (#4490) 
Signed-off-by: syd520zy <529477025@qq.com>
Signed-off-by: faceless void <100186390+syd520zy@users.noreply.github.com>
Signed-off-by: Nick Cao <ncao@redhat.com>
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Signed-off-by: kTorp <Kristoffer.Torp@amd.com>
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: Hyoseop Song <crad_on25@naver.com>
Signed-off-by: Hyoseop Song  <crad_on25@naver.com>
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Chendi.Xue <chendi.xue@intel.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Kristoffer <kristoffer.torp@amd.com>
Co-authored-by: Junhong Liu <ljh_lbj@163.com>
Co-authored-by: lishunyang <lishunyang12@163.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Hyoseop Song <crad_on25@naver.com>
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-06-22 06:42:25 +00:00
王金旭CursorClaude Opus 4.8
 		ad07f98b73 [Feat] Streaming TTS word-level timestamps via forced aligner (#4034) 
Signed-off-by: wjinxu <1299461899@qq.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-22 04:51:45 +00:00
rein yang afbcc51672 [Model] Aura support - Non async chunk path (#4257) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Signed-off-by: R2-Y <ruiruyang2@gmail.com>
 2026-06-21 22:08:26 +08:00
Shin 1292c47804 [BugFix]Fix diffusion profile RPC None results (#4372) 
Signed-off-by: yixiaoer <shin@yixiaoer.sg>
 2026-06-21 17:14:21 +08:00
MaciejBalaNV a1e1ac0155 Cosmos3 transfer (#4379) 
Signed-off-by: Maciej Bala <mbala@nvidia.com>
Signed-off-by: MaciejBalaNV <mbala@nvidia.com>
 2026-06-21 09:59:18 +08:00
Ian Eaves 951ae28eb0 [Frontend] Configurable video storage backends with TTL (#2531) 
Signed-off-by: Ian Eaves <ian.k.eaves@gmail.com>
 2026-06-19 17:41:22 -06:00
Yueqian Lin 208932210f [Chore][TTS] Reference recipes.vllm.ai recipes, remove dead code, and fix stale references (#4567) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-06-19 15:05:26 +08:00
amy-why-3459 f05e03aea8 [Perf][DreamZero]dreamzero supports CUDA graph, torch.compile, and DiT Caching. (#4213) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-06-19 14:24:37 +08:00
汪志鹏CursorHongsheng LiuClaude Sonnet 4.6
 		f75c437a60 [Refactor]Base class for dit pipelines with unified parameter declaration[1/N] (#4225) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-18 17:14:18 +08:00
NumberWanandlishunyang12 d744940b18 Refactor: extract OmniStreamingVideoHandler base and QwenOmniStreamingVideoHandler (#4424) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
Signed-off-by: lishunyang12 <125541396+lishunyang12@users.noreply.github.com>
Co-authored-by: lishunyang12 <125541396+lishunyang12@users.noreply.github.com>
 2026-06-17 15:07:45 +00:00
blondeandClaude Opus 4.6 8fa0394d29 [Bugfix] Wire log_stats to AsyncOmni and add missing token metrics for non-text requests (#4482) 
Signed-off-by: blonde <164845202+blondeCS@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-06-17 21:31:44 +08:00
Peiqi YinHongsheng LiuChenguang ZHENG
 		64a5bb92de [Refactor] Omni Stage Runtime and Distributed Replica Control Plane (#3855) 
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Chenguang ZHENG <645327136@qq.com>
 2026-06-17 14:13:20 +08:00