anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
2,960 Commits 235 Branches 31 Tags
85d1f92ee58de7cf65bcfd9a02fc0a4c3e3dc9d3
Commit Graph
634 Commits
Author SHA1 Message Date
tzhouamandClaude Opus 5 08884c4cfb rebase: adapt to vLLM 0.29 callback, embedding and video decoder changes 
Four unrelated 0.29 surface changes, each reproduced locally against the
pinned f5c3cc24 build:

- PromptUpdateDetails.is_embed now takes only the full token list. The
  Bagel img2img replacement still declared (tok, seq), which crashed
  stage-0 engine core startup and produced the Distributed and Qwen3-Omni
  "no live replica" failures rather than anything KV related.

- dispatch_unquantized_gemm gained a linear_backend argument and is called
  with it. The CPU-forcing test stubs took no arguments; they now accept
  and ignore whatever upstream passes so a further signature change does
  not break them again.

- VocabParallelEmbedding gained a fused kernel that engages whenever
  tp_size > 1 on CUDA, and it has no CPU implementation. The encoder TP
  fixtures drive CPU tensors under a faked TP group, so they pin the
  portable masked path.

- The pyav video decoder backend was removed; valid options are now
  opencv, torchcodec, pynvvideocodec and deepstream. The OpenAI video
  endpoint hardcoded pyav, which turned every video request into a 400.
  It moves to opencv, the upstream default.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146Xu5Sgj4vXxLxvLjV5YPj
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-09-10 00:41:53 +08:00
tzhouam f9879239ba chore: satisfy changed-file policy 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-09-10 00:41:52 +08:00
tzhouam 87a1fd8da2 rebase: adapt to vLLM v0.29 APIs 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-09-10 00:41:52 +08:00
Shaun WalshClaude Opus 4.6Nick Cao
 		0e602509aa [BugFix] Add field validators for /v1/audio/generate request (#4741) 
Signed-off-by: Shaun Walsh <shaunwalsh24@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Nick Cao <ncao@redhat.com>
 2026-09-09 13:38:42 +00:00
Allen WuandTRAE CLI f2e30f46b6 [Bugfix][TTS] Isolate shared Higgs v3 reference encode from request cancellation (#7076) 
Signed-off-by: Allen Wu <allenwu2795@gmail.com>
Co-authored-by: TRAE CLI <traecli@bytedance.com>
 2026-09-08 20:32:05 -07:00
Jim Ban 358b7d79c1 [Bugfix][Frontend] Enforce image pixel limits for video input references (#6963) 
Signed-off-by: BANANASJIM <bananasjim1@gmail.com>
 2026-09-08 20:31:54 -07:00
Ziming Wang 43b697541a [Bugfix][Frontend] Clear the P0 multimodal cache through the renderer (#7003) 
Signed-off-by: ZenAlexa <zimingwang945@gmail.com>
 2026-09-08 20:31:42 -07:00
Rakesh KariyaCursorClaude Opus 5
 		2bae550c04 [Frontend] Opt-in WebSocket TTS split_granularity and session seed (#7046) 
Signed-off-by: rk9595 <rakesh.kariya@somaiya.edu>
Signed-off-by: Rakesh Kariya <rakesh.kariya@somaiya.edu>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-08 19:15:59 -07:00
zhengjiaandCursor 81c1c460b3 [Bugfix][Frontend] Accept the msgpack-numpy package's numpy markers on the OpenPI endpoint (#6051) 
Signed-off-by: zjli2013 <leezhengjiang@126.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-08 19:15:48 -07:00
NancyFyongandNancyFyong e74fe3c6e8 [Model] Add Audio8 TTS Preview 0.6B (DualAR, 44.1 kHz codec) (#6157) 
Signed-off-by: NancyFyong <NancyFyong@users.noreply.github.com>
Co-authored-by: NancyFyong <NancyFyong@users.noreply.github.com>
 2026-09-08 19:00:25 -07:00
boatman cdf027874c [Feat][OmniVoice]Support Varlen Attn, Request-Batch and Step-Execution (#6408) 
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com>
 2026-09-08 18:58:38 -07:00
Harry MellorHongsheng LiuClaude Opus 5
 		fdaf4e31bd Add vLLM-Omni library info to Hugging Face Hub requests (#5381) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-08 10:08:38 -06:00
liangmenghuangandClaude Fable 5.1 7a80b79698 [Bugfix][Model] GR00T-N1.7: honor the per-request seed for flow-matching noise (#7253) 
Signed-off-by: liangmengh <liangmengh@nvidia.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-08 15:53:20 +00:00
LHXuuu c9c25b33c3 [Frontend] Add server-side VAD for turn-based omni models (#6618) 
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com>
 2026-09-08 03:17:00 -07:00
tlysanhuo 7c2e30afd3 [Bugfix] Bound omni benchmark per-request timeout to 15 min by default (#7130) 
Signed-off-by: tly <2200895168@qq.com>
 2026-09-08 02:00:25 -07:00
Chenguang Zheng f3fabe4145 [Bugfix] Keep --disable-log-stats out of stage config resolution (#7237) 
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
 2026-09-08 16:19:07 +08:00
Chenguang Zheng 087ac46318 [Refactor 1/N] Centralize Omni config resolution (#5140) 
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
 2026-09-08 10:30:26 +08:00
ZacheryAU b1ec136bd7 [Frontend] Add omni benchmark support for image and video endpoints (#4728) 2026-09-08 09:28:21 +08:00
boatman 043f03d576 [Migrate]Move model-specific helpers out of serving_speech (#6697) 
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com>
 2026-09-08 08:59:45 +08:00
FredHuangNV 85b0bad2d3 [Frontend][Model] Add Cosmos3 control uploads to video API (#7027) 
Signed-off-by: Fred Huang <frhuang@nvidia.com>
 2026-09-07 01:38:56 -07:00
SYLAR ea3cf99894 [diffusion][feature] Add component-selective offload policies (#5929) 
Signed-off-by: lishunyang12 <lishunyang12@163.com>
 2026-09-06 05:40:09 +08:00
chickeytonClaude Fable 5Gao Han
 		5d7fdf9350 [Core] Graduate MiniCPM-o 4.5 and PersonaPlex full-duplex serving out of experimental (#6196) 
Signed-off-by: chickeyton <ngton2014@gmail.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-09-05 08:20:21 +00:00
Hongsheng Liu d4096e01fe [Model] Add native MAGI-2 Preview diffusion support (#5918) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-09-05 15:32:30 +08:00
BruceLoveDecimalandClaude Opus 5 1e74807c35 [Feature][Moss-TTS] Reference Audio Preprocessing (#4982) 
Signed-off-by: BruceLoveDecimal <39156883+BruceLoveDecimal@users.noreply.github.com>
Signed-off-by: liuqihao <liuqihao970610@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 16:36:26 -07:00
LOGO127 b6b992ac7b [Bugfix] Fix Higgs Audio v3 voice-clone token validation (#7065) 
Signed-off-by: luozijian <luozijian0924@gmail.com>
 2026-09-04 16:06:47 -07:00
boatman 348fd0fc07 [Feat][dots.tts] Add Online Serving Support (#6235) 
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com>
 2026-09-04 14:44:10 -07:00
Yuekai ZhangandClaude Fable 5 4179b0cae1 [Perf][NemotronVoiceChat] Fast defaults for all three deploy profiles: 2.8x offline, realtime full-duplex audio (#6354) 
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-09-03 18:49:19 -07:00
Eugene Zhao ec22c7ac06 [Bugfix] Restore Omni health route ownership (#6723) 
Signed-off-by: qujing226 <qujing226@gmail.com>
Signed-off-by: Eugene Zhao <113281545+qujing226@users.noreply.github.com>
 2026-09-03 00:54:36 -07:00
Chiquan MoandMciG-ggg 2b47faf601 [Bugfix] Validate --stage-overrides shape in parse_stage_overrides (drop field allowlist) (#6230) 
Signed-off-by: Chiquan Mo <mcigggg399026@gmail.com>
Signed-off-by: MciG-ggg <MciG-ggg@users.noreply.github.com>
Co-authored-by: MciG-ggg <MciG-ggg@users.noreply.github.com>
 2026-09-02 23:19:08 -07:00
maithilijoshi20andmaithilijoshi20 63ed0fdef4 [Bugfix] Fix vLLM serve error response imports (#6707) 
Signed-off-by: maithilijoshi20 <maithilijoshi2003@gmail.com>
Signed-off-by: maithilijoshi20 <97733343+maithilijoshi20@users.noreply.github.com>
Co-authored-by: maithilijoshi20 <maithilijoshi2003@gmail.com>
 2026-09-02 15:48:01 -07:00
GXINandHongsheng Liu 3ab582c993 [Bugfix][Qwen3-TTS] Reject task mismatches before EngineCore dispatch (#6113) 
Signed-off-by: XIN GAO <1037396230@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-02 11:16:55 -07:00
AuFlowandHongsheng Liu 49c6fc1540 [Frontend] Add configurable output sample rate to Speech API (#6553) 
Signed-off-by: 郑倩倩 <2458983186@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-01 17:25:09 +08:00
Yuxuan RhoandHongsheng Liu 7992c2213e [Config] Remove diffusion_batch_size and use max_num_seqs instead. (#6484) 
Signed-off-by: LyxWxj <yuxuanlou621@foxmail.com>
Signed-off-by: Yuxuan Rho <104440472+LyxWxj@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-01 01:02:54 +00:00
XRay2016 8e26dce1f4 [Diffusion] Add Boogu Edit-Turbo support docs and tests (#6701) 
Signed-off-by: xRay2016 <1150722393@qq.com>
 2026-09-01 08:41:23 +08:00
汪志鹏andHongsheng Liu 86b85c078b [Feature][MiniMax-H3] Fuse the FastH3 four-step adapter at load time (#6714) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-31 12:04:58 +08:00
akshatvishu f321cdbe16 [Bugfix] Use Seed-TTS reference audio with MiniCPM-o 4.5 (#6628) 
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
 2026-08-31 10:27:34 +08:00
SYLAR 759aa4ffeb [Diffusion][Performance] Optimize MiniMax-H3 video output transfer (#6824) 
Signed-off-by: lishunyang12 <lishunyang12@163.com>
 2026-08-31 07:07:34 +08:00
mokekeandmokashliu 1a9d598d2b [Frontend] Encode interleaved video frames in parallel (#6776) 
Signed-off-by: mokashliu <mokashliu@tencent.com>
Co-authored-by: mokashliu <mokashliu@tencent.com>
 2026-08-31 07:04:27 +08:00
baonudesifeizhai 8be601ed7f [Diffusion] Add native SymmMem Fast Ulysses transport (#6340) 
Signed-off-by: roG0d <baonudesifeizhai@gmail.com>
 2026-08-30 04:14:21 +00:00
YancyandAsthenia 73893e73ea [Bugfix] Fix create_error_response import across vLLM versions (#6773) 
Signed-off-by: Asthenia <asthenia0412@gmail.com>
Co-authored-by: Asthenia <asthenia0412@gmail.com>
 2026-08-29 17:01:51 +00:00
Sy03 3adee400a0 [Bugfix][Qwen3-TTS] Handle codec generations that exhaust their token budget (#6728) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-08-29 08:59:05 -07:00
Nguyen Kha Nhat LongCursorSamit
 		bb5bf8fd0a [Bugfix] Fix colocate-async sleep admission race and deliver AR abort tokens (#6367) 
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Signed-off-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-08-29 09:11:39 +00:00
Jiatai Wang f66f05633c [Bugfix] Honor request seed in MOSS-TTS adapters (#6543) 
Signed-off-by: JiataiWang <wangjiatai@proton.me>
 2026-08-28 22:44:13 -07:00
Bob Tian d934a5f7b8 [Bugfix] Report engine-queued requests as waiting in vllm_omni:num_requests gauges (#6549) 
Signed-off-by: Bob Tian <bobzetian@google.com>
 2026-08-29 03:58:26 +00:00
Zhang BowenandHongsheng Liu 46cfa8435b [Model][Frontend] MiniMax-H3: Parallelize MP4 response frame conversion (#6499) 
Signed-off-by: MosCloud <bwzhang1991@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-28 21:22:00 -06:00
Honghan Zhu f584729f24 [BugFix] Fix local media access for diffusion speech (#6622) 
Signed-off-by: zhuhh97 <zhuhonghan@huawei.com>
 2026-08-29 02:01:24 +00:00
Canlin Guolishunyang12yuanwu
 		b6a4baa659 [Feature] Support text encoder Disaggregation for H3 (#5885) 
Signed-off-by: Canlin Guo <canlinguosdu@gmail.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Signed-off-by: yuanwu <yuan.wu@intel.com>
Co-authored-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: yuanwu <yuan.wu@intel.com>
 2026-08-28 14:27:05 +08:00
Eugene Zhao f8f8aa8eb7 [Bugfix] Honor stage selection in online profiler endpoints (#6609) 
Signed-off-by: qujing226 <qujing226@gmail.com>
 2026-08-28 11:41:47 +08:00
Zhanshenzhuiandamy-why-3459 c6df39e9f7 [Benchmark][MiniCPM-o] Add Omni-DuplexEval support (#6634) 
Signed-off-by: Zhanshenzhui <99891002+zyforsure@users.noreply.github.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-08-27 16:31:50 +08:00
Zhou TaichangandClaude Opus 5 651b03225b [Rebase] Rebase to vllm 0.28.0 (#6606) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-27 06:12:55 +00:00