anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
2,957 Commits 235 Branches 31 Tags
08884c4cfbbeb48b23ce67e265885b6c7774ccdd
Commit Graph
514 Commits
Author SHA1 Message Date
tzhouamandClaude Opus 5 08884c4cfb rebase: adapt to vLLM 0.29 callback, embedding and video decoder changes 
Four unrelated 0.29 surface changes, each reproduced locally against the
pinned f5c3cc24 build:

- PromptUpdateDetails.is_embed now takes only the full token list. The
  Bagel img2img replacement still declared (tok, seq), which crashed
  stage-0 engine core startup and produced the Distributed and Qwen3-Omni
  "no live replica" failures rather than anything KV related.

- dispatch_unquantized_gemm gained a linear_backend argument and is called
  with it. The CPU-forcing test stubs took no arguments; they now accept
  and ignore whatever upstream passes so a further signature change does
  not break them again.

- VocabParallelEmbedding gained a fused kernel that engages whenever
  tp_size > 1 on CUDA, and it has no CPU implementation. The encoder TP
  fixtures drive CPU tensors under a faked TP group, so they pin the
  portable masked path.

- The pyav video decoder backend was removed; valid options are now
  opencv, torchcodec, pynvvideocodec and deepstream. The OpenAI video
  endpoint hardcoded pyav, which turned every video request into a 400.
  It moves to opencv, the upstream default.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146Xu5Sgj4vXxLxvLjV5YPj
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-09-10 00:41:53 +08:00
tzhouam f9879239ba chore: satisfy changed-file policy 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-09-10 00:41:52 +08:00
tzhouam 87a1fd8da2 rebase: adapt to vLLM v0.29 APIs 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-09-10 00:41:52 +08:00
zyz111222 6e79d582e3 [Bugfix][Ascend] fix npu 310/a5 bugs (#6685) 
Signed-off-by: zouyizhou <zouyizhou@huawei.com>
 2026-09-09 19:03:09 +08:00
NancyFyongandNancyFyong e74fe3c6e8 [Model] Add Audio8 TTS Preview 0.6B (DualAR, 44.1 kHz codec) (#6157) 
Signed-off-by: NancyFyong <NancyFyong@users.noreply.github.com>
Co-authored-by: NancyFyong <NancyFyong@users.noreply.github.com>
 2026-09-08 19:00:25 -07:00
boatman cdf027874c [Feat][OmniVoice]Support Varlen Attn, Request-Batch and Step-Execution (#6408) 
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com>
 2026-09-08 18:58:38 -07:00
eval d7678e2bb3 [Bugfix][MiniCPM-o] Fix the audio_embeds input path (#5730) 
Signed-off-by: eval-dev <0xe5bca0@gmail.com>
Signed-off-by: eval <74645252+eval-dev@users.noreply.github.com>
 2026-09-08 18:41:42 -07:00
Zhichao Zhang 5078c7ad66 [Bugfix][MiMo-Audio] Align code2wav decode with tokenizer device (#6539) 
Signed-off-by: chaosansui <zzc15560846421@163.com>
Signed-off-by: Zhichao Zhang <60429419+smartDream-chao@users.noreply.github.com>
 2026-09-08 18:38:11 -07:00
Harry MellorHongsheng LiuClaude Opus 5
 		fdaf4e31bd Add vLLM-Omni library info to Hugging Face Hub requests (#5381) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-08 10:08:38 -06:00
Shuolei Wang 5378b77be4 [Model][JoyAI-VL-Interaction] Add native multi-stage pipeline (#5352) 
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
 2026-09-08 05:07:08 -07:00
Deep Shahandamy-why-3459 1413eaab7c [Bugfix][Qwen3-Omni] Read code-predictor RoPE theta from rope_parameters (#7228) 
Signed-off-by: Deep Shah <deep@socratic.co>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-09-08 10:25:27 +08:00
Yueqian Lin e1c59edec6 [Refactor] fish_speech: vendor DAC codec modules, drop external fish-speech dependency (#4871) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-09-08 09:22:36 +08:00
boatman 043f03d576 [Migrate]Move model-specific helpers out of serving_speech (#6697) 
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com>
 2026-09-08 08:59:45 +08:00
Pujitha PaladuguPujitha Paladuguamy-why-3459
 		cf36f5e474 [Bugfix][MiniCPM-o] Allow ragged audio_feature_lens across a batch (#7071) 
Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
Co-authored-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-09-07 23:51:13 +08:00
BruceLoveDecimal 5cd925631c [Feature][LingBot World]stepwise execution (#6844) 
Signed-off-by: BruceLoveDecimal <liuqihao970610@gmail.com>
Signed-off-by: liuqihao <liuqihao970610@gmail.com>
 2026-09-07 13:38:23 +08:00
NATUREandamy-why-3459 ff6e906a25 [Model] Preserve MiniCPM-o Talker full attention until capacity (#6799) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-09-06 22:25:35 +08:00
Hongsheng Liu 715b8b8746 [Model] MiniMax-H3: validate versioned text-conditioning handoff (#6720) 
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-09-06 21:45:10 +08:00
SYLARandhsliuustc0106 14d2d00536 [Bugfix] Fix Hunyuan Image-3 expert mapping contract (#7021) 
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
Co-authored-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-09-06 21:43:37 +08:00
chickeytonClaude Fable 5Gao Han
 		5d7fdf9350 [Core] Graduate MiniCPM-o 4.5 and PersonaPlex full-duplex serving out of experimental (#6196) 
Signed-off-by: chickeyton <ngton2014@gmail.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-09-05 08:20:21 +00:00
boatman 66bbebd9d3 [Bugfix] Fix fallback loading for IndexTTS external models (#6137) 
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com>
 2026-09-04 19:58:33 -07:00
BruceLoveDecimalandClaude Opus 5 1e74807c35 [Feature][Moss-TTS] Reference Audio Preprocessing (#4982) 
Signed-off-by: BruceLoveDecimal <39156883+BruceLoveDecimal@users.noreply.github.com>
Signed-off-by: liuqihao <liuqihao970610@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 16:36:26 -07:00
LOGO127 b6b992ac7b [Bugfix] Fix Higgs Audio v3 voice-clone token validation (#7065) 
Signed-off-by: luozijian <luozijian0924@gmail.com>
 2026-09-04 16:06:47 -07:00
Inesh Reddy ChappidiandGao Han 66d938100b [Bugfix] Give exact-shape codec models an unpadded input_ids view (#6775) 
Signed-off-by: Inesh Reddy <ineshreddy249@gmail.com>
Signed-off-by: IneshReddy249 <ineshreddy249@gmail.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-09-04 15:28:49 -07:00
boatman 348fd0fc07 [Feat][dots.tts] Add Online Serving Support (#6235) 
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com>
 2026-09-04 14:44:10 -07:00
Yuekai ZhangandClaude Fable 5 4179b0cae1 [Perf][NemotronVoiceChat] Fast defaults for all three deploy profiles: 2.8x offline, realtime full-duplex audio (#6354) 
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-09-03 18:49:19 -07:00
Jim Ban 711daacabf [Bugfix][TTS] Make CosyVoice3 TRT plan publication concurrency-safe (#6955) 
Signed-off-by: BANANASJIM <bananasjim1@gmail.com>
 2026-09-03 06:44:42 +00:00
Tai An d1a5a7c178 [Bugfix][Qwen2.5-Omni] Fix two stale call sites in the speech helpers (#6886) 
Signed-off-by: Anai-Guo <antai12232931@outlook.com>
 2026-09-02 21:41:05 -07:00
Joshna-MedisettyandCursor 24c49cf392 [Bugfix][XPU] Add MoTRMSNorm.forward_xpu delegating to forward_native (#5568) 
Signed-off-by: Joshna Medisetty <joshna.medisetty@intel.com>
Signed-off-by: Joshna-Medisetty <joshna.medisetty@intel.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-02 23:06:17 -05:00
ZhanshenzhuiandSy03 48ae412a2f [model]Add experimental Mage-VL full-duplex adapter and offline integration (#6537) 
Signed-off-by: Zhanshenzhui <99891002+zyforsure@users.noreply.github.com>
Co-authored-by: Sy03 <1370724210@qq.com>
 2026-09-02 20:25:32 -07:00
DanaerLee ca6b4e2112 [Perf][Bugfix][OmniVoice] Restore float16 serving, and fuse the generator hot loop (#6317) 
Signed-off-by: MrlixiangWE <mrdanaer@gmail.com>
 2026-09-02 19:54:55 +00:00
Jim Ban b9a88bf51e [Bugfix][TTS] Isolate concurrent StepAudio2 TRT builds (#6957) 
Signed-off-by: BANANASJIM <bananasjim1@gmail.com>
 2026-09-02 19:53:02 +00:00
GXINandHongsheng Liu 3ab582c993 [Bugfix][Qwen3-TTS] Reject task mismatches before EngineCore dispatch (#6113) 
Signed-off-by: XIN GAO <1037396230@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-02 11:16:55 -07:00
jingchengtian 67ef0b64b9 [Hardware][Ascend] Use npu_rotary_mul fused kernel for MOSS-TTS codec RoPE (#6908) 
Signed-off-by: jingchengtian <tjc1995@126.com>
 2026-09-02 08:58:23 +00:00
Rakesh Kariya b864374089 [Bugfix][MiMo-Audio] Restore make_empty_intermediate_tensors on the talker for vLLM 0.28 (#6803) 
Signed-off-by: rk9595 <rakesh.kariya@somaiya.edu>
 2026-08-31 15:26:35 +08:00
akshatvishu f321cdbe16 [Bugfix] Use Seed-TTS reference audio with MiniCPM-o 4.5 (#6628) 
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
 2026-08-31 10:27:34 +08:00
akshatvishu c41925689d [BugFix][Higgs-Audio-V3] Fix transcript and concurrent sampling tests (#6422) 
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
 2026-08-30 08:52:15 +00:00
Yueqian Lin a6b559cf54 [Bugfix][MiniMax-Music3] Resolve stage subdirs against a real snapshot (#6640) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-08-30 06:51:24 +00:00
NATURE 87736a3cc5 [Bugfix][MiniCPM-o] Bound full-duplex Talker context with sliding recompute (#6626) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
 2026-08-30 00:49:42 +08:00
psv666andGao Han 5c47fd0bc8 [Bugfix][Qwen3-Omni] Handle missing packed modules mapping (#6748) 
Signed-off-by: psv666 <2693925048@qq.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-08-29 15:24:49 +08:00
abrahamzewoudieHongsheng Liuamy-why-3459
 		143ccc5a2f Fix Qwen3-Omni audio encoder TP when heads are not divisible by TP size (#4322) 
Signed-off-by: abrahamzewoudie <azewoudi@amd.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-08-29 14:52:02 +08:00
boatman bfe8967ca2 [Bugfix][StepAudio2] Fix async-chunk metadata mismatch and boundary glitches (#5917) 
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com>
 2026-08-28 21:45:24 -07:00
WallbreazzzCanlin GuoCanlin Guo
 		be335a86f9 fix(npu): extend bf16 autocast to NPU in MossAudioTokenizer decode (#6664) 
Signed-off-by: Wallbreazzz <110282866+Wallbreazzz@users.noreply.github.com>
Signed-off-by: Canlin Guo <961750412@qq.com>
Co-authored-by: Canlin Guo <961750412@qq.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
 2026-08-28 04:01:49 -07:00
Sy03 009b992c5b [Bugfix][MiniCPM-o] Isolate duplex handoff failures and barge-in (#6170) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-08-28 15:46:41 +08:00
Canlin Guolishunyang12yuanwu
 		b6a4baa659 [Feature] Support text encoder Disaggregation for H3 (#5885) 
Signed-off-by: Canlin Guo <canlinguosdu@gmail.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Signed-off-by: yuanwu <yuan.wu@intel.com>
Co-authored-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: yuanwu <yuan.wu@intel.com>
 2026-08-28 14:27:05 +08:00
BruceLoveDecimalandamy-why-3459 ab8d45d3cc [Bugfix][MiniCPM-o] Retire CFM DiT CUDA graphs a generation at a time (#6587) 
Signed-off-by: BruceLoveDecimal <liuqihao970610@gmail.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-08-28 14:07:57 +08:00
Hongsheng Liuandhsliuustc0106 4245fd6955 [Fix] Restore MammothModa2 support (#6694) 
Signed-off-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
Co-authored-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
 2026-08-28 08:32:32 +08:00
Zhou TaichangandClaude Opus 5 651b03225b [Rebase] Rebase to vllm 0.28.0 (#6606) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-27 06:12:55 +00:00
Sy03andsy03 432b2c6ca3 [Model] Add native full-duplex Nemotron VoiceChat serving (#6089) 
Signed-off-by: Sysy <1370724210@qq.com>
Signed-off-by: sy03 <137072421@qq.com>
Co-authored-by: sy03 <137072421@qq.com>
 2026-08-26 13:19:20 -07:00
Allen WuTRAE CLISy03TRAE CLI
 		f90e992dc1 [Model] Optimize CosyVoice3 TensorRT stream handoff (#5673) 
Signed-off-by: Allen Wu <allenwu2795@gmail.com>
Co-authored-by: TRAE CLI <noreply@bytedance.com>
Co-authored-by: Sy03 <1370724210@qq.com>
Co-authored-by: TRAE CLI <traecli@bytedance.com>
 2026-08-25 22:51:46 -07:00
wy17003andHongsheng Liu b454e34966 [Refactor][2/N Scheduler]Replace _FULL_PAYLOAD_INPUT_STAGES with a resolved stage transport capability. (#6149) 
Signed-off-by: wy17003 <yaow17003@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-25 15:24:59 +08:00