anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
1,420 Commits 235 Branches 31 Tags
05883ff47065c5cc080d9b2df8191e71596fbd67
Commit Graph
329 Commits
Author SHA1 Message Date
Yueqian Lin 74522ca3ec Merge branch 'main' into feat/moss-tts-nano 
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-04-24 11:00:13 -04:00
Sy03shimei.jmhSYLAR
 		7b876324c2 [Feature] Streaming video input with EVS frame filtering (RFC #2201 Phase 2-4) (#2342) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: shimei.jmh <shimei.jmh@taobao.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-04-24 22:54:35 +08:00
汪志鹏andSYLAR 37e5954acf [Config Refactor]: Remove bagel yaml (#2936) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-04-24 21:21:30 +08:00
Hacoandlishunyang 22ffb85063 [Config Refactor] sentinel default precedence (#3078) 
Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: xiaohajiayou <923390377@qq.com>
Co-authored-by: lishunyang <lishunyang12@163.com>
 2026-04-24 20:07:24 +08:00
Lancer 0fe76857ac [Bugfix] fix GLM-Image multi-stage online generation fail (#3084) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2026-04-24 20:00:14 +08:00
Yuanheng ZhaoLHXuuuHongsheng Liu
 		3a32cd6780 [Model] Ming-flash-omni-2.0 Omni-Speech and TTS (#2890) 
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com>
Co-authored-by: LHXuuu <xulianhao.xlh@antgroup.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-04-23 12:04:52 +08:00
WeiQing Chen f327cd1c89 [Bugfix] Fix default sampling params for /v1/videos on main (#3049) 
Signed-off-by: david6666666 <530634352@qq.com>
 2026-04-23 11:31:13 +08:00
amy-why-3459 3aa84c4742 [Tests]Add accuracy benchmark L4 test cases (#2843) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-04-23 09:40:43 +08:00
wuhang fd9a0ee27f [Feature] Failure message shows more details (#2961) 
Signed-off-by: wuhang <wuhang6@huawei.com>
 2026-04-23 04:32:43 +08:00
NumberWan ead87aa521 [BugFix] Surface diffusion metrics in chat completions; sanitize Bagel img2img mm kwargs (#2932) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
 2026-04-22 16:57:01 +00:00
Zeyu Huang | 黃澤宇 d8cc7a0409 [feat]: General diffusers adapter backend to run diffusion models (#2724) 
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
 2026-04-23 00:07:18 +08:00
Chen-Yo SunandYueqian Lin 8e1bbc9ab5 [Feature][Voxtral TTS] Support per-model extra sampling params & add cfg_alpha for Voxtral TTS (#2338) 
Signed-off-by: Chen-Yo Sun <chenyo.sun@mistral.ai>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
 2026-04-22 03:49:50 +00:00
Yueqian Lin 26bbe7b527 fix(moss_tts_nano): address PR review feedback 
- Wire ref_audio through /v1/audio/speech: _build_moss_tts_params is now
  async, resolves ref_audio via MediaConnector and passes the waveform as
  prompt_audio_array so the model can materialise a temp WAV for upstream
  inference_stream (previously voice cloning via the REST endpoint was a
  no-op). Serving now also requires ref_text when ref_audio is provided.
- Fix per-request EOS in batched decode: replace the shared
  _ar_emit_stop_token bool with a _ar_last_chunk_flags list so
  compute_logits emits EOS per row; finished requests no longer wait for
  the slowest peer in a max_num_seqs=4 batch.
- Snapshot and restore CPU+CUDA RNG state around torch.manual_seed to
  limit global-state bleed; add comment noting deterministic output
  under concurrent batching is best-effort (upstream inference_stream
  uses the global RNG).
- Align _DEFAULT_MODE with the offline example and tests ("voice_clone").
- Clean up outdated "buffer first" comment in _create_stream_gen; document
  single-threaded AR-worker assumption for _stream_gens; add one-liner
  explaining the load_weights iterator drain.

Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-04-21 17:58:50 -04:00
Yueqian Lin 3bd77d3f8f Merge branch 'main' into feat/moss-tts-nano 
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-04-21 17:50:21 -04:00
WeiQing Chen 0cdec8210f [Bugfix] ComfyUI image-to-image DALL-E endpoint cases #2886 (#2980) 
Signed-off-by: david6666666 <530634352@qq.com>
 2026-04-21 21:37:52 +08:00
Yueqian LinandSYLAR 3f504b45d2 [Config Refactor] Migrate 5 TTS models (VoxCPM2 / CosyVoice3 / MiMo Audio / Voxtral TTS / Fish Speech S2 Pro) to Pipeline + Deploy schema (#2958) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-04-21 21:17:53 +08:00
Daniel HuangChenguang ZhengZhou TaichangHongsheng Liubjf-frz
 		52b53367ef [Enhancement] Engine runtime errors (#2426) 
Signed-off-by: Daniel Huang <daniel1.huang@intel.com>
Signed-off-by: Daniel Huang <pilotflyer824@gmail.com>
Signed-off-by: bjf-frz <frz123db@gmail.com>
Co-authored-by: Chenguang Zheng <645327136@qq.com>
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: bjf-frz <frz123db@gmail.com>
 2026-04-21 14:46:05 +08:00
Vensen e076378df1 [Feat][sleepmode] add omni sleepmode and ack protocol (#2022) 2026-04-21 08:19:53 +08:00
Jared WenandHongsheng Liu 7e28eda929 [Bugfix] Fix GLM-Image output dimensions and image edit pipeline (#2320) 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: Jared Wen <w13431838023@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-04-20 19:24:06 +08:00
amy-why-3459 0393c58533 [BugFix] Fixed the issue where --no-async-chunk was not working. (#2934) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-04-20 13:50:17 +08:00
HacoSamitSYLAR
 		93beef17aa [Bugfix] diffusion end points allow model mismatch (#2805) 
Signed-off-by: xiaohajiayou <923390377@qq.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-04-20 00:21:03 +08:00
SYLARreidliu41xiaohajiayouAlex BrooksHongsheng Liu
 		a3903810fe [Config Refactor][2/N] Pipeline + Deploy Config Schema (#2383) 
Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: xiaohajiayou <75477391+xiaohajiayou@users.noreply.github.com>
Co-authored-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-04-19 14:36:00 +12:00
Sy03 a683b1dd75 [Bugfix][VoxCPM2] Fix voice-clone decode loop by padding prefill prompt (#2894) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-04-18 14:04:44 +00:00
ChenWenjing 4124a1f858 [Feature]Support audio streaming input and output-phase2 (#2581) 
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
 2026-04-18 15:42:41 +08:00
Mike QiuMike_QiuHongsheng Liu
 		6b7be88cd5 Fix NoneType error of outputs (#2315) 
Signed-off-by: Mike_Qiu <qiudayu.qdy@antgroup.com>
Co-authored-by: Mike_Qiu <qiudayu.qdy@antgroup.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-04-17 20:10:24 +08:00
LancerandHongsheng Liu cf75ae6ed5 [Bugfix] Fix image quality in /v1/images/generations for multi-stage pipeline (#2267) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-04-17 20:09:41 +08:00
WeiQing ChenandGao Han f658bcb0bf [Bugfix] Limit Qwen-Image-Edit-2511 input image count (#2840) 
Signed-off-by: david6666666 <530634352@qq.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-04-17 20:07:30 +08:00
Bingyu (Spencer) Liu a5a4998ece [Feature] Support Prefill-Decode disaggregation via vLLM KV transfer (#2220) 
Signed-off-by: LiuBingyu <liubingyu62@gmail.com>
 2026-04-17 18:43:17 +08:00
Peiqi Yin bbd6a443ed [Core] Refactor CFG companion tracker and use in Orchestrator (#2623) 
Signed-off-by: yinpe <11810305@mail.sustech.edu.cn>
 2026-04-17 08:13:00 +00:00
LancerandHongsheng Liu bf64a6b841 [Feature] Flux2 klein inpaint (#1180) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-04-17 07:55:00 +08:00
Sy03 7d64a7c996 [BugFix][VoxCPM2]: split multichar Chinese tokens to match training tokenization (#2832) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-04-16 10:07:48 -05:00
Mike QiuMike_QiuClaude Opus 4.6
 		2ec91d4dfd [FIX] Preserve YAML default stop words when request sends empty list (#2855) 
Signed-off-by: Mike_Qiu <qiudayu.qdy@antgroup.com>
Co-authored-by: Mike_Qiu <qiudayu.qdy@antgroup.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-04-16 14:40:34 +00:00
NATUREandHongsheng Liu 4d816ff1de [Feature] Bagel: Support tp+cfg parallel using mooncake transfer engine connector (#2705) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-04-16 16:25:13 +08:00
IsleOfDawnlightCeleste-jqlyj-jjjYueqian Lin
 		4bf4c63147 Add voxcpm model support. (#2467) 
Signed-off-by: Celeste-jq <591998922@qq.com>
Signed-off-by: lyj-jjj <liuyingjun5@huawei.com>
Signed-off-by: IsleOfDawnlight <stellamou@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Celeste-jq <591998922@qq.com>
Co-authored-by: lyj-jjj <liuyingjun5@huawei.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
 2026-04-15 15:04:58 +08:00
WeiQing Chen 6c6551dff8 [Feature] feat: add video frame interpolation postprocess (#2555) 
Signed-off-by: David Chen <530634352@qq.com>
 2026-04-15 11:52:20 +08:00
Samit 61a3cbdff5 [Perf] Eliminate Hop 3 IPC overhead for single-stage diffusion via inline execution (#2736) 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
 2026-04-15 11:46:06 +08:00
amy-why-3459 227bab3038 [Benchmark]Omni-modality model accuracy benchmark(Daily-Omni & seed-tts-eval) (#2558) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-04-15 11:07:35 +08:00
rongfu.leng 9e46a79c17 [Bugfix] Handle numpy array outputs when generate image (#1680) 
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
 2026-04-15 08:52:32 +08:00
Yueqian Lin fc81b8504a feat(moss-tts-nano): add online serving, Gradio demo, and fix /v1/audio/speech integration 
- Register moss_tts_nano as TTS model stage in serving_speech.py
  (model detection, validation, prompt building)
- Fix registry mod_relname: moss_tts_nano -> modeling_moss_tts_nano
- Fix stage config: is_comprehension=true (required for generate task)
- Fix default mode: voice_clone -> continuation (built-in presets)
- Add compute_logits stub for VllmModelForTextGeneration protocol
- Remove unused _sentinel nn.Parameter
- Add Gradio demo with AudioWorklet streaming player (48kHz stereo)
- Add online/offline serving docs and launch scripts

TODO: Single-stage generation models don't support true streaming
(progressive audio chunks). Current TTFP = full generation time.
Needs multi-step scheduling support in GPUGenerationModelRunner.

Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-04-14 13:06:44 -04:00
Bvicii 4fb078a031 [Bugfix] Reject /v1/audio/speech for Qwen omni models (#2763) 
Signed-off-by: Bvicii <yizhanhuang2002@gmail.com>
 2026-04-14 12:53:06 +00:00
Samit 644edac0b6 [Perf] Optimize MP4 encoding latency in video generation (#2735) 
Signed-off-by: samithuang <285365963@qq.com>
 2026-04-14 14:20:38 +08:00
Sy03Yueqian LinYueqian Lin
 		14f7910900 [Perf]: Speedup VoxCPM2 TTS performance and Support PagedAttention (#2690) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-04-14 09:16:47 +12:00
amy-why-3459 2b70e89535 [Revert] Revert "[Log] Wire stat loggers into AsyncOmniEngine to match AsyncLL… (#2716) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-04-13 14:55:16 +08:00
Canlin Guo 5d58abb11e [Log] Wire stat loggers into AsyncOmniEngine to match AsyncLLM (#2551) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-04-12 19:54:23 +08:00
ReidandHongsheng Liu 38dfe56fdf [Bugfix] Validate speaker in chat endpoint and fix case-insensitive lookup (#2407) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-04-12 00:14:16 -04:00
Juan Pablo Zuluaga f7e8df9dbd [TTS][OmniVoice] Add voice cloning support for OmniVoice TTS (#2676) 
Signed-off-by: JuanPZuluaga <juanz9312@gmail.com>
 2026-04-11 12:14:03 -04:00
teithandYueqian Lin 001f2e32e2 [BUG FIX]: prevent EngineCore crash when Qwen TTS Base task is missing ref_text (#2203) 
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-04-11 02:46:38 +00:00
Nick CaoandClaude 2bc183f6f0 [Refactor] Remove dependency on librosa (#2273) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-11 02:11:31 +08:00
bjf-frz fbb5dd5794 [Bugfix]: modify diffusion pipeline profiler result in videos (#2647) 
Signed-off-by: bjf-frz <frz123db@gmail.com>
 2026-04-10 16:48:06 +08:00
wuhangandDidan Deng cb91cbe61e [Entrypoint][Refactor]Stage CLI Refactor (#2020) 
Signed-off-by: wuhang <wuhang6@huawei.com>
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-04-10 14:40:37 +08:00