anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
1,370 Commits 235 Branches 31 Tags
26bbe7b527c7248a060e6fd06d7ff1f47005cbfd
Commit Graph
1370 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Yueqian Lin 26bbe7b527 fix(moss_tts_nano): address PR review feedback 
- Wire ref_audio through /v1/audio/speech: _build_moss_tts_params is now
  async, resolves ref_audio via MediaConnector and passes the waveform as
  prompt_audio_array so the model can materialise a temp WAV for upstream
  inference_stream (previously voice cloning via the REST endpoint was a
  no-op). Serving now also requires ref_text when ref_audio is provided.
- Fix per-request EOS in batched decode: replace the shared
  _ar_emit_stop_token bool with a _ar_last_chunk_flags list so
  compute_logits emits EOS per row; finished requests no longer wait for
  the slowest peer in a max_num_seqs=4 batch.
- Snapshot and restore CPU+CUDA RNG state around torch.manual_seed to
  limit global-state bleed; add comment noting deterministic output
  under concurrent batching is best-effort (upstream inference_stream
  uses the global RNG).
- Align _DEFAULT_MODE with the offline example and tests ("voice_clone").
- Clean up outdated "buffer first" comment in _create_stream_gen; document
  single-threaded AR-worker assumption for _stream_gens; add one-liner
  explaining the load_weights iterator drain.

Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-04-21 17:58:50 -04:00
Yueqian Lin 3bd77d3f8f Merge branch 'main' into feat/moss-tts-nano 
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-04-21 17:50:21 -04:00
NumberWan b60a661b3f [Feature] Load Balancer - Add LeastQueueLengthBalancer RoundRobinBalancer (#2448) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
 2026-04-21 18:17:28 +00:00
Nick Cao 339a6da28c [Bugfix] treewide: drop references to librosa (#2996) 
Signed-off-by: Nick Cao <ncao@redhat.com>
 2026-04-21 16:04:15 +00:00
Hongsheng Liu 25b451a552 Codex revert pr reviewer (#2959) 
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
 2026-04-21 21:43:15 +08:00
WeiQing Chen 0cdec8210f [Bugfix] ComfyUI image-to-image DALL-E endpoint cases #2886 (#2980) 
Signed-off-by: david6666666 <530634352@qq.com>
 2026-04-21 21:37:52 +08:00
Yueqian LinandSYLAR 3f504b45d2 [Config Refactor] Migrate 5 TTS models (VoxCPM2 / CosyVoice3 / MiMo Audio / Voxtral TTS / Fish Speech S2 Pro) to Pipeline + Deploy schema (#2958) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-04-21 21:17:53 +08:00
fan2956 c2578f7131 [Bugfix] Fix VAE parallelism dist.gather performance bottleneck on NPU (#2969) 
Signed-off-by: fan2956 <zhoufan53@huawei.com>
 2026-04-21 20:35:02 +08:00
bjf-frz 5f029ee4d5 [Enhancement]remove duplicate video preprocess in Wan2.2 pipeline (#2963) 
Signed-off-by: bjf-frz <frz123db@gmail.com>
 2026-04-21 20:33:09 +08:00
fywc b001151e0d [Model] Add HSDP support for Stable-Audio-Open (#2982) 
Signed-off-by: hanzheli <hanzheli@kuaishou.com>
 2026-04-21 18:13:20 +08:00
Haco ad7c966b1d [BugFix] add missing subtalker sampling config to Qwen3-TTS deploy YAML (#2940) 
Signed-off-by: xiaohajiayou <923390377@qq.com>
 2026-04-21 14:47:22 +08:00
Daniel HuangChenguang ZhengZhou TaichangHongsheng Liubjf-frz
 		52b53367ef [Enhancement] Engine runtime errors (#2426) 
Signed-off-by: Daniel Huang <daniel1.huang@intel.com>
Signed-off-by: Daniel Huang <pilotflyer824@gmail.com>
Signed-off-by: bjf-frz <frz123db@gmail.com>
Co-authored-by: Chenguang Zheng <645327136@qq.com>
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: bjf-frz <frz123db@gmail.com>
 2026-04-21 14:46:05 +08:00
amy-why-3459 c1ba86ab5b [BugFix] Fix the issue with stream=True (#2955) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-04-21 11:51:39 +08:00
wangyu de4e47237a [CI][Bugfix] Improve cosine similarity calculation by incorporating length harmony adjustment in text comparison (#2964) 
Signed-off-by: wangyu <410167048@qq.com>
 2026-04-21 11:50:50 +08:00
Vensen e076378df1 [Feat][sleepmode] add omni sleepmode and ack protocol (#2022) 2026-04-21 08:19:53 +08:00
SYLAR 618268d6b9 [Bugfix] Truncate mimo-audio code2wav prompt to MAX_CODE2WAV_TOKENS (#2693) 
Signed-off-by: lishunyang <lishunyang12@163.com>
 2026-04-20 16:40:29 +00:00
Didan Deng 400690f63b [CI] Remove small resolution test in Qwen-Image Perf test when vae patch parallel is enabled (#2872) 
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-04-20 16:29:32 +00:00
Ayush Agarwal dc8a9e2f17 [Model] feat: FastGen DMD2-distilled Wan 2.1 pipelines (T2V, I2V) (#2749) 
Signed-off-by: ayushag <ayushag@nvidia.com>
 2026-04-20 21:21:43 +08:00
bjf-frz 71d81d475a [Enhancement] add pytorch profiler ops and memory record (#2472) 
Signed-off-by: bjf-frz <frz123db@gmail.com>
 2026-04-20 20:58:14 +08:00
Sheral Kumar 461bddca66 CI: publish Omni images to a separate Docker Hub repository (#2829) 
Signed-off-by: Sheral Kumar <shekumar@amd.com>
 2026-04-20 20:52:22 +08:00
John Liu BUAA 6128f6df1f [Example] Add Hunyuan-Image3 end2end.py and README.md (#2590) 
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com>
 2026-04-20 20:15:14 +08:00
Canlin Guo 23b2a95df6 [Docs] Add Wan2.2 image-to-video recipe for Ascend NPU (A2/A3) (#2919) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-04-20 19:35:48 +08:00
Jared WenandHongsheng Liu 7e28eda929 [Bugfix] Fix GLM-Image output dimensions and image edit pipeline (#2320) 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: Jared Wen <w13431838023@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-04-20 19:24:06 +08:00
WeiQing Chen c2859e9cd7 [Revert] drop Wan2.2 prompt-length enforcement from #2847 (#2877) 
Signed-off-by: david6666666 <530634352@qq.com>
 2026-04-20 19:19:38 +08:00
fywc d613864dbf [Model] Add HSDP support for LTX-2 (#2899) 
Signed-off-by: hanzheli <hanzheli@kuaishou.com>
Signed-off-by: fywc <hanzheli@kuaishou.com>
 2026-04-20 18:51:16 +08:00
wangyu 8a9add1c81 [CI] Restructure vLLM-Omni Test Layout, Fixture Scope, and Support Modules (#2620) 
Signed-off-by: wangyu <410167048@qq.com>
 2026-04-20 14:23:42 +08:00
amy-why-3459 0393c58533 [BugFix] Fixed the issue where --no-async-chunk was not working. (#2934) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-04-20 13:50:17 +08:00
rein yang d435fe0707 [Docs] update documents (#2921) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
 2026-04-20 12:55:04 +08:00
fan2956 78f237e603 [Perf] Optimize Wan2.2 device free on image preprocess (#2852) 
Signed-off-by: fan2956 <zhoufan53@huawei.com>
 2026-04-20 02:36:22 +00:00
SYLAR cd384d95f3 [Config Refactor 2.5/N] Centralize pipeline registry (#2915) 
Signed-off-by: lishunyang <lishunyang12@163.com>
 2026-04-20 01:18:03 +08:00
Yuanheng ZhaoandDidan Deng 68f28f9631 [Feat] Support layerwise CPU offloading for more videogen models (#2018) 
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-04-20 00:43:53 +08:00
HacoSamitSYLAR
 		93beef17aa [Bugfix] diffusion end points allow model mismatch (#2805) 
Signed-off-by: xiaohajiayou <923390377@qq.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-04-20 00:21:03 +08:00
Will.houandwillamhou 15684518d0 perf(helios): replace strided RoPE with stack+flatten for contiguous memory (#2474) 
Signed-off-by: willamhou <willamhou@ceresman.com>
Co-authored-by: willamhou <willamhou@ceresman.com>
 2026-04-19 20:28:48 +08:00
Sy03 26edc7f2fb [Bugfix][VoxCPM2]: Fix vectorized_gather OOB under concurrent prefill+decode batches (#2903) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-04-19 20:39:14 +12:00
SYLARreidliu41xiaohajiayouAlex BrooksHongsheng Liu
 		a3903810fe [Config Refactor][2/N] Pipeline + Deploy Config Schema (#2383) 
Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: xiaohajiayou <75477391+xiaohajiayou@users.noreply.github.com>
Co-authored-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-04-19 14:36:00 +12:00
Sy03 a683b1dd75 [Bugfix][VoxCPM2] Fix voice-clone decode loop by padding prefill prompt (#2894) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-04-18 14:04:44 +00:00
Canlin Guo 9313f37056 [Agent] Add NPU main2main skill (#2858) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-04-18 21:57:39 +08:00
LancerandSamit 9cf1fe71f4 [Feat] cache-dit for GLM-Image (#1399) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-04-18 21:16:04 +08:00
ChenWenjing fe6cec6ba3 [doc][skip ci] remove redundant content in readme (#2901) 
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
 2026-04-18 18:18:12 +08:00
汪志鹏andHongsheng Liu 768931e6a4 [BugFix]: Fix multi-stage cfg bug (#2801) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-04-18 09:43:22 +00:00
ChenWenjing 4124a1f858 [Feature]Support audio streaming input and output-phase2 (#2581) 
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
 2026-04-18 15:42:41 +08:00
Nick Cao d2c23d7bd0 [Refactor] Remove resampy dependency (#2891) 
Signed-off-by: Nick Cao <ncao@redhat.com>
 2026-04-18 03:20:36 +00:00
WeiQing Chenanddavid6666666 4f71f7309e [Docs] Update WeChat QR code for community support (#2895) 
Signed-off-by: david6666666 <david6666666@users.noreply.github.com>
Co-authored-by: david6666666 <david6666666@users.noreply.github.com>
 2026-04-18 10:30:03 +08:00
Hongsheng LiuandCopilot 5a68c217e2 [skip ci][recipe] draft vllm-omni recipes (#2646) 
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-04-18 10:29:17 +08:00
Alex Brooks f346f2fdc6 Add Teacache Support for LongCat Image (#1487) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-04-18 07:52:35 +08:00
Joshna-MedisettyandHongsheng Liu b5ddff7242 Nextstep online e2e (#2107) 
Signed-off-by: Joshna Medisetty <joshna.medisetty@intel.com>
Signed-off-by: Joshna-Medisetty <joshna.medisetty@intel.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-04-18 07:51:50 +08:00
ChenWenjingYueqian Linlinyueqian
 		1637dbaaf3 [skip CI][Docs] Add Qwen3-Omni and Qwen3-TTS performance blog and figures (#1837) 
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: linyueqian <linyueqian@outlook.com>
 2026-04-18 07:50:35 +08:00
Nick CaoandClaude f2edb81a59 [Cleanup] Remove dead runtime.defaults config parameters (#2343) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-17 23:36:41 +08:00
Lancer 64d368d3fc [Bugfix] fix CI failure (#2884) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2026-04-17 23:32:19 +08:00
wangyu b4add5bd87 [CI] Skip test_bagel[parallel_tp_2] and test_wan22_i2v_online_serving_generates_video[wan22_i2v_usp2_hsdp2] (#2883) 
Signed-off-by: wangyu <410167048@qq.com>
 2026-04-17 23:10:51 +08:00