anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
1,420 Commits 235 Branches 31 Tags
05883ff47065c5cc080d9b2df8191e71596fbd67
Commit Graph
309 Commits
Author SHA1 Message Date
Yueqian Lin 74522ca3ec Merge branch 'main' into feat/moss-tts-nano 
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-04-24 11:00:13 -04:00
Sy03shimei.jmhSYLAR
 		7b876324c2 [Feature] Streaming video input with EVS frame filtering (RFC #2201 Phase 2-4) (#2342) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: shimei.jmh <shimei.jmh@taobao.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-04-24 22:54:35 +08:00
汪志鹏andSYLAR 37e5954acf [Config Refactor]: Remove bagel yaml (#2936) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-04-24 21:21:30 +08:00
Hacoandlishunyang 22ffb85063 [Config Refactor] sentinel default precedence (#3078) 
Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: xiaohajiayou <923390377@qq.com>
Co-authored-by: lishunyang <lishunyang12@163.com>
 2026-04-24 20:07:24 +08:00
bjf-frz e24f86ae2e [Enhancement]modify profiling.md (#3051) 
Signed-off-by: bjf-frz <frz123db@gmail.com>
 2026-04-24 16:30:24 +08:00
Baoyuan Qi 5632b78b3d [Feature] XiaomiMiMo/MiMo-V2.5-ASR support (#3089) 
Signed-off-by: Baoyuan Qi <qibaoyuan@xiaomi.com>
 2026-04-24 13:55:58 +08:00
Lancer d20a6b5b39 [chore] refine the examples (#3095) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2026-04-24 13:40:38 +08:00
wuhang 2e067bbc24 [Docs] CLI Docs updates (#2978) 
Signed-off-by: wuhang <wuhang6@huawei.com>
 2026-04-24 11:25:22 +08:00
Yuanheng ZhaoLHXuuuHongsheng Liu
 		3a32cd6780 [Model] Ming-flash-omni-2.0 Omni-Speech and TTS (#2890) 
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com>
Co-authored-by: LHXuuu <xulianhao.xlh@antgroup.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-04-23 12:04:52 +08:00
Zeyu Huang | 黃澤宇 d8cc7a0409 [feat]: General diffusers adapter backend to run diffusion models (#2724) 
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
 2026-04-23 00:07:18 +08:00
Liang LvandSYLAR ee15f39ea7 [AutoRound] Support Qwen Omni W4A16 quantization model (#2670) 
Signed-off-by: lvliang-intel <liang1.lv@intel.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-04-22 19:54:45 +08:00
Jared WenandSYLAR 554233208f [ConfigRefactor] GLM-Image (#2977) 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: Jared Wen <w13431838023@gmail.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-04-22 15:09:28 +08:00
Chen-Yo SunandYueqian Lin 8e1bbc9ab5 [Feature][Voxtral TTS] Support per-model extra sampling params & add cfg_alpha for Voxtral TTS (#2338) 
Signed-off-by: Chen-Yo Sun <chenyo.sun@mistral.ai>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
 2026-04-22 03:49:50 +00:00
Yueqian Lin b03a41a7f9 refactor(moss_tts_nano): migrate to Pipeline + Deploy schema 
Follow-up to #2958. MOSS-TTS-Nano was authored before the schema refactor
and still shipped as a legacy stage_configs yaml; this aligns it with the
same layout the 5 migrated TTS models now use.

- vllm_omni/model_executor/models/moss_tts_nano/pipeline.py declares
  MOSS_TTS_NANO_PIPELINE (single LLM_AR stage, owns_tokenizer, audio
  output, stop_token_ids=[2] as a hard EOS backstop).
- vllm_omni/deploy/moss_tts_nano.yaml holds runtime knobs (max_num_seqs,
  gpu_memory_utilization, enforce_eager, default_sampling_params,
  skip_mm_profiling); trust_remote_code stays at deploy top-level.
- vllm_omni/config/pipeline_registry.py registers the entry so the lazy
  registry can resolve it.
- moss_tts_nano/__init__.py exports MossTTSNanoForGeneration (VoxCPM2 pattern).
- Removed vllm_omni/model_executor/stage_configs/moss_tts_nano.yaml.

Examples, shell scripts, READMEs, and buildkite-invoked tests are
updated to use `vllm serve <model> --omni` / `--deploy-config`
(auto-load kicks in; no --stage-configs-path or --trust-remote-code).

Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-04-21 18:07:50 -04:00
Yueqian Lin 3bd77d3f8f Merge branch 'main' into feat/moss-tts-nano 
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-04-21 17:50:21 -04:00
Nick Cao 339a6da28c [Bugfix] treewide: drop references to librosa (#2996) 
Signed-off-by: Nick Cao <ncao@redhat.com>
 2026-04-21 16:04:15 +00:00
Yueqian LinandSYLAR 3f504b45d2 [Config Refactor] Migrate 5 TTS models (VoxCPM2 / CosyVoice3 / MiMo Audio / Voxtral TTS / Fish Speech S2 Pro) to Pipeline + Deploy schema (#2958) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-04-21 21:17:53 +08:00
fywc b001151e0d [Model] Add HSDP support for Stable-Audio-Open (#2982) 
Signed-off-by: hanzheli <hanzheli@kuaishou.com>
 2026-04-21 18:13:20 +08:00
John Liu BUAA 6128f6df1f [Example] Add Hunyuan-Image3 end2end.py and README.md (#2590) 
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com>
 2026-04-20 20:15:14 +08:00
Jared WenandHongsheng Liu 7e28eda929 [Bugfix] Fix GLM-Image output dimensions and image edit pipeline (#2320) 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: Jared Wen <w13431838023@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-04-20 19:24:06 +08:00
Yuanheng ZhaoandDidan Deng 68f28f9631 [Feat] Support layerwise CPU offloading for more videogen models (#2018) 
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-04-20 00:43:53 +08:00
SYLARreidliu41xiaohajiayouAlex BrooksHongsheng Liu
 		a3903810fe [Config Refactor][2/N] Pipeline + Deploy Config Schema (#2383) 
Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: xiaohajiayou <75477391+xiaohajiayou@users.noreply.github.com>
Co-authored-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-04-19 14:36:00 +12:00
Sy03 a683b1dd75 [Bugfix][VoxCPM2] Fix voice-clone decode loop by padding prefill prompt (#2894) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-04-18 14:04:44 +00:00
ChenWenjing 4124a1f858 [Feature]Support audio streaming input and output-phase2 (#2581) 
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
 2026-04-18 15:42:41 +08:00
Nick Cao d2c23d7bd0 [Refactor] Remove resampy dependency (#2891) 
Signed-off-by: Nick Cao <ncao@redhat.com>
 2026-04-18 03:20:36 +00:00
Nick CaoandClaude f2edb81a59 [Cleanup] Remove dead runtime.defaults config parameters (#2343) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-17 23:36:41 +08:00
Yuanheng Zhao c0ccbb872f [Model] Add Ming-flash-omni-2.0 Thinker Stage (#1822) 
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
 2026-04-17 19:10:02 +08:00
Alex Brooks d463978098 [Model] Support Flux1 Schnell (#2528) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-04-17 16:02:30 +08:00
Nick CaoandClaude 1e3bb36521 [Refactor] Remove sox from dependencies (#2745) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-17 01:24:47 +00:00
TaffyOfficialTaffyOfficialClaude Opus 4.6
 		c3ca5daafb Feat/Add HunyuanImage-3.0-Instruct ar part support: (#2713) 
Signed-off-by: TaffyOfficial <2324465096@qq.com>
Co-authored-by: TaffyOfficial <2324465096@qq.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-16 23:15:25 +08:00
Yuanheng Zhao de5f8a23b2 [Doc][Misc] Update DreamID-Omni Example; Add DreamID-Omni post process function (#2809) 
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
 2026-04-16 03:26:48 +00:00
IsleOfDawnlightCeleste-jqlyj-jjjYueqian Lin
 		4bf4c63147 Add voxcpm model support. (#2467) 
Signed-off-by: Celeste-jq <591998922@qq.com>
Signed-off-by: lyj-jjj <liuyingjun5@huawei.com>
Signed-off-by: IsleOfDawnlight <stellamou@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Celeste-jq <591998922@qq.com>
Co-authored-by: lyj-jjj <liuyingjun5@huawei.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
 2026-04-15 15:04:58 +08:00
Yueqian Lin 02e5dc747d [Perf] VoxCPM2: streaming VAE + compile optimization (45% RTF reduction) (#2758) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-04-15 09:58:25 +08:00
Yueqian Lin 033e4041be fix(moss-tts-nano): remove unused voice variable (ruff F841) 
The upstream inference_stream() API has no voice/spk parameter;
voice preset selection is not yet wired into the call. Remove the
dead assignment to silence ruff F841.

Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-04-14 13:11:43 -04:00
Yueqian Lin fc81b8504a feat(moss-tts-nano): add online serving, Gradio demo, and fix /v1/audio/speech integration 
- Register moss_tts_nano as TTS model stage in serving_speech.py
  (model detection, validation, prompt building)
- Fix registry mod_relname: moss_tts_nano -> modeling_moss_tts_nano
- Fix stage config: is_comprehension=true (required for generate task)
- Fix default mode: voice_clone -> continuation (built-in presets)
- Add compute_logits stub for VllmModelForTextGeneration protocol
- Remove unused _sentinel nn.Parameter
- Add Gradio demo with AudioWorklet streaming player (48kHz stereo)
- Add online/offline serving docs and launch scripts

TODO: Single-stage generation models don't support true streaming
(progressive audio chunks). Current TTFP = full generation time.
Needs multi-step scheduling support in GPUGenerationModelRunner.

Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-04-14 13:06:44 -04:00
Yueqian Lin da6b1b5741 feat(moss-tts-nano): add single-stage MOSS-TTS-Nano TTS support 
Integrates OpenMOSS-Team/MOSS-TTS-Nano (0.1B) as a single
GPUGenerationWorker stage.  Both the AR LM and MOSS-Audio-Tokenizer-Nano
codec run inside MossTTSNanoForGeneration.forward(), removing the need for
an inter-stage connector.

Key design choices:
- Weights loaded in load_weights() not __init__ (avoids pre-CUDA alloc)
- trust_remote_code delegates to upstream HF model classes
- codec path read from config.audio_tokenizer_pretrained_name_or_path
- inference_stream() collects progressive audio chunks for low latency
- 48 kHz stereo output; voice clone + continuation modes supported


Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-04-14 13:06:12 -04:00
Zhang JianandClaude Opus 4.6 17acd0589a [Test] Add Stable Audio offline e2e TeaCache Test (#2377) 
Signed-off-by: Zhang <jianmusings@gmail.com>
Signed-off-by: Zhang Jian <jianmusings@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-14 15:05:12 +08:00
iancarrasco-b10 48c30bc399 [Qwen3-TTS] Remove hardcoded distributed_executor_backend to improve single-GPU performance (#2604) 
Signed-off-by: Ian Carrasco <ian.carrasco@baseten.co>
 2026-04-14 14:49:13 +08:00
Sy03Yueqian LinYueqian Lin
 		14f7910900 [Perf]: Speedup VoxCPM2 TTS performance and Support PagedAttention (#2690) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-04-14 09:16:47 +12:00
汪志鹏 2c67c30550 [Bagel]: Support think mode in single stage deployment of Bagel (#2650) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-04-13 11:15:49 +00:00
Yueqian Lin d9e745ce2c [Fix] VoxCPM2: support raw audio for voice cloning via OpenAI API (#2720) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-04-13 06:30:56 +00:00
Jiaqian LiuandCanlin Guo 8097747a5d [Model] Adapt Wan2.2-I2V-A14B via LightX2V offline conversion path (#2134) 
Signed-off-by: Celeste-jq <591998922@qq.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
 2026-04-13 14:20:04 +08:00
zhou zhuoxin 16041ab550 [Refactor] Extend CFG Parallel to support 3 or 4 branch dispatch across M GPUs (#2423) 2026-04-12 22:06:49 +08:00
Yuanheng ZhaoHongsheng LiuSYLAR
 		c20cac86ce [Bugfix] Restore user config/runtime stage init timeout (#2519) 
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-04-12 00:48:47 +08:00
Samit d1fef41266 [Doc] Add LTX-2 online serving deployment recipes with optimization benchmarks (#1971) 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
 2026-04-11 16:50:03 +08:00
Yueqian LinandSYLAR a41174e083 [Model] VoxCPM2 native AR TTS support (#2658) 
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: SYLAR <lishunyang12@users.noreply.github.com>
 2026-04-11 10:21:26 +08:00
Nick CaoandClaude 2bc183f6f0 [Refactor] Remove dependency on librosa (#2273) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-11 02:11:31 +08:00
wuhangandDidan Deng cb91cbe61e [Entrypoint][Refactor]Stage CLI Refactor (#2020) 
Signed-off-by: wuhang <wuhang6@huawei.com>
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-04-10 14:40:37 +08:00
JohnJan f3f2dc590c [Feature]: support Flux.2-dev CFG-Parallel (#2010) 2026-04-10 13:55:21 +08:00
Sy03 694be6f2e1 [Bugfix] Fix Fish Speech voice clone FileNotFoundError on multi-GPU (#2606) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-04-09 20:34:34 +00:00