anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
1,271 Commits 235 Branches 31 Tags
033e4041bebcf4c7d1e4363867ce6bc438fa2e1f
Commit Graph
276 Commits
Author SHA1 Message Date
Yueqian Lin 033e4041be fix(moss-tts-nano): remove unused voice variable (ruff F841) 
The upstream inference_stream() API has no voice/spk parameter;
voice preset selection is not yet wired into the call. Remove the
dead assignment to silence ruff F841.

Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-04-14 13:11:43 -04:00
Yueqian Lin fc81b8504a feat(moss-tts-nano): add online serving, Gradio demo, and fix /v1/audio/speech integration 
- Register moss_tts_nano as TTS model stage in serving_speech.py
  (model detection, validation, prompt building)
- Fix registry mod_relname: moss_tts_nano -> modeling_moss_tts_nano
- Fix stage config: is_comprehension=true (required for generate task)
- Fix default mode: voice_clone -> continuation (built-in presets)
- Add compute_logits stub for VllmModelForTextGeneration protocol
- Remove unused _sentinel nn.Parameter
- Add Gradio demo with AudioWorklet streaming player (48kHz stereo)
- Add online/offline serving docs and launch scripts

TODO: Single-stage generation models don't support true streaming
(progressive audio chunks). Current TTFP = full generation time.
Needs multi-step scheduling support in GPUGenerationModelRunner.

Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-04-14 13:06:44 -04:00
Yueqian Lin da6b1b5741 feat(moss-tts-nano): add single-stage MOSS-TTS-Nano TTS support 
Integrates OpenMOSS-Team/MOSS-TTS-Nano (0.1B) as a single
GPUGenerationWorker stage.  Both the AR LM and MOSS-Audio-Tokenizer-Nano
codec run inside MossTTSNanoForGeneration.forward(), removing the need for
an inter-stage connector.

Key design choices:
- Weights loaded in load_weights() not __init__ (avoids pre-CUDA alloc)
- trust_remote_code delegates to upstream HF model classes
- codec path read from config.audio_tokenizer_pretrained_name_or_path
- inference_stream() collects progressive audio chunks for low latency
- 48 kHz stereo output; voice clone + continuation modes supported


Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-04-14 13:06:12 -04:00
Zhang JianandClaude Opus 4.6 17acd0589a [Test] Add Stable Audio offline e2e TeaCache Test (#2377) 
Signed-off-by: Zhang <jianmusings@gmail.com>
Signed-off-by: Zhang Jian <jianmusings@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-14 15:05:12 +08:00
iancarrasco-b10 48c30bc399 [Qwen3-TTS] Remove hardcoded distributed_executor_backend to improve single-GPU performance (#2604) 
Signed-off-by: Ian Carrasco <ian.carrasco@baseten.co>
 2026-04-14 14:49:13 +08:00
Sy03Yueqian LinYueqian Lin
 		14f7910900 [Perf]: Speedup VoxCPM2 TTS performance and Support PagedAttention (#2690) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-04-14 09:16:47 +12:00
汪志鹏 2c67c30550 [Bagel]: Support think mode in single stage deployment of Bagel (#2650) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-04-13 11:15:49 +00:00
Yueqian Lin d9e745ce2c [Fix] VoxCPM2: support raw audio for voice cloning via OpenAI API (#2720) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-04-13 06:30:56 +00:00
Jiaqian LiuandCanlin Guo 8097747a5d [Model] Adapt Wan2.2-I2V-A14B via LightX2V offline conversion path (#2134) 
Signed-off-by: Celeste-jq <591998922@qq.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
 2026-04-13 14:20:04 +08:00
zhou zhuoxin 16041ab550 [Refactor] Extend CFG Parallel to support 3 or 4 branch dispatch across M GPUs (#2423) 2026-04-12 22:06:49 +08:00
Yuanheng ZhaoHongsheng LiuSYLAR
 		c20cac86ce [Bugfix] Restore user config/runtime stage init timeout (#2519) 
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-04-12 00:48:47 +08:00
Samit d1fef41266 [Doc] Add LTX-2 online serving deployment recipes with optimization benchmarks (#1971) 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
 2026-04-11 16:50:03 +08:00
Yueqian LinandSYLAR a41174e083 [Model] VoxCPM2 native AR TTS support (#2658) 
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: SYLAR <lishunyang12@users.noreply.github.com>
 2026-04-11 10:21:26 +08:00
Nick CaoandClaude 2bc183f6f0 [Refactor] Remove dependency on librosa (#2273) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-11 02:11:31 +08:00
wuhangandDidan Deng cb91cbe61e [Entrypoint][Refactor]Stage CLI Refactor (#2020) 
Signed-off-by: wuhang <wuhang6@huawei.com>
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-04-10 14:40:37 +08:00
JohnJan f3f2dc590c [Feature]: support Flux.2-dev CFG-Parallel (#2010) 2026-04-10 13:55:21 +08:00
Sy03 694be6f2e1 [Bugfix] Fix Fish Speech voice clone FileNotFoundError on multi-GPU (#2606) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-04-09 20:34:34 +00:00
Dogeun KimYejoon Leeaidas
 		e2b0ee4b87 [Model] Add Dynin-omni model in vllm-omni (#1759) 
Signed-off-by: kdg6245@snu.ac.kr <kdg6245@snu.ac.kr>
Signed-off-by: Yejoon Lee (IPAI) <leeyejoon@snu.ac.kr>
Signed-off-by: aidas (arpa-kt) <aidaslab@gmail.com>
Signed-off-by: Dogeun Kim <82812668+DOGEUNNKIM@users.noreply.github.com>
Co-authored-by: Yejoon Lee (IPAI) <leeyejoon@snu.ac.kr>
Co-authored-by: aidas (arpa-kt) <aidaslab@gmail.com>
 2026-04-09 16:37:34 +08:00
汪志鹏 9d87229a30 [MagiHuman] Fix audio sample rate and fps propagation for online serving (#2554) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-04-09 06:52:12 +00:00
Lancer 8609bc8ed9 [Feat] image2image for Z-Image (#1580) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
Signed-off-by: Lancer <402430575@qq.com>
 2026-04-08 14:41:24 +08:00
Yuanheng Zhao 8a55d3d01f [Feat] Enable Layerwise CPU offloading for SD3.5, Ovis-Image, Nextstep_1, LongCat-Image (#2339) 
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
 2026-04-08 09:55:09 +08:00
Markus / Markandmarksverdhai feefdaee9f [Bugfix] Accept 'speaker' as alias for 'voice' in TTS speech API (#2424) 
Signed-off-by: marksverdhei <marksverdhei@hotmail.com>
Signed-off-by: marksverdhai <249650165+marksverdhai@users.noreply.github.com>
Co-authored-by: marksverdhai <249650165+marksverdhai@users.noreply.github.com>
 2026-04-07 15:07:01 +00:00
skf 9584dd6e69 [HunyuanImage3] Align system_prompt support with official implementation (#2270) 
Signed-off-by: skf1999 <13234016272@163.com>
 2026-04-07 15:37:17 +08:00
erfgss 5d4c9ec4fa [bugfix]bugfix dreamid (#2125) 
Signed-off-by: Chen Yang <2082464740@qq.com>
Signed-off-by: erfgss <97771661+erfgss@users.noreply.github.com>
 2026-04-07 12:06:58 +08:00
汪志鹏 5b2c4f909d [BugFix] Add bagel text2text/img2text think mode support (#2503) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-04-07 13:29:26 +12:00
汪志鹏andDidan Deng 0059ec878c [Model Support]: Magihuman support (#2301) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-04-04 09:29:09 +00:00
+11
Juan Pablo ZuluagaJuanPZuluagaYi LiuHongsheng LiuCanlin GuoBinh TangBinh TangDidan Dengrein yangzhumingjue138ChenWenjingvraiti汪志鹏Sy03chickeytonJerry SongLidang JiangClaude Opus 4.6Alex BrookslinyueqianYueqian Lin
 		f50c5a413f [Qwen3TTS] [TTS] [Feat] Refactor voice cache manager (#2108) 
Signed-off-by: JuanPZuluaga <juanz9312@gmal.com>
Signed-off-by: yiliu30 <yi4.liu@intel.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: Binh Tang <tangbinhna@gmail.com>
Signed-off-by: Binh Tang <binht@netflix.com>
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
Signed-off-by: vraiti <vraiti@redhat.com>
Signed-off-by: Songrui625 <songrui625@gmail.com>
Signed-off-by: Lidang Jiang <lidangjiang@gmail.com>
Signed-off-by: Lidang-Jiang <lidangjiang@gmail.com>
Signed-off-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: JuanPZuluaga <juanz9312@gmal.com>
Co-authored-by: Yi Liu <yi4.liu@intel.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
Co-authored-by: Binh Tang <tangbinhna@gmail.com>
Co-authored-by: Binh Tang <binht@netflix.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: rein yang <73573651+R2-Y@users.noreply.github.com>
Co-authored-by: zhumingjue138 <zhumingjue@huawei.com>
Co-authored-by: ChenWenjing <54166744+Shirley125@users.noreply.github.com>
Co-authored-by: vraiti <vraiti@redhat.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Sy03 <1370724210@qq.com>
Co-authored-by: chickeyton <ngton2014@gmail.com>
Co-authored-by: Jerry Song <46962917+Songrui625@users.noreply.github.com>
Co-authored-by: Lidang Jiang <119769478+Lidang-Jiang@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-04-03 16:51:58 -04:00
Yueqian Lin 0e83ebe1d4 [OmniVoice] Add two-stage TTS serving support (#2463) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-04-03 20:41:20 +00:00
Jerry Song cd71567b06 [Model] Add two stages inference for model LTX-2 distilled. (#2260) 
Signed-off-by: Songrui625 <songrui625@gmail.com>
 2026-04-03 23:17:49 +08:00
汪志鹏 fa275fd1be [BAGEL] [Feature]: Add thinking mode in Bagel multi-stage serving (#2447) 2026-04-03 11:47:57 +08:00
ChenWenjingandHongsheng Liu 728cf6d023 [Feature] add session based audio streaming input (#2208) 
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-04-02 22:51:41 +08:00
Binh TangBinh TangDidan Deng
 		458f402323 Add VACE support for WAN 2.1 conditional video generation (#1885) 
Signed-off-by: Binh Tang <tangbinhna@gmail.com>
Signed-off-by: Binh Tang <binht@netflix.com>
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: Binh Tang <binht@netflix.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-04-02 15:42:50 +08:00
NATURE d40840b114 [Perf] Bagel KV-ready early forwarding and time step consistency for /v1/chat/completions (#2398) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
 2026-04-01 08:50:45 +00:00
Samit 7b965a7b6c [Feat][Benchmark] Add synchronous video generation endpoint POST /v1/videos/sync for benchmark test (#2049) 
Signed-off-by: samithuang <285365963@qq.com>
 2026-04-01 10:47:33 +08:00
汪志鹏 369f301a1b [BugFix] Add BAGEL single-stage diffusion config and fix multiple <im_start><im_end> bug (#2381) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-04-01 09:19:13 +08:00
noobHappylife 7cb7b3e8ad Fix Qwen3-TTS gradio demo (#2372) 
Signed-off-by: noobhappylife <aratar1991@hotmail.com>
 2026-03-31 11:42:09 +00:00
chickeyton fa0792a0aa [Doc] Add transformers version requirement in GLM-Image example doc (#2265) 
Signed-off-by: chickeyton <ngton2014@gmail.com>
 2026-03-29 15:56:51 +00:00
rongfu.lengandHongsheng Liu 9d9e6abae6 [Feat] add helios fp8 quantization (#1916) 
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-03-29 06:35:56 +00:00
Lidang Jiang 168bea2440 [Misc] Make gradio an optional dependency and upgrade to >=6.7.0 (#2221) 
Signed-off-by: Lidang-Jiang <lidangjiang@gmail.com>
Signed-off-by: Lidang Jiang <lidangjiang@gmail.com>
 2026-03-28 12:33:55 -04:00
amy-why-3459 eced989b40 [Docs] Modify the documentation description for streaming output (#2300) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-03-28 11:24:15 +00:00
Ding Zuhao 5cf6870277 [Bugfix][HunyuanImage3.0] Fix default guidance_scale from 1.0 to 4.0 and port GPU MoE ForwardContext fix from NPU (#2142) 
Signed-off-by: Ding Zuhao <e1583181@u.nus.edu>
 2026-03-28 13:49:03 +08:00
Yuanheng Zhao a353594e2e [Misc] Clean up unused diffusion timing args in examples (#2266) 
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
 2026-03-28 07:03:17 +08:00
Sy03 d9912eb6e9 [Bugfix] Fix Fish Speech S2 Pro prompt handling for truncated audio & emotion tag (#2268) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-03-28 07:00:00 +08:00
dengyunyangand汪志鹏 c0d90b64c6 [Feature] support sp for hunyuan (#2163) 
Signed-off-by: dengyunyang <584797741@qq.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-03-27 17:28:13 +08:00
Reid 6f87a9382f Frontend] Support --dtype in qwen3_omni offline e2e script (#2246) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
 2026-03-27 03:12:11 +00:00
SamitZeyu Huang | 黃澤宇gcanlin
 		d96d63cb7f [Doc] Improve diffusion generation parameter docs for online serving (#2051) 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-26 15:12:16 +00:00
Juan Pablo ZuluagaandYueqian Lin 574ec99ef4 [Qwen3TTS][ServingSpeech] Bugfix/voice upload and add optional ref_text (#2046) 
Signed-off-by: JuanPZuluaga <juanz9312@gmail.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-03-26 14:26:52 +00:00
rein yang 8305477750 [Bugfix] remove default sampling parameters (#2173) 2026-03-25 20:34:13 -07:00
Baoyuan Qi 807f3133d7 [skip ci][Docs] Add FlashAttention requirement for audio generation to prevent noise-only outputs in mimo-audio model (#2205) 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2026-03-26 10:21:00 +08:00
Markus / MarkYueqian Linlinyueqian
 		29936a3edd [Frontend] Speaker embedding support for speech and voices APIs (#1227) 
Signed-off-by: marksverdhei <marksverdhei@hotmail.com>
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: linyueqian <linyueqian@outlook.com>
 2026-03-25 19:02:48 +00:00