anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
957 Commits 235 Branches 31 Tags
034d579d770bb39fa2fb8d2bd1caca27f1affae3
Commit Graph
180 Commits
Author SHA1 Message Date
tzhouam 034d579d77 Update vLLM precompiled wheel version and refactor various components for improved functionality and error handling. Changes include modifying the get_name method in DiffusionQuantizationConfig to use class-level attributes, enhancing metadata handling in StageEngineCoreClient, and adjusting the AsyncOmni class constructor for better parameter management. Additionally, improved error handling in omni_snapshot_download and optimized initial chunk size computation in talker2code2wav_async_chunk for dynamic load adaptation. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-19 05:14:34 +00:00
tzhouam 342a209924 Merge remote-tracking branch 'origin/main' into dev/vllm-align 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-19 02:06:26 +00:00
Juan Pablo ZuluagapabloHongsheng Liu
 		fc5e8f825e [Optim][Qwen3TTS] big boost model throughput+latency high concurrency (#1852) 
Signed-off-by: pablo <pablo@agigo.ai>
Signed-off-by: JuanPZuluaga <juanz9312@gmail.com>
Co-authored-by: pablo <pablo@agigo.ai>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-03-18 12:22:05 -04:00
Taichang Zhou dc6f2fb81f Enhance cu_seqlens computation in Qwen3Omni_VisionTransformer by switching to tensor operations for improved performance and compatibility. Added error handling to fallback on a vectorized implementation if necessary. This update aligns with upstream changes and optimizes the handling of sequence lengths during attention execution. 2026-03-18 23:27:55 +08:00
Taichang Zhou 7a68bd37b7 Refactor cu_seqlens handling in Qwen3Omni_VisionTransformer to align with upstream vLLM changes. The sequence length helper now retains cu_seqlens in numpy format and converts it for attention execution, improving compatibility and performance. 2026-03-18 23:23:41 +08:00
汪志鹏 61e170c69a [Feature]: Remove some useless hf_overrides in yaml (#1898) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-03-18 19:46:44 +08:00
Y. FisherandCanlin Guo f52b51533a [Feature] Add cache-dit support for HunyuanImage3 (#1848) 
Signed-off-by: KexiongYu <yukexiong1@huawei.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
 2026-03-18 10:18:27 +00:00
Chenguang ZhengyinpePeiqi YinwuhangPeiqi Yinlinyueqianprincepridelishunyang
 		3158e9d28f [Entrypoint][Refactor] vLLM-Omni Entrypoint Refactoring (#1908) 
Signed-off-by: yinpe <11810305@mail.sustech.edu.cn>
Signed-off-by: Peiqi Yin <yinpeiqi809@gmail.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: wuhang <whlbx@hotmail.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
Signed-off-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: yinpe <11810305@mail.sustech.edu.cn>
Co-authored-by: Peiqi Yin <yinpeiqi809@gmail.com>
Co-authored-by: wuhang <whlbx@hotmail.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: lishunyang <lishunyang12@163.com>
 2026-03-18 17:33:53 +08:00
Yueqian Lin fe786f1a19 Fix Base voice clone streaming quality and stop-token crash (#1945) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-18 06:54:12 +08:00
Taichang Zhou 8a31e7865d Increase timeout for Diffusion Model Test and Diffusion Images API LoRA E2E in Buildkite configurations to 30 minutes. Also, introduce a new timeout constant for diffusion initialization in relevant test files to enhance stability during testing. Update OmniServer initialization to utilize the new timeout settings. 2026-03-17 23:43:46 +08:00
tzhouam 722ad6bbae rebase: align vllm-omni with vLLM 0a0a1a198be8 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-17 03:26:42 +00:00
tzhouam 0eccb77c4f Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-03-17 02:48:50 +00:00
Juan Pablo Zuluaga cf328eb8e8 [Bug][Qwen3TTS][Streaming] remove dynamic initial chunk and only compute on initial request (#1930) 
Signed-off-by: JuanPZuluaga <juanz9312@gmail.com>
 2026-03-16 15:23:09 -04:00
rongfu.leng 4036cd547c [Misc] removed qwen3_tts.py as it is out-dated (#1926) 
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
 2026-03-16 13:09:04 -04:00
tzhouam a26e568dfc Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-03-16 07:30:43 +00:00
Canlin Guo 9718a9e369 [NPU] Upgrade to v0.17.0 (#1890) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-16 14:42:59 +08:00
tzhouam 3a508d7b7a rebase: align vllm-omni with vLLM e9163b536e72 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-16 02:32:39 +00:00
tzhouam 97899c6c98 Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-03-13 19:03:57 +00:00
汪志鹏 e0b71056ba [BugFix]: Fix bagel online inference bug (#1804) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-03-13 08:58:44 +00:00
tzhouam d3db476f2d Merge branch 'main' into dev/vllm-align 2026-03-13 06:51:22 +00:00
tzhouam e17fb0e97e Refactor OpenAI serving components and enhance multimodal handling 
- Updated import for OpenAIServingEmbedding in api_server.py.
- Enhanced omni_init_app_state to initialize renderer for engine_client.
- Added handle_oov_mm_token parameter to multiple model classes for better multimodal token handling.
- Improved comments for clarity in various model files.
- Adjusted GPUARModelRunner to ensure proper handling of late interaction runner attributes.

Signed-off-by: [Your Name] <[Your Email]>

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-13 01:55:17 +00:00
Yueqian Lin 366b3368ea Add Fish Speech S2 Pro support with online serving and voice cloning (#1798) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-12 16:20:52 -04:00
Yueqian Lin 9c6fbd76be Enable async_scheduling by default for Qwen3-TTS (#1853) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-12 12:30:20 -04:00
usberkeleyprinceprideHongsheng Liu
 		b3cced310a [bugfix] Add Interleaved 2D Rotary Embedding for HunyuanImage3 (#1784) 
Signed-off-by: Bradley <bradley.b.pitt@gmail.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-03-12 21:33:03 +08:00
tzhouam 11b6c16007 rebase: align vllm-omni with vLLM 84e436ed1c94 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-11 19:24:44 +00:00
tzhouam 08a2673c66 rebase: align vllm-omni with vLLM 84e436ed1c94 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-11 07:29:52 +00:00
Junhong Liu 2ee4a07adb [Refactor][Perf] Qwen3-omni: code predictor with re-prefill + SDPA and eliminate decode hot-path CPU round-trips (#1758) 
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
 2026-03-11 07:52:21 +08:00
Juan Pablo Zuluagaandpablo 1d0a97f0d1 [feat][Qwen3TTS] Simple dynamic TTFA based on Code2Wav load (#1714) 
Signed-off-by: pablo <pablo@agigo.ai>
Co-authored-by: pablo <pablo@agigo.ai>
 2026-03-10 21:02:33 +00:00
Baoyuan Qi 2fdc647ed7 [BUGFIX] Add compatibility for mimo-audio with vLLM 0.17.0 (#1752) 
Signed-off-by: Baoyuan Qi <qibaoyuan@xiaomi.com>
 2026-03-10 20:57:26 +00:00
Sy03 761eff930a [Fix][Qwen3-TTS] Preserve ref_code decoder context for Base ICL (#1731) 2026-03-09 13:36:27 -04:00
汪志鹏 7271116973 [Bagel]: Support multistage img2img (#1669) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-03-09 22:23:50 +08:00
Zhou Taichang汪志鹏Hongsheng LiuCyrus Leung
 		89b527e533 Dev/rebase v0170 (#1639) 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-03-09 17:29:38 +08:00
zhaotyerlinyueqianYueqian Lin
 		41964cb90a feat(tts): add voice upload API for Qwen3-TTS (#1201) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-03-09 11:02:11 +08:00
Yueqian Lin a9f694b08a [Bugfix] Use upstream MediaConnector for ref_audio resolution (#1661) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-09 01:38:26 +00:00
Roman Koshkin 0e432e524c [Bugfix] (qwen3_tts): enable batched offline inference by fixing tens… (#1417) 
Signed-off-by: Roman Koshkin <roman.koshkin@sbintuitions.co.jp>
 2026-03-09 09:17:10 +08:00
Yueqian Lin 3bc89de0c0 Add non-async chunk support for Qwen3-TTS (#1678) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-06 14:45:32 +08:00
Baoyuan Qi 32fd084ae2 [MiMo-Audio] Bugfix tp lg than 1 (#1688) 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2026-03-06 14:45:08 +08:00
Sy03 90ec499bd5 [Refactor][Perf] Qwen3-TTS: re-prefill Code Predictor with torch.compile + enable Code2Wav decoder CUDA Graph (#1617) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-03-06 18:21:55 +13:00
Du BinlinyueqianYueqian Lin
 		fd51841230 perf: replace per-element .item() GPU syncs with batch .tolist() in TTS code predictor (#1614) 
Signed-off-by: dubin555 <dubin555@gmail.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-03-06 12:22:01 +08:00
Jack Leegoogle-labs-jules[bot]JackLeeHal
 		c0b714a8ac [Feature] Support flexible task_type configuration for Qwen3-TTS models (#1197) 
Signed-off-by: Jack Lee <695697442@qq.com>
Signed-off-by: jackleehal <695697442@qq.com>
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: JackLeeHal <15664755+JackLeeHal@users.noreply.github.com>
 2026-03-05 13:39:09 -05:00
zhanqiuhuandYueqian Lin e643aaec15 [Bugfix] Fix all-silence TTS output: use float32 for speech tokenizer decoder (#1664) 
Signed-off-by: Zhanqiu Hu <zh338@cornell.edu>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-03-06 00:04:26 +08:00
Juan Pablo Zuluagaandpablo 6a45efb287 [Feat][Qwen3TTS] reduce TTFA with flexible initial phase (#1583) 
Signed-off-by: pablo <pablo@agigo.ai>
Co-authored-by: pablo <pablo@agigo.ai>
 2026-03-05 23:14:41 +08:00
Ziming Huang 070ea0ddc3 [Optimize][Qwen3-Omni] Reduce inter-packet latency in async chunk (#1656) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
 2026-03-05 22:26:30 +08:00
dsinghviandHongsheng Liu b63e7c7c9a [Model] Fun cosy voice3-0.5-b-2512 (#498) 
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-03-04 19:54:59 -05:00
ShiCheng DengcatcatiwzbiHongsheng LiudsinghviCanlin GuoPeiqi Yin汪志鹏
 		1612948062 add support for MammothModa2 model (#336) 
Signed-off-by: HonestDeng <2958906959@qq.com>
Signed-off-by: iwzbi <iwzbi@zju.edu.cn>
Signed-off-by: iwzbi <wzbi@zju.edu.cn>
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: catcat <108673086+iwzbi@users.noreply.github.com>
Co-authored-by: iwzbi <iwzbi@zju.edu.cn>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: dsinghvi <divyanshsinghvi@gmail.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-03-04 22:52:56 +08:00
usberkeleyandprincepride 84338494d2 [Model] Add Hunyuan Image3 AR Support (#759) 
Signed-off-by: Bradley <bradley.b.pitt@gmail.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: princepride <wangzhipeng628@gmail.com>
 2026-03-04 21:49:38 +08:00
Yueqian Lin a3240fd484 [Bugfix] Fix transformers 5.x compat issues in online TTS serving (#1536) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-04 01:28:32 +08:00
Junhong Liu 742d845b8e Make chunk_size and left_context_size configurable via YAML for async chunking (#1423) 
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Signed-off-by: Junhong Liu <ljh_lbj@163.com>
 2026-03-03 22:06:52 +08:00
zhanqiuhu 80c89e00cb [Bugfix] Fix Qwen3-TTS code predictor crash due to missing vLLM config context (#1619) 
Signed-off-by: Zhanqiu <zh338@cornell.edu>
 2026-03-03 11:02:13 +08:00
Baoyuan Qi f371f7bf05 [bugfix] Fix unexpected argument 'is_finished' in function llm2code2wav_async_chunk of mimo-audio (#1570) 
Signed-off-by: Baoyuan Qi <qibaoyuan@126.com>
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2026-03-02 21:49:06 +08:00