anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
759 Commits 235 Branches 31 Tags
663976b8ca77366bb794e75d4e323d59bc405ae2
Commit Graph
478 Commits
Author SHA1 Message Date
tzhouam 663976b8ca Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-02-20 01:38:44 +00:00
Chendi.Xue 0b577a7263 [v0.16.0][BUG FIX]Fix hunyuan MOE after update to 0.16.0 (#1401) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
 2026-02-19 23:57:11 +13:00
tzhouam b867bacd24 rebase 2025/02/16 
Qwen3 Omni offline/online passed

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-16 03:21:21 +00:00
Zhou Taichang汪志鹏Hongsheng LiuIsotr0pyZJY0516youkaichao
 		75770c9309 [Rebase] Rebase to vllm v0.16.0 (#1357)
ComfyUI Integration -- Publish to Comfy registry / Publish Custom Node to registry (push) Failing after 44s
Details 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: ZJY0516 <zhu.jiangyun@foxmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2026-02-13 18:05:07 +08:00
tzhouam 15099119dd Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0.16.0 2026-02-13 09:55:18 +00:00
汪志鹏 630e84ef93 [BugFix]: Revert utils change (#1369) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-02-13 17:54:09 +08:00
tzhouam f24273dc03 Update vLLM version to 0.16.0 in quickstart and CUDA installation documentation, and adjust parameters in multimodal generation script for improved performance. Refactor request logging in serving_chat.py for better input tracking. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 09:50:11 +00:00
tzhouam 8a4eee9bed Enhance configuration and debugging: Added asyncio mode to pytest settings, increased timeout for Qwen3-TTS E2E test, and improved logging in GPU model runner for better traceability of input modalities. Updated Dockerfile.ci to streamline VLLM installation process. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 06:39:28 +00:00
tzhouam 31e9fdbb57 Refactor chat_utils to remove unused multi-modal classes and add async audio extraction from video. Update README to reflect new entry point for audio extraction. Enhance serving_chat to process audio from video URLs when specified. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 03:27:16 +00:00
tzhouam 39a045f439 debug Qwen 3 Omni cuda graph 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 02:57:05 +00:00
tzhouam 7e9cae56fc Refactor Dockerfile.ci to use precompiled VLLM installation and remove unused parse.py file. Update serving_chat.py to streamline rendering logic and enhance compatibility with new API. Modify OmniInputPreprocessor to directly handle SingletonDictPrompt, improving input processing efficiency. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 02:28:25 +00:00
Hongsheng Liu 622c6f5f14 Merge branch 'main' into dev/rebase-v0.16.0 2026-02-13 06:05:57 +08:00
Taichang Zhou a6d633c319 debug 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-12 23:46:20 +08:00
tzhouam ec3814f587 debug 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-12 14:20:57 +00:00
tzhouam 59a7602d6e debug 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-12 13:55:08 +00:00
liuzhenwei 00bd07b82b [XPU] Update Bagel's flash_attn_varlen_func to fa utils (#1295) 
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
 2026-02-12 21:33:06 +08:00
Taichang Zhou 85a7b558ab rebase to v0.16.0 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-12 21:30:54 +08:00
Taichang Zhou 7458ba44b7 merge main 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-12 21:21:03 +08:00
Taichang ZhouandCursor e2ced80d59 Merge origin/main into dev/rebase-v0.16.0. 
Resolve arg_utils merge conflict and align CI base image tag to v0.16.0 for the vLLM rebase track.

Co-authored-by: Cursor <cursoragent@cursor.com>

Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-12 21:18:03 +08:00
rein yangandHongsheng Liu 36a79717c6 change qwen3-omni open cudagraph by default (#1352) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-12 20:49:34 +08:00
dsinghviand汪志鹏 c6f314df4f [Feature]: Diffusion sleep to use process level memory calculation (#1276) 
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-02-12 12:17:54 +00:00
Ding Zuhaoand汪志鹏 b16965dfa7 [Feature][Bugfix] Add CFG feature to Bagel (#1310) 
Signed-off-by: Ding Zuhao <e1583181@u.nus.edu>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-02-12 19:48:30 +08:00
knlnguyen1802 9994594782 [Misc] Support WorkerWrapperBase and CustomPipeline for Diffusion Worker (#764) 
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
 2026-02-12 19:23:51 +08:00
tzhouam f8959bcc6e Temp fix for metrics error in test_qwen3_tts 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-12 10:02:19 +00:00
Yuanheng ZhaoandSamit 8bf849c399 Refactor CPU Offloading Backend Pattern (#1223) 
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-02-12 17:29:55 +08:00
John Liu BUAA f117a07271 [Bugfix] reused metrics to modify the API Server token statistics in Stream Response (#1301) 
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com>
 2026-02-12 17:24:01 +08:00
tzhouam 86e20769fd Add SSL context support in benchmark function for improved connection handling 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-12 08:52:27 +00:00
ChenWenjingBhanu068Gao Han
 		ab1acb27ff [Feature]async scheduling to overlap chunk IO and compute (#951) 
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
Co-authored-by: Bhanu068 <voutharoja.bhanu06@gmail.com>
Co-authored-by: Gao Han <gaohan19@huawei.com>
 2026-02-12 16:05:02 +08:00
Sihao Li 71c9e77606 [Feature] add Tensor Parallelism to SD_3.5 (#1336) 
Signed-off-by: GG-li <3226868735@qq.com>
 2026-02-12 07:33:23 +00:00
tzhouam 3e89c99001 Merge remote-tracking branch 'remotes/origin/main' into dev/vllm-align 2026-02-12 07:16:25 +00:00
tzhouam c751f0d684 Update Dockerfile.ci to install a new version of vllm from a different extra index URL. Refactor import statement in serving_chat.py to use the correct BaseRenderer module path. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-12 06:37:17 +00:00
SamitandFlora Feng 9c00852d85 [Feature] Support Wan2.2 T2V and I2V Online Serving with OpenAI /v1/videos API (#1073) 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
Signed-off-by: SamitHuang <285365963@qq.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-02-12 14:28:32 +08:00
Ziming HuangHongsheng LiuCopilot
 		0b56fba7b3 [Pref] Support Qwen3 Omni code2wav batch infernce with async chunk (#1246) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: Ziming Huang <1520787127@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-02-12 11:50:10 +08:00
tzhouamandCursor 50f2b1e854 Merge origin/main into dev/vllm-align 
Co-authored-by: Cursor <cursoragent@cursor.com>

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-12 03:41:38 +00:00
tzhouam 0099eebb3f Refactor OmniEngineArgs and AsyncOmniEngineArgs to improve compatibility with partial payloads by using getattr for various attributes. Update GPUGenerationModelRunner and OmniGPUModelRunner to replace the activate_lora parameter with num_active_loras for better clarity and backward compatibility. This change enhances the handling of LoRA activation during model runs. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-12 03:33:50 +00:00
Junhong Liu 82fd2c3637 Fix yield token metrics and opt metrics record stats (#1292) 2026-02-12 00:52:05 +08:00
SYLAR 830d261f9d [Feature]: FP8 Quantization Support for DiT (#1034) 
Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-02-11 21:01:35 +08:00
Yueqian Lin 5fb7d137c7 [Bugfix][Qwen3-TTS] Preserve original model ID in omni_snapshot_download (#1318) 2026-02-11 18:24:45 +08:00
Ekagra Ranjan dd302b63c4 [Bugfix][Qwen3-TTS] Fix task type (#1317) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
 2026-02-11 15:49:15 +08:00
Zhou Taichang a854f0b7ff [Debug] Add trigger to concurrent stage init (#1274) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-11 14:18:18 +08:00
tzhouam 3663e803d1 rebase 2026/02/11 
vllm: 4a1550d22d7058e129d0e1257e726b3bf4a77025

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-11 02:43:49 +00:00
rein yang 963b64e7ae [Bugfix] fix precision issues of qwen3-omni when enable async_chunk without system prompt (#1288) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
 2026-02-11 10:03:41 +08:00
tzhouam 279b6b5050 Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-02-11 01:44:54 +00:00
SYLARandHongsheng Liu 8a9644c821 [Bugfix] Fix image edit RoPE crash when explicit height/width are provided (#1265) 
Signed-off-by: lishunyang <lishunyang12@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-10 21:18:36 +00:00
汪志鹏 8228b5a8fe [Bagel]: Support TP (#1293) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-02-11 00:17:44 +13:00
Canlin Guo e192b0a6c5 [Misc] Add per-request generator_device to online image gen and edit (#1183) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-02-10 18:47:08 +08:00
dongbo910220andhsliuustc0106 43e8747c00 [Feat]: support VAE patch parallelism (#756) 
Signed-off-by: dongbo910220 <1275604947@qq.com>
Co-authored-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-02-10 17:36:10 +08:00
Yueqian Lin f55ce9c793 [Test] Add e2e tests for Qwen3-TTS speech endpoint (#1206) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-02-10 16:33:39 +08:00
amy-why-3459 ee624d0686 [BugFix] Fixed the issue where ignore_eos was not working. (#1286) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-02-10 16:08:20 +08:00
zhou zhuoxinandCopilot 109bb97b2c [Fix]Ensure HuggingFace downloads complete before initialization. (#1213) 
Signed-off-by: zhou zhuoxin <zhouzhuoxin1508@outlook.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-02-10 16:05:12 +08:00