anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
594 Commits 235 Branches 31 Tags
ff8b3dbbc8a4227dd689ddde760b9594761dd929
Commit Graph
594 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
tzhouam ff8b3dbbc8 update qwen 2.5 Omni 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-02 03:01:11 +00:00
tzhouam 1297a46040 Merge remote-tracking branch 'origin/main' into dev/vllm-align 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-02 01:54:12 +00:00
ceanna93andanna df3b347cd9 [Misc] Align error handling with upstream vLLM v0.14.0 (#1122) 
Signed-off-by: anna <lee.anna@navercorp.com>
Co-authored-by: anna <lee.anna@navercorp.com>
 2026-02-01 21:59:08 +08:00
Roger Wang de2cac95e8 [Doc] First stable release of vLLM-Omni (#1129) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2026-02-01 00:21:57 -08:00
Roger Wang 81774fd933 [Misc] Bump version to 0.14.0 (#1128) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2026-02-01 00:09:43 -08:00
汪志鹏andHongsheng Liu de0a8b057b [CI]: Bagel E2E Smoked Test (#1074) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-31 22:08:22 +13:00
汪志鹏andHongsheng Liu 0f0e94760c [TeaCache]: Add Coefficient Estimation (#940) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-31 22:06:17 +13:00
Canlin Guomuziyuhui666Hongsheng Liu
 		ed89c8b043 [Bugfix] Fix NPU SDPA attention mask shape and semantics (#1031) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: muziyuhui666 <111362884+muziyuhui666@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
v0.14.0 		2026-01-31 14:14:15 +08:00
dengyunyang 70a5de9dd4 [FEATURE] /v1/images/edit interface (#1101) 
Signed-off-by: dengyunyang <584797741@qq.com>
 2026-01-31 14:13:36 +08:00
Canlin Guo 722a69e8d1 [NPU] Align with GPUModelRunner (#1114) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-01-31 04:26:34 +00:00
Juan Pablo ZuluagaHongsheng LiuWeiQing Chen
 		2e6895ff0e [Bugfix][Qwen3TTS] Load speaker_id/voices from model configuration (#1079) 
Signed-off-by: pablo <juanz9312@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
 2026-01-31 10:42:09 +08:00
wangyuwangyu31577Hongsheng Liu
 		07f4c3fdd6 [Bugfix] fix benchmark audio timing error and add benchmark test (#1109) 
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com>
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-31 10:15:26 +08:00
TJian e503a41941 [CI] [ROCm] Quick fix amd ci (#1116) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-01-31 09:46:23 +08:00
Markus / Mark 425952e7d3 [BugFix] Fix Qwen3 TTS 0.6B profile run hang (#995) (#1082) 2026-01-31 08:00:03 +08:00
Didan DengandHongsheng Liu 23cf48d7bb [Perf]: CFG parallel abstraction (#851) 
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-30 16:23:44 +00:00
Jiangyun ZhuandHongsheng Liu 8ad39570b8 [Bugfix] fix request_id of image generation in api server (#1112) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-31 00:05:34 +08:00
Ziming Huangram16gHongsheng Liu
 		f6cfc0d1e3 [BugFix] Fix Qwen3 Omni talker mtp torch.compile startup error (#1104) 
Signed-off-by: ram16g <anlianfengjie@163.com>
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Co-authored-by: ram16g <anlianfengjie@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-30 23:25:49 +08:00
Canlin Guo 4eeea68cff [Platform] Add supports_torch_inductor interface (#1108) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-01-30 15:21:43 +00:00
XU MingshiandHongsheng Liu be835c4483 [BugFix] Fix modulate_index shape error in Qwen-Image-Edit Task (#1100) 
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-30 13:44:58 +00:00
amy-why-3459andHongsheng Liu e6b9d52905 [BugFix] Modify max_tokens and modify the log and fix #1103 (#1097) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-31 01:50:23 +13:00
tzhouam 7aea8e8d91 Add OmniMRotaryEmbedding for multimodal position computation 
- Introduced OmniMRotaryEmbedding to extend MRotaryEmbedding with methods for computing input positions across various multimodal scenarios, including image, video, and audio inputs.
- Updated patch.py to replace the original MRotaryEmbedding with OmniMRotaryEmbedding in the module patching logic.
- Created new files for the OmniMRotaryEmbedding implementation and its associated methods.

Signed-off-by: [Your Name] <[Your Email]>

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-01-30 10:03:02 +00:00
tzhouam 3208afd82b Refactor OmniRequestOutput to encapsulate multimodal output 
- Changed the multimodal_output field to a private attribute _multimodal_output for better encapsulation.
- Added a property method to access multimodal_output, allowing for a clearer interface and improved handling of request outputs.

Signed-off-by: [Your Name] <[Your Email]>

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-01-30 10:02:22 +00:00
tzhouam 2b009b3cef Fix audio tensor extraction in end2end.py files across multiple examples 
- Updated the audio tensor extraction logic to access the correct output structure in Qwen2.5 Omni, Qwen3 Omni, and Qwen3 TTS implementations.
- Ensured consistency in handling audio outputs for improved functionality.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-01-30 10:00:14 +00:00
Yuanheng ZhaoHongsheng LiuCopilot
 		2923d5a91e [Feature] Support DiT Layerwise (Blockwise) CPU Offloading (#858) 
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-01-30 17:35:09 +08:00
WeiQing Chen be580d5c24 [Misc] update wechat image (#1096) 2026-01-30 17:06:02 +08:00
Pierre LE GUEN 2c294a750b fix: UnboundLocalError for role in streaming audio/image responses (#784) 
Signed-off-by: Pierre Le Guen <26087574+PierreLeGuen@users.noreply.github.com>
 2026-01-30 16:27:04 +08:00
NATURE 790e34e375 [Bugfix] Remove Mooncake/Yuanrong connector import warning (#1091) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
 2026-01-30 13:26:29 +08:00
tzhouam 4faf190e28 Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-01-30 03:49:04 +00:00
wangyuwangyu31577Hongsheng Liu
 		7bfd3c069d [Benchmark] Add vLLM-Omni Omni model online benchmark (#780) 
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com>
Signed-off-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-30 11:41:09 +08:00
Andy Zhou c7fb59fd28 [Fix] Increase max wait time for server readiness to accommodate model loading (#1089) 
Signed-off-by: Andy Zhou <46011930+AndyZhou952@users.noreply.github.com>
 2026-01-30 11:27:19 +08:00
tzhouam ca57f510c7 Refactor OmniInputPreprocessor to remove unused embed processing logic 
- Eliminated the _process_embeds method and related imports from preprocess.py, streamlining the input preprocessing workflow.
- Updated the main processing logic to focus solely on token inputs, enhancing clarity and maintainability.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-01-30 03:02:38 +00:00
tzhouam 4b895459e4 Merge branch 'main' into dev/vllm-align 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-01-30 02:53:03 +00:00
tzhouam 62dbf07fb6 Refactor OpenAI serving modules for improved functionality and organization 
- Reintroduced the load_chat_template import in api_server.py for better template handling.
- Updated the logic in serving_chat.py to streamline the resolution of chat template content formats, enhancing clarity and functionality.
- Adjusted the application of chat templates to ensure proper handling of tools and chat templates.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-01-30 02:44:13 +00:00
tzhouam 104e105c93 Refactor multimodal processing imports and update configuration handling in Qwen3OmniMoeThinker 
- Changed the import path for multimodal processing to improve module organization.
- Removed the unused multimodal configuration parameter from the Qwen3OmniMoeThinker initialization, streamlining the class definition.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-01-30 02:43:41 +00:00
tzhouam ef21879c3a Refactor GPU model runner to streamline multimodal configuration handling 
- Removed the unused import of supports_mm_encoder_only from interfaces.
- Updated the dummy run check to utilize the multimodal configuration directly, enhancing clarity and future extensibility.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-01-30 02:43:26 +00:00
Ding Zuhao 0e07eb6ff9 [Doc][Bagel] Add BAGEL-7B-MoT documentation and edit the default stage configuration (#987) 
Signed-off-by: Ding Zuhao <e1583181@u.nus.edu>
Signed-off-by: jzz <e1583181@u.nus.edu>
 2026-01-30 12:56:10 +13:00
dengyunyang 47a0969976 [Bugfix] pr conflict fix, bugfix ignore mm data from stages to async omni (#1025) 
Signed-off-by: dengyunyang <584797741@qq.com>
 2026-01-29 17:09:34 +00:00
Canlin Guo 2c4974c6ea [Bugfix][Test] Re-enable the log simple tests (#1065) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-01-30 00:55:53 +08:00
dongbo910220 3eecd2057b [Fix] stabilize diffusion images LoRA E2E across CI drift (#1075) 
Signed-off-by: dongbo910220 <1275604947@qq.com>
 2026-01-30 00:28:29 +08:00
Fanli LinandCopilot bfbf3e5b69 [Feat] Support XPU Backend in vLLM-Omni (#191) 
Signed-off-by: Fanli Lin <fanli.lin@intel.com>
Signed-off-by: Fanli Lin <fanli0116@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-01-30 04:33:05 +13:00
Fanli LinandSamit c7f89ef085 [Wan2.2] Optimize memory usage with conditional transformer loading (#980) 
Signed-off-by: Lin, Fanli <fanli.lin@intel.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-01-29 22:02:24 +08:00
Ziming HuangandHongsheng Liu a909d36415 [Perf] Qwen3 Omni talker mtp optimization (#1005) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-29 21:23:19 +08:00
dsinghvi f8dd566abd [BugFix] Fix to use correct attn backend (#1038) 
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
 2026-01-29 21:20:39 +08:00
Ziming Huang c48a9f33ad [CI] Change Qwen3 Omni stage placement strategy (#1072) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
 2026-01-30 02:10:09 +13:00
JohnJan ee0f7a7797 [feat]: adapt batch request for flux (#1028) 
Signed-off-by: wuzhongjian wuzhongjian_yewu@cmss.chinamobile.com
 2026-01-29 20:19:31 +08:00
Canlin Guo 0c8b5ac2d5 [Bugfix] Set VaeImageProcessor do_convert_rgb True (#1032) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-01-29 12:02:39 +00:00
amy-why-3459andRein Yang dd518973de [BugFix] Fix the accuracy issue of multimodal input. (#1020) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
 2026-01-29 19:30:11 +08:00
tzhouam 45880a30f0 Refactor chat_utils and OpenAI modules for improved functionality and organization 
- Updated import statements in chat_utils.py to replace the internal content format type with the public one for better clarity.
- Refactored api_server.py to streamline the chat template loading process by replacing the previous method with a direct function call.
- Enhanced serving_chat.py by reintroducing safe_apply_chat_template functions for better template handling.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-01-29 09:00:43 +00:00
XU MingshiandHongsheng Liu 5b8ade8446 [Diffusion][Bugfix] Fix the flash_attn backends selection logic (#983) 
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-29 16:44:58 +08:00
tzhouam 2ac33489b5 Update Dockerfile.ci to install vllm package using the correct URL format 
- Modified the pip install command for vllm to use the correct URL format for direct installation from the specified commit hash, ensuring a more reliable installation process.
- Removed the fallback option for pre-release installation to streamline the installation command.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-01-29 08:25:51 +00:00