anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
607 Commits 235 Branches 31 Tags
2eda72b5dcec8383b79633a8fc3dbca5f5e51adb
Commit Graph
607 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Taichang Zhou 2eda72b5dc Refactor comments and formatting in Qwen3 TTS configuration and modeling files for improved readability. 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-02 21:51:01 +08:00
Taichang Zhou 22ca74432b Merge branch 'main' into dev/vllm-align 2026-02-02 21:35:45 +08:00
Taichang Zhou 354777d5e7 Merge branch 'main' of https://github.com/vllm-project/vllm-omni 2026-02-02 21:34:42 +08:00
Aliciaandprincepride 47b11c6b13 [CI] Temporarily remove slow tests. (#1143) 
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: princepride <wangzhipeng628@gmail.com>
 2026-02-02 20:36:56 +08:00
Rustam Khadipash dc6177520e [Feature] add Tensor Parallelism to LongCat-Image(-Edit) (#926) 
Signed-off-by: Rustam Khadipash <16683750+hadipash@users.noreply.github.com>
 2026-02-02 10:28:25 +00:00
汪志鹏 b6c846cc95 Add stage_init_timeout=300 (#1153) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-02-02 17:58:48 +08:00
tzhouam 9194c1dd84 Update base image in Dockerfile.rocm from v0.14.0 to v0.15.0 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-02 08:41:35 +00:00
tzhouam ce9105267f Enhance error handling in TTS generation and improve audio output compatibility by supporting both "audio" and "model_outputs" keys in response. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-02 08:40:56 +00:00
tzhouam 39f7cade27 Fix audio sample rate access in end2end.py for correct output retrieval 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-02 08:38:24 +00:00
tzhouam 08fa85a1ad Update audio tensor access in offline inference tests and increase image fetch timeout for CI 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-02 06:10:54 +00:00
tzhouam 6568a78a26 Remove unused tokenizer argument from OmniInputProcessor class docstring 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-02 03:12:43 +00:00
tzhouam ff8b3dbbc8 update qwen 2.5 Omni 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-02 03:01:11 +00:00
tzhouam 1297a46040 Merge remote-tracking branch 'origin/main' into dev/vllm-align 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-02 01:54:12 +00:00
ceanna93andanna df3b347cd9 [Misc] Align error handling with upstream vLLM v0.14.0 (#1122) 
Signed-off-by: anna <lee.anna@navercorp.com>
Co-authored-by: anna <lee.anna@navercorp.com>
 2026-02-01 21:59:08 +08:00
Roger Wang de2cac95e8 [Doc] First stable release of vLLM-Omni (#1129) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2026-02-01 00:21:57 -08:00
Roger Wang 81774fd933 [Misc] Bump version to 0.14.0 (#1128) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2026-02-01 00:09:43 -08:00
汪志鹏andHongsheng Liu de0a8b057b [CI]: Bagel E2E Smoked Test (#1074) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-31 22:08:22 +13:00
汪志鹏andHongsheng Liu 0f0e94760c [TeaCache]: Add Coefficient Estimation (#940) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-31 22:06:17 +13:00
Canlin Guomuziyuhui666Hongsheng Liu
 		ed89c8b043 [Bugfix] Fix NPU SDPA attention mask shape and semantics (#1031) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: muziyuhui666 <111362884+muziyuhui666@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
v0.14.0 		2026-01-31 14:14:15 +08:00
dengyunyang 70a5de9dd4 [FEATURE] /v1/images/edit interface (#1101) 
Signed-off-by: dengyunyang <584797741@qq.com>
 2026-01-31 14:13:36 +08:00
Canlin Guo 722a69e8d1 [NPU] Align with GPUModelRunner (#1114) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-01-31 04:26:34 +00:00
Taichang Zhou e033498600 Merge branch 'main' of https://github.com/vllm-project/vllm-omni 2026-01-31 10:59:36 +08:00
Juan Pablo ZuluagaHongsheng LiuWeiQing Chen
 		2e6895ff0e [Bugfix][Qwen3TTS] Load speaker_id/voices from model configuration (#1079) 
Signed-off-by: pablo <juanz9312@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
 2026-01-31 10:42:09 +08:00
wangyuwangyu31577Hongsheng Liu
 		07f4c3fdd6 [Bugfix] fix benchmark audio timing error and add benchmark test (#1109) 
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com>
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-31 10:15:26 +08:00
TJian e503a41941 [CI] [ROCm] Quick fix amd ci (#1116) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-01-31 09:46:23 +08:00
Markus / Mark 425952e7d3 [BugFix] Fix Qwen3 TTS 0.6B profile run hang (#995) (#1082) 2026-01-31 08:00:03 +08:00
Didan DengandHongsheng Liu 23cf48d7bb [Perf]: CFG parallel abstraction (#851) 
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-30 16:23:44 +00:00
Jiangyun ZhuandHongsheng Liu 8ad39570b8 [Bugfix] fix request_id of image generation in api server (#1112) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-31 00:05:34 +08:00
Ziming Huangram16gHongsheng Liu
 		f6cfc0d1e3 [BugFix] Fix Qwen3 Omni talker mtp torch.compile startup error (#1104) 
Signed-off-by: ram16g <anlianfengjie@163.com>
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Co-authored-by: ram16g <anlianfengjie@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-30 23:25:49 +08:00
Canlin Guo 4eeea68cff [Platform] Add supports_torch_inductor interface (#1108) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-01-30 15:21:43 +00:00
XU MingshiandHongsheng Liu be835c4483 [BugFix] Fix modulate_index shape error in Qwen-Image-Edit Task (#1100) 
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-30 13:44:58 +00:00
amy-why-3459andHongsheng Liu e6b9d52905 [BugFix] Modify max_tokens and modify the log and fix #1103 (#1097) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-31 01:50:23 +13:00
tzhouam 7aea8e8d91 Add OmniMRotaryEmbedding for multimodal position computation 
- Introduced OmniMRotaryEmbedding to extend MRotaryEmbedding with methods for computing input positions across various multimodal scenarios, including image, video, and audio inputs.
- Updated patch.py to replace the original MRotaryEmbedding with OmniMRotaryEmbedding in the module patching logic.
- Created new files for the OmniMRotaryEmbedding implementation and its associated methods.

Signed-off-by: [Your Name] <[Your Email]>

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-01-30 10:03:02 +00:00
tzhouam 3208afd82b Refactor OmniRequestOutput to encapsulate multimodal output 
- Changed the multimodal_output field to a private attribute _multimodal_output for better encapsulation.
- Added a property method to access multimodal_output, allowing for a clearer interface and improved handling of request outputs.

Signed-off-by: [Your Name] <[Your Email]>

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-01-30 10:02:22 +00:00
tzhouam 2b009b3cef Fix audio tensor extraction in end2end.py files across multiple examples 
- Updated the audio tensor extraction logic to access the correct output structure in Qwen2.5 Omni, Qwen3 Omni, and Qwen3 TTS implementations.
- Ensured consistency in handling audio outputs for improved functionality.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-01-30 10:00:14 +00:00
Yuanheng ZhaoHongsheng LiuCopilot
 		2923d5a91e [Feature] Support DiT Layerwise (Blockwise) CPU Offloading (#858) 
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-01-30 17:35:09 +08:00
WeiQing Chen be580d5c24 [Misc] update wechat image (#1096) 2026-01-30 17:06:02 +08:00
Pierre LE GUEN 2c294a750b fix: UnboundLocalError for role in streaming audio/image responses (#784) 
Signed-off-by: Pierre Le Guen <26087574+PierreLeGuen@users.noreply.github.com>
 2026-01-30 16:27:04 +08:00
NATURE 790e34e375 [Bugfix] Remove Mooncake/Yuanrong connector import warning (#1091) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
 2026-01-30 13:26:29 +08:00
tzhouam 4faf190e28 Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-01-30 03:49:04 +00:00
wangyuwangyu31577Hongsheng Liu
 		7bfd3c069d [Benchmark] Add vLLM-Omni Omni model online benchmark (#780) 
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com>
Signed-off-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-30 11:41:09 +08:00
Andy Zhou c7fb59fd28 [Fix] Increase max wait time for server readiness to accommodate model loading (#1089) 
Signed-off-by: Andy Zhou <46011930+AndyZhou952@users.noreply.github.com>
 2026-01-30 11:27:19 +08:00
tzhouam ca57f510c7 Refactor OmniInputPreprocessor to remove unused embed processing logic 
- Eliminated the _process_embeds method and related imports from preprocess.py, streamlining the input preprocessing workflow.
- Updated the main processing logic to focus solely on token inputs, enhancing clarity and maintainability.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-01-30 03:02:38 +00:00
tzhouam 4b895459e4 Merge branch 'main' into dev/vllm-align 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-01-30 02:53:03 +00:00
tzhouam 62dbf07fb6 Refactor OpenAI serving modules for improved functionality and organization 
- Reintroduced the load_chat_template import in api_server.py for better template handling.
- Updated the logic in serving_chat.py to streamline the resolution of chat template content formats, enhancing clarity and functionality.
- Adjusted the application of chat templates to ensure proper handling of tools and chat templates.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-01-30 02:44:13 +00:00
tzhouam 104e105c93 Refactor multimodal processing imports and update configuration handling in Qwen3OmniMoeThinker 
- Changed the import path for multimodal processing to improve module organization.
- Removed the unused multimodal configuration parameter from the Qwen3OmniMoeThinker initialization, streamlining the class definition.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-01-30 02:43:41 +00:00
tzhouam ef21879c3a Refactor GPU model runner to streamline multimodal configuration handling 
- Removed the unused import of supports_mm_encoder_only from interfaces.
- Updated the dummy run check to utilize the multimodal configuration directly, enhancing clarity and future extensibility.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-01-30 02:43:26 +00:00
Ding Zuhao 0e07eb6ff9 [Doc][Bagel] Add BAGEL-7B-MoT documentation and edit the default stage configuration (#987) 
Signed-off-by: Ding Zuhao <e1583181@u.nus.edu>
Signed-off-by: jzz <e1583181@u.nus.edu>
 2026-01-30 12:56:10 +13:00
dengyunyang 47a0969976 [Bugfix] pr conflict fix, bugfix ignore mm data from stages to async omni (#1025) 
Signed-off-by: dengyunyang <584797741@qq.com>
 2026-01-29 17:09:34 +00:00
Canlin Guo 2c4974c6ea [Bugfix][Test] Re-enable the log simple tests (#1065) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-01-30 00:55:53 +08:00