Commit Graph
500 Commits
Author SHA1 Message Date
tzhouam 22e01c416b Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-02-25 08:15:55 +00:00
Zhou Taichang 296d995ec2 [CI] Remove overwhelming debug log (#1463)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-25 12:06:01 +08:00
Juan Pablo Zuluagaandpablo 4de077ea62 [Qwen3TTS][Feat] Code2Wav batched decoding (#1426)
Signed-off-by: pablo <pablo@agigo.ai>
Co-authored-by: pablo <pablo@agigo.ai>
2026-02-25 11:16:46 +13:00
Junhong LiuandHongsheng Liu c0908dd3fa [Bugfix] Forward all vllm-omni serve command parameters to model (#985)
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Signed-off-by: Junhong Liu <ljh_lbj@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-02-24 22:20:01 +08:00
36b8f80266 [Feature] Support Stage Based Deployment CLI (#939)
Signed-off-by: wuhang <wuhang6@huawei.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: wuhang <whlbx@hotmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-02-24 16:55:23 +08:00
tzhouam 04ccb062fc Add signature compatibility tests for OmniLLM class
- Introduced tests to ensure method signatures in OmniLLM match those of the base LLM class, preventing runtime TypeErrors due to parameter mismatches.
- Updated the _run_engine method in OmniLLM to include an optional output_type parameter.

Signed-off-by: [Your Name] <your.email@example.com>

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-24 06:58:39 +00:00
Canlin GuoandHongsheng Liu af11b02081 [Minor][Refactor] Pass seq_token_counts explicitly (#1425)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-02-24 14:10:00 +08:00
tzhouam 861698f88f Update Dockerfile for vllm wheel version and enhance SHM connector tests
- Updated Dockerfile to use a new vllm wheel version.
- Modified test_put_get_inline to clarify that it always uses SHM and updated metadata assertions.
- Added handling for speculative configuration in GPU model runners to delay clearing connector metadata.
- Refactored payload decoding in OmniGPUModelRunner for better clarity and maintainability.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-24 05:45:48 +00:00
tzhouam e22ae62396 Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-02-24 03:23:20 +00:00
tzhouam 9070f15f52 debug qwen tts
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-24 03:17:11 +00:00
NATURE 1589931fa6 [Feature] vLLM-Omni RDMA connector (#1019)
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
2026-02-24 11:13:52 +08:00
Ding ZuhaoandWang Zhipeng: princepride d6a3551a62 [Bug] [Bagel] Fix kv transfer bug (#1437)
Signed-off-by: Ding Zuhao <e1583181@u.nus.edu>
Co-authored-by: Wang Zhipeng: princepride <wangzhipeng628@gmail.com>
2026-02-23 15:58:22 +00:00
zhou zhuoxin 40f72bb870 [Feat]: add ASCII art logo for vLLM-Omni (#1430) 2026-02-23 22:10:09 +08:00
Ding Zuhaoand汪志鹏 0ff443f069 [Feature][BAGEL] Combine multi-branch cfg into a single batch to accelerate inference. (#1429)
Signed-off-by: Ding Zuhao <e1583181@u.nus.edu>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
2026-02-23 17:41:47 +08:00
tzhouam bf69cd94bb Rebase 2026/02/23
Update Dockerfile for vllm wheel version and enhance input processing in Omni models. Added decoding for byte messages in async requests, included new metrics in benchmarks, and improved pause state handling in the scheduler. Refined input and output processing methods for better multimodal support and error handling.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-23 03:36:01 +00:00
tzhouam 292da4b0d8 Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-02-23 02:09:09 +00:00
Hongsheng Liu a3f2d4cbfe Revert "[misc] Feature/pr reviewer auto trigger&update model" (#1432) 2026-02-23 07:52:09 +08:00
Hongsheng LiuandClaude Opus 4.6 267170115b [misc] Feature/pr reviewer auto trigger&update model (#1431)
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
Signed-off-by: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Hunter Liu <hunter@liu.sh>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 23:33:45 +08:00
Sy03andHongsheng Liu efbe4116ff [Feature] : Support disaggregated inference pipeline for Qwen3_TTS (#1161)
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-02-21 07:36:44 +08:00
tzhouam 9869ea007e Update vLLM wheel version in Dockerfile and enhance input processing for omni models. Added functionality to reinject omni-specific fields into processed prompts and improved handling of multimodal audio extraction in serving_chat.py. Adjusted SharedMemoryConnector logic for payload size checks and refined input preprocessing methods for better compatibility with multimodal data.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-20 08:34:53 +00:00
Alex Brooks 5295f0f81b [Bugfix] Fix Hardcoded Datatypes in Z-image (#1393)
Signed-off-by: Alex Brooks <albrooks@redhat.com>
2026-02-20 06:55:24 +00:00
Chendi.Xue d2f015edce [0.16.0] remove cuda hard-code for Hunyuan Image3 (#1402)
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
2026-02-20 13:23:31 +08:00
tzhouam 663976b8ca Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-02-20 01:38:44 +00:00
Chendi.Xue 0b577a7263 [v0.16.0][BUG FIX]Fix hunyuan MOE after update to 0.16.0 (#1401)
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
2026-02-19 23:57:11 +13:00
tzhouam b867bacd24 rebase 2025/02/16
Qwen3 Omni offline/online passed

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-16 03:21:21 +00:00
75770c9309 [Rebase] Rebase to vllm v0.16.0 (#1357)
ComfyUI Integration -- Publish to Comfy registry / Publish Custom Node to registry (push) Failing after 44s
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: ZJY0516 <zhu.jiangyun@foxmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
2026-02-13 18:05:07 +08:00
tzhouam 15099119dd Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0.16.0 2026-02-13 09:55:18 +00:00
汪志鹏 630e84ef93 [BugFix]: Revert utils change (#1369)
Signed-off-by: princepride <wangzhipeng628@gmail.com>
2026-02-13 17:54:09 +08:00
tzhouam f24273dc03 Update vLLM version to 0.16.0 in quickstart and CUDA installation documentation, and adjust parameters in multimodal generation script for improved performance. Refactor request logging in serving_chat.py for better input tracking.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-13 09:50:11 +00:00
tzhouam 8a4eee9bed Enhance configuration and debugging: Added asyncio mode to pytest settings, increased timeout for Qwen3-TTS E2E test, and improved logging in GPU model runner for better traceability of input modalities. Updated Dockerfile.ci to streamline VLLM installation process.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-13 06:39:28 +00:00
tzhouam 31e9fdbb57 Refactor chat_utils to remove unused multi-modal classes and add async audio extraction from video. Update README to reflect new entry point for audio extraction. Enhance serving_chat to process audio from video URLs when specified.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-13 03:27:16 +00:00
tzhouam 39a045f439 debug Qwen 3 Omni cuda graph
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-13 02:57:05 +00:00
tzhouam 7e9cae56fc Refactor Dockerfile.ci to use precompiled VLLM installation and remove unused parse.py file. Update serving_chat.py to streamline rendering logic and enhance compatibility with new API. Modify OmniInputPreprocessor to directly handle SingletonDictPrompt, improving input processing efficiency.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-13 02:28:25 +00:00
Hongsheng Liu 622c6f5f14 Merge branch 'main' into dev/rebase-v0.16.0 2026-02-13 06:05:57 +08:00
Taichang Zhou a6d633c319 debug
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
2026-02-12 23:46:20 +08:00
tzhouam ec3814f587 debug
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-12 14:20:57 +00:00
tzhouam 59a7602d6e debug
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-12 13:55:08 +00:00
liuzhenwei 00bd07b82b [XPU] Update Bagel's flash_attn_varlen_func to fa utils (#1295)
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
2026-02-12 21:33:06 +08:00
Taichang Zhou 85a7b558ab rebase to v0.16.0
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
2026-02-12 21:30:54 +08:00
Taichang Zhou 7458ba44b7 merge main
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
2026-02-12 21:21:03 +08:00
Taichang ZhouandCursor e2ced80d59 Merge origin/main into dev/rebase-v0.16.0.
Resolve arg_utils merge conflict and align CI base image tag to v0.16.0 for the vLLM rebase track.

Co-authored-by: Cursor <cursoragent@cursor.com>

Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
2026-02-12 21:18:03 +08:00
rein yangandHongsheng Liu 36a79717c6 change qwen3-omni open cudagraph by default (#1352)
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-02-12 20:49:34 +08:00
dsinghviand汪志鹏 c6f314df4f [Feature]: Diffusion sleep to use process level memory calculation (#1276)
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
2026-02-12 12:17:54 +00:00
Ding Zuhaoand汪志鹏 b16965dfa7 [Feature][Bugfix] Add CFG feature to Bagel (#1310)
Signed-off-by: Ding Zuhao <e1583181@u.nus.edu>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
2026-02-12 19:48:30 +08:00
knlnguyen1802 9994594782 [Misc] Support WorkerWrapperBase and CustomPipeline for Diffusion Worker (#764)
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
2026-02-12 19:23:51 +08:00
tzhouam f8959bcc6e Temp fix for metrics error in test_qwen3_tts
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-12 10:02:19 +00:00
Yuanheng ZhaoandSamit 8bf849c399 Refactor CPU Offloading Backend Pattern (#1223)
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Samit <285365963@qq.com>
2026-02-12 17:29:55 +08:00
John Liu BUAA f117a07271 [Bugfix] reused metrics to modify the API Server token statistics in Stream Response (#1301)
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com>
2026-02-12 17:24:01 +08:00
tzhouam 86e20769fd Add SSL context support in benchmark function for improved connection handling
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-12 08:52:27 +00:00
ab1acb27ff [Feature]async scheduling to overlap chunk IO and compute (#951)
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
Co-authored-by: Bhanu068 <voutharoja.bhanu06@gmail.com>
Co-authored-by: Gao Han <gaohan19@huawei.com>
2026-02-12 16:05:02 +08:00