tzhouam
e6d4cdd71a
Refactor model mode initialization and enhance error handling in AsyncOmni. Update worker type retrieval in omni_stage and add audio token handling in Qwen3OmniMoe classes for improved robustness.
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-09 06:17:08 +00:00
tzhouam
8c41492307
Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0169
2026-03-06 03:02:50 +00:00
Ziming Huang
070ea0ddc3
[Optimize][Qwen3-Omni] Reduce inter-packet latency in async chunk ( #1656 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
2026-03-05 22:26:30 +08:00
tzhouam
2398a7c1d7
Merge origin/main into dev/rebase-v0170
...
Resolve conflict in omni_generation_scheduler.py: keep both
num_cached_tokens tracking and additional_information caching.
Made-with: Cursor
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-03 14:31:05 +00:00
Junhong Liu
742d845b8e
Make chunk_size and left_context_size configurable via YAML for async chunking ( #1423 )
...
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com >
Signed-off-by: Junhong Liu <ljh_lbj@163.com >
2026-03-03 22:06:52 +08:00
tzhouam
3e539fcb6a
Merge remote-tracking branch 'origin/main' into dev/vllm-align
...
Made-with: Cursor
# Conflicts:
# vllm_omni/entrypoints/openai/serving_chat.py
# vllm_omni/worker/gpu_model_runner.py
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-03 02:57:27 +00:00
汪志鹏 and Cursor
e37a89ffcb
[Feature]: Support cfg kv-cache transfer in multi-stage ( #1422 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-03-02 05:06:28 +00:00
tzhouam
26744bcb35
Merge origin/main into dev/vllm-align
...
Resolved conflicts:
- vllm_omni/inputs/preprocess.py: took origin/main's elif mm_processor_kwargs branch, fixed undefined mm_uuids reference
- vllm_omni/model_executor/models/qwen3_tts/qwen3_tts_code_predictor_vllm.py: took origin/main's NPU support
Made-with: Cursor
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-02 02:10:32 +00:00
汪志鹏
3d9fa8d53f
[BugFix]: fix a lot of bug ( #1565 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
2026-02-28 16:26:56 +08:00
Sy03
0cc3481acf
[Feat] : Support Async chunk cleanup ( #1087 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
2026-02-27 14:39:33 +08:00
amy-why-3459
66c3e04263
[Performance]Qwen3-Omni performance optimization ( #1378 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-02-26 21:21:36 +08:00
tzhouam
8d0c46487e
Merge origin/main into dev/vllm-align
...
Resolve conflicts:
- docker/Dockerfile.ci: Keep wheel install section, update hash to 6042e66cd5
- vllm_omni/platforms/npu/worker/npu_model_runner.py: Accept main's refactoring
Made-with: Cursor
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-26 08:19:53 +00:00
junuxyz and Hongsheng Liu
caed95fb45
[Bugfix] Fix tuple/list KV cache extraction crash ( #1405 )
...
Signed-off-by: junuxyz <216036880+junuxyz@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-02-26 15:20:40 +08:00
tzhouam
861698f88f
Update Dockerfile for vllm wheel version and enhance SHM connector tests
...
- Updated Dockerfile to use a new vllm wheel version.
- Modified test_put_get_inline to clarify that it always uses SHM and updated metadata assertions.
- Added handling for speculative configuration in GPU model runners to delay clearing connector metadata.
- Refactored payload decoding in OmniGPUModelRunner for better clarity and maintainability.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-24 05:45:48 +00:00
tzhouam
e22ae62396
Merge remote-tracking branch 'origin/main' into dev/vllm-align
2026-02-24 03:23:20 +00:00
tzhouam
9070f15f52
debug qwen tts
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-24 03:17:11 +00:00
NATURE
1589931fa6
[Feature] vLLM-Omni RDMA connector ( #1019 )
...
Signed-off-by: natureofnature <wzliu@connect.hku.hk >
2026-02-24 11:13:52 +08:00
tzhouam
292da4b0d8
Merge remote-tracking branch 'origin/main' into dev/vllm-align
2026-02-23 02:09:09 +00:00
Sy03 and Hongsheng Liu
efbe4116ff
[Feature] : Support disaggregated inference pipeline for Qwen3_TTS ( #1161 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-02-21 07:36:44 +08:00
tzhouam
9869ea007e
Update vLLM wheel version in Dockerfile and enhance input processing for omni models. Added functionality to reinject omni-specific fields into processed prompts and improved handling of multimodal audio extraction in serving_chat.py. Adjusted SharedMemoryConnector logic for payload size checks and refined input preprocessing methods for better compatibility with multimodal data.
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-20 08:34:53 +00:00
75770c9309
[Rebase] Rebase to vllm v0.16.0 ( #1357 )
...
ComfyUI Integration -- Publish to Comfy registry / Publish Custom Node to registry (push) Failing after 44s
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: ZJY0516 <zhu.jiangyun@foxmail.com >
Co-authored-by: youkaichao <youkaichao@gmail.com >
2026-02-13 18:05:07 +08:00
tzhouam
ec3814f587
debug
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-12 14:20:57 +00:00
Taichang Zhou and Cursor
e2ced80d59
Merge origin/main into dev/rebase-v0.16.0.
...
Resolve arg_utils merge conflict and align CI base image tag to v0.16.0 for the vLLM rebase track.
Co-authored-by: Cursor <cursoragent@cursor.com >
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
2026-02-12 21:18:03 +08:00
ab1acb27ff
[Feature]async scheduling to overlap chunk IO and compute ( #951 )
...
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn >
Co-authored-by: Bhanu068 <voutharoja.bhanu06@gmail.com >
Co-authored-by: Gao Han <gaohan19@huawei.com >
2026-02-12 16:05:02 +08:00
tzhouam
064595a97c
Merge remote-tracking branch 'origin/main' into dev/vllm-align
2026-02-10 02:22:49 +00:00
Junhong Liu and Hongsheng Liu
9af6fb9a1f
[Feature] Opt metrics structure ( #891 )
...
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com >
Signed-off-by: Junhong Liu <ljh_lbj@163.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-02-09 19:17:32 +08:00
tzhouam
7e4a604d4d
reabse 2025/2/9
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-09 03:07:43 +00:00
amy-why-3459
5304bc1e16
[Refactor] Refactor async chunk and fix the shape mismatch issue ( #1151 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-02-04 00:00:56 +08:00
amy-why-3459 and Hongsheng Liu
e6b9d52905
[BugFix] Modify max_tokens and modify the log and fix #1103 ( #1097 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-31 01:50:23 +13:00
NATURE
790e34e375
[Bugfix] Remove Mooncake/Yuanrong connector import warning ( #1091 )
...
Signed-off-by: natureofnature <wzliu@connect.hku.hk >
2026-01-30 13:26:29 +08:00
amy-why-3459 and Rein Yang
dd518973de
[BugFix] Fix the accuracy issue of multimodal input. ( #1020 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
Co-authored-by: Rein Yang <ruiruyang2@gmail.com >
2026-01-29 19:30:11 +08:00
汪志鹏
741f7e2826
[Core]: KV Cache Transfer Encapsulation ( #979 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
2026-01-28 23:12:45 +08:00
amy-why-3459
d2b3a24af3
[BugFix] Modify the method of obtaining external_request_id ( #961 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-01-27 20:56:02 +08:00
yangsonglin13
16be03b909
[Feature] Implement YuanrongConnector based on OmniConnectorBase ( #716 )
...
Signed-off-by: yangsonglin13 <yangsonglin566@gmail.com >
2026-01-27 20:17:02 +08:00
Yueqian Lin
77ff875170
Support Qwen3 tts online serving ( #968 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
2026-01-27 14:37:21 +08:00
52bf1e2f40
[Feature]Support async computation and communication across stages by chunks ( #727 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
Signed-off-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: CHEN <116010019@link.cuhk.edu.cn >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-26 11:21:24 +08:00
汪志鹏 and wzliu
7f821be57b
Support Bagel Model ( #726 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: wzliu <wzliu@connect.hku.hk >
2026-01-22 23:09:55 +13:00
Gao Han
9c7de5373a
[Feature]Basic version of supporting streaming output ( #367 )
...
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk >
2026-01-01 03:02:04 +13:00
Ziming Huang and Hongsheng Liu
ad0224f2e6
[Feature] Support Omni serving abort request ( #486 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2025-12-31 16:14:03 +08:00
Canlin Guo
97a5860681
[Security][Feature] Use Msgpack in OmniSerializer instead of pickle ( #310 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2025-12-22 13:21:16 +13:00
8939eb835d
[Rebase] Rebase to vllm 0.12.0 ( #335 )
...
Signed-off-by: Rein Yang <ruiruyang2@gmail.com >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Co-authored-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2025-12-19 16:42:02 +08:00
Cyrus Leung
c091a03d82
[Chore] Bump Python version and apply formatting changes for v0.12 release ( #126 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2025-12-16 15:19:40 +08:00
NATURE
f995211da0
[Feature] Omni Connector + ray supported ( #215 )
...
Signed-off-by: wzliu <wzliu@connect.hku.hk >
2025-12-10 20:15:03 +08:00
Gao Han
c5a89302f1
[Feature] Support online inference ( #64 )
...
Signed-off-by: Gaohan123 <gaohan19@huawei.com >
2025-11-17 14:44:10 +08:00
Samit
8cafb4294d
[Docs] Setup Documentation System and Re-organize Dependencies ( #49 )
...
Signed-off-by: SamitHuang <285365963@qq.com >
2025-11-15 21:45:43 +08:00
hsliu
8e075d4917
update dirs
2025-09-11 15:56:28 +08:00
tzhouam