tzhouam
1ccffce2bc
rebase to updated commit
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-06 02:32:26 +00:00
tzhouam
395eab8984
Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0169
2026-03-05 02:29:14 +00:00
1612948062
add support for MammothModa2 model ( #336 )
...
Signed-off-by: HonestDeng <2958906959@qq.com >
Signed-off-by: iwzbi <iwzbi@zju.edu.cn >
Signed-off-by: iwzbi <wzbi@zju.edu.cn >
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: catcat <108673086+iwzbi@users.noreply.github.com >
Co-authored-by: iwzbi <iwzbi@zju.edu.cn >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: dsinghvi <divyanshsinghvi@gmail.com >
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com >
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
2026-03-04 22:52:56 +08:00
tzhouam
4914466f0b
Update Dockerfile.ci to install vLLM wheel version 3dd142adb7f7fa3b0b8345ee046ba36a12d9e530 and remove ExtractHiddenStatesProposer references from GPU model runners for cleaner code.
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-03 15:09:26 +00:00
tzhouam
98926707f5
rebase 2026/3/3
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-03 03:50:34 +00:00
tzhouam
473d3ecd29
Update vLLM wheel version and adjust GPU memory utilization in CI configuration
...
- Updated the vLLM wheel commit hash in Dockerfile.ci to a60985b07eaf0fefa4c353ad2309b745707591ed.
- Modified the mock shared memory handle name in test_basic_connectors.py for clarity.
- Increased GPU memory utilization to 0.9 and adjusted max_num_batched_tokens to 4096 in qwen2_5_omni_ci.yaml to enable testing on H800.
- Refactored GPU model runner classes to remove deprecated activate_lora parameter and ensure compatibility with num_active_loras.
- Enhanced request handling in the OmniGPUModelRunner class to improve streaming request updates.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-02 03:08:45 +00:00
tzhouam
26744bcb35
Merge origin/main into dev/vllm-align
...
Resolved conflicts:
- vllm_omni/inputs/preprocess.py: took origin/main's elif mm_processor_kwargs branch, fixed undefined mm_uuids reference
- vllm_omni/model_executor/models/qwen3_tts/qwen3_tts_code_predictor_vllm.py: took origin/main's NPU support
Made-with: Cursor
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-02 02:10:32 +00:00
amy-why-3459
66c3e04263
[Performance]Qwen3-Omni performance optimization ( #1378 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-02-26 21:21:36 +08:00
tzhouam
a252918008
[Update] Upgrade vLLM wheel version and enhance metrics handling
...
- Updated the vLLM wheel to commit 24650715105a26b41c88f4164777b5e1a0f3200b in Dockerfile.ci.
- Modified the `calculate_metrics` function to handle `tokenizer` as optional (can be None) in metrics.py.
- Adjusted the `benchmark` function to accept `tokenizer` as optional in patch.py.
- Enhanced condition checks in OmniARScheduler to ensure valid token generation in omni_ar_scheduler.py.
- Updated input processing to include supported tasks in input_processor.py.
- Cleaned up unnecessary code in output_processor.py.
- Added support for SSL ciphers in the API server configuration in api_server.py.
- Enabled speculative decoding in GPU model runners for improved performance in gpu_generation_model_runner.py and gpu_model_runner.py.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-25 10:02:23 +00:00
tzhouam
22e01c416b
Merge remote-tracking branch 'origin/main' into dev/vllm-align
2026-02-25 08:15:55 +00:00
Canlin Guo and Hongsheng Liu
af11b02081
[Minor][Refactor] Pass seq_token_counts explicitly ( #1425 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-02-24 14:10:00 +08:00
tzhouam
861698f88f
Update Dockerfile for vllm wheel version and enhance SHM connector tests
...
- Updated Dockerfile to use a new vllm wheel version.
- Modified test_put_get_inline to clarify that it always uses SHM and updated metadata assertions.
- Added handling for speculative configuration in GPU model runners to delay clearing connector metadata.
- Refactored payload decoding in OmniGPUModelRunner for better clarity and maintainability.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-24 05:45:48 +00:00
tzhouam
bf69cd94bb
Rebase 2026/02/23
...
Update Dockerfile for vllm wheel version and enhance input processing in Omni models. Added decoding for byte messages in async requests, included new metrics in benchmarks, and improved pause state handling in the scheduler. Refined input and output processing methods for better multimodal support and error handling.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-23 03:36:01 +00:00
Sy03 and Hongsheng Liu
efbe4116ff
[Feature] : Support disaggregated inference pipeline for Qwen3_TTS ( #1161 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-02-21 07:36:44 +08:00
75770c9309
[Rebase] Rebase to vllm v0.16.0 ( #1357 )
...
ComfyUI Integration -- Publish to Comfy registry / Publish Custom Node to registry (push) Failing after 44s
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: ZJY0516 <zhu.jiangyun@foxmail.com >
Co-authored-by: youkaichao <youkaichao@gmail.com >
2026-02-13 18:05:07 +08:00
0b56fba7b3
[Pref] Support Qwen3 Omni code2wav batch infernce with async chunk ( #1246 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Signed-off-by: Ziming Huang <1520787127@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2026-02-12 11:50:10 +08:00
amy-why-3459
5304bc1e16
[Refactor] Refactor async chunk and fix the shape mismatch issue ( #1151 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-02-04 00:00:56 +08:00
Zhou Taichang and 汪志鹏
70ef548448
Dev/rebase v0.15.0 ( #1159 )
...
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
2026-02-03 12:24:45 +08:00
52bf1e2f40
[Feature]Support async computation and communication across stages by chunks ( #727 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
Signed-off-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: CHEN <116010019@link.cuhk.edu.cn >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-26 11:21:24 +08:00
Gao Han
a54c323952
[Model] Support Qwen3-TTS model series ( #895 )
...
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk >
2026-01-22 16:57:37 +08:00
+13
776c3a794e
Dev/rebase 0.14.0 ( #847 )
...
Signed-off-by: JaredforReal <w13431838023@gmail.com >
Signed-off-by: root <root@hk01dgx039.cm.cluster >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: TangPeng <85704592@qq.com >
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com >
Signed-off-by: mxuax <mxuax@connect.ust.hk >
Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: iwzbi <wzbi@zju.edu.cn >
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
Signed-off-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com >
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com >
Signed-off-by: samithuang <285365963@qq.com >
Signed-off-by: linyueqian <linyueqian@outlook.com >
Signed-off-by: David Chen <530634352@qq.com >
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com >
Signed-off-by: Dinesh G <G.Dinesh@ibm.com >
Signed-off-by: gDINESH13 <dinesh13g@gmail.com >
Signed-off-by: root <root@hk01dgx028.cm.cluster >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: JaredforReal <w13431838023@gmail.com >
Co-authored-by: root <root@hk01dgx039.cm.cluster >
Co-authored-by: JustQJ <37905360+JustQJ@users.noreply.github.com >
Co-authored-by: XU Mingshi <91017482+mxuax@users.noreply.github.com >
Co-authored-by: Sihyeon Jang <uneedsihyeon@gmail.com >
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
Co-authored-by: catcat <108673086+iwzbi@users.noreply.github.com >
Co-authored-by: Ziming Huang <hzm414167@alibaba-inc.com >
Co-authored-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com >
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com >
Co-authored-by: Samit <285365963@qq.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com >
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com >
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: John Liu BUAA <liukecheng97@gmail.com >
Co-authored-by: D!NE$H <67671800+gDINESH13@users.noreply.github.com >
Co-authored-by: root <root@hk01dgx028.cm.cluster >
2026-01-21 00:25:25 +08:00
dsinghvi and Hongsheng Liu
df4909630c
model forward call logic fixes ( #495 )
...
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2025-12-27 14:22:07 +00:00
8939eb835d
[Rebase] Rebase to vllm 0.12.0 ( #335 )
...
Signed-off-by: Rein Yang <ruiruyang2@gmail.com >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Co-authored-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2025-12-19 16:42:02 +08:00
Alicia
a33bbcedea
[Fix] Resolve comments & update vLLM-Omni name usages. ( #122 )
...
Signed-off-by: WANG Cong <115451386+congw729@users.noreply.github.com >
2025-12-01 08:05:24 +08:00
R2-Y and Rein
0a4e5a8710
[Feature] add support for Qwen3-omni ( #55 )
...
Signed-off-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: Rein <rein@Reins-MacBook-Air.local >
2025-11-30 04:44:01 +08:00
tzhouam