tzhouam
36a507e071
Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0170
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-04 08:26:56 +00:00
Yueqian Lin
a3240fd484
[Bugfix] Fix transformers 5.x compat issues in online TTS serving ( #1536 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
2026-03-04 01:28:32 +08:00
tzhouam
2398a7c1d7
Merge origin/main into dev/rebase-v0170
...
Resolve conflict in omni_generation_scheduler.py: keep both
num_cached_tokens tracking and additional_information caching.
Made-with: Cursor
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-03 14:31:05 +00:00
Junhong Liu
742d845b8e
Make chunk_size and left_context_size configurable via YAML for async chunking ( #1423 )
...
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com >
Signed-off-by: Junhong Liu <ljh_lbj@163.com >
2026-03-03 22:06:52 +08:00
tzhouam
26744bcb35
Merge origin/main into dev/vllm-align
...
Resolved conflicts:
- vllm_omni/inputs/preprocess.py: took origin/main's elif mm_processor_kwargs branch, fixed undefined mm_uuids reference
- vllm_omni/model_executor/models/qwen3_tts/qwen3_tts_code_predictor_vllm.py: took origin/main's NPU support
Made-with: Cursor
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-02 02:10:32 +00:00
Sy03
0cc3481acf
[Feat] : Support Async chunk cleanup ( #1087 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
2026-02-27 14:39:33 +08:00
tzhouam
8d0c46487e
Merge origin/main into dev/vllm-align
...
Resolve conflicts:
- docker/Dockerfile.ci: Keep wheel install section, update hash to 6042e66cd5
- vllm_omni/platforms/npu/worker/npu_model_runner.py: Accept main's refactoring
Made-with: Cursor
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-26 08:19:53 +00:00
Junhong Liu and Gao Han
919ad28da5
[BugFix] process request.num_cached_tokens if it equals to the initial value ( #1468 )
...
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com >
Co-authored-by: Gao Han <gaohan19@huawei.com >
2026-02-26 10:59:18 +08:00
tzhouam
a252918008
[Update] Upgrade vLLM wheel version and enhance metrics handling
...
- Updated the vLLM wheel to commit 24650715105a26b41c88f4164777b5e1a0f3200b in Dockerfile.ci.
- Modified the `calculate_metrics` function to handle `tokenizer` as optional (can be None) in metrics.py.
- Adjusted the `benchmark` function to accept `tokenizer` as optional in patch.py.
- Enhanced condition checks in OmniARScheduler to ensure valid token generation in omni_ar_scheduler.py.
- Updated input processing to include supported tasks in input_processor.py.
- Cleaned up unnecessary code in output_processor.py.
- Added support for SSL ciphers in the API server configuration in api_server.py.
- Enabled speculative decoding in GPU model runners for improved performance in gpu_generation_model_runner.py and gpu_model_runner.py.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-25 10:02:23 +00:00
tzhouam
861698f88f
Update Dockerfile for vllm wheel version and enhance SHM connector tests
...
- Updated Dockerfile to use a new vllm wheel version.
- Modified test_put_get_inline to clarify that it always uses SHM and updated metadata assertions.
- Added handling for speculative configuration in GPU model runners to delay clearing connector metadata.
- Refactored payload decoding in OmniGPUModelRunner for better clarity and maintainability.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-24 05:45:48 +00:00
tzhouam
9070f15f52
debug qwen tts
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-24 03:17:11 +00:00
tzhouam
bf69cd94bb
Rebase 2026/02/23
...
Update Dockerfile for vllm wheel version and enhance input processing in Omni models. Added decoding for byte messages in async requests, included new metrics in benchmarks, and improved pause state handling in the scheduler. Refined input and output processing methods for better multimodal support and error handling.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-23 03:36:01 +00:00
Sy03 and Hongsheng Liu
efbe4116ff
[Feature] : Support disaggregated inference pipeline for Qwen3_TTS ( #1161 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-02-21 07:36:44 +08:00
75770c9309
[Rebase] Rebase to vllm v0.16.0 ( #1357 )
...
ComfyUI Integration -- Publish to Comfy registry / Publish Custom Node to registry (push) Failing after 44s
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: ZJY0516 <zhu.jiangyun@foxmail.com >
Co-authored-by: youkaichao <youkaichao@gmail.com >
2026-02-13 18:05:07 +08:00
ab1acb27ff
[Feature]async scheduling to overlap chunk IO and compute ( #951 )
...
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn >
Co-authored-by: Bhanu068 <voutharoja.bhanu06@gmail.com >
Co-authored-by: Gao Han <gaohan19@huawei.com >
2026-02-12 16:05:02 +08:00
amy-why-3459
d1cf4c4c0f
[Bugfix] async_chunk rebase v0.15.0 ( #1195 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-02-04 20:50:09 +08:00
Zhou Taichang and 汪志鹏
70ef548448
Dev/rebase v0.15.0 ( #1159 )
...
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
2026-02-03 12:24:45 +08:00
amy-why-3459
d2b3a24af3
[BugFix] Modify the method of obtaining external_request_id ( #961 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-01-27 20:56:02 +08:00
52bf1e2f40
[Feature]Support async computation and communication across stages by chunks ( #727 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
Signed-off-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: CHEN <116010019@link.cuhk.edu.cn >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-26 11:21:24 +08:00
汪志鹏 and wzliu
7f821be57b
Support Bagel Model ( #726 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: wzliu <wzliu@connect.hku.hk >
2026-01-22 23:09:55 +13:00
Gao Han
a54c323952
[Model] Support Qwen3-TTS model series ( #895 )
...
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk >
2026-01-22 16:57:37 +08:00
+13
776c3a794e
Dev/rebase 0.14.0 ( #847 )
...
Signed-off-by: JaredforReal <w13431838023@gmail.com >
Signed-off-by: root <root@hk01dgx039.cm.cluster >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: TangPeng <85704592@qq.com >
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com >
Signed-off-by: mxuax <mxuax@connect.ust.hk >
Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: iwzbi <wzbi@zju.edu.cn >
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
Signed-off-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com >
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com >
Signed-off-by: samithuang <285365963@qq.com >
Signed-off-by: linyueqian <linyueqian@outlook.com >
Signed-off-by: David Chen <530634352@qq.com >
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com >
Signed-off-by: Dinesh G <G.Dinesh@ibm.com >
Signed-off-by: gDINESH13 <dinesh13g@gmail.com >
Signed-off-by: root <root@hk01dgx028.cm.cluster >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: JaredforReal <w13431838023@gmail.com >
Co-authored-by: root <root@hk01dgx039.cm.cluster >
Co-authored-by: JustQJ <37905360+JustQJ@users.noreply.github.com >
Co-authored-by: XU Mingshi <91017482+mxuax@users.noreply.github.com >
Co-authored-by: Sihyeon Jang <uneedsihyeon@gmail.com >
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
Co-authored-by: catcat <108673086+iwzbi@users.noreply.github.com >
Co-authored-by: Ziming Huang <hzm414167@alibaba-inc.com >
Co-authored-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com >
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com >
Co-authored-by: Samit <285365963@qq.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com >
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com >
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: John Liu BUAA <liukecheng97@gmail.com >
Co-authored-by: D!NE$H <67671800+gDINESH13@users.noreply.github.com >
Co-authored-by: root <root@hk01dgx028.cm.cluster >
2026-01-21 00:25:25 +08:00
8939eb835d
[Rebase] Rebase to vllm 0.12.0 ( #335 )
...
Signed-off-by: Rein Yang <ruiruyang2@gmail.com >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Co-authored-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2025-12-19 16:42:02 +08:00
Cyrus Leung
c091a03d82
[Chore] Bump Python version and apply formatting changes for v0.12 release ( #126 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2025-12-16 15:19:40 +08:00
Lancer
31fedcfa2b
[Misc] fix yaml config ( #320 )
...
Signed-off-by: Lancer <maruixiang6688@gmail.com >
2025-12-16 02:40:30 +00:00
汪志鹏
6c333161a4
[BugFix]Remove duplicate code ( #309 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
2025-12-15 01:42:55 +00:00
Hongsheng Liu and WeiQing Chen
f22944eea4
[Doc] fix docs Feature Design and Module Design ( #283 )
...
Signed-off-by: hsliu <liuhongsheng4@huawei.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
2025-12-12 14:52:53 +08:00
Zhou Taichang and Gao Han
c70b167f70
[Engine] Remove Diffusion_XX which duplicates with Generation_XX ( #163 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Co-authored-by: Gao Han <gaohan19@huawei.com >
2025-12-05 03:04:23 +00:00
Samit
04987ffa7a
[Bugfix] Fix Qwen-omni Online Inference Bug caused by check_stop and long sequence ( #112 )
...
Signed-off-by: samithuang <285365963@qq.com >
2025-11-30 23:23:45 +08:00
Canlin Guo
ec9e04027d
[Feat] Add NPU Backend support for vLLM-Omni ( #89 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2025-11-30 20:39:06 +08:00
R2-Y and Rein
0a4e5a8710
[Feature] add support for Qwen3-omni ( #55 )
...
Signed-off-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: Rein <rein@Reins-MacBook-Air.local >
2025-11-30 04:44:01 +08:00
Roger Wang and Gao Han
a5800cd2c2
[Doc] Misc documentation polishing ( #98 )
...
Signed-off-by: Roger Wang <hey@rogerw.io >
Signed-off-by: Gao Han <gaohan19@huawei.com >
Co-authored-by: Gao Han <gaohan19@huawei.com >
2025-11-29 20:42:39 +08:00
Gao Han
db81ee9f69
[Docs] optimize and supplement docs system ( #86 )
...
Signed-off-by: Gaohan123 <gaohan19@huawei.com >
2025-11-28 11:03:52 +08:00
Jiangyun Zhu
5707fc78d5
[CI] fix ci and format existing code ( #71 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
2025-11-18 20:54:26 +08:00
Gaohan123
1aa5ae61cb
fix reviews
...
Signed-off-by: Gaohan123 <gaohan19@huawei.com >
2025-11-05 23:26:58 +08:00
Gaohan123
6e1478222e
Dependency refractored to vLLM v0.11.0
...
Signed-off-by: Gaohan123 <gaohan19@huawei.com >
2025-11-04 22:06:41 +08:00
Gaohan123
6cf73c4e80
Remove unused files and fix some bugs
2025-10-25 18:02:11 +08:00
Gaohan123
171f3c1ea7
fix conflicts
2025-10-24 16:49:19 +08:00
Gaohan123
a04e945166
add precommit
2025-10-24 16:20:58 +08:00
tzhouam
eb006f66e5
Enhance OmniScheduler error handling with logging
...
- Added logging for exceptions when wrapping scheduled requests in OmniScheduler.
- Removed unnecessary blank lines for cleaner code structure.
2025-10-24 12:00:36 +08:00
tzhouam
87b3ae7e70
Merge branch 'feat/omni-schedulers' of https://github.com/tzhouam/vllm-omni into feat/omni-schedulers
2025-10-24 11:57:45 +08:00
tzhouam
c23d24ca19
Refactor comments in diffusion_scheduler.py for clarity and consistency
...
- Translated comments from Chinese to English for better accessibility.
- Improved clarity of comments related to request handling and scheduling logic.
- Ensured comments align with the code functionality and maintain a uniform style.
2025-10-24 11:57:39 +08:00
Zhou Taichang and Copilot
9ebc1dd652
Apply suggestion from @Copilot
...
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
2025-10-24 10:41:54 +08:00
tzhouam
4acaef3ccb
Merge branch 'feat/omni-schedulers' of https://github.com/tzhouam/vllm-omni into feat/omni-schedulers
2025-10-24 09:41:29 +08:00
tzhouam
c37cd6f0e3
delete unused imports
2025-10-24 09:41:20 +08:00
Zhou Taichang and Copilot
b207bf89f2
Update vllm_omni/core/sched/diffusion_scheduler.py
...
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
2025-10-22 17:44:20 +08:00
Zhou Taichang and Copilot
8d6a360f96
Update vllm_omni/core/sched/diffusion_scheduler.py
...
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
2025-10-22 17:13:45 +08:00
tzhouam
19220aae82
Add scheduling components for vLLM-omni
...
- Introduced `DiffusionScheduler` to handle scheduling for diffusion models, allowing immediate request completion.
- Added `OmniNewRequestData` to encapsulate additional request data, including prompt embeddings and extra information.
- Updated `OmniScheduler` to ensure compatibility with new request data structures.
- Created `output.py` to define new data classes for enriched request handling.
- Enhanced `__init__.py` in the engine module to support new payload structures for prompt embeddings and additional information.
2025-10-22 15:05:23 +08:00
hsliu
c330e5e8f3
del unnessearcy files
2025-09-29 17:49:32 +08:00
hsliu
81f04d3cf9
update ar_dit_test
2025-09-28 21:58:48 +08:00