tzhouam
b867bacd24
rebase 2025/02/16
...
Qwen3 Omni offline/online passed
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-16 03:21:21 +00:00
tzhouam
15099119dd
Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0.16.0
2026-02-13 09:55:18 +00:00
汪志鹏
630e84ef93
[BugFix]: Revert utils change ( #1369 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
2026-02-13 17:54:09 +08:00
tzhouam
f24273dc03
Update vLLM version to 0.16.0 in quickstart and CUDA installation documentation, and adjust parameters in multimodal generation script for improved performance. Refactor request logging in serving_chat.py for better input tracking.
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-13 09:50:11 +00:00
tzhouam
31e9fdbb57
Refactor chat_utils to remove unused multi-modal classes and add async audio extraction from video. Update README to reflect new entry point for audio extraction. Enhance serving_chat to process audio from video URLs when specified.
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-13 03:27:16 +00:00
tzhouam
7e9cae56fc
Refactor Dockerfile.ci to use precompiled VLLM installation and remove unused parse.py file. Update serving_chat.py to streamline rendering logic and enhance compatibility with new API. Modify OmniInputPreprocessor to directly handle SingletonDictPrompt, improving input processing efficiency.
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-13 02:28:25 +00:00
Taichang Zhou
a6d633c319
debug
...
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
2026-02-12 23:46:20 +08:00
tzhouam
59a7602d6e
debug
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-12 13:55:08 +00:00
Taichang Zhou
85a7b558ab
rebase to v0.16.0
...
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
2026-02-12 21:30:54 +08:00
Taichang Zhou and Cursor
e2ced80d59
Merge origin/main into dev/rebase-v0.16.0.
...
Resolve arg_utils merge conflict and align CI base image tag to v0.16.0 for the vLLM rebase track.
Co-authored-by: Cursor <cursoragent@cursor.com >
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
2026-02-12 21:18:03 +08:00
dsinghvi and 汪志鹏
c6f314df4f
[Feature]: Diffusion sleep to use process level memory calculation ( #1276 )
...
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
2026-02-12 12:17:54 +00:00
knlnguyen1802
9994594782
[Misc] Support WorkerWrapperBase and CustomPipeline for Diffusion Worker ( #764 )
...
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com >
2026-02-12 19:23:51 +08:00
tzhouam
f8959bcc6e
Temp fix for metrics error in test_qwen3_tts
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-12 10:02:19 +00:00
Yuanheng Zhao and Samit
8bf849c399
Refactor CPU Offloading Backend Pattern ( #1223 )
...
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com >
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com >
Signed-off-by: Samit <285365963@qq.com >
Co-authored-by: Samit <285365963@qq.com >
2026-02-12 17:29:55 +08:00
ab1acb27ff
[Feature]async scheduling to overlap chunk IO and compute ( #951 )
...
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn >
Co-authored-by: Bhanu068 <voutharoja.bhanu06@gmail.com >
Co-authored-by: Gao Han <gaohan19@huawei.com >
2026-02-12 16:05:02 +08:00
tzhouam
3e89c99001
Merge remote-tracking branch 'remotes/origin/main' into dev/vllm-align
2026-02-12 07:16:25 +00:00
tzhouam
c751f0d684
Update Dockerfile.ci to install a new version of vllm from a different extra index URL. Refactor import statement in serving_chat.py to use the correct BaseRenderer module path.
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-12 06:37:17 +00:00
Samit and Flora Feng
9c00852d85
[Feature] Support Wan2.2 T2V and I2V Online Serving with OpenAI /v1/videos API ( #1073 )
...
Signed-off-by: samithuang <285365963@qq.com >
Signed-off-by: Samit <285365963@qq.com >
Signed-off-by: SamitHuang <285365963@qq.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-02-12 14:28:32 +08:00
tzhouam and Cursor
50f2b1e854
Merge origin/main into dev/vllm-align
...
Co-authored-by: Cursor <cursoragent@cursor.com >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-12 03:41:38 +00:00
Junhong Liu
82fd2c3637
Fix yield token metrics and opt metrics record stats ( #1292 )
2026-02-12 00:52:05 +08:00
Yueqian Lin
5fb7d137c7
[Bugfix][Qwen3-TTS] Preserve original model ID in omni_snapshot_download ( #1318 )
2026-02-11 18:24:45 +08:00
Zhou Taichang
a854f0b7ff
[Debug] Add trigger to concurrent stage init ( #1274 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-11 14:18:18 +08:00
tzhouam
3663e803d1
rebase 2026/02/11
...
vllm: 4a1550d22d7058e129d0e1257e726b3bf4a77025
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-11 02:43:49 +00:00
rein yang
963b64e7ae
[Bugfix] fix precision issues of qwen3-omni when enable async_chunk without system prompt ( #1288 )
...
Signed-off-by: Rein Yang <ruiruyang2@gmail.com >
2026-02-11 10:03:41 +08:00
tzhouam
279b6b5050
Merge remote-tracking branch 'origin/main' into dev/vllm-align
2026-02-11 01:44:54 +00:00
Canlin Guo
e192b0a6c5
[Misc] Add per-request generator_device to online image gen and edit ( #1183 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2026-02-10 18:47:08 +08:00
amy-why-3459
ee624d0686
[BugFix] Fixed the issue where ignore_eos was not working. ( #1286 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-02-10 16:08:20 +08:00
zhou zhuoxin and Copilot
109bb97b2c
[Fix]Ensure HuggingFace downloads complete before initialization. ( #1213 )
...
Signed-off-by: zhou zhuoxin <zhouzhuoxin1508@outlook.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2026-02-10 16:05:12 +08:00
tzhouam
064595a97c
Merge remote-tracking branch 'origin/main' into dev/vllm-align
2026-02-10 02:22:49 +00:00
Jiaping Wu
5fea4823b7
[Model] SupportHunyuanImage3 Diffusion Model in vllm-omni ( #1085 )
...
Signed-off-by: Semmer2 <semmer@live.cn >
2026-02-10 03:03:44 +13:00
Junhong Liu and Hongsheng Liu
9af6fb9a1f
[Feature] Opt metrics structure ( #891 )
...
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com >
Signed-off-by: Junhong Liu <ljh_lbj@163.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-02-09 19:17:32 +08:00
Chenguang Zheng
c6b526bc2e
fix: add diffusion offload args to OmniConfig group instead of serve_parser ( #1271 )
...
Signed-off-by: Chenguang ZHENG <645327136@qq.com >
2026-02-09 11:46:00 +08:00
tzhouam
7e4a604d4d
reabse 2025/2/9
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-09 03:07:43 +00:00
tzhouam and Cursor
03c6a58f69
Merge origin/main into dev/vllm-align; resolve omni_stage.py conflicts
...
Co-authored-by: Cursor <cursoragent@cursor.com >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-09 01:52:31 +00:00
dsinghvi
59024bee62
Process-Scoped GPU Memory Accounting ( #1204 )
...
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
2026-02-08 22:43:02 +08:00
ApsarasX
87847a214a
[Bugfix] Ensure seed=0 is correctly handled in image edit ( #1248 )
...
Signed-off-by: ApsarasX <apsarax@outlook.com >
2026-02-07 07:09:54 +08:00
ApsarasX
10f841bb98
[Bugfix] Fix '--max-generated-image-size' cli args type ( #1249 )
...
Signed-off-by: ApsarasX <apsarax@outlook.com >
2026-02-06 21:44:51 +08:00
tzhouam
41e22e2770
[Refactor] Clean up code by removing unnecessary lines and adding missing imports in async_omni_llm.py, omni_stage.py, omni.py, and api_server.py
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-05 11:02:03 +00:00
tzhouam
24b04f58a0
rebase 2026/2/5
2026-02-05 10:52:28 +00:00
Junhong Liu and Hongsheng Liu
959b3ca8c8
[Bugfix] E2E metric incorrect qwen3-omni with async chunk feature ( #1018 )
...
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com >
Signed-off-by: Junhong Liu <ljh_lbj@163.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-02-04 15:27:30 +08:00
Zhou Taichang
d6f93b0f37
[Debug] Correct Unreasonable Long Timeout ( #1175 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-03 08:49:51 +00:00
Zhou Taichang and 汪志鹏
70ef548448
Dev/rebase v0.15.0 ( #1159 )
...
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
2026-02-03 12:24:45 +08:00
ceanna93 and anna
df3b347cd9
[Misc] Align error handling with upstream vLLM v0.14.0 ( #1122 )
...
Signed-off-by: anna <lee.anna@navercorp.com >
Co-authored-by: anna <lee.anna@navercorp.com >
2026-02-01 21:59:08 +08:00
dengyunyang
70a5de9dd4
[FEATURE] /v1/images/edit interface ( #1101 )
...
Signed-off-by: dengyunyang <584797741@qq.com >
2026-01-31 14:13:36 +08:00
2e6895ff0e
[Bugfix][Qwen3TTS] Load speaker_id/voices from model configuration ( #1079 )
...
Signed-off-by: pablo <juanz9312@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
2026-01-31 10:42:09 +08:00
Didan Deng and Hongsheng Liu
23cf48d7bb
[Perf]: CFG parallel abstraction ( #851 )
...
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-30 16:23:44 +00:00
Jiangyun Zhu and Hongsheng Liu
8ad39570b8
[Bugfix] fix request_id of image generation in api server ( #1112 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-31 00:05:34 +08:00
2923d5a91e
[Feature] Support DiT Layerwise (Blockwise) CPU Offloading ( #858 )
...
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2026-01-30 17:35:09 +08:00
Pierre LE GUEN
2c294a750b
fix: UnboundLocalError for role in streaming audio/image responses ( #784 )
...
Signed-off-by: Pierre Le Guen <26087574+PierreLeGuen@users.noreply.github.com >
2026-01-30 16:27:04 +08:00
7bfd3c069d
[Benchmark] Add vLLM-Omni Omni model online benchmark ( #780 )
...
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com >
Signed-off-by: wangyu <53896905+yenuo26@users.noreply.github.com >
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-30 11:41:09 +08:00
tzhouam