tzhouam
bf69cd94bb
Rebase 2026/02/23
...
Update Dockerfile for vllm wheel version and enhance input processing in Omni models. Added decoding for byte messages in async requests, included new metrics in benchmarks, and improved pause state handling in the scheduler. Refined input and output processing methods for better multimodal support and error handling.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-23 03:36:01 +00:00
tzhouam
292da4b0d8
Merge remote-tracking branch 'origin/main' into dev/vllm-align
2026-02-23 02:09:09 +00:00
Hongsheng Liu
a3f2d4cbfe
Revert "[misc] Feature/pr reviewer auto trigger&update model" ( #1432 )
2026-02-23 07:52:09 +08:00
Hongsheng Liu and Claude Opus 4.6
267170115b
[misc] Feature/pr reviewer auto trigger&update model ( #1431 )
...
Signed-off-by: hsliu <liuhongsheng4@huawei.com >
Signed-off-by: Claude Opus 4.6 <noreply@anthropic.com >
Signed-off-by: Hunter Liu <hunter@liu.sh >
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
2026-02-22 23:33:45 +08:00
Sy03 and Hongsheng Liu
efbe4116ff
[Feature] : Support disaggregated inference pipeline for Qwen3_TTS ( #1161 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-02-21 07:36:44 +08:00
tzhouam
9869ea007e
Update vLLM wheel version in Dockerfile and enhance input processing for omni models. Added functionality to reinject omni-specific fields into processed prompts and improved handling of multimodal audio extraction in serving_chat.py. Adjusted SharedMemoryConnector logic for payload size checks and refined input preprocessing methods for better compatibility with multimodal data.
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-20 08:34:53 +00:00
Alex Brooks
5295f0f81b
[Bugfix] Fix Hardcoded Datatypes in Z-image ( #1393 )
...
Signed-off-by: Alex Brooks <albrooks@redhat.com >
2026-02-20 06:55:24 +00:00
Chendi.Xue
d2f015edce
[0.16.0] remove cuda hard-code for Hunyuan Image3 ( #1402 )
...
Signed-off-by: Chendi Xue <chendi.xue@intel.com >
2026-02-20 13:23:31 +08:00
tzhouam
663976b8ca
Merge remote-tracking branch 'origin/main' into dev/vllm-align
2026-02-20 01:38:44 +00:00
Chendi.Xue
0b577a7263
[v0.16.0][BUG FIX]Fix hunyuan MOE after update to 0.16.0 ( #1401 )
...
Signed-off-by: Chendi Xue <chendi.xue@intel.com >
2026-02-19 23:57:11 +13:00
tzhouam
b867bacd24
rebase 2025/02/16
...
Qwen3 Omni offline/online passed
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-16 03:21:21 +00:00
75770c9309
[Rebase] Rebase to vllm v0.16.0 ( #1357 )
...
ComfyUI Integration -- Publish to Comfy registry / Publish Custom Node to registry (push) Failing after 44s
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: ZJY0516 <zhu.jiangyun@foxmail.com >
Co-authored-by: youkaichao <youkaichao@gmail.com >
2026-02-13 18:05:07 +08:00
tzhouam
15099119dd
Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0.16.0
2026-02-13 09:55:18 +00:00
汪志鹏
630e84ef93
[BugFix]: Revert utils change ( #1369 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
2026-02-13 17:54:09 +08:00
tzhouam
f24273dc03
Update vLLM version to 0.16.0 in quickstart and CUDA installation documentation, and adjust parameters in multimodal generation script for improved performance. Refactor request logging in serving_chat.py for better input tracking.
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-13 09:50:11 +00:00
tzhouam
8a4eee9bed
Enhance configuration and debugging: Added asyncio mode to pytest settings, increased timeout for Qwen3-TTS E2E test, and improved logging in GPU model runner for better traceability of input modalities. Updated Dockerfile.ci to streamline VLLM installation process.
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-13 06:39:28 +00:00
tzhouam
31e9fdbb57
Refactor chat_utils to remove unused multi-modal classes and add async audio extraction from video. Update README to reflect new entry point for audio extraction. Enhance serving_chat to process audio from video URLs when specified.
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-13 03:27:16 +00:00
tzhouam
39a045f439
debug Qwen 3 Omni cuda graph
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-13 02:57:05 +00:00
tzhouam
7e9cae56fc
Refactor Dockerfile.ci to use precompiled VLLM installation and remove unused parse.py file. Update serving_chat.py to streamline rendering logic and enhance compatibility with new API. Modify OmniInputPreprocessor to directly handle SingletonDictPrompt, improving input processing efficiency.
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-13 02:28:25 +00:00
Hongsheng Liu
622c6f5f14
Merge branch 'main' into dev/rebase-v0.16.0
2026-02-13 06:05:57 +08:00
Taichang Zhou
a6d633c319
debug
...
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
2026-02-12 23:46:20 +08:00
tzhouam
ec3814f587
debug
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-12 14:20:57 +00:00
tzhouam
59a7602d6e
debug
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-12 13:55:08 +00:00
liuzhenwei
00bd07b82b
[XPU] Update Bagel's flash_attn_varlen_func to fa utils ( #1295 )
...
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com >
2026-02-12 21:33:06 +08:00
Taichang Zhou
85a7b558ab
rebase to v0.16.0
...
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
2026-02-12 21:30:54 +08:00
Taichang Zhou
7458ba44b7
merge main
...
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
2026-02-12 21:21:03 +08:00
Taichang Zhou and Cursor
e2ced80d59
Merge origin/main into dev/rebase-v0.16.0.
...
Resolve arg_utils merge conflict and align CI base image tag to v0.16.0 for the vLLM rebase track.
Co-authored-by: Cursor <cursoragent@cursor.com >
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
2026-02-12 21:18:03 +08:00
rein yang and Hongsheng Liu
36a79717c6
change qwen3-omni open cudagraph by default ( #1352 )
...
Signed-off-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-02-12 20:49:34 +08:00
dsinghvi and 汪志鹏
c6f314df4f
[Feature]: Diffusion sleep to use process level memory calculation ( #1276 )
...
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
2026-02-12 12:17:54 +00:00
Ding Zuhao and 汪志鹏
b16965dfa7
[Feature][Bugfix] Add CFG feature to Bagel ( #1310 )
...
Signed-off-by: Ding Zuhao <e1583181@u.nus.edu >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
2026-02-12 19:48:30 +08:00
knlnguyen1802
9994594782
[Misc] Support WorkerWrapperBase and CustomPipeline for Diffusion Worker ( #764 )
...
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com >
2026-02-12 19:23:51 +08:00
tzhouam
f8959bcc6e
Temp fix for metrics error in test_qwen3_tts
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-12 10:02:19 +00:00
Yuanheng Zhao and Samit
8bf849c399
Refactor CPU Offloading Backend Pattern ( #1223 )
...
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com >
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com >
Signed-off-by: Samit <285365963@qq.com >
Co-authored-by: Samit <285365963@qq.com >
2026-02-12 17:29:55 +08:00
John Liu BUAA
f117a07271
[Bugfix] reused metrics to modify the API Server token statistics in Stream Response ( #1301 )
...
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com >
2026-02-12 17:24:01 +08:00
tzhouam
86e20769fd
Add SSL context support in benchmark function for improved connection handling
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-12 08:52:27 +00:00
ab1acb27ff
[Feature]async scheduling to overlap chunk IO and compute ( #951 )
...
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn >
Co-authored-by: Bhanu068 <voutharoja.bhanu06@gmail.com >
Co-authored-by: Gao Han <gaohan19@huawei.com >
2026-02-12 16:05:02 +08:00
Sihao Li
71c9e77606
[Feature] add Tensor Parallelism to SD_3.5 ( #1336 )
...
Signed-off-by: GG-li <3226868735@qq.com >
2026-02-12 07:33:23 +00:00
tzhouam
3e89c99001
Merge remote-tracking branch 'remotes/origin/main' into dev/vllm-align
2026-02-12 07:16:25 +00:00
tzhouam
c751f0d684
Update Dockerfile.ci to install a new version of vllm from a different extra index URL. Refactor import statement in serving_chat.py to use the correct BaseRenderer module path.
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-12 06:37:17 +00:00
Samit and Flora Feng
9c00852d85
[Feature] Support Wan2.2 T2V and I2V Online Serving with OpenAI /v1/videos API ( #1073 )
...
Signed-off-by: samithuang <285365963@qq.com >
Signed-off-by: Samit <285365963@qq.com >
Signed-off-by: SamitHuang <285365963@qq.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-02-12 14:28:32 +08:00
0b56fba7b3
[Pref] Support Qwen3 Omni code2wav batch infernce with async chunk ( #1246 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Signed-off-by: Ziming Huang <1520787127@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2026-02-12 11:50:10 +08:00
tzhouam and Cursor
50f2b1e854
Merge origin/main into dev/vllm-align
...
Co-authored-by: Cursor <cursoragent@cursor.com >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-12 03:41:38 +00:00
tzhouam
0099eebb3f
Refactor OmniEngineArgs and AsyncOmniEngineArgs to improve compatibility with partial payloads by using getattr for various attributes. Update GPUGenerationModelRunner and OmniGPUModelRunner to replace the activate_lora parameter with num_active_loras for better clarity and backward compatibility. This change enhances the handling of LoRA activation during model runs.
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-12 03:33:50 +00:00
Junhong Liu
82fd2c3637
Fix yield token metrics and opt metrics record stats ( #1292 )
2026-02-12 00:52:05 +08:00
SYLAR
830d261f9d
[Feature]: FP8 Quantization Support for DiT ( #1034 )
...
Signed-off-by: lishunyang <lishunyang12@163.com >
Signed-off-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
2026-02-11 21:01:35 +08:00
Yueqian Lin
5fb7d137c7
[Bugfix][Qwen3-TTS] Preserve original model ID in omni_snapshot_download ( #1318 )
2026-02-11 18:24:45 +08:00
Ekagra Ranjan
dd302b63c4
[Bugfix][Qwen3-TTS] Fix task type ( #1317 )
...
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com >
2026-02-11 15:49:15 +08:00
Zhou Taichang
a854f0b7ff
[Debug] Add trigger to concurrent stage init ( #1274 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-11 14:18:18 +08:00
tzhouam
3663e803d1
rebase 2026/02/11
...
vllm: 4a1550d22d7058e129d0e1257e726b3bf4a77025
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-11 02:43:49 +00:00
rein yang
963b64e7ae
[Bugfix] fix precision issues of qwen3-omni when enable async_chunk without system prompt ( #1288 )
...
Signed-off-by: Rein Yang <ruiruyang2@gmail.com >
2026-02-11 10:03:41 +08:00