anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
770 Commits 235 Branches 31 Tags
bf69cd94bbc1ee755ae82c075002c75ec9d296d3
Commit Graph
770 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
tzhouam bf69cd94bb Rebase 2026/02/23 
Update Dockerfile for vllm wheel version and enhance input processing in Omni models. Added decoding for byte messages in async requests, included new metrics in benchmarks, and improved pause state handling in the scheduler. Refined input and output processing methods for better multimodal support and error handling.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-23 03:36:01 +00:00
tzhouam 292da4b0d8 Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-02-23 02:09:09 +00:00
Hongsheng Liu a3f2d4cbfe Revert "[misc] Feature/pr reviewer auto trigger&update model" (#1432) 2026-02-23 07:52:09 +08:00
Hongsheng LiuandClaude Opus 4.6 267170115b [misc] Feature/pr reviewer auto trigger&update model (#1431) 
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
Signed-off-by: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Hunter Liu <hunter@liu.sh>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-22 23:33:45 +08:00
UsamaKenway d8a32a6a87 [Misc] Add Qwen2.5-Omni-3B model support to Gradio demo (#1382) 
Signed-off-by: UsamaKenway <usamakenway@gmail.com>
 2026-02-22 21:17:09 +08:00
Hongsheng LiuandClaude Opus 4.6 6c85ddcb96 [Feature] Add automated PR reviewer bot with GLM integration (#1424) 
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
Signed-off-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-22 13:53:59 +08:00
Sy03andHongsheng Liu efbe4116ff [Feature] : Support disaggregated inference pipeline for Qwen3_TTS (#1161) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-21 07:36:44 +08:00
tzhouam 9869ea007e Update vLLM wheel version in Dockerfile and enhance input processing for omni models. Added functionality to reinject omni-specific fields into processed prompts and improved handling of multimodal audio extraction in serving_chat.py. Adjusted SharedMemoryConnector logic for payload size checks and refined input preprocessing methods for better compatibility with multimodal data. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-20 08:34:53 +00:00
Alex Brooks 5295f0f81b [Bugfix] Fix Hardcoded Datatypes in Z-image (#1393) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-02-20 06:55:24 +00:00
Yan MaandDaniel Huang 4f9731c920 [XPU] Add XPU Dockerfile and related docs (#1162) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
Signed-off-by: Daniel Huang <daniel1.huang@intel.com>
Co-authored-by: Daniel Huang <daniel1.huang@intel.com>
 2026-02-20 13:33:48 +08:00
Chendi.Xue d2f015edce [0.16.0] remove cuda hard-code for Hunyuan Image3 (#1402) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
 2026-02-20 13:23:31 +08:00
tzhouam 663976b8ca Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-02-20 01:38:44 +00:00
Chendi.Xue 0b577a7263 [v0.16.0][BUG FIX]Fix hunyuan MOE after update to 0.16.0 (#1401) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
 2026-02-19 23:57:11 +13:00
tzhouam b867bacd24 rebase 2025/02/16 
Qwen3 Omni offline/online passed

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-16 03:21:21 +00:00
tzhouam 93eeb1a4c8 Merge remote-tracking branch 'origin/main' into dev/rebase-v0.16.0 2026-02-15 01:24:23 +00:00
wangyu a42b7487b0 [Test] Fix expansion and example test case for qwen3-omni (#1358) 
Signed-off-by: yenuo26 <410167048@qq.com>
 2026-02-15 00:43:38 +13:00
Zhou Taichang汪志鹏Hongsheng LiuIsotr0pyZJY0516youkaichao
 		75770c9309 [Rebase] Rebase to vllm v0.16.0 (#1357)
ComfyUI Integration -- Publish to Comfy registry / Publish Custom Node to registry (push) Failing after 44s
Details 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: ZJY0516 <zhu.jiangyun@foxmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
v0.16.0rc1 		2026-02-13 18:05:07 +08:00
tzhouam 3051fb10b5 reverse change in SamplingParams 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 09:59:03 +00:00
tzhouam 15099119dd Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0.16.0 2026-02-13 09:55:18 +00:00
汪志鹏 630e84ef93 [BugFix]: Revert utils change (#1369) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-02-13 17:54:09 +08:00
tzhouam f24273dc03 Update vLLM version to 0.16.0 in quickstart and CUDA installation documentation, and adjust parameters in multimodal generation script for improved performance. Refactor request logging in serving_chat.py for better input tracking. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 09:50:11 +00:00
tzhouam f50cf07c74 Update Dockerfile.ci to allow the installation of 'pplx_kernels' to continue even if the compatible wheel is not found, enhancing build resilience. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 07:22:07 +00:00
tzhouam bc0a2598ed Update Dockerfile.ci to replace 'uv pip' with 'pip' for 'pplx_kernels' installation, ensuring consistency in package management across the CI environment. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 07:11:40 +00:00
tzhouam 963deaf5ad Refactor Dockerfile.ci to streamline package installations by using 'uv pip' consistently for both 'pplx_kernels' and VLLM dependencies, ensuring a more uniform CI environment setup. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 06:55:10 +00:00
tzhouam 3abec0173c Update Dockerfile.ci to force-reinstall the 'pplx_kernels' package, ensuring the latest version is used in the CI environment. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 06:41:41 +00:00
tzhouam 8a4eee9bed Enhance configuration and debugging: Added asyncio mode to pytest settings, increased timeout for Qwen3-TTS E2E test, and improved logging in GPU model runner for better traceability of input modalities. Updated Dockerfile.ci to streamline VLLM installation process. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 06:39:28 +00:00
tzhouam 31e9fdbb57 Refactor chat_utils to remove unused multi-modal classes and add async audio extraction from video. Update README to reflect new entry point for audio extraction. Enhance serving_chat to process audio from video URLs when specified. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 03:27:16 +00:00
tzhouam 39a045f439 debug Qwen 3 Omni cuda graph 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 02:57:05 +00:00
tzhouam 7e9cae56fc Refactor Dockerfile.ci to use precompiled VLLM installation and remove unused parse.py file. Update serving_chat.py to streamline rendering logic and enhance compatibility with new API. Modify OmniInputPreprocessor to directly handle SingletonDictPrompt, improving input processing efficiency. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 02:28:25 +00:00
tzhouam 31c5465e00 Update .gitignore to exclude .wav output files and remove audio_0.wav 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 01:52:06 +00:00
Hongsheng Liu 622c6f5f14 Merge branch 'main' into dev/rebase-v0.16.0 2026-02-13 06:05:57 +08:00
Taichang Zhou a6d633c319 debug 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-12 23:46:20 +08:00
wangyu f4867ec175 [Test] Add Omni Model Performance Benchmark Test (#1321) 
Signed-off-by: yenuo26 <410167048@qq.com>
Signed-off-by: wangyu <53896905+yenuo26@users.noreply.github.com>
 2026-02-12 23:11:57 +08:00
tzhouam ec3814f587 debug 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-12 14:20:57 +00:00
tzhouam 59a7602d6e debug 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-12 13:55:08 +00:00
liuzhenwei 00bd07b82b [XPU] Update Bagel's flash_attn_varlen_func to fa utils (#1295) 
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
 2026-02-12 21:33:06 +08:00
Taichang Zhou 85a7b558ab rebase to v0.16.0 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-12 21:30:54 +08:00
Taichang Zhou 3dc19bc305 0.16.0 docker not released yet 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-12 21:25:53 +08:00
Taichang Zhou 7458ba44b7 merge main 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-12 21:21:03 +08:00
Taichang ZhouandCursor e2ced80d59 Merge origin/main into dev/rebase-v0.16.0. 
Resolve arg_utils merge conflict and align CI base image tag to v0.16.0 for the vLLM rebase track.

Co-authored-by: Cursor <cursoragent@cursor.com>

Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-12 21:18:03 +08:00
Taichang Zhou 1705edcce9 Refactor model path in test_qwen3_tts for improved clarity and maintainability 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-12 21:12:53 +08:00
rein yangandHongsheng Liu 36a79717c6 change qwen3-omni open cudagraph by default (#1352) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-12 20:49:34 +08:00
dsinghviand汪志鹏 c6f314df4f [Feature]: Diffusion sleep to use process level memory calculation (#1276) 
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-02-12 12:17:54 +00:00
Ding Zuhaoand汪志鹏 b16965dfa7 [Feature][Bugfix] Add CFG feature to Bagel (#1310) 
Signed-off-by: Ding Zuhao <e1583181@u.nus.edu>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-02-12 19:48:30 +08:00
knlnguyen1802 9994594782 [Misc] Support WorkerWrapperBase and CustomPipeline for Diffusion Worker (#764) 
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
 2026-02-12 19:23:51 +08:00
WeiQing Chen e427da022b [Misc] wechat image update (#1354) 
Signed-off-by: David Chen <530634352@qq.com>
 2026-02-12 18:46:56 +08:00
dengyunyang b0a5ffa100 [Bugfix] remove Tongyi-MAI/Z-Image-Turbo related test from L2 ci (#1348) 
Signed-off-by: dengyunyang <584797741@qq.com>
 2026-02-12 18:44:05 +08:00
Aliciaandyenuo26 17e238c7ea [DOC] Doc for CI test - Details about five level stucture and some other files. (#1167) 
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
Co-authored-by: yenuo26 <410167048@qq.com>
 2026-02-12 18:43:51 +08:00
tzhouam f8959bcc6e Temp fix for metrics error in test_qwen3_tts 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-12 10:02:19 +00:00
tzhouam db40563b88 Update Dockerfile.ci to pin nvidia-cublas-cu12 and add numpy installation, ensuring proper dependency management for the build environment. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-12 09:52:51 +00:00