anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
793 Commits 235 Branches 31 Tags
22e01c416bb235b2a0123da2858f8cfd384ebeca
Commit Graph
54 Commits
Author SHA1 Message Date
tzhouam 861698f88f Update Dockerfile for vllm wheel version and enhance SHM connector tests 
- Updated Dockerfile to use a new vllm wheel version.
- Modified test_put_get_inline to clarify that it always uses SHM and updated metadata assertions.
- Added handling for speculative configuration in GPU model runners to delay clearing connector metadata.
- Refactored payload decoding in OmniGPUModelRunner for better clarity and maintainability.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-24 05:45:48 +00:00
tzhouam e22ae62396 Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-02-24 03:23:20 +00:00
TJian 5722333fba [ROCM] [CI] fix dockerfile.rocm to support nightly build and also fix amd ci v0.16.0rc1 (#1380) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-02-23 15:00:12 +08:00
tzhouam bf69cd94bb Rebase 2026/02/23 
Update Dockerfile for vllm wheel version and enhance input processing in Omni models. Added decoding for byte messages in async requests, included new metrics in benchmarks, and improved pause state handling in the scheduler. Refined input and output processing methods for better multimodal support and error handling.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-23 03:36:01 +00:00
tzhouam 292da4b0d8 Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-02-23 02:09:09 +00:00
tzhouam 9869ea007e Update vLLM wheel version in Dockerfile and enhance input processing for omni models. Added functionality to reinject omni-specific fields into processed prompts and improved handling of multimodal audio extraction in serving_chat.py. Adjusted SharedMemoryConnector logic for payload size checks and refined input preprocessing methods for better compatibility with multimodal data. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-20 08:34:53 +00:00
Yan MaandDaniel Huang 4f9731c920 [XPU] Add XPU Dockerfile and related docs (#1162) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
Signed-off-by: Daniel Huang <daniel1.huang@intel.com>
Co-authored-by: Daniel Huang <daniel1.huang@intel.com>
 2026-02-20 13:33:48 +08:00
tzhouam b867bacd24 rebase 2025/02/16 
Qwen3 Omni offline/online passed

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-16 03:21:21 +00:00
Zhou Taichang汪志鹏Hongsheng LiuIsotr0pyZJY0516youkaichao
 		75770c9309 [Rebase] Rebase to vllm v0.16.0 (#1357)
ComfyUI Integration -- Publish to Comfy registry / Publish Custom Node to registry (push) Failing after 44s
Details 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: ZJY0516 <zhu.jiangyun@foxmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2026-02-13 18:05:07 +08:00
tzhouam f50cf07c74 Update Dockerfile.ci to allow the installation of 'pplx_kernels' to continue even if the compatible wheel is not found, enhancing build resilience. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 07:22:07 +00:00
tzhouam bc0a2598ed Update Dockerfile.ci to replace 'uv pip' with 'pip' for 'pplx_kernels' installation, ensuring consistency in package management across the CI environment. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 07:11:40 +00:00
tzhouam 963deaf5ad Refactor Dockerfile.ci to streamline package installations by using 'uv pip' consistently for both 'pplx_kernels' and VLLM dependencies, ensuring a more uniform CI environment setup. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 06:55:10 +00:00
tzhouam 3abec0173c Update Dockerfile.ci to force-reinstall the 'pplx_kernels' package, ensuring the latest version is used in the CI environment. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 06:41:41 +00:00
tzhouam 8a4eee9bed Enhance configuration and debugging: Added asyncio mode to pytest settings, increased timeout for Qwen3-TTS E2E test, and improved logging in GPU model runner for better traceability of input modalities. Updated Dockerfile.ci to streamline VLLM installation process. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 06:39:28 +00:00
tzhouam 7e9cae56fc Refactor Dockerfile.ci to use precompiled VLLM installation and remove unused parse.py file. Update serving_chat.py to streamline rendering logic and enhance compatibility with new API. Modify OmniInputPreprocessor to directly handle SingletonDictPrompt, improving input processing efficiency. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 02:28:25 +00:00
Taichang Zhou 85a7b558ab rebase to v0.16.0 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-12 21:30:54 +08:00
Taichang Zhou 3dc19bc305 0.16.0 docker not released yet 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-12 21:25:53 +08:00
Taichang ZhouandCursor e2ced80d59 Merge origin/main into dev/rebase-v0.16.0. 
Resolve arg_utils merge conflict and align CI base image tag to v0.16.0 for the vLLM rebase track.

Co-authored-by: Cursor <cursoragent@cursor.com>

Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-12 21:18:03 +08:00
tzhouam db40563b88 Update Dockerfile.ci to pin nvidia-cublas-cu12 and add numpy installation, ensuring proper dependency management for the build environment. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-12 09:52:51 +00:00
tzhouam 8c1df5c704 update flashinfer 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-12 08:52:06 +00:00
tzhouam c751f0d684 Update Dockerfile.ci to install a new version of vllm from a different extra index URL. Refactor import statement in serving_chat.py to use the correct BaseRenderer module path. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-12 06:37:17 +00:00
Taichang Zhou cd2ffbf2a0 update dockerfile.ci 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-11 23:39:15 +08:00
Taichang Zhou 514bd46f2e Uninstall old nvidia-cublas-cu12 version before reinstalling the pinned version to ensure consistent dependency management in Dockerfile.ci. 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-11 22:34:05 +08:00
Taichang Zhou 169d94dab9 Refactor Dockerfile.ci to use bash for Python executable resolution during vllm-omni installation and pin nvidia-cublas-cu12 version to 12.9.1.4 for consistent dependency management. 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-11 21:31:09 +08:00
Taichang Zhou 268a58e4e8 Update pyproject.toml to remove redundant transformers package installation and adjust Dockerfile.ci to install the current package in editable mode. Also, update fa3-fwd package version in requirements/cuda.txt to 0.0.2 for consistency. 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-11 21:18:53 +08:00
Taichang Zhou eb05967451 Pin nvidia-cublas-cu12 to version 12.9.1.4 in Dockerfile.ci to ensure consistent dependency management in the CI environment. 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-11 20:31:57 +08:00
tzhouam 05784d3dd0 Update pyproject.toml to include transformers package and remove redundant installation step in Dockerfile.ci for cleaner CI setup. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-11 06:27:51 +00:00
tzhouam b7f297c8c3 Add transformers package installation in Dockerfile.ci to ensure compatibility with the CI environment. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-11 06:15:00 +00:00
tzhouam 379f61d038 Enhance Dockerfile.ci by adding a debug step to print key dependency versions for CI troubleshooting, while commenting out the flashinfer version sync logic for clarity. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-11 06:01:01 +00:00
tzhouam 31b6584373 Refine flashinfer version retrieval in Dockerfile.ci to enhance reliability and maintain consistency in package installations 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-11 04:56:50 +00:00
tzhouam 50cea0b29c Refactor flashinfer version retrieval in Dockerfile.ci for improved accuracy and consistency in package installations 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-11 04:34:52 +00:00
tzhouam e897640b43 Update Dockerfile.ci to enhance flashinfer version retrieval logic and ensure package installation consistency 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-11 04:11:11 +00:00
tzhouam 3663e803d1 rebase 2026/02/11 
vllm: 4a1550d22d7058e129d0e1257e726b3bf4a77025

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-11 02:43:49 +00:00
tzhouam 279b6b5050 Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-02-11 01:44:54 +00:00
Yueqian Lin f55ce9c793 [Test] Add e2e tests for Qwen3-TTS speech endpoint (#1206) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-02-10 16:33:39 +08:00
tzhouam 3c5b53358c Update Dockerfile.ci to ensure consistent formatting of flashinfer version extraction commands for improved readability and maintainability. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-10 03:53:55 +00:00
tzhouam 58d09ed656 reabse 2026/02/10 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-10 03:36:09 +00:00
tzhouam e0f540fe98 Enhance Dockerfile.ci to synchronize flashinfer runtime package versions with installed flashinfer-python, ensuring compatibility across components. Additionally, improve error handling in ring_globals.py for flashinfer import failures, logging warnings for fallback scenarios. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-10 02:17:00 +00:00
tzhouam 670240ccd4 Enhance Dockerfile.ci to allow pre-release vLLM installations by adding --prerelease=allow flag for compatibility with nightly/commit wheels. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-09 10:01:39 +00:00
tzhouam c9fb2dc7c7 Update vLLM commit reference in Dockerfile.ci to f97ca671766c5201404e9fc812e35bf2c4e95a01 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-09 08:09:21 +00:00
tzhouam 7f54fd2f54 Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-02-06 02:26:06 +00:00
TJianPopSoda2002gcanlin
 		78a5aae299 [Hardware] [Feat] Setup platform dependent package installation (#1046) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: PopSoda2002 <zhouhp.me@gmail.com>
Co-authored-by: gcanlin <canlinguosdu@gmail.com>
 2026-02-05 21:47:41 +08:00
tzhouam 24b04f58a0 rebase 2026/2/5 2026-02-05 10:52:28 +00:00
Canlin Guo f95f8e936d [NPU] Update Dockerfile and docs for v0.14.0 (#671) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-02-04 15:10:50 +08:00
Zhou Taichang 7fcbaa794c [Debug] Clear Dockerfile.ci to accelerate build image (#1172) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-03 13:43:45 +08:00
Zhou Taichangand汪志鹏 70ef548448 Dev/rebase v0.15.0 (#1159) 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-02-03 12:24:45 +08:00
TJian 28d13e0dee [Doc] Fix the version in the documentation to v0.14.0rc1; fix dockerfile.rocm entrypoint (#913) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-01-23 03:37:54 +00:00
TJian 4869311f2f [ROCm] [CI] Optimize Dockerfilerocm and reduce build time on CI (#885) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-01-22 03:39:45 +13:00
+13
Zhou TaichangJaredforRealrootJustQJXU MingshiSihyeon JangJiangyun ZhucatcatZiming HuangYuhan LiuAliciaSamitYueqian LinWeiQing ChenPeiqi YinDidan DengHongsheng Liuwangyuwangyu31577汪志鹏John Liu BUAAD!NE$Hroot
 		776c3a794e Dev/rebase 0.14.0 (#847) 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: root <root@hk01dgx039.cm.cluster>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: TangPeng <85704592@qq.com>
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: iwzbi <wzbi@zju.edu.cn>
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Signed-off-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com>
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com>
Signed-off-by: Dinesh G <G.Dinesh@ibm.com>
Signed-off-by: gDINESH13 <dinesh13g@gmail.com>
Signed-off-by: root <root@hk01dgx028.cm.cluster>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: JaredforReal <w13431838023@gmail.com>
Co-authored-by: root <root@hk01dgx039.cm.cluster>
Co-authored-by: JustQJ <37905360+JustQJ@users.noreply.github.com>
Co-authored-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Co-authored-by: Sihyeon Jang <uneedsihyeon@gmail.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: catcat <108673086+iwzbi@users.noreply.github.com>
Co-authored-by: Ziming Huang <hzm414167@alibaba-inc.com>
Co-authored-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: John Liu BUAA <liukecheng97@gmail.com>
Co-authored-by: D!NE$H <67671800+gDINESH13@users.noreply.github.com>
Co-authored-by: root <root@hk01dgx028.cm.cluster>
 2026-01-21 00:25:25 +08:00
wangyuandwangyu31577 4e23bfff9e [Test] Add full test for Qwen3-Omni-30B-A3B-Instruct (#720) 
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com>
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com>
 2026-01-16 21:16:43 +08:00