anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
873 Commits 235 Branches 31 Tags
3e539fcb6a017fa9d8ac86f4b8d6eb852ff0dde4
Commit Graph
70 Commits
Author SHA1 Message Date
tzhouam e0497550c4 Update Dockerfile.ci to install a specific version of flashinfer-cubin (0.6.4) directly, simplifying the installation process and ensuring compatibility with the current setup. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 02:51:35 +00:00
tzhouam 029bb56e3a Update Dockerfile.ci to use importlib.metadata for retrieving the current version of flashinfer, enhancing compatibility with the latest Python standards. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 02:28:24 +00:00
tzhouam 65169baa9d Update Dockerfile.ci to create a symbolic link for Python 3, ensuring compatibility with Python scripts during the build process. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 01:58:35 +00:00
tzhouam 908e3fe761 Update Dockerfile.ci to dynamically install the current version of flashinfer-cubin, ensuring compatibility with the latest changes in the flashinfer package. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-02 07:00:59 +00:00
tzhouam 8fb247a481 Update Dockerfile.ci to install specific version of flashinfer-cubin (0.6.4) and ensure vllm installation remains intact. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-02 06:48:58 +00:00
tzhouam be4226bae0 Remove unnecessary uninstallation of nvidia-cublas-cu12 and streamline vllm installation in Dockerfile.ci 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-02 06:38:24 +00:00
tzhouam 473d3ecd29 Update vLLM wheel version and adjust GPU memory utilization in CI configuration 
- Updated the vLLM wheel commit hash in Dockerfile.ci to a60985b07eaf0fefa4c353ad2309b745707591ed.
- Modified the mock shared memory handle name in test_basic_connectors.py for clarity.
- Increased GPU memory utilization to 0.9 and adjusted max_num_batched_tokens to 4096 in qwen2_5_omni_ci.yaml to enable testing on H800.
- Refactored GPU model runner classes to remove deprecated activate_lora parameter and ensure compatibility with num_active_loras.
- Enhanced request handling in the OmniGPUModelRunner class to improve streaming request updates.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-02 03:08:45 +00:00
tzhouam 26744bcb35 Merge origin/main into dev/vllm-align 
Resolved conflicts:
- vllm_omni/inputs/preprocess.py: took origin/main's elif mm_processor_kwargs branch, fixed undefined mm_uuids reference
- vllm_omni/model_executor/models/qwen3_tts/qwen3_tts_code_predictor_vllm.py: took origin/main's NPU support

Made-with: Cursor

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-02 02:10:32 +00:00
Canlin Guo c81266703a [NPU][Bugfix] Align GPU side and recover qwen3-tts (#1564) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-02-28 12:46:13 +08:00
TJian 7ac8d613f6 [Version] Auto generate version using setuptool_scm (#1224) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-02-27 14:34:27 +08:00
TJian 881c675087 [ROCm] [CI] [Docker] Point to use the latest vLLM v0.16.0 stable version (#1500) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-02-26 19:16:14 +08:00
tzhouam 8d0c46487e Merge origin/main into dev/vllm-align 
Resolve conflicts:
- docker/Dockerfile.ci: Keep wheel install section, update hash to 6042e66cd5
- vllm_omni/platforms/npu/worker/npu_model_runner.py: Accept main's refactoring

Made-with: Cursor

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-26 08:19:53 +00:00
Zhou Taichang 986b356077 [CI] Update Dockerfile for vllm-omni CI image and remove obsolete dep… (#1491) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-26 14:36:03 +08:00
Canlin Guo 22c8cde328 [NPU] Upgrade to v0.16.0 (#1375) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-02-26 14:26:50 +08:00
wangyu 1ae2e1e4c4 [Test] Reduce Perf test case and fix modify stage config (#1449) 
Signed-off-by: yenuo26 <410167048@qq.com>
 2026-02-26 14:20:57 +08:00
tzhouam a252918008 [Update] Upgrade vLLM wheel version and enhance metrics handling 
- Updated the vLLM wheel to commit 24650715105a26b41c88f4164777b5e1a0f3200b in Dockerfile.ci.
- Modified the `calculate_metrics` function to handle `tokenizer` as optional (can be None) in metrics.py.
- Adjusted the `benchmark` function to accept `tokenizer` as optional in patch.py.
- Enhanced condition checks in OmniARScheduler to ensure valid token generation in omni_ar_scheduler.py.
- Updated input processing to include supported tasks in input_processor.py.
- Cleaned up unnecessary code in output_processor.py.
- Added support for SSL ciphers in the API server configuration in api_server.py.
- Enabled speculative decoding in GPU model runners for improved performance in gpu_generation_model_runner.py and gpu_model_runner.py.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-25 10:02:23 +00:00
tzhouam 861698f88f Update Dockerfile for vllm wheel version and enhance SHM connector tests 
- Updated Dockerfile to use a new vllm wheel version.
- Modified test_put_get_inline to clarify that it always uses SHM and updated metadata assertions.
- Added handling for speculative configuration in GPU model runners to delay clearing connector metadata.
- Refactored payload decoding in OmniGPUModelRunner for better clarity and maintainability.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-24 05:45:48 +00:00
tzhouam e22ae62396 Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-02-24 03:23:20 +00:00
TJian 5722333fba [ROCM] [CI] fix dockerfile.rocm to support nightly build and also fix amd ci v0.16.0rc1 (#1380) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-02-23 15:00:12 +08:00
tzhouam bf69cd94bb Rebase 2026/02/23 
Update Dockerfile for vllm wheel version and enhance input processing in Omni models. Added decoding for byte messages in async requests, included new metrics in benchmarks, and improved pause state handling in the scheduler. Refined input and output processing methods for better multimodal support and error handling.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-23 03:36:01 +00:00
tzhouam 292da4b0d8 Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-02-23 02:09:09 +00:00
tzhouam 9869ea007e Update vLLM wheel version in Dockerfile and enhance input processing for omni models. Added functionality to reinject omni-specific fields into processed prompts and improved handling of multimodal audio extraction in serving_chat.py. Adjusted SharedMemoryConnector logic for payload size checks and refined input preprocessing methods for better compatibility with multimodal data. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-20 08:34:53 +00:00
Yan MaandDaniel Huang 4f9731c920 [XPU] Add XPU Dockerfile and related docs (#1162) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
Signed-off-by: Daniel Huang <daniel1.huang@intel.com>
Co-authored-by: Daniel Huang <daniel1.huang@intel.com>
 2026-02-20 13:33:48 +08:00
tzhouam b867bacd24 rebase 2025/02/16 
Qwen3 Omni offline/online passed

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-16 03:21:21 +00:00
Zhou Taichang汪志鹏Hongsheng LiuIsotr0pyZJY0516youkaichao
 		75770c9309 [Rebase] Rebase to vllm v0.16.0 (#1357)
ComfyUI Integration -- Publish to Comfy registry / Publish Custom Node to registry (push) Failing after 44s
Details 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: ZJY0516 <zhu.jiangyun@foxmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2026-02-13 18:05:07 +08:00
tzhouam f50cf07c74 Update Dockerfile.ci to allow the installation of 'pplx_kernels' to continue even if the compatible wheel is not found, enhancing build resilience. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 07:22:07 +00:00
tzhouam bc0a2598ed Update Dockerfile.ci to replace 'uv pip' with 'pip' for 'pplx_kernels' installation, ensuring consistency in package management across the CI environment. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 07:11:40 +00:00
tzhouam 963deaf5ad Refactor Dockerfile.ci to streamline package installations by using 'uv pip' consistently for both 'pplx_kernels' and VLLM dependencies, ensuring a more uniform CI environment setup. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 06:55:10 +00:00
tzhouam 3abec0173c Update Dockerfile.ci to force-reinstall the 'pplx_kernels' package, ensuring the latest version is used in the CI environment. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 06:41:41 +00:00
tzhouam 8a4eee9bed Enhance configuration and debugging: Added asyncio mode to pytest settings, increased timeout for Qwen3-TTS E2E test, and improved logging in GPU model runner for better traceability of input modalities. Updated Dockerfile.ci to streamline VLLM installation process. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 06:39:28 +00:00
tzhouam 7e9cae56fc Refactor Dockerfile.ci to use precompiled VLLM installation and remove unused parse.py file. Update serving_chat.py to streamline rendering logic and enhance compatibility with new API. Modify OmniInputPreprocessor to directly handle SingletonDictPrompt, improving input processing efficiency. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 02:28:25 +00:00
Taichang Zhou 85a7b558ab rebase to v0.16.0 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-12 21:30:54 +08:00
Taichang Zhou 3dc19bc305 0.16.0 docker not released yet 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-12 21:25:53 +08:00
Taichang ZhouandCursor e2ced80d59 Merge origin/main into dev/rebase-v0.16.0. 
Resolve arg_utils merge conflict and align CI base image tag to v0.16.0 for the vLLM rebase track.

Co-authored-by: Cursor <cursoragent@cursor.com>

Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-12 21:18:03 +08:00
tzhouam db40563b88 Update Dockerfile.ci to pin nvidia-cublas-cu12 and add numpy installation, ensuring proper dependency management for the build environment. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-12 09:52:51 +00:00
tzhouam 8c1df5c704 update flashinfer 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-12 08:52:06 +00:00
tzhouam c751f0d684 Update Dockerfile.ci to install a new version of vllm from a different extra index URL. Refactor import statement in serving_chat.py to use the correct BaseRenderer module path. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-12 06:37:17 +00:00
Taichang Zhou cd2ffbf2a0 update dockerfile.ci 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-11 23:39:15 +08:00
Taichang Zhou 514bd46f2e Uninstall old nvidia-cublas-cu12 version before reinstalling the pinned version to ensure consistent dependency management in Dockerfile.ci. 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-11 22:34:05 +08:00
Taichang Zhou 169d94dab9 Refactor Dockerfile.ci to use bash for Python executable resolution during vllm-omni installation and pin nvidia-cublas-cu12 version to 12.9.1.4 for consistent dependency management. 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-11 21:31:09 +08:00
Taichang Zhou 268a58e4e8 Update pyproject.toml to remove redundant transformers package installation and adjust Dockerfile.ci to install the current package in editable mode. Also, update fa3-fwd package version in requirements/cuda.txt to 0.0.2 for consistency. 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-11 21:18:53 +08:00
Taichang Zhou eb05967451 Pin nvidia-cublas-cu12 to version 12.9.1.4 in Dockerfile.ci to ensure consistent dependency management in the CI environment. 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-11 20:31:57 +08:00
tzhouam 05784d3dd0 Update pyproject.toml to include transformers package and remove redundant installation step in Dockerfile.ci for cleaner CI setup. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-11 06:27:51 +00:00
tzhouam b7f297c8c3 Add transformers package installation in Dockerfile.ci to ensure compatibility with the CI environment. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-11 06:15:00 +00:00
tzhouam 379f61d038 Enhance Dockerfile.ci by adding a debug step to print key dependency versions for CI troubleshooting, while commenting out the flashinfer version sync logic for clarity. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-11 06:01:01 +00:00
tzhouam 31b6584373 Refine flashinfer version retrieval in Dockerfile.ci to enhance reliability and maintain consistency in package installations 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-11 04:56:50 +00:00
tzhouam 50cea0b29c Refactor flashinfer version retrieval in Dockerfile.ci for improved accuracy and consistency in package installations 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-11 04:34:52 +00:00
tzhouam e897640b43 Update Dockerfile.ci to enhance flashinfer version retrieval logic and ensure package installation consistency 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-11 04:11:11 +00:00
tzhouam 3663e803d1 rebase 2026/02/11 
vllm: 4a1550d22d7058e129d0e1257e726b3bf4a77025

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-11 02:43:49 +00:00
tzhouam 279b6b5050 Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-02-11 01:44:54 +00:00