anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
793 Commits 235 Branches 31 Tags
22e01c416bb235b2a0123da2858f8cfd384ebeca
Commit Graph
793 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
tzhouam 22e01c416b Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-02-25 08:15:55 +00:00
Didan Deng 1e86404456 [Doc] Refine Diffusion Tutorial Documents (#1305) 
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-02-25 15:43:08 +08:00
WeiQing Chen bf9754a8b1 [Misc] update wechat image (#1464) 
Signed-off-by: David Chen <530634352@qq.com>
 2026-02-25 12:06:27 +08:00
Zhou Taichang 296d995ec2 [CI] Remove overwhelming debug log (#1463) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-25 12:06:01 +08:00
Juan Pablo Zuluagaandpablo 4de077ea62 [Qwen3TTS][Feat] Code2Wav batched decoding (#1426) 
Signed-off-by: pablo <pablo@agigo.ai>
Co-authored-by: pablo <pablo@agigo.ai>
 2026-02-25 11:16:46 +13:00
汪志鹏 82e1bf2804 [Doc]: Add bagel single/multi node usage with mooncake document (#1450) 2026-02-24 15:54:57 +00:00
Junhong LiuandHongsheng Liu c0908dd3fa [Bugfix] Forward all vllm-omni serve command parameters to model (#985) 
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Signed-off-by: Junhong Liu <ljh_lbj@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-24 22:20:01 +08:00
Junhong LiuandHongsheng Liu a8cec43ce6 [Doc] Optimize vLLM-Omni metrics documentation (#1311) 
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Signed-off-by: Junhong Liu <ljh_lbj@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-24 20:27:29 +08:00
wuhang汪志鹏Copilot
 		36b8f80266 [Feature] Support Stage Based Deployment CLI (#939) 
Signed-off-by: wuhang <wuhang6@huawei.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: wuhang <whlbx@hotmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-02-24 16:55:23 +08:00
Nicolò LucchesiandHongsheng Liu 06d347b47f [Misc] Extend Diffusion Benchmark script to other backends (#875) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-24 16:44:33 +08:00
tzhouam 81b9e40bbf Enhance signature compatibility checks in OmniLLM tests 
- Improved the test_overridden_method_signatures_compatible function to better handle missing parameters in method signatures.
- Added logic to account for VAR_KEYWORD parameters, ensuring more accurate detection of signature mismatches.

Signed-off-by: [Your Name] <your.email@example.com>

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-24 07:56:47 +00:00
tzhouam 04ccb062fc Add signature compatibility tests for OmniLLM class 
- Introduced tests to ensure method signatures in OmniLLM match those of the base LLM class, preventing runtime TypeErrors due to parameter mismatches.
- Updated the _run_engine method in OmniLLM to include an optional output_type parameter.

Signed-off-by: [Your Name] <your.email@example.com>

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-24 06:58:39 +00:00
Canlin GuoandHongsheng Liu af11b02081 [Minor][Refactor] Pass seq_token_counts explicitly (#1425) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-24 14:10:00 +08:00
tzhouam 861698f88f Update Dockerfile for vllm wheel version and enhance SHM connector tests 
- Updated Dockerfile to use a new vllm wheel version.
- Modified test_put_get_inline to clarify that it always uses SHM and updated metadata assertions.
- Added handling for speculative configuration in GPU model runners to delay clearing connector metadata.
- Refactored payload decoding in OmniGPUModelRunner for better clarity and maintainability.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-24 05:45:48 +00:00
tzhouam e22ae62396 Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-02-24 03:23:20 +00:00
tzhouam 9070f15f52 debug qwen tts 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-24 03:17:11 +00:00
NATURE 1589931fa6 [Feature] vLLM-Omni RDMA connector (#1019) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
 2026-02-24 11:13:52 +08:00
Alicia 7b02b856a6 [CI] Set L2 & L3 tests running conditions. (#1344) 
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-02-24 10:50:10 +08:00
Ding ZuhaoandWang Zhipeng: princepride d6a3551a62 [Bug] [Bagel] Fix kv transfer bug (#1437) 
Signed-off-by: Ding Zuhao <e1583181@u.nus.edu>
Co-authored-by: Wang Zhipeng: princepride <wangzhipeng628@gmail.com>
 2026-02-23 15:58:22 +00:00
zhou zhuoxin 40f72bb870 [Feat]: add ASCII art logo for vLLM-Omni (#1430) 2026-02-23 22:10:09 +08:00
Ding Zuhaoand汪志鹏 0ff443f069 [Feature][BAGEL] Combine multi-branch cfg into a single batch to accelerate inference. (#1429) 
Signed-off-by: Ding Zuhao <e1583181@u.nus.edu>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-02-23 17:41:47 +08:00
TJian 5722333fba [ROCM] [CI] fix dockerfile.rocm to support nightly build and also fix amd ci v0.16.0rc1 (#1380) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-02-23 15:00:12 +08:00
Zhou Taichang b42287d14b [Doc] Update GPU installation commands (#1434) 2026-02-23 13:54:57 +08:00
tzhouam bf69cd94bb Rebase 2026/02/23 
Update Dockerfile for vllm wheel version and enhance input processing in Omni models. Added decoding for byte messages in async requests, included new metrics in benchmarks, and improved pause state handling in the scheduler. Refined input and output processing methods for better multimodal support and error handling.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-23 03:36:01 +00:00
tzhouam 292da4b0d8 Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-02-23 02:09:09 +00:00
Hongsheng Liu a3f2d4cbfe Revert "[misc] Feature/pr reviewer auto trigger&update model" (#1432) 2026-02-23 07:52:09 +08:00
Hongsheng LiuandClaude Opus 4.6 267170115b [misc] Feature/pr reviewer auto trigger&update model (#1431) 
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
Signed-off-by: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Hunter Liu <hunter@liu.sh>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-22 23:33:45 +08:00
UsamaKenway d8a32a6a87 [Misc] Add Qwen2.5-Omni-3B model support to Gradio demo (#1382) 
Signed-off-by: UsamaKenway <usamakenway@gmail.com>
 2026-02-22 21:17:09 +08:00
Hongsheng LiuandClaude Opus 4.6 6c85ddcb96 [Feature] Add automated PR reviewer bot with GLM integration (#1424) 
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
Signed-off-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-22 13:53:59 +08:00
Sy03andHongsheng Liu efbe4116ff [Feature] : Support disaggregated inference pipeline for Qwen3_TTS (#1161) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-21 07:36:44 +08:00
tzhouam 9869ea007e Update vLLM wheel version in Dockerfile and enhance input processing for omni models. Added functionality to reinject omni-specific fields into processed prompts and improved handling of multimodal audio extraction in serving_chat.py. Adjusted SharedMemoryConnector logic for payload size checks and refined input preprocessing methods for better compatibility with multimodal data. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-20 08:34:53 +00:00
Alex Brooks 5295f0f81b [Bugfix] Fix Hardcoded Datatypes in Z-image (#1393) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-02-20 06:55:24 +00:00
Yan MaandDaniel Huang 4f9731c920 [XPU] Add XPU Dockerfile and related docs (#1162) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
Signed-off-by: Daniel Huang <daniel1.huang@intel.com>
Co-authored-by: Daniel Huang <daniel1.huang@intel.com>
 2026-02-20 13:33:48 +08:00
Chendi.Xue d2f015edce [0.16.0] remove cuda hard-code for Hunyuan Image3 (#1402) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
 2026-02-20 13:23:31 +08:00
tzhouam 663976b8ca Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-02-20 01:38:44 +00:00
Chendi.Xue 0b577a7263 [v0.16.0][BUG FIX]Fix hunyuan MOE after update to 0.16.0 (#1401) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
 2026-02-19 23:57:11 +13:00
tzhouam b867bacd24 rebase 2025/02/16 
Qwen3 Omni offline/online passed

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-16 03:21:21 +00:00
tzhouam 93eeb1a4c8 Merge remote-tracking branch 'origin/main' into dev/rebase-v0.16.0 2026-02-15 01:24:23 +00:00
wangyu a42b7487b0 [Test] Fix expansion and example test case for qwen3-omni (#1358) 
Signed-off-by: yenuo26 <410167048@qq.com>
 2026-02-15 00:43:38 +13:00
Zhou Taichang汪志鹏Hongsheng LiuIsotr0pyZJY0516youkaichao
 		75770c9309 [Rebase] Rebase to vllm v0.16.0 (#1357)
ComfyUI Integration -- Publish to Comfy registry / Publish Custom Node to registry (push) Failing after 44s
Details 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: ZJY0516 <zhu.jiangyun@foxmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
v0.16.0rc1 		2026-02-13 18:05:07 +08:00
tzhouam 3051fb10b5 reverse change in SamplingParams 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 09:59:03 +00:00
tzhouam 15099119dd Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0.16.0 2026-02-13 09:55:18 +00:00
汪志鹏 630e84ef93 [BugFix]: Revert utils change (#1369) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-02-13 17:54:09 +08:00
tzhouam f24273dc03 Update vLLM version to 0.16.0 in quickstart and CUDA installation documentation, and adjust parameters in multimodal generation script for improved performance. Refactor request logging in serving_chat.py for better input tracking. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 09:50:11 +00:00
tzhouam f50cf07c74 Update Dockerfile.ci to allow the installation of 'pplx_kernels' to continue even if the compatible wheel is not found, enhancing build resilience. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 07:22:07 +00:00
tzhouam bc0a2598ed Update Dockerfile.ci to replace 'uv pip' with 'pip' for 'pplx_kernels' installation, ensuring consistency in package management across the CI environment. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 07:11:40 +00:00
tzhouam 963deaf5ad Refactor Dockerfile.ci to streamline package installations by using 'uv pip' consistently for both 'pplx_kernels' and VLLM dependencies, ensuring a more uniform CI environment setup. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 06:55:10 +00:00
tzhouam 3abec0173c Update Dockerfile.ci to force-reinstall the 'pplx_kernels' package, ensuring the latest version is used in the CI environment. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 06:41:41 +00:00
tzhouam 8a4eee9bed Enhance configuration and debugging: Added asyncio mode to pytest settings, increased timeout for Qwen3-TTS E2E test, and improved logging in GPU model runner for better traceability of input modalities. Updated Dockerfile.ci to streamline VLLM installation process. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 06:39:28 +00:00
tzhouam 31e9fdbb57 Refactor chat_utils to remove unused multi-modal classes and add async audio extraction from video. Update README to reflect new entry point for audio extraction. Enhance serving_chat to process audio from video URLs when specified. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 03:27:16 +00:00