anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
750 Commits 235 Branches 31 Tags
f24273dc03d8eadedd681c68ee62a49a889b7deb
Commit Graph
750 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
tzhouam f24273dc03 Update vLLM version to 0.16.0 in quickstart and CUDA installation documentation, and adjust parameters in multimodal generation script for improved performance. Refactor request logging in serving_chat.py for better input tracking. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 09:50:11 +00:00
tzhouam f50cf07c74 Update Dockerfile.ci to allow the installation of 'pplx_kernels' to continue even if the compatible wheel is not found, enhancing build resilience. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 07:22:07 +00:00
tzhouam bc0a2598ed Update Dockerfile.ci to replace 'uv pip' with 'pip' for 'pplx_kernels' installation, ensuring consistency in package management across the CI environment. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 07:11:40 +00:00
tzhouam 963deaf5ad Refactor Dockerfile.ci to streamline package installations by using 'uv pip' consistently for both 'pplx_kernels' and VLLM dependencies, ensuring a more uniform CI environment setup. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 06:55:10 +00:00
tzhouam 3abec0173c Update Dockerfile.ci to force-reinstall the 'pplx_kernels' package, ensuring the latest version is used in the CI environment. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 06:41:41 +00:00
tzhouam 8a4eee9bed Enhance configuration and debugging: Added asyncio mode to pytest settings, increased timeout for Qwen3-TTS E2E test, and improved logging in GPU model runner for better traceability of input modalities. Updated Dockerfile.ci to streamline VLLM installation process. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 06:39:28 +00:00
tzhouam 31e9fdbb57 Refactor chat_utils to remove unused multi-modal classes and add async audio extraction from video. Update README to reflect new entry point for audio extraction. Enhance serving_chat to process audio from video URLs when specified. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 03:27:16 +00:00
tzhouam 39a045f439 debug Qwen 3 Omni cuda graph 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 02:57:05 +00:00
tzhouam 7e9cae56fc Refactor Dockerfile.ci to use precompiled VLLM installation and remove unused parse.py file. Update serving_chat.py to streamline rendering logic and enhance compatibility with new API. Modify OmniInputPreprocessor to directly handle SingletonDictPrompt, improving input processing efficiency. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 02:28:25 +00:00
tzhouam 31c5465e00 Update .gitignore to exclude .wav output files and remove audio_0.wav 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-13 01:52:06 +00:00
Hongsheng Liu 622c6f5f14 Merge branch 'main' into dev/rebase-v0.16.0 2026-02-13 06:05:57 +08:00
Taichang Zhou a6d633c319 debug 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-12 23:46:20 +08:00
wangyu f4867ec175 [Test] Add Omni Model Performance Benchmark Test (#1321) 
Signed-off-by: yenuo26 <410167048@qq.com>
Signed-off-by: wangyu <53896905+yenuo26@users.noreply.github.com>
 2026-02-12 23:11:57 +08:00
tzhouam ec3814f587 debug 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-12 14:20:57 +00:00
tzhouam 59a7602d6e debug 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-12 13:55:08 +00:00
liuzhenwei 00bd07b82b [XPU] Update Bagel's flash_attn_varlen_func to fa utils (#1295) 
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
 2026-02-12 21:33:06 +08:00
Taichang Zhou 85a7b558ab rebase to v0.16.0 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-12 21:30:54 +08:00
Taichang Zhou 3dc19bc305 0.16.0 docker not released yet 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-12 21:25:53 +08:00
Taichang Zhou 7458ba44b7 merge main 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-12 21:21:03 +08:00
Taichang ZhouandCursor e2ced80d59 Merge origin/main into dev/rebase-v0.16.0. 
Resolve arg_utils merge conflict and align CI base image tag to v0.16.0 for the vLLM rebase track.

Co-authored-by: Cursor <cursoragent@cursor.com>

Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-12 21:18:03 +08:00
Taichang Zhou 1705edcce9 Refactor model path in test_qwen3_tts for improved clarity and maintainability 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-12 21:12:53 +08:00
rein yangandHongsheng Liu 36a79717c6 change qwen3-omni open cudagraph by default (#1352) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-12 20:49:34 +08:00
dsinghviand汪志鹏 c6f314df4f [Feature]: Diffusion sleep to use process level memory calculation (#1276) 
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-02-12 12:17:54 +00:00
Ding Zuhaoand汪志鹏 b16965dfa7 [Feature][Bugfix] Add CFG feature to Bagel (#1310) 
Signed-off-by: Ding Zuhao <e1583181@u.nus.edu>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-02-12 19:48:30 +08:00
knlnguyen1802 9994594782 [Misc] Support WorkerWrapperBase and CustomPipeline for Diffusion Worker (#764) 
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
 2026-02-12 19:23:51 +08:00
WeiQing Chen e427da022b [Misc] wechat image update (#1354) 
Signed-off-by: David Chen <530634352@qq.com>
 2026-02-12 18:46:56 +08:00
dengyunyang b0a5ffa100 [Bugfix] remove Tongyi-MAI/Z-Image-Turbo related test from L2 ci (#1348) 
Signed-off-by: dengyunyang <584797741@qq.com>
 2026-02-12 18:44:05 +08:00
Aliciaandyenuo26 17e238c7ea [DOC] Doc for CI test - Details about five level stucture and some other files. (#1167) 
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
Co-authored-by: yenuo26 <410167048@qq.com>
 2026-02-12 18:43:51 +08:00
tzhouam f8959bcc6e Temp fix for metrics error in test_qwen3_tts 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-12 10:02:19 +00:00
tzhouam db40563b88 Update Dockerfile.ci to pin nvidia-cublas-cu12 and add numpy installation, ensuring proper dependency management for the build environment. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-12 09:52:51 +00:00
Yuanheng ZhaoandSamit 8bf849c399 Refactor CPU Offloading Backend Pattern (#1223) 
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-02-12 17:29:55 +08:00
John Liu BUAA f117a07271 [Bugfix] reused metrics to modify the API Server token statistics in Stream Response (#1301) 
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com>
 2026-02-12 17:24:01 +08:00
tzhouam 86e20769fd Add SSL context support in benchmark function for improved connection handling 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-12 08:52:27 +00:00
tzhouam 8c1df5c704 update flashinfer 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-12 08:52:06 +00:00
ChenWenjingBhanu068Gao Han
 		ab1acb27ff [Feature]async scheduling to overlap chunk IO and compute (#951) 
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
Co-authored-by: Bhanu068 <voutharoja.bhanu06@gmail.com>
Co-authored-by: Gao Han <gaohan19@huawei.com>
 2026-02-12 16:05:02 +08:00
Sihao Li 71c9e77606 [Feature] add Tensor Parallelism to SD_3.5 (#1336) 
Signed-off-by: GG-li <3226868735@qq.com>
 2026-02-12 07:33:23 +00:00
tzhouam 3e89c99001 Merge remote-tracking branch 'remotes/origin/main' into dev/vllm-align 2026-02-12 07:16:25 +00:00
tzhouam c751f0d684 Update Dockerfile.ci to install a new version of vllm from a different extra index URL. Refactor import statement in serving_chat.py to use the correct BaseRenderer module path. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-12 06:37:17 +00:00
SamitandFlora Feng 9c00852d85 [Feature] Support Wan2.2 T2V and I2V Online Serving with OpenAI /v1/videos API (#1073) 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
Signed-off-by: SamitHuang <285365963@qq.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-02-12 14:28:32 +08:00
rein yang a1aed18b80 update qwen3-omni & qwen2.5-onmi openai client (#1304) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
 2026-02-12 12:10:08 +08:00
Ziming HuangHongsheng LiuCopilot
 		0b56fba7b3 [Pref] Support Qwen3 Omni code2wav batch infernce with async chunk (#1246) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: Ziming Huang <1520787127@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-02-12 11:50:10 +08:00
tzhouamandCursor 50f2b1e854 Merge origin/main into dev/vllm-align 
Co-authored-by: Cursor <cursoragent@cursor.com>

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-12 03:41:38 +00:00
tzhouam 0099eebb3f Refactor OmniEngineArgs and AsyncOmniEngineArgs to improve compatibility with partial payloads by using getattr for various attributes. Update GPUGenerationModelRunner and OmniGPUModelRunner to replace the activate_lora parameter with num_active_loras for better clarity and backward compatibility. This change enhances the handling of LoRA activation during model runs. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-12 03:33:50 +00:00
wangyuwangyu31577Hongsheng Liu
 		20a8fe1dcc [Test] L2 & L3 Test Case Stratification Design for Omni Model (#1272) 
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com>
Signed-off-by: yenuo26 <410167048@qq.com>
Signed-off-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-12 09:56:35 +08:00
Junhong Liu 82fd2c3637 Fix yield token metrics and opt metrics record stats (#1292) 2026-02-12 00:52:05 +08:00
Taichang Zhou cd2ffbf2a0 update dockerfile.ci 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-11 23:39:15 +08:00
Taichang Zhou 9fbffc6137 Update requirements/cuda.txt to clarify the pinning of nvidia-cublas-cu12 version 12.9.1.4 due to dependency resolution issues with torch. This comment aids in understanding the force-reinstall strategy in Dockerfile.ci. 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-11 22:44:11 +08:00
Taichang Zhou 514bd46f2e Uninstall old nvidia-cublas-cu12 version before reinstalling the pinned version to ensure consistent dependency management in Dockerfile.ci. 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-11 22:34:05 +08:00
Taichang Zhou 169d94dab9 Refactor Dockerfile.ci to use bash for Python executable resolution during vllm-omni installation and pin nvidia-cublas-cu12 version to 12.9.1.4 for consistent dependency management. 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-11 21:31:09 +08:00
Taichang Zhou 268a58e4e8 Update pyproject.toml to remove redundant transformers package installation and adjust Dockerfile.ci to install the current package in editable mode. Also, update fa3-fwd package version in requirements/cuda.txt to 0.0.2 for consistency. 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-02-11 21:18:53 +08:00