anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
808 Commits 235 Branches 31 Tags
8d0c46487ecb9a49d0c5e249a91f81166fc045ef
Commit Graph
808 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
tzhouam 8d0c46487e Merge origin/main into dev/vllm-align 
Resolve conflicts:
- docker/Dockerfile.ci: Keep wheel install section, update hash to 6042e66cd5
- vllm_omni/platforms/npu/worker/npu_model_runner.py: Accept main's refactoring

Made-with: Cursor

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-26 08:19:53 +00:00
junuxyzandHongsheng Liu caed95fb45 [Bugfix] Fix tuple/list KV cache extraction crash (#1405) 
Signed-off-by: junuxyz <216036880+junuxyz@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-26 15:20:40 +08:00
Yuanheng Zhao 904b97d5bd [Fix][Chore] Qwen3-TTS Modeling Minor Code Sanity Improvements (#1482) 
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
 2026-02-26 14:39:08 +08:00
Zhou Taichang 986b356077 [CI] Update Dockerfile for vllm-omni CI image and remove obsolete dep… (#1491) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-26 14:36:03 +08:00
Canlin Guo 22c8cde328 [NPU] Upgrade to v0.16.0 (#1375) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-02-26 14:26:50 +08:00
wangyu 1ae2e1e4c4 [Test] Reduce Perf test case and fix modify stage config (#1449) 
Signed-off-by: yenuo26 <410167048@qq.com>
 2026-02-26 14:20:57 +08:00
亦瑾 5e7974c15b [Bugfix] Fix SDPA attention mask dtype and shape (Fix #857) (#1349) 
Signed-off-by: jader <yjader@foxmail.com>
 2026-02-26 11:54:28 +08:00
Junhong LiuandGao Han 919ad28da5 [BugFix] process request.num_cached_tokens if it equals to the initial value (#1468) 
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Co-authored-by: Gao Han <gaohan19@huawei.com>
 2026-02-26 10:59:18 +08:00
Sangchun Ha 315daeba8c [Bugfix] Fix case-sensitive task_type matching in Qwen3TTSModelForGeneration (#1455) 
Signed-off-by: Sangchun Ha <seomk9896@gmail.com>
 2026-02-26 10:27:13 +08:00
Zhou Taichang 8e950daf6a [Debug] Multi-Request for Qwen 3 Omni use_audio_in_video (#1433) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-26 09:26:25 +08:00
Yanick Schraner 97f4c5b1e7 [Bugfix] Add TTS configuration options (#1177) 
Signed-off-by: Yanick Schraner <yanick.schraner@bs.ch>
 2026-02-26 00:14:44 +08:00
Dong WandHongsheng Liu 1eebb374f1 [Model]Add new nextstep_1(Diffusion) model(only T2I) (#612) 
Signed-off-by: Dong Wang <dongw2019@gmail.com>
Signed-off-by: sniper35 <dongw2019@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-26 00:09:11 +08:00
SHIJIN ZHANG 466a7f7a62 [Bugfix]: Fix merging updated additional information to ensure dict type (#1296) 
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com>
 2026-02-26 00:06:49 +08:00
Junhong Liu 43939df2a5 [Bugfix] Robust Audio Data Handling in _create_audio_choice (#1222) 
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
 2026-02-25 22:59:49 +08:00
tzhouam a252918008 [Update] Upgrade vLLM wheel version and enhance metrics handling 
- Updated the vLLM wheel to commit 24650715105a26b41c88f4164777b5e1a0f3200b in Dockerfile.ci.
- Modified the `calculate_metrics` function to handle `tokenizer` as optional (can be None) in metrics.py.
- Adjusted the `benchmark` function to accept `tokenizer` as optional in patch.py.
- Enhanced condition checks in OmniARScheduler to ensure valid token generation in omni_ar_scheduler.py.
- Updated input processing to include supported tasks in input_processor.py.
- Cleaned up unnecessary code in output_processor.py.
- Added support for SSL ciphers in the API server configuration in api_server.py.
- Enabled speculative decoding in GPU model runners for improved performance in gpu_generation_model_runner.py and gpu_model_runner.py.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-25 10:02:23 +00:00
tzhouam 22e01c416b Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-02-25 08:15:55 +00:00
Didan Deng 1e86404456 [Doc] Refine Diffusion Tutorial Documents (#1305) 
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-02-25 15:43:08 +08:00
WeiQing Chen bf9754a8b1 [Misc] update wechat image (#1464) 
Signed-off-by: David Chen <530634352@qq.com>
 2026-02-25 12:06:27 +08:00
Zhou Taichang 296d995ec2 [CI] Remove overwhelming debug log (#1463) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-25 12:06:01 +08:00
Juan Pablo Zuluagaandpablo 4de077ea62 [Qwen3TTS][Feat] Code2Wav batched decoding (#1426) 
Signed-off-by: pablo <pablo@agigo.ai>
Co-authored-by: pablo <pablo@agigo.ai>
 2026-02-25 11:16:46 +13:00
汪志鹏 82e1bf2804 [Doc]: Add bagel single/multi node usage with mooncake document (#1450) 2026-02-24 15:54:57 +00:00
Junhong LiuandHongsheng Liu c0908dd3fa [Bugfix] Forward all vllm-omni serve command parameters to model (#985) 
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Signed-off-by: Junhong Liu <ljh_lbj@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-24 22:20:01 +08:00
Junhong LiuandHongsheng Liu a8cec43ce6 [Doc] Optimize vLLM-Omni metrics documentation (#1311) 
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Signed-off-by: Junhong Liu <ljh_lbj@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-24 20:27:29 +08:00
wuhang汪志鹏Copilot
 		36b8f80266 [Feature] Support Stage Based Deployment CLI (#939) 
Signed-off-by: wuhang <wuhang6@huawei.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: wuhang <whlbx@hotmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-02-24 16:55:23 +08:00
Nicolò LucchesiandHongsheng Liu 06d347b47f [Misc] Extend Diffusion Benchmark script to other backends (#875) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-24 16:44:33 +08:00
tzhouam 81b9e40bbf Enhance signature compatibility checks in OmniLLM tests 
- Improved the test_overridden_method_signatures_compatible function to better handle missing parameters in method signatures.
- Added logic to account for VAR_KEYWORD parameters, ensuring more accurate detection of signature mismatches.

Signed-off-by: [Your Name] <your.email@example.com>

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-24 07:56:47 +00:00
tzhouam 04ccb062fc Add signature compatibility tests for OmniLLM class 
- Introduced tests to ensure method signatures in OmniLLM match those of the base LLM class, preventing runtime TypeErrors due to parameter mismatches.
- Updated the _run_engine method in OmniLLM to include an optional output_type parameter.

Signed-off-by: [Your Name] <your.email@example.com>

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-24 06:58:39 +00:00
Canlin GuoandHongsheng Liu af11b02081 [Minor][Refactor] Pass seq_token_counts explicitly (#1425) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-24 14:10:00 +08:00
tzhouam 861698f88f Update Dockerfile for vllm wheel version and enhance SHM connector tests 
- Updated Dockerfile to use a new vllm wheel version.
- Modified test_put_get_inline to clarify that it always uses SHM and updated metadata assertions.
- Added handling for speculative configuration in GPU model runners to delay clearing connector metadata.
- Refactored payload decoding in OmniGPUModelRunner for better clarity and maintainability.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-24 05:45:48 +00:00
tzhouam e22ae62396 Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-02-24 03:23:20 +00:00
tzhouam 9070f15f52 debug qwen tts 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-24 03:17:11 +00:00
NATURE 1589931fa6 [Feature] vLLM-Omni RDMA connector (#1019) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
 2026-02-24 11:13:52 +08:00
Alicia 7b02b856a6 [CI] Set L2 & L3 tests running conditions. (#1344) 
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-02-24 10:50:10 +08:00
Ding ZuhaoandWang Zhipeng: princepride d6a3551a62 [Bug] [Bagel] Fix kv transfer bug (#1437) 
Signed-off-by: Ding Zuhao <e1583181@u.nus.edu>
Co-authored-by: Wang Zhipeng: princepride <wangzhipeng628@gmail.com>
 2026-02-23 15:58:22 +00:00
zhou zhuoxin 40f72bb870 [Feat]: add ASCII art logo for vLLM-Omni (#1430) 2026-02-23 22:10:09 +08:00
Ding Zuhaoand汪志鹏 0ff443f069 [Feature][BAGEL] Combine multi-branch cfg into a single batch to accelerate inference. (#1429) 
Signed-off-by: Ding Zuhao <e1583181@u.nus.edu>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-02-23 17:41:47 +08:00
TJian 5722333fba [ROCM] [CI] fix dockerfile.rocm to support nightly build and also fix amd ci v0.16.0rc1 (#1380) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-02-23 15:00:12 +08:00
Zhou Taichang b42287d14b [Doc] Update GPU installation commands (#1434) 2026-02-23 13:54:57 +08:00
tzhouam bf69cd94bb Rebase 2026/02/23 
Update Dockerfile for vllm wheel version and enhance input processing in Omni models. Added decoding for byte messages in async requests, included new metrics in benchmarks, and improved pause state handling in the scheduler. Refined input and output processing methods for better multimodal support and error handling.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-23 03:36:01 +00:00
tzhouam 292da4b0d8 Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-02-23 02:09:09 +00:00
Hongsheng Liu a3f2d4cbfe Revert "[misc] Feature/pr reviewer auto trigger&update model" (#1432) 2026-02-23 07:52:09 +08:00
Hongsheng LiuandClaude Opus 4.6 267170115b [misc] Feature/pr reviewer auto trigger&update model (#1431) 
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
Signed-off-by: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Hunter Liu <hunter@liu.sh>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-22 23:33:45 +08:00
UsamaKenway d8a32a6a87 [Misc] Add Qwen2.5-Omni-3B model support to Gradio demo (#1382) 
Signed-off-by: UsamaKenway <usamakenway@gmail.com>
 2026-02-22 21:17:09 +08:00
Hongsheng LiuandClaude Opus 4.6 6c85ddcb96 [Feature] Add automated PR reviewer bot with GLM integration (#1424) 
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
Signed-off-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-22 13:53:59 +08:00
Sy03andHongsheng Liu efbe4116ff [Feature] : Support disaggregated inference pipeline for Qwen3_TTS (#1161) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-21 07:36:44 +08:00
tzhouam 9869ea007e Update vLLM wheel version in Dockerfile and enhance input processing for omni models. Added functionality to reinject omni-specific fields into processed prompts and improved handling of multimodal audio extraction in serving_chat.py. Adjusted SharedMemoryConnector logic for payload size checks and refined input preprocessing methods for better compatibility with multimodal data. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-20 08:34:53 +00:00
Alex Brooks 5295f0f81b [Bugfix] Fix Hardcoded Datatypes in Z-image (#1393) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-02-20 06:55:24 +00:00
Yan MaandDaniel Huang 4f9731c920 [XPU] Add XPU Dockerfile and related docs (#1162) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
Signed-off-by: Daniel Huang <daniel1.huang@intel.com>
Co-authored-by: Daniel Huang <daniel1.huang@intel.com>
 2026-02-20 13:33:48 +08:00
Chendi.Xue d2f015edce [0.16.0] remove cuda hard-code for Hunyuan Image3 (#1402) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
 2026-02-20 13:23:31 +08:00
tzhouam 663976b8ca Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-02-20 01:38:44 +00:00