anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
3,465 Commits 235 Branches 31 Tags
main
Commit Graph
32 Commits
Author SHA1 Message Date
Zhou Taichang 0b0d2d7009 [Rebase] Align vLLM-Omni with vLLM v0.31.0 (#8459) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-10-05 14:11:01 +08:00
Sy03andzack 1ea0ee3b38 [Model] Add opt-in packed Flow and batched HiFT inference for CosyVoice3 (#8224) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: zack <huixindaddy@yahoo.com>
Co-authored-by: zack <huixindaddy@yahoo.com>
 2026-09-29 15:48:36 +00:00
Zhou TaichangClaude Opus 5Gao Han
 		59db6a4285 [Rebase] Rebase to vLLM 0.30.0 (#7820) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-09-22 15:52:01 +00:00
Zhou TaichangandClaude Opus 5 2ce15ab7a0 [Rebase] Rebase to vllm 0.27.0 (#5976) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-11 15:23:46 +00:00
Zhou TaichangandClaude Opus 4.8 d3c47efc4f [Rebase] Rebase to vllm v0.25.0 (#5042) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-12 23:39:53 +08:00
Zhou Taichang a560ed184d [Rebase] Rebase to vllm v0.24.0 (#4709) 2026-06-30 19:41:06 +08:00
vraitiHongsheng LiuClaude Sonnet 4.5
 		77d395150b Remove unregister_vllm_metrics no-op patch (#4396) 
Signed-off-by: vraiti <vraiti@redhat.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
 2026-06-18 10:13:49 +08:00
Yihong Jin 2f64c712fd [Quantization][Qwen3-Omni] Enable NVFP4 W4A4 serving on Blackwell (#4025) 
Signed-off-by: Yihong Jin <yihongjin404@gmail.com>
 2026-06-18 10:01:46 +08:00
SYLAR b5352a5869 [Quant][Perf] Default Blackwell FP8 GEMM to quack CuteDSL fused-bias kernel (#4241) 
Signed-off-by: WeiQing Chen <david6666666@users.noreply.github.com>
 2026-06-08 19:44:20 +00:00
SYLAR 29b18ea298 [Quant] ModelOpt FP8 for Wan2.2 & HunyuanVideo-1.5 video-gen (#3305) 
Signed-off-by: WeiQing Chen <david6666666@users.noreply.github.com>
 2026-06-08 02:53:06 +08:00
WeiQing Chenandlishunyang12 6e8ef213d8 [BugFix] Support ModelOpt FP8 under batched diffusion serving (#4155) 
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: lishunyang12 <lishunyang12@163.com>
 2026-06-05 06:25:01 +08:00
LHXuuuandvraiti 41a795b5d6 [Metrics] Add audio SLOs + cross-stage transfer families + per-(stage, replica) wrap for upstream vllm:* (#3576) 
Signed-off-by: vraiti <vraiti@redhat.com>
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com>
Co-authored-by: vraiti <vraiti@redhat.com>
 2026-06-01 20:20:48 +08:00
+1
Zhou TaichangClaude Opus 4.7skfZheng WengangHongsheng LiuWeiQing ChenSYLARYueqian Linnainiu258JasonYueqian Lin
 		f92d84fcf9 [Rebase] Rebase to vllm releases/v0.22.0 (#3891) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: ZhengWG <zwg0606@gmail.com>
Signed-off-by: david6666666 <530634352@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Signed-off-by: nainiu258 <cperfect02@163.com>
Signed-off-by: nainiu258 <101917677+nainiu258@users.noreply.github.com>
Signed-off-by: JasonJ2021 <jasonj@zju.edu.cn>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: skf <54565339+skf-1999@users.noreply.github.com>
Co-authored-by: Zheng Wengang <zwg0606@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: nainiu258 <101917677+nainiu258@users.noreply.github.com>
Co-authored-by: Jason <72191212+JasonJ2021@users.noreply.github.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-06-01 11:08:12 +08:00
NATUREandHongsheng Liu 82a0b3a467 [2/5] [core]refactor communication layer: PR 2 of 5 Qwen3 Omni non async (#2677) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-05-15 14:14:08 +08:00
Zhou TaichangandYuanheng Zhao 7f2b1e9386 [Rebase] Rebase to vllm 0.20.0 (#3232) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Co-authored-by: Yuanheng Zhao <54058983+yuanheng-zhao@users.noreply.github.com>
 2026-04-30 09:56:59 +12:00
ChenWenjing 4124a1f858 [Feature]Support audio streaming input and output-phase2 (#2581) 
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
 2026-04-18 15:42:41 +08:00
TaffyOfficialTaffyOfficialClaude Opus 4.6
 		c3ca5daafb Feat/Add HunyuanImage-3.0-Instruct ar part support: (#2713) 
Signed-off-by: TaffyOfficial <2324465096@qq.com>
Co-authored-by: TaffyOfficial <2324465096@qq.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-16 23:15:25 +08:00
Zhou Taichang 2804a85e58 [Rebase] Rebase to vllm v0.19.0 (#2475) 2026-04-04 15:03:32 +08:00
汪志鹏 1bd1d9d618 Remove mm_prefix_lm patch because vllm==0.18.0 already support (#2062) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-03-21 22:32:00 +08:00
汪志鹏 7271116973 [Bagel]: Support multistage img2img (#1669) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-03-09 22:23:50 +08:00
Jared Wen汪志鹏Hongsheng Liu
 		4146095623 [Perf] GLM Image (#920) 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: Jared Wen <w13431838023@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-27 10:29:14 +08:00
ChenWenjingBhanu068Gao Han
 		ab1acb27ff [Feature]async scheduling to overlap chunk IO and compute (#951) 
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
Co-authored-by: Bhanu068 <voutharoja.bhanu06@gmail.com>
Co-authored-by: Gao Han <gaohan19@huawei.com>
 2026-02-12 16:05:02 +08:00
Zhou Taichangand汪志鹏 70ef548448 Dev/rebase v0.15.0 (#1159) 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-02-03 12:24:45 +08:00
Canlin Guo cadd77262d [NPU] Upgrade to v0.14.0 (#820) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-01-27 16:45:32 +08:00
Canlin GuoandHongsheng Liu fe3d9a6a81 [NPU][Upgrade] Upgrade to v0.12.0 (#447) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-26 14:10:23 +08:00
dengyunyang bd5c81cb7e [BugFix] Failed to print DEBUG level log (#307) 
Signed-off-by: dengyunyang <584797741@qq.com>
 2025-12-15 07:13:36 +00:00
Mengqing Cao 00b1a9a1a0 [Misc] Temporarily support downloading models from ModelScope by snapshot download (#132) 
Signed-off-by: MengqingCao <cmq0113@163.com>
 2025-12-01 17:13:23 +00:00
R2-YandRein 0a4e5a8710 [Feature] add support for Qwen3-omni (#55) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Rein <rein@Reins-MacBook-Air.local>
 2025-11-30 04:44:01 +08:00
Jiangyun Zhu 5707fc78d5 [CI] fix ci and format existing code (#71) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-11-18 20:54:26 +08:00
Gaohan123 6e1478222e Dependency refractored to vLLM v0.11.0 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-04 22:06:41 +08:00
Gaohan123 171f3c1ea7 fix conflicts 2025-10-24 16:49:19 +08:00
tzhouam 83e7f6ba12 Add Omni model components and input processing for hidden states support 
- Introduced `OmniModelRunnerOutput` and `OmniRequestOutput` classes for handling multimodal outputs.
- Implemented `OmniEngineCoreRequest` and `OmniInputPreprocessor` to manage prompt embeddings and additional information.
- Created `OmniTokensPrompt`, `OmniTokenInputs`, and `OmniEmbedsPrompt` for enhanced token handling.
- Added parsing and preprocessing functions to support new input structures.
- Removed the obsolete `diffusion_engine.py` file as part of the refactor.
 2025-10-22 15:16:25 +08:00