anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
873 Commits 235 Branches 31 Tags
3e539fcb6a017fa9d8ac86f4b8d6eb852ff0dde4
Commit Graph
873 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
tzhouam 3e539fcb6a Merge remote-tracking branch 'origin/main' into dev/vllm-align 
Made-with: Cursor

# Conflicts:
#	vllm_omni/entrypoints/openai/serving_chat.py
#	vllm_omni/worker/gpu_model_runner.py

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 02:57:27 +00:00
tzhouam e0497550c4 Update Dockerfile.ci to install a specific version of flashinfer-cubin (0.6.4) directly, simplifying the installation process and ensuring compatibility with the current setup. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 02:51:35 +00:00
tzhouam 029bb56e3a Update Dockerfile.ci to use importlib.metadata for retrieving the current version of flashinfer, enhancing compatibility with the latest Python standards. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 02:28:24 +00:00
tzhouam 65169baa9d Update Dockerfile.ci to create a symbolic link for Python 3, ensuring compatibility with Python scripts during the build process. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 01:58:35 +00:00
Alex BrooksandSYLAR a09b6499f8 [Bugfix][Model] Fix LongCat Image Config Handling / Layer Creation (#1485) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-03-03 00:50:08 +08:00
Samit 22a51a7cdb [Feature][Wan2.2] Speed up diffusion model startup by multi-thread weight loading (#1504) 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
 2026-03-02 14:53:25 +00:00
rongfu.leng a4b2e48718 [Bugfix] Import InputPreprocessor into Renderer (#1566) 
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
 2026-03-02 21:54:15 +08:00
Baoyuan Qi f371f7bf05 [bugfix] Fix unexpected argument 'is_finished' in function llm2code2wav_async_chunk of mimo-audio (#1570) 
Signed-off-by: Baoyuan Qi <qibaoyuan@126.com>
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2026-03-02 21:49:06 +08:00
rein yang f082f6eedb [Bugfix] fix kernel error for qwen3-omni (#1602) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
 2026-03-02 18:12:30 +08:00
Ting FU 1ca198eaff [BugFix] Fix load_weights error when loading HunyuanImage3.0 (#1598) 
Signed-off-by: Semmer2 <semmer@live.cn>
 2026-03-02 08:40:48 +00:00
tzhouam 908e3fe761 Update Dockerfile.ci to dynamically install the current version of flashinfer-cubin, ensuring compatibility with the latest changes in the flashinfer package. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-02 07:00:59 +00:00
tzhouam 8fb247a481 Update Dockerfile.ci to install specific version of flashinfer-cubin (0.6.4) and ensure vllm installation remains intact. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-02 06:48:58 +00:00
tzhouam be4226bae0 Remove unnecessary uninstallation of nvidia-cublas-cu12 and streamline vllm installation in Dockerfile.ci 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-02 06:38:24 +00:00
汪志鹏andCursor e37a89ffcb [Feature]: Support cfg kv-cache transfer in multi-stage (#1422) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-03-02 05:06:28 +00:00
tzhouam 473d3ecd29 Update vLLM wheel version and adjust GPU memory utilization in CI configuration 
- Updated the vLLM wheel commit hash in Dockerfile.ci to a60985b07eaf0fefa4c353ad2309b745707591ed.
- Modified the mock shared memory handle name in test_basic_connectors.py for clarity.
- Increased GPU memory utilization to 0.9 and adjusted max_num_batched_tokens to 4096 in qwen2_5_omni_ci.yaml to enable testing on H800.
- Refactored GPU model runner classes to remove deprecated activate_lora parameter and ensure compatibility with num_active_loras.
- Enhanced request handling in the OmniGPUModelRunner class to improve streaming request updates.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-02 03:08:45 +00:00
dsinghviandZhou Taichang 68764cc4f4 [Refactor]: Phase1 for rebasing_additional_info (#1394) 
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2026-03-02 10:19:41 +08:00
tzhouam 26744bcb35 Merge origin/main into dev/vllm-align 
Resolved conflicts:
- vllm_omni/inputs/preprocess.py: took origin/main's elif mm_processor_kwargs branch, fixed undefined mm_uuids reference
- vllm_omni/model_executor/models/qwen3_tts/qwen3_tts_code_predictor_vllm.py: took origin/main's NPU support

Made-with: Cursor

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-02 02:10:32 +00:00
Roger Wang fec0182ac2 0.16.0 release (#1576) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2026-02-28 21:28:27 -08:00
汪志鹏 3d9fa8d53f [BugFix]: fix a lot of bug (#1565) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
v0.16.0 		2026-02-28 16:26:56 +08:00
wangyu cd2234a1bf [CI] Modify some CI test cases to run on L4 environment to reduce H100 resource usage. (#1543) 
Signed-off-by: yenuo26 <410167048@qq.com>
Signed-off-by: wangyu <53896905+yenuo26@users.noreply.github.com>
 2026-02-28 14:41:51 +08:00
Ting FU b0156d8e1c [BugFix] Fix unexpected crash when init OmniDiffusion (#1562) 
Signed-off-by: Semmer2 <semmer@live.cn>
 2026-02-28 14:21:49 +08:00
Canlin Guo c81266703a [NPU][Bugfix] Align GPU side and recover qwen3-tts (#1564) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-02-28 12:46:13 +08:00
Yueqian Lin 37775ffe60 fix(qwen3-tts): fix Base ICL voice clone producing corrupted audio (#1554) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-02-28 11:49:47 +08:00
rein yang bfcfee7867 [Docs] update async chunk docs diagram [skip ci] (#1530) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
 2026-02-28 10:42:50 +08:00
Zhou Taichang fe0c02ee78 [Debug] Merge vllm pull 35368 (#1534) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-28 10:39:25 +08:00
Junhong Liu 42d86af1f1 Fix no embed text spk tokens (#1540) 
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
 2026-02-28 09:33:58 +08:00
Alicia 11f59f600d [CI][skip ci]Update H100 image link based on #1518 (#1538) 
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-02-27 18:25:18 +08:00
AliciaandYueqian Lin 64f5ef615c [Doc] [skip ci]Sync. (#1363) 
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-02-27 17:28:35 +08:00
Zhou Taichang bc13f7cf38 [Doc] Update installation instructions for vllm 0.16.0 (#1505) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-27 16:06:30 +08:00
Isotr0py 1f862ce04f [Chore] Cleanup dead code in GGUF DiT code path (#1533) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-02-27 07:17:27 +00:00
Chendi.Xueandgcanlin 482f9b89a1 [XPU][NPU][ROCM] enable cpu_offloading flag for non_cuda (#1488) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Co-authored-by: gcanlin <canlinguosdu@gmail.com>
 2026-02-27 14:52:15 +08:00
SHIJIN ZHANGand齐保元 ba9594724e [Bugfix] Fix redundant finished req status updating on OmniGenerationScheduler (#1510) 
Signed-off-by: shijin zhang <75300765+Dovis01@users.noreply.github.com>
Co-authored-by: 齐保元 <qibaoyuan@xiaomi.com>
 2026-02-27 14:42:38 +08:00
Canlin GuoHongsheng LiuNicolò Lucchesi
 		eaef3a31a4 [Profiler] Support online profiling (#1136) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: Canlin Guo <961750412@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>
 2026-02-27 14:41:20 +08:00
Sy03 0cc3481acf [Feat] : Support Async chunk cleanup (#1087) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-02-27 14:39:33 +08:00
TJian 7ac8d613f6 [Version] Auto generate version using setuptool_scm (#1224) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-02-27 14:34:27 +08:00
Ekagra RanjanandHongsheng Liu aa54569033 Add benchmark for v1/audio/speech non-streaming (#1408) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-27 14:31:52 +08:00
WeiQing ChenIsotr0pyIsotr0pyHongsheng Liu
 		0b884ebc6c [Feature]: Native GGUF Quantization Support for DiT (#1285) 
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-27 11:39:14 +08:00
+13
Baoyuan Qiwangyuwangyu31577Zhang ShijinHongsheng LiuSihao LiXU MingshiCanlin GuoIsotr0pyJohnJanWeiQing Chendongbo910220Jiangyun ZhuJunhong LiuTJianshijin zhangZhou TaichangrootPrajwal AShijin Zhangdingningning dingCopilot
 		b19c227fb3 [New Model]: XiaomiMiMo/MiMo-Audio-7B-Instruct support (#750) 
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com>
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: GG-li <3226868735@qq.com>
Signed-off-by: Sihao Li <111170255+GG-li@users.noreply.github.com>
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: Baoyuan Qi <qibaoyuan@126.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: wuzhongjian <wuzhongjian_yewu@cmss.chinamobile.com>
Signed-off-by: dongbo910220 <1275604947@qq.com>
Signed-off-by: dongbo910220 <32610838+dongbo910220@users.noreply.github.com>
Signed-off-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Signed-off-by: baoyuan qi <qibaoyuan@126.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Prajwal A <prajwalanagani@gmail.com>
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com>
Signed-off-by: 丁宁 <nndding@gmail.com>
Signed-off-by: SHIJIN ZHANG <75300765+Dovis01@users.noreply.github.com>
Signed-off-by: dingning<dingning7@xiaomi.com>
Signed-off-by: dingning <dingning7@xiaomi.com>
Signed-off-by: dingning <dingning@xiaomi.com>
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com>
Co-authored-by: Zhang Shijin <zhangshijin@xiaomi.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Sihao Li <111170255+GG-li@users.noreply.github.com>
Co-authored-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: JohnJan <wuzhongjian_yewu@cmss.chinamobile.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: dongbo910220 <32610838+dongbo910220@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: Junhong Liu <ljh_lbj@163.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
Co-authored-by: shijin zhang <zsj1364226740@gmail.com>
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: root <root@hk01dgx028.cm.cluster>
Co-authored-by: Prajwal A <34590600+LawJarp-A@users.noreply.github.com>
Co-authored-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com>
Co-authored-by: dingning <dingning7@xiaomi.com>
Co-authored-by: ning ding <nndding@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-02-27 11:19:49 +08:00
Xu Luandxulusjb c9b31a4c1a feat(qwen3-tts): Add CUDA Graph support for speech tokenizer decoder (#1205) 
Signed-off-by: xulusjb <fdukeshik@gmail.com>
Co-authored-by: xulusjb <fdukeshik@gmail.com>
 2026-02-27 10:48:25 +08:00
rein yang 443b80e06c [skip ci][Doc] add design docs for async chunk in qwen3-omni (#962) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
 2026-02-27 10:46:13 +08:00
Jared Wen汪志鹏Hongsheng Liu
 		4146095623 [Perf] GLM Image (#920) 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: Jared Wen <w13431838023@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-27 10:29:14 +08:00
dongbo910220 f6ed1bc4e7 diffusion: enable VAE patch parallel for SD3.5 (#1428) 
Signed-off-by: dongbo910220 <1275604947@qq.com>
 2026-02-27 10:28:30 +08:00
Canlin Guo 3b0c21f94d [Platform] Enable layerwise offload on all hardware (#1492) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-02-27 06:47:28 +08:00
Chendi.XueandHongsheng Liu 43a9e48ff7 [XPU] fix qwen_omni after rebase to v0.16.0 (#1416) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-27 06:44:48 +08:00
amy-why-3459Hongsheng LiuCanlin Guo
 		e1337c89c7 [BugFix] Restore talker's config (#1524) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Canlin Guo <961750412@qq.com>
 2026-02-27 06:44:11 +08:00
wuhang 4bee33744f [Bugfix] Use uds for zmq address if not set --stage-id (#1522) 
Signed-off-by: wuhang <wuhang6@huawei.com>
 2026-02-26 21:44:07 +08:00
zhumingjue138 c22019b93a [CI] fixed CI timeout (#1460) 
Signed-off-by: zhumingjue <zhumingjue@huawei.com>
Signed-off-by: zhumingjue138 <zhumingjue@huawei.com>
 2026-02-26 21:31:36 +08:00
Canlin GuoandHongsheng Liu f53152d340 [Feature] Support HSDP for diffusion models (#1339) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-26 21:31:10 +08:00
amy-why-3459 66c3e04263 [Performance]Qwen3-Omni performance optimization (#1378) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-02-26 21:21:36 +08:00
Zeyu Huang | 黃澤宇andSamit b44a615625 [Doc][Test][Misc] ComfyUI test, more screenshot, and code cleaning (#1435) 
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-02-26 21:03:54 +08:00