anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
955 Commits 235 Branches 31 Tags
e6d4cdd71a46ab97f5682d0cff6cd3e76db73400
Commit Graph
196 Commits
Author SHA1 Message Date
Taichang Zhou c8b695f181 smaller talker gpu util to avoid oom 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-03-08 01:48:42 +08:00
Taichang Zhou f405240513 Change pytest fixture scope from "module" to "class" for omni_server and omni_server_no_async_chunk in test_qwen3_tts.py to improve test isolation and resource management. 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-03-07 14:50:59 +08:00
Taichang Zhou 0c2464db20 revert unnecessary changes 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-03-07 13:26:32 +08:00
Taichang Zhou 5511bc14c7 Merge origin/main into dev/rebase-v0170 
- Resolved conflict in vllm_omni/engine/output_processor.py: use tracing_enabled for vLLM v0.17.0 API
- Resolved conflict in vllm_omni/model_executor/models/qwen3_tts/qwen3_tts_code_predictor_vllm.py: keep correct _CodePredictorDecoderLayer.forward()
- Update Dockerfile.ci vLLM wheel commit to b31e9326a (releases/v0.17.0)

Made-with: Cursor

Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-03-07 13:18:33 +08:00
Yueqian Lin 3bc89de0c0 Add non-async chunk support for Qwen3-TTS (#1678) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-06 14:45:32 +08:00
tzhouam 8c41492307 Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0169 2026-03-06 03:02:50 +00:00
Ding Zuhaoand汪志鹏 128e6f4b97 [Feature][Bagel] Add CFG parallel mode (#1578) 
Signed-off-by: Ding Zuhao <e1583181@u.nus.edu>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-03-06 09:55:54 +08:00
Yueqian Lin 8536dce759 [Cleanup] Move cosyvoice3 tests to model subdirectory (#1666) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-05 15:28:22 -05:00
zhanqiuhuandYueqian Lin e643aaec15 [Bugfix] Fix all-silence TTS output: use float32 for speech tokenizer decoder (#1664) 
Signed-off-by: Zhanqiu Hu <zh338@cornell.edu>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-03-06 00:04:26 +08:00
WeiQing Chen 83fe7d005b [Model] support LTX-2 text-to-video image-to-video (#841) 
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
 2026-03-05 23:31:48 +08:00
Juan Pablo Zuluagaandpablo 6a45efb287 [Feat][Qwen3TTS] reduce TTFA with flexible initial phase (#1583) 
Signed-off-by: pablo <pablo@agigo.ai>
Co-authored-by: pablo <pablo@agigo.ai>
 2026-03-05 23:14:41 +08:00
Alicia e674d60008 [CI] Remove high concurrency tests before issue #1374 fixed. (#1683) 
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-03-05 17:46:37 +08:00
tzhouam 5be08baed3 Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0170 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-05 08:53:26 +00:00
Daniel Huang 24c80070d6 Replace hard-coded cuda generator with current_omni_platform.device_type (#1677) 
Signed-off-by: Daniel Huang <daniel1.huang@intel.com>
 2026-03-05 06:53:30 +00:00
tzhouam 395eab8984 Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0169 2026-03-05 02:29:14 +00:00
dsinghviandHongsheng Liu b63e7c7c9a [Model] Fun cosy voice3-0.5-b-2512 (#498) 
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-03-04 19:54:59 -05:00
ShiCheng DengcatcatiwzbiHongsheng LiudsinghviCanlin GuoPeiqi Yin汪志鹏
 		1612948062 add support for MammothModa2 model (#336) 
Signed-off-by: HonestDeng <2958906959@qq.com>
Signed-off-by: iwzbi <iwzbi@zju.edu.cn>
Signed-off-by: iwzbi <wzbi@zju.edu.cn>
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: catcat <108673086+iwzbi@users.noreply.github.com>
Co-authored-by: iwzbi <iwzbi@zju.edu.cn>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: dsinghvi <divyanshsinghvi@gmail.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-03-04 22:52:56 +08:00
Alex BrooksandCanlin Guo 11a5842bd1 [Bugfix] Fix Dtype Parsing (#1391) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: Canlin Guo <961750412@qq.com>
 2026-03-04 22:49:28 +08:00
amy-why-3459 46b7e0cf9f [Test][Qwen3-Omni]Modify Qwen3-Omni benchmark test cases (#1628) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-03-04 22:42:47 +08:00
tzhouam 36a507e071 Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0170 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-04 08:26:56 +00:00
Yueqian Lin 35bd3ff7d2 [Bugfix] Add TTS request validation to prevent engine crashes (#1641) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-04 14:27:02 +08:00
dongbo910220Hongsheng LiuJiangyun Zhu
 		675c35db97 [Refactor] lora: reuse load_weights packed mapping (#991) 
Signed-off-by: dongbo910220 <1275604947@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
 2026-03-04 02:58:43 +00:00
tzhouam 2398a7c1d7 Merge origin/main into dev/rebase-v0170 
Resolve conflict in omni_generation_scheduler.py: keep both
num_cached_tokens tracking and additional_information caching.

Made-with: Cursor

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 14:31:05 +00:00
tzhouam 2da9d28f1b Enhance reproducibility in Ulysses attention tests by replacing current_omni_platform.seed_everything with torch's manual_seed and adding support for CUDA devices. Introduce get_supported_tasks method in AsyncOmni class to dynamically determine available tasks based on stage capabilities. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 09:45:51 +00:00
dengyunyang d275c0c993 [FEATURE] Tile/Patch parallelism refactor for easily support other models (#1366) 
Signed-off-by: dengyunyang <584797741@qq.com>
 2026-03-03 15:48:51 +08:00
Alicia e7f2c88701 [CI] Add scripts for bechmark collection and email distribution. (#1307) 
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-03-02 23:00:33 -08:00
tzhouam 3e539fcb6a Merge remote-tracking branch 'origin/main' into dev/vllm-align 
Made-with: Cursor

# Conflicts:
#	vllm_omni/entrypoints/openai/serving_chat.py
#	vllm_omni/worker/gpu_model_runner.py

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 02:57:27 +00:00
Samit 22a51a7cdb [Feature][Wan2.2] Speed up diffusion model startup by multi-thread weight loading (#1504) 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
 2026-03-02 14:53:25 +00:00
汪志鹏andCursor e37a89ffcb [Feature]: Support cfg kv-cache transfer in multi-stage (#1422) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-03-02 05:06:28 +00:00
tzhouam 473d3ecd29 Update vLLM wheel version and adjust GPU memory utilization in CI configuration 
- Updated the vLLM wheel commit hash in Dockerfile.ci to a60985b07eaf0fefa4c353ad2309b745707591ed.
- Modified the mock shared memory handle name in test_basic_connectors.py for clarity.
- Increased GPU memory utilization to 0.9 and adjusted max_num_batched_tokens to 4096 in qwen2_5_omni_ci.yaml to enable testing on H800.
- Refactored GPU model runner classes to remove deprecated activate_lora parameter and ensure compatibility with num_active_loras.
- Enhanced request handling in the OmniGPUModelRunner class to improve streaming request updates.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-02 03:08:45 +00:00
dsinghviandZhou Taichang 68764cc4f4 [Refactor]: Phase1 for rebasing_additional_info (#1394) 
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2026-03-02 10:19:41 +08:00
tzhouam 26744bcb35 Merge origin/main into dev/vllm-align 
Resolved conflicts:
- vllm_omni/inputs/preprocess.py: took origin/main's elif mm_processor_kwargs branch, fixed undefined mm_uuids reference
- vllm_omni/model_executor/models/qwen3_tts/qwen3_tts_code_predictor_vllm.py: took origin/main's NPU support

Made-with: Cursor

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-02 02:10:32 +00:00
wangyu cd2234a1bf [CI] Modify some CI test cases to run on L4 environment to reduce H100 resource usage. (#1543) 
Signed-off-by: yenuo26 <410167048@qq.com>
Signed-off-by: wangyu <53896905+yenuo26@users.noreply.github.com>
 2026-02-28 14:41:51 +08:00
Zhou Taichang fe0c02ee78 [Debug] Merge vllm pull 35368 (#1534) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-28 10:39:25 +08:00
Chendi.Xueandgcanlin 482f9b89a1 [XPU][NPU][ROCM] enable cpu_offloading flag for non_cuda (#1488) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Co-authored-by: gcanlin <canlinguosdu@gmail.com>
 2026-02-27 14:52:15 +08:00
Sy03 0cc3481acf [Feat] : Support Async chunk cleanup (#1087) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-02-27 14:39:33 +08:00
WeiQing ChenIsotr0pyIsotr0pyHongsheng Liu
 		0b884ebc6c [Feature]: Native GGUF Quantization Support for DiT (#1285) 
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-27 11:39:14 +08:00
+13
Baoyuan Qiwangyuwangyu31577Zhang ShijinHongsheng LiuSihao LiXU MingshiCanlin GuoIsotr0pyJohnJanWeiQing Chendongbo910220Jiangyun ZhuJunhong LiuTJianshijin zhangZhou TaichangrootPrajwal AShijin Zhangdingningning dingCopilot
 		b19c227fb3 [New Model]: XiaomiMiMo/MiMo-Audio-7B-Instruct support (#750) 
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com>
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: GG-li <3226868735@qq.com>
Signed-off-by: Sihao Li <111170255+GG-li@users.noreply.github.com>
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: Baoyuan Qi <qibaoyuan@126.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: wuzhongjian <wuzhongjian_yewu@cmss.chinamobile.com>
Signed-off-by: dongbo910220 <1275604947@qq.com>
Signed-off-by: dongbo910220 <32610838+dongbo910220@users.noreply.github.com>
Signed-off-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Signed-off-by: baoyuan qi <qibaoyuan@126.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Prajwal A <prajwalanagani@gmail.com>
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com>
Signed-off-by: 丁宁 <nndding@gmail.com>
Signed-off-by: SHIJIN ZHANG <75300765+Dovis01@users.noreply.github.com>
Signed-off-by: dingning<dingning7@xiaomi.com>
Signed-off-by: dingning <dingning7@xiaomi.com>
Signed-off-by: dingning <dingning@xiaomi.com>
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com>
Co-authored-by: Zhang Shijin <zhangshijin@xiaomi.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Sihao Li <111170255+GG-li@users.noreply.github.com>
Co-authored-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: JohnJan <wuzhongjian_yewu@cmss.chinamobile.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: dongbo910220 <32610838+dongbo910220@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: Junhong Liu <ljh_lbj@163.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
Co-authored-by: shijin zhang <zsj1364226740@gmail.com>
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: root <root@hk01dgx028.cm.cluster>
Co-authored-by: Prajwal A <34590600+LawJarp-A@users.noreply.github.com>
Co-authored-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com>
Co-authored-by: dingning <dingning7@xiaomi.com>
Co-authored-by: ning ding <nndding@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-02-27 11:19:49 +08:00
Xu Luandxulusjb c9b31a4c1a feat(qwen3-tts): Add CUDA Graph support for speech tokenizer decoder (#1205) 
Signed-off-by: xulusjb <fdukeshik@gmail.com>
Co-authored-by: xulusjb <fdukeshik@gmail.com>
 2026-02-27 10:48:25 +08:00
Canlin Guo 3b0c21f94d [Platform] Enable layerwise offload on all hardware (#1492) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-02-27 06:47:28 +08:00
Canlin GuoandHongsheng Liu f53152d340 [Feature] Support HSDP for diffusion models (#1339) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-26 21:31:10 +08:00
amy-why-3459 66c3e04263 [Performance]Qwen3-Omni performance optimization (#1378) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-02-26 21:21:36 +08:00
Zeyu Huang | 黃澤宇andSamit b44a615625 [Doc][Test][Misc] ComfyUI test, more screenshot, and code cleaning (#1435) 
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-02-26 21:03:54 +08:00
knlnguyen1802andHongsheng Liu 5361353be5 [Bugfix] Race condition in MultiprocExecutor when concurent access to Scheduler (#1448) 
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-26 21:03:38 +08:00
Juan Pablo ZuluagapabloHongsheng Liu
 		03c01e0415 [Qwen3TTS][Feat] Streaming output (#1438) 
Signed-off-by: pablo <pablo@agigo.ai>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: pablo <pablo@agigo.ai>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-26 21:02:35 +08:00
Yan Ma cf1f4c6338 [XPU] Enable FLASH_ATTN on XPU (#1332) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-02-26 20:28:09 +08:00
Alex Brooks bc26de834e [Bugfix] Fix LoRA Scaling on Active Adapters (#1421) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-02-26 17:22:35 +08:00
Yuanheng Zhao ed08a216c1 [Misc] Migrate L1 tests to use pytest-mock (#1315) 
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
 2026-02-26 17:08:39 +08:00
tzhouam 8d0c46487e Merge origin/main into dev/vllm-align 
Resolve conflicts:
- docker/Dockerfile.ci: Keep wheel install section, update hash to 6042e66cd5
- vllm_omni/platforms/npu/worker/npu_model_runner.py: Accept main's refactoring

Made-with: Cursor

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-26 08:19:53 +00:00
junuxyzandHongsheng Liu caed95fb45 [Bugfix] Fix tuple/list KV cache extraction crash (#1405) 
Signed-off-by: junuxyz <216036880+junuxyz@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-26 15:20:40 +08:00