anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
937 Commits 235 Branches 31 Tags
5511bc14c790719091d59add089d0ec4bb252c34
Commit Graph
937 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Taichang Zhou 5511bc14c7 Merge origin/main into dev/rebase-v0170 
- Resolved conflict in vllm_omni/engine/output_processor.py: use tracing_enabled for vLLM v0.17.0 API
- Resolved conflict in vllm_omni/model_executor/models/qwen3_tts/qwen3_tts_code_predictor_vllm.py: keep correct _CodePredictorDecoderLayer.forward()
- Update Dockerfile.ci vLLM wheel commit to b31e9326a (releases/v0.17.0)

Made-with: Cursor

Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-03-07 13:18:33 +08:00
Taichang Zhou 3a6b2e27be [Update] Bump VLLM base tag to v0.17.0 in CI Dockerfile 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-03-07 13:14:10 +08:00
Didan Deng 30191ed112 [Feature]: Add vae-patch-parallel CLI argument in online serving (#1716) 
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-03-07 01:45:02 +08:00
erfgss b7fcc9d07e [Profile] Adding metrics for Diffusion/DiT Single diffusion Pipeline (#668) 
Signed-off-by: Chen Yang <2082464740@qq.com>
Signed-off-by: erfgss <97771661+erfgss@users.noreply.github.com>
 2026-03-06 15:39:19 +08:00
Yueqian Lin 411fe05bd4 [skip CI][Docs] Add TTS model developer guide (#1693) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-06 14:49:19 +08:00
JohnJanandHongsheng Liu 92adcdeab4 [feature]: support flux2.klein cache_dit (#1209) 
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-03-06 14:46:57 +08:00
Canlin Guo 846167b237 [1/N][Refactor] Clean up dead code in output processor (#1579) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-06 14:45:53 +08:00
Yueqian Lin 3bc89de0c0 Add non-async chunk support for Qwen3-TTS (#1678) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-06 14:45:32 +08:00
Baoyuan Qi 32fd084ae2 [MiMo-Audio] Bugfix tp lg than 1 (#1688) 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2026-03-06 14:45:08 +08:00
Sy03 90ec499bd5 [Refactor][Perf] Qwen3-TTS: re-prefill Code Predictor with torch.compile + enable Code2Wav decoder CUDA Graph (#1617) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-03-06 18:21:55 +13:00
Du BinlinyueqianYueqian Lin
 		fd51841230 perf: replace per-element .item() GPU syncs with batch .tolist() in TTS code predictor (#1614) 
Signed-off-by: dubin555 <dubin555@gmail.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-03-06 12:22:01 +08:00
tzhouam 8c41492307 Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0169 2026-03-06 03:02:50 +00:00
tzhouam 44ac28c386 trigger RTD rebuild 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-06 03:02:07 +00:00
tzhouam 1ccffce2bc rebase to updated commit 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-06 02:32:26 +00:00
Ding Zuhaoand汪志鹏 128e6f4b97 [Feature][Bagel] Add CFG parallel mode (#1578) 
Signed-off-by: Ding Zuhao <e1583181@u.nus.edu>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-03-06 09:55:54 +08:00
Yueqian Lin 8536dce759 [Cleanup] Move cosyvoice3 tests to model subdirectory (#1666) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-05 15:28:22 -05:00
Jack Leegoogle-labs-jules[bot]JackLeeHal
 		c0b714a8ac [Feature] Support flexible task_type configuration for Qwen3-TTS models (#1197) 
Signed-off-by: Jack Lee <695697442@qq.com>
Signed-off-by: jackleehal <695697442@qq.com>
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: JackLeeHal <15664755+JackLeeHal@users.noreply.github.com>
 2026-03-05 13:39:09 -05:00
zhanqiuhuandYueqian Lin e643aaec15 [Bugfix] Fix all-silence TTS output: use float32 for speech tokenizer decoder (#1664) 
Signed-off-by: Zhanqiu Hu <zh338@cornell.edu>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-03-06 00:04:26 +08:00
Canlin Guo d5ddc190f9 [UX] Add progress bar for diffusion models (#1652) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-05 23:56:14 +08:00
wangyu ea317e4552 [Doc] Add the test guide document. [skip ci] (#1376) 
Signed-off-by: yenuo26 <410167048@qq.com>
 2026-03-05 23:41:36 +08:00
Lidang Jiang 28c2200b47 [BugFix] Return proper HTTP status for ErrorResponse in create_speech (#1687) 
Signed-off-by: Lidang-Jiang <lidangjiang@gmail.com>
 2026-03-05 23:38:20 +08:00
WeiQing Chen 83fe7d005b [Model] support LTX-2 text-to-video image-to-video (#841) 
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
 2026-03-05 23:31:48 +08:00
Juan Pablo Zuluagaandpablo 6a45efb287 [Feat][Qwen3TTS] reduce TTFA with flexible initial phase (#1583) 
Signed-off-by: pablo <pablo@agigo.ai>
Co-authored-by: pablo <pablo@agigo.ai>
 2026-03-05 23:14:41 +08:00
Ziming Huang 070ea0ddc3 [Optimize][Qwen3-Omni] Reduce inter-packet latency in async chunk (#1656) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
 2026-03-05 22:26:30 +08:00
Alicia e674d60008 [CI] Remove high concurrency tests before issue #1374 fixed. (#1683) 
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-03-05 17:46:37 +08:00
rongfu.leng 562136d390 [Misc] remove logits_processor_pattern this field, because vllm have … (#1675) 
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
 2026-03-05 09:36:19 +00:00
Alex Brooks fa944bd8e8 [BugFix] Fix LongCat Sequence Parallelism / Small Cleanup (#1631) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-03-05 17:31:00 +08:00
tzhouam 5be08baed3 Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0170 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-05 08:53:26 +00:00
Daniel Huang 24c80070d6 Replace hard-coded cuda generator with current_omni_platform.device_type (#1677) 
Signed-off-by: Daniel Huang <daniel1.huang@intel.com>
 2026-03-05 06:53:30 +00:00
zhou zhuoxin 58eb1c68db [FP8 Quantization] Add FP8 quantization support for Flux transformer (#1640) 
Signed-off-by: zhou zhuoxin <zhouzhuoxin1508@outlook.com>
 2026-03-05 11:08:34 +08:00
tzhouam 395eab8984 Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0169 2026-03-05 02:29:14 +00:00
jiangmengyu18Hongsheng LiuSYLAR
 		5493e75877 [NPU] [Features] [Bugfix] Support mindiesd adaln (#1537) 
Signed-off-by: jiangmengyu18 <451528648@qq.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-03-05 09:01:19 +08:00
SYLAR c316153b49 [Bugfix] Enable torch.compile for low noise model (transformer_2) (#1541) 
Signed-off-by: lishunyang <lishunyang12@163.com>
 2026-03-05 09:00:34 +08:00
dsinghviandHongsheng Liu b63e7c7c9a [Model] Fun cosy voice3-0.5-b-2512 (#498) 
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-03-04 19:54:59 -05:00
ShiCheng DengcatcatiwzbiHongsheng LiudsinghviCanlin GuoPeiqi Yin汪志鹏
 		1612948062 add support for MammothModa2 model (#336) 
Signed-off-by: HonestDeng <2958906959@qq.com>
Signed-off-by: iwzbi <iwzbi@zju.edu.cn>
Signed-off-by: iwzbi <wzbi@zju.edu.cn>
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: catcat <108673086+iwzbi@users.noreply.github.com>
Co-authored-by: iwzbi <iwzbi@zju.edu.cn>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: dsinghvi <divyanshsinghvi@gmail.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-03-04 22:52:56 +08:00
Yan Ma 599045a94c [XPU] fix UMD version in docker file (#1545) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-03-04 22:52:00 +08:00
Alex BrooksandCanlin Guo 11a5842bd1 [Bugfix] Fix Dtype Parsing (#1391) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: Canlin Guo <961750412@qq.com>
 2026-03-04 22:49:28 +08:00
amy-why-3459 46b7e0cf9f [Test][Qwen3-Omni]Modify Qwen3-Omni benchmark test cases (#1628) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-03-04 22:42:47 +08:00
usberkeleyandprincepride 84338494d2 [Model] Add Hunyuan Image3 AR Support (#759) 
Signed-off-by: Bradley <bradley.b.pitt@gmail.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: princepride <wangzhipeng628@gmail.com>
 2026-03-04 21:49:38 +08:00
Hongsheng Liu b84d73be4b docs: Announce vllm-omni-skills community project (#1651) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
 2026-03-04 20:56:50 +08:00
tzhouam 36a507e071 Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0170 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-04 08:26:56 +00:00
WeiQing Chen 0c6a10ebb6 [Misc] update wechat (#1649) 
Signed-off-by: David Chen <530634352@qq.com>
 2026-03-04 15:34:00 +08:00
Alicia 29eecc2da8 [CI] Fix ASCII codes. (#1647) 
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-03-04 14:39:53 +08:00
Yueqian Lin 35bd3ff7d2 [Bugfix] Add TTS request validation to prevent engine crashes (#1641) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-04 14:27:02 +08:00
Lancer 4082202484 [chore] add _repeated_blocks for regional compilation support (#1642) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2026-03-04 14:26:06 +08:00
汪志鹏andgcanlin f9ec07f0e2 [Model]: Helios support (#1604) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-04 14:17:15 +08:00
dongbo910220Hongsheng LiuJiangyun Zhu
 		675c35db97 [Refactor] lora: reuse load_weights packed mapping (#991) 
Signed-off-by: dongbo910220 <1275604947@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
 2026-03-04 02:58:43 +00:00
tzhouam ae818f4b45 Fix Qwen3Omni VisionTransformer compatibility with updated vLLM Qwen2_5_VisionAttention 
The upstream vLLM Qwen2_5_VisionAttention.forward() now requires a
`sequence_lengths` argument, but Qwen3_VisionBlock.forward() does not
pass it, causing a TypeError at init time.

Instead of patching the vLLM package directly, introduce a
Qwen3Omni_VisionTransformer subclass in vllm-omni that overrides
forward() to properly compute and propagate sequence_lengths via
MMEncoderAttention (following qwen3_vl.py's pattern). Both thinker
and talker now import the fixed subclass.

Also bump the vLLM wheel to e196028dbaa41b.

Made-with: Cursor

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-04 02:51:02 +00:00
Yueqian Lin a3240fd484 [Bugfix] Fix transformers 5.x compat issues in online TTS serving (#1536) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-04 01:28:32 +08:00
tzhouam 4914466f0b Update Dockerfile.ci to install vLLM wheel version 3dd142adb7f7fa3b0b8345ee046ba36a12d9e530 and remove ExtractHiddenStatesProposer references from GPU model runners for cleaner code. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 15:09:26 +00:00