anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
927 Commits 235 Branches 31 Tags
3a6b2e27becc526309fbe1ea55aaac89810ce951
Commit Graph
927 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Taichang Zhou 3a6b2e27be [Update] Bump VLLM base tag to v0.17.0 in CI Dockerfile 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-03-07 13:14:10 +08:00
tzhouam 8c41492307 Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0169 2026-03-06 03:02:50 +00:00
tzhouam 44ac28c386 trigger RTD rebuild 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-06 03:02:07 +00:00
tzhouam 1ccffce2bc rebase to updated commit 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-06 02:32:26 +00:00
Ding Zuhaoand汪志鹏 128e6f4b97 [Feature][Bagel] Add CFG parallel mode (#1578) 
Signed-off-by: Ding Zuhao <e1583181@u.nus.edu>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-03-06 09:55:54 +08:00
Yueqian Lin 8536dce759 [Cleanup] Move cosyvoice3 tests to model subdirectory (#1666) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-05 15:28:22 -05:00
Jack Leegoogle-labs-jules[bot]JackLeeHal
 		c0b714a8ac [Feature] Support flexible task_type configuration for Qwen3-TTS models (#1197) 
Signed-off-by: Jack Lee <695697442@qq.com>
Signed-off-by: jackleehal <695697442@qq.com>
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: JackLeeHal <15664755+JackLeeHal@users.noreply.github.com>
 2026-03-05 13:39:09 -05:00
zhanqiuhuandYueqian Lin e643aaec15 [Bugfix] Fix all-silence TTS output: use float32 for speech tokenizer decoder (#1664) 
Signed-off-by: Zhanqiu Hu <zh338@cornell.edu>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-03-06 00:04:26 +08:00
Canlin Guo d5ddc190f9 [UX] Add progress bar for diffusion models (#1652) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-05 23:56:14 +08:00
wangyu ea317e4552 [Doc] Add the test guide document. [skip ci] (#1376) 
Signed-off-by: yenuo26 <410167048@qq.com>
 2026-03-05 23:41:36 +08:00
Lidang Jiang 28c2200b47 [BugFix] Return proper HTTP status for ErrorResponse in create_speech (#1687) 
Signed-off-by: Lidang-Jiang <lidangjiang@gmail.com>
 2026-03-05 23:38:20 +08:00
WeiQing Chen 83fe7d005b [Model] support LTX-2 text-to-video image-to-video (#841) 
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
 2026-03-05 23:31:48 +08:00
Juan Pablo Zuluagaandpablo 6a45efb287 [Feat][Qwen3TTS] reduce TTFA with flexible initial phase (#1583) 
Signed-off-by: pablo <pablo@agigo.ai>
Co-authored-by: pablo <pablo@agigo.ai>
 2026-03-05 23:14:41 +08:00
Ziming Huang 070ea0ddc3 [Optimize][Qwen3-Omni] Reduce inter-packet latency in async chunk (#1656) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
 2026-03-05 22:26:30 +08:00
Alicia e674d60008 [CI] Remove high concurrency tests before issue #1374 fixed. (#1683) 
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-03-05 17:46:37 +08:00
rongfu.leng 562136d390 [Misc] remove logits_processor_pattern this field, because vllm have … (#1675) 
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
 2026-03-05 09:36:19 +00:00
Alex Brooks fa944bd8e8 [BugFix] Fix LongCat Sequence Parallelism / Small Cleanup (#1631) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-03-05 17:31:00 +08:00
tzhouam 5be08baed3 Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0170 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-05 08:53:26 +00:00
Daniel Huang 24c80070d6 Replace hard-coded cuda generator with current_omni_platform.device_type (#1677) 
Signed-off-by: Daniel Huang <daniel1.huang@intel.com>
 2026-03-05 06:53:30 +00:00
zhou zhuoxin 58eb1c68db [FP8 Quantization] Add FP8 quantization support for Flux transformer (#1640) 
Signed-off-by: zhou zhuoxin <zhouzhuoxin1508@outlook.com>
 2026-03-05 11:08:34 +08:00
tzhouam 395eab8984 Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0169 2026-03-05 02:29:14 +00:00
jiangmengyu18Hongsheng LiuSYLAR
 		5493e75877 [NPU] [Features] [Bugfix] Support mindiesd adaln (#1537) 
Signed-off-by: jiangmengyu18 <451528648@qq.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-03-05 09:01:19 +08:00
SYLAR c316153b49 [Bugfix] Enable torch.compile for low noise model (transformer_2) (#1541) 
Signed-off-by: lishunyang <lishunyang12@163.com>
 2026-03-05 09:00:34 +08:00
dsinghviandHongsheng Liu b63e7c7c9a [Model] Fun cosy voice3-0.5-b-2512 (#498) 
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-03-04 19:54:59 -05:00
ShiCheng DengcatcatiwzbiHongsheng LiudsinghviCanlin GuoPeiqi Yin汪志鹏
 		1612948062 add support for MammothModa2 model (#336) 
Signed-off-by: HonestDeng <2958906959@qq.com>
Signed-off-by: iwzbi <iwzbi@zju.edu.cn>
Signed-off-by: iwzbi <wzbi@zju.edu.cn>
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: catcat <108673086+iwzbi@users.noreply.github.com>
Co-authored-by: iwzbi <iwzbi@zju.edu.cn>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: dsinghvi <divyanshsinghvi@gmail.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-03-04 22:52:56 +08:00
Yan Ma 599045a94c [XPU] fix UMD version in docker file (#1545) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-03-04 22:52:00 +08:00
Alex BrooksandCanlin Guo 11a5842bd1 [Bugfix] Fix Dtype Parsing (#1391) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: Canlin Guo <961750412@qq.com>
 2026-03-04 22:49:28 +08:00
amy-why-3459 46b7e0cf9f [Test][Qwen3-Omni]Modify Qwen3-Omni benchmark test cases (#1628) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-03-04 22:42:47 +08:00
usberkeleyandprincepride 84338494d2 [Model] Add Hunyuan Image3 AR Support (#759) 
Signed-off-by: Bradley <bradley.b.pitt@gmail.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: princepride <wangzhipeng628@gmail.com>
 2026-03-04 21:49:38 +08:00
Hongsheng Liu b84d73be4b docs: Announce vllm-omni-skills community project (#1651) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
 2026-03-04 20:56:50 +08:00
tzhouam 36a507e071 Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0170 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-04 08:26:56 +00:00
WeiQing Chen 0c6a10ebb6 [Misc] update wechat (#1649) 
Signed-off-by: David Chen <530634352@qq.com>
 2026-03-04 15:34:00 +08:00
Alicia 29eecc2da8 [CI] Fix ASCII codes. (#1647) 
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-03-04 14:39:53 +08:00
Yueqian Lin 35bd3ff7d2 [Bugfix] Add TTS request validation to prevent engine crashes (#1641) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-04 14:27:02 +08:00
Lancer 4082202484 [chore] add _repeated_blocks for regional compilation support (#1642) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2026-03-04 14:26:06 +08:00
汪志鹏andgcanlin f9ec07f0e2 [Model]: Helios support (#1604) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-04 14:17:15 +08:00
dongbo910220Hongsheng LiuJiangyun Zhu
 		675c35db97 [Refactor] lora: reuse load_weights packed mapping (#991) 
Signed-off-by: dongbo910220 <1275604947@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
 2026-03-04 02:58:43 +00:00
tzhouam ae818f4b45 Fix Qwen3Omni VisionTransformer compatibility with updated vLLM Qwen2_5_VisionAttention 
The upstream vLLM Qwen2_5_VisionAttention.forward() now requires a
`sequence_lengths` argument, but Qwen3_VisionBlock.forward() does not
pass it, causing a TypeError at init time.

Instead of patching the vLLM package directly, introduce a
Qwen3Omni_VisionTransformer subclass in vllm-omni that overrides
forward() to properly compute and propagate sequence_lengths via
MMEncoderAttention (following qwen3_vl.py's pattern). Both thinker
and talker now import the fixed subclass.

Also bump the vLLM wheel to e196028dbaa41b.

Made-with: Cursor

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-04 02:51:02 +00:00
Yueqian Lin a3240fd484 [Bugfix] Fix transformers 5.x compat issues in online TTS serving (#1536) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-04 01:28:32 +08:00
tzhouam 4914466f0b Update Dockerfile.ci to install vLLM wheel version 3dd142adb7f7fa3b0b8345ee046ba36a12d9e530 and remove ExtractHiddenStatesProposer references from GPU model runners for cleaner code. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 15:09:26 +00:00
tzhouam 2398a7c1d7 Merge origin/main into dev/rebase-v0170 
Resolve conflict in omni_generation_scheduler.py: keep both
num_cached_tokens tracking and additional_information caching.

Made-with: Cursor

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 14:31:05 +00:00
Junhong Liu 742d845b8e Make chunk_size and left_context_size configurable via YAML for async chunking (#1423) 
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Signed-off-by: Junhong Liu <ljh_lbj@163.com>
 2026-03-03 22:06:52 +08:00
tzhouam 16824aa295 Add max_query_len parameter to _LocalPredictorKVCache for improved query handling 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 14:04:07 +00:00
Yuanheng Zhao b13761ed26 [Bugfix] Fix filepath resolution for model with subdir and GLM-Image generation (#1609) 
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
 2026-03-03 23:40:53 +13:00
tzhouam 2da9d28f1b Enhance reproducibility in Ulysses attention tests by replacing current_omni_platform.seed_everything with torch's manual_seed and adding support for CUDA devices. Introduce get_supported_tasks method in AsyncOmni class to dynamically determine available tasks based on stage capabilities. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 09:45:51 +00:00
tzhouam 4d8a76e721 Reintroduce installation of flashinfer-jit-cache in Dockerfile.ci to enhance dependency management for the CI environment, ensuring compatibility with the latest requirements. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 09:23:09 +00:00
dengyunyang d275c0c993 [FEATURE] Tile/Patch parallelism refactor for easily support other models (#1366) 
Signed-off-by: dengyunyang <584797741@qq.com>
 2026-03-03 15:48:51 +08:00
Alicia e7f2c88701 [CI] Add scripts for bechmark collection and email distribution. (#1307) 
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-03-02 23:00:33 -08:00
tzhouam a17078974f Remove flashinfer-jit-cache installation from Dockerfile.ci to streamline dependency management and avoid potential conflicts. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 07:00:18 +00:00
tzhouam 8dcf1061f1 Update Dockerfile.ci to install additional dependencies including flashinfer-jit-cache, nvidia-cublas-cu12, and numpy, ensuring a more comprehensive environment setup for development. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 06:47:10 +00:00