anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
911 Commits 235 Branches 31 Tags
1ccffce2bc7c5967805f17a109cf0725f5c7c906
Commit Graph
911 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
tzhouam 1ccffce2bc rebase to updated commit 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-06 02:32:26 +00:00
tzhouam 5be08baed3 Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0170 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-05 08:53:26 +00:00
Daniel Huang 24c80070d6 Replace hard-coded cuda generator with current_omni_platform.device_type (#1677) 
Signed-off-by: Daniel Huang <daniel1.huang@intel.com>
 2026-03-05 06:53:30 +00:00
zhou zhuoxin 58eb1c68db [FP8 Quantization] Add FP8 quantization support for Flux transformer (#1640) 
Signed-off-by: zhou zhuoxin <zhouzhuoxin1508@outlook.com>
 2026-03-05 11:08:34 +08:00
tzhouam 395eab8984 Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0169 2026-03-05 02:29:14 +00:00
jiangmengyu18Hongsheng LiuSYLAR
 		5493e75877 [NPU] [Features] [Bugfix] Support mindiesd adaln (#1537) 
Signed-off-by: jiangmengyu18 <451528648@qq.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-03-05 09:01:19 +08:00
SYLAR c316153b49 [Bugfix] Enable torch.compile for low noise model (transformer_2) (#1541) 
Signed-off-by: lishunyang <lishunyang12@163.com>
 2026-03-05 09:00:34 +08:00
dsinghviandHongsheng Liu b63e7c7c9a [Model] Fun cosy voice3-0.5-b-2512 (#498) 
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-03-04 19:54:59 -05:00
ShiCheng DengcatcatiwzbiHongsheng LiudsinghviCanlin GuoPeiqi Yin汪志鹏
 		1612948062 add support for MammothModa2 model (#336) 
Signed-off-by: HonestDeng <2958906959@qq.com>
Signed-off-by: iwzbi <iwzbi@zju.edu.cn>
Signed-off-by: iwzbi <wzbi@zju.edu.cn>
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: catcat <108673086+iwzbi@users.noreply.github.com>
Co-authored-by: iwzbi <iwzbi@zju.edu.cn>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: dsinghvi <divyanshsinghvi@gmail.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-03-04 22:52:56 +08:00
Yan Ma 599045a94c [XPU] fix UMD version in docker file (#1545) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-03-04 22:52:00 +08:00
Alex BrooksandCanlin Guo 11a5842bd1 [Bugfix] Fix Dtype Parsing (#1391) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: Canlin Guo <961750412@qq.com>
 2026-03-04 22:49:28 +08:00
amy-why-3459 46b7e0cf9f [Test][Qwen3-Omni]Modify Qwen3-Omni benchmark test cases (#1628) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-03-04 22:42:47 +08:00
usberkeleyandprincepride 84338494d2 [Model] Add Hunyuan Image3 AR Support (#759) 
Signed-off-by: Bradley <bradley.b.pitt@gmail.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: princepride <wangzhipeng628@gmail.com>
 2026-03-04 21:49:38 +08:00
Hongsheng Liu b84d73be4b docs: Announce vllm-omni-skills community project (#1651) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
 2026-03-04 20:56:50 +08:00
tzhouam 36a507e071 Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0170 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-04 08:26:56 +00:00
WeiQing Chen 0c6a10ebb6 [Misc] update wechat (#1649) 
Signed-off-by: David Chen <530634352@qq.com>
 2026-03-04 15:34:00 +08:00
Alicia 29eecc2da8 [CI] Fix ASCII codes. (#1647) 
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-03-04 14:39:53 +08:00
Yueqian Lin 35bd3ff7d2 [Bugfix] Add TTS request validation to prevent engine crashes (#1641) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-04 14:27:02 +08:00
Lancer 4082202484 [chore] add _repeated_blocks for regional compilation support (#1642) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2026-03-04 14:26:06 +08:00
汪志鹏andgcanlin f9ec07f0e2 [Model]: Helios support (#1604) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-04 14:17:15 +08:00
dongbo910220Hongsheng LiuJiangyun Zhu
 		675c35db97 [Refactor] lora: reuse load_weights packed mapping (#991) 
Signed-off-by: dongbo910220 <1275604947@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
 2026-03-04 02:58:43 +00:00
tzhouam ae818f4b45 Fix Qwen3Omni VisionTransformer compatibility with updated vLLM Qwen2_5_VisionAttention 
The upstream vLLM Qwen2_5_VisionAttention.forward() now requires a
`sequence_lengths` argument, but Qwen3_VisionBlock.forward() does not
pass it, causing a TypeError at init time.

Instead of patching the vLLM package directly, introduce a
Qwen3Omni_VisionTransformer subclass in vllm-omni that overrides
forward() to properly compute and propagate sequence_lengths via
MMEncoderAttention (following qwen3_vl.py's pattern). Both thinker
and talker now import the fixed subclass.

Also bump the vLLM wheel to e196028dbaa41b.

Made-with: Cursor

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-04 02:51:02 +00:00
Yueqian Lin a3240fd484 [Bugfix] Fix transformers 5.x compat issues in online TTS serving (#1536) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-04 01:28:32 +08:00
tzhouam 4914466f0b Update Dockerfile.ci to install vLLM wheel version 3dd142adb7f7fa3b0b8345ee046ba36a12d9e530 and remove ExtractHiddenStatesProposer references from GPU model runners for cleaner code. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 15:09:26 +00:00
tzhouam 2398a7c1d7 Merge origin/main into dev/rebase-v0170 
Resolve conflict in omni_generation_scheduler.py: keep both
num_cached_tokens tracking and additional_information caching.

Made-with: Cursor

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 14:31:05 +00:00
Junhong Liu 742d845b8e Make chunk_size and left_context_size configurable via YAML for async chunking (#1423) 
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Signed-off-by: Junhong Liu <ljh_lbj@163.com>
 2026-03-03 22:06:52 +08:00
tzhouam 16824aa295 Add max_query_len parameter to _LocalPredictorKVCache for improved query handling 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 14:04:07 +00:00
Yuanheng Zhao b13761ed26 [Bugfix] Fix filepath resolution for model with subdir and GLM-Image generation (#1609) 
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
 2026-03-03 23:40:53 +13:00
tzhouam 2da9d28f1b Enhance reproducibility in Ulysses attention tests by replacing current_omni_platform.seed_everything with torch's manual_seed and adding support for CUDA devices. Introduce get_supported_tasks method in AsyncOmni class to dynamically determine available tasks based on stage capabilities. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 09:45:51 +00:00
tzhouam 4d8a76e721 Reintroduce installation of flashinfer-jit-cache in Dockerfile.ci to enhance dependency management for the CI environment, ensuring compatibility with the latest requirements. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 09:23:09 +00:00
dengyunyang d275c0c993 [FEATURE] Tile/Patch parallelism refactor for easily support other models (#1366) 
Signed-off-by: dengyunyang <584797741@qq.com>
 2026-03-03 15:48:51 +08:00
Alicia e7f2c88701 [CI] Add scripts for bechmark collection and email distribution. (#1307) 
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-03-02 23:00:33 -08:00
tzhouam a17078974f Remove flashinfer-jit-cache installation from Dockerfile.ci to streamline dependency management and avoid potential conflicts. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 07:00:18 +00:00
tzhouam 8dcf1061f1 Update Dockerfile.ci to install additional dependencies including flashinfer-jit-cache, nvidia-cublas-cu12, and numpy, ensuring a more comprehensive environment setup for development. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 06:47:10 +00:00
Yuanheng Zhao 3b2b5012eb [Doc] Fix links in the configuration doc (#1615) 
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
 2026-03-03 04:52:51 +00:00
Zhou Taichang 90a48f4444 [Debug] Enable curl retry aligned with openai (#1539) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 11:56:42 +08:00
tzhouam 98926707f5 rebase 2026/3/3 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 03:50:34 +00:00
zhanqiuhu 80c89e00cb [Bugfix] Fix Qwen3-TTS code predictor crash due to missing vLLM config context (#1619) 
Signed-off-by: Zhanqiu <zh338@cornell.edu>
 2026-03-03 11:02:13 +08:00
tzhouam 3e539fcb6a Merge remote-tracking branch 'origin/main' into dev/vllm-align 
Made-with: Cursor

# Conflicts:
#	vllm_omni/entrypoints/openai/serving_chat.py
#	vllm_omni/worker/gpu_model_runner.py

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 02:57:27 +00:00
tzhouam e0497550c4 Update Dockerfile.ci to install a specific version of flashinfer-cubin (0.6.4) directly, simplifying the installation process and ensuring compatibility with the current setup. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 02:51:35 +00:00
tzhouam 029bb56e3a Update Dockerfile.ci to use importlib.metadata for retrieving the current version of flashinfer, enhancing compatibility with the latest Python standards. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 02:28:24 +00:00
tzhouam 65169baa9d Update Dockerfile.ci to create a symbolic link for Python 3, ensuring compatibility with Python scripts during the build process. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 01:58:35 +00:00
Alex BrooksandSYLAR a09b6499f8 [Bugfix][Model] Fix LongCat Image Config Handling / Layer Creation (#1485) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-03-03 00:50:08 +08:00
Samit 22a51a7cdb [Feature][Wan2.2] Speed up diffusion model startup by multi-thread weight loading (#1504) 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
 2026-03-02 14:53:25 +00:00
rongfu.leng a4b2e48718 [Bugfix] Import InputPreprocessor into Renderer (#1566) 
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
 2026-03-02 21:54:15 +08:00
Baoyuan Qi f371f7bf05 [bugfix] Fix unexpected argument 'is_finished' in function llm2code2wav_async_chunk of mimo-audio (#1570) 
Signed-off-by: Baoyuan Qi <qibaoyuan@126.com>
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2026-03-02 21:49:06 +08:00
rein yang f082f6eedb [Bugfix] fix kernel error for qwen3-omni (#1602) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
 2026-03-02 18:12:30 +08:00
Ting FU 1ca198eaff [BugFix] Fix load_weights error when loading HunyuanImage3.0 (#1598) 
Signed-off-by: Semmer2 <semmer@live.cn>
 2026-03-02 08:40:48 +00:00
tzhouam 908e3fe761 Update Dockerfile.ci to dynamically install the current version of flashinfer-cubin, ensuring compatibility with the latest changes in the flashinfer package. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-02 07:00:59 +00:00
tzhouam 8fb247a481 Update Dockerfile.ci to install specific version of flashinfer-cubin (0.6.4) and ensure vllm installation remains intact. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-02 06:48:58 +00:00