Commit Graph
594 Commits
Author SHA1 Message Date
tzhouam 8c41492307 Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0169 2026-03-06 03:02:50 +00:00
tzhouam 1ccffce2bc rebase to updated commit
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-03-06 02:32:26 +00:00
Ding Zuhaoand汪志鹏 128e6f4b97 [Feature][Bagel] Add CFG parallel mode (#1578)
Signed-off-by: Ding Zuhao <e1583181@u.nus.edu>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
2026-03-06 09:55:54 +08:00
c0b714a8ac [Feature] Support flexible task_type configuration for Qwen3-TTS models (#1197)
Signed-off-by: Jack Lee <695697442@qq.com>
Signed-off-by: jackleehal <695697442@qq.com>
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: JackLeeHal <15664755+JackLeeHal@users.noreply.github.com>
2026-03-05 13:39:09 -05:00
zhanqiuhuandYueqian Lin e643aaec15 [Bugfix] Fix all-silence TTS output: use float32 for speech tokenizer decoder (#1664)
Signed-off-by: Zhanqiu Hu <zh338@cornell.edu>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
2026-03-06 00:04:26 +08:00
Canlin Guo d5ddc190f9 [UX] Add progress bar for diffusion models (#1652)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
2026-03-05 23:56:14 +08:00
Lidang Jiang 28c2200b47 [BugFix] Return proper HTTP status for ErrorResponse in create_speech (#1687)
Signed-off-by: Lidang-Jiang <lidangjiang@gmail.com>
2026-03-05 23:38:20 +08:00
WeiQing Chen 83fe7d005b [Model] support LTX-2 text-to-video image-to-video (#841)
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
2026-03-05 23:31:48 +08:00
Juan Pablo Zuluagaandpablo 6a45efb287 [Feat][Qwen3TTS] reduce TTFA with flexible initial phase (#1583)
Signed-off-by: pablo <pablo@agigo.ai>
Co-authored-by: pablo <pablo@agigo.ai>
2026-03-05 23:14:41 +08:00
Ziming Huang 070ea0ddc3 [Optimize][Qwen3-Omni] Reduce inter-packet latency in async chunk (#1656)
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
2026-03-05 22:26:30 +08:00
rongfu.leng 562136d390 [Misc] remove logits_processor_pattern this field, because vllm have … (#1675)
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
2026-03-05 09:36:19 +00:00
Alex Brooks fa944bd8e8 [BugFix] Fix LongCat Sequence Parallelism / Small Cleanup (#1631)
Signed-off-by: Alex Brooks <albrooks@redhat.com>
2026-03-05 17:31:00 +08:00
tzhouam 5be08baed3 Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0170
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-03-05 08:53:26 +00:00
zhou zhuoxin 58eb1c68db [FP8 Quantization] Add FP8 quantization support for Flux transformer (#1640)
Signed-off-by: zhou zhuoxin <zhouzhuoxin1508@outlook.com>
2026-03-05 11:08:34 +08:00
tzhouam 395eab8984 Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0169 2026-03-05 02:29:14 +00:00
5493e75877 [NPU] [Features] [Bugfix] Support mindiesd adaln (#1537)
Signed-off-by: jiangmengyu18 <451528648@qq.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
2026-03-05 09:01:19 +08:00
SYLAR c316153b49 [Bugfix] Enable torch.compile for low noise model (transformer_2) (#1541)
Signed-off-by: lishunyang <lishunyang12@163.com>
2026-03-05 09:00:34 +08:00
dsinghviandHongsheng Liu b63e7c7c9a [Model] Fun cosy voice3-0.5-b-2512 (#498)
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-03-04 19:54:59 -05:00
1612948062 add support for MammothModa2 model (#336)
Signed-off-by: HonestDeng <2958906959@qq.com>
Signed-off-by: iwzbi <iwzbi@zju.edu.cn>
Signed-off-by: iwzbi <wzbi@zju.edu.cn>
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: catcat <108673086+iwzbi@users.noreply.github.com>
Co-authored-by: iwzbi <iwzbi@zju.edu.cn>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: dsinghvi <divyanshsinghvi@gmail.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
2026-03-04 22:52:56 +08:00
Alex BrooksandCanlin Guo 11a5842bd1 [Bugfix] Fix Dtype Parsing (#1391)
Signed-off-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: Canlin Guo <961750412@qq.com>
2026-03-04 22:49:28 +08:00
usberkeleyandprincepride 84338494d2 [Model] Add Hunyuan Image3 AR Support (#759)
Signed-off-by: Bradley <bradley.b.pitt@gmail.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: princepride <wangzhipeng628@gmail.com>
2026-03-04 21:49:38 +08:00
tzhouam 36a507e071 Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0170
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-03-04 08:26:56 +00:00
Yueqian Lin 35bd3ff7d2 [Bugfix] Add TTS request validation to prevent engine crashes (#1641)
Signed-off-by: linyueqian <linyueqian@outlook.com>
2026-03-04 14:27:02 +08:00
Lancer 4082202484 [chore] add _repeated_blocks for regional compilation support (#1642)
Signed-off-by: Lancer <maruixiang6688@gmail.com>
2026-03-04 14:26:06 +08:00
汪志鹏andgcanlin f9ec07f0e2 [Model]: Helios support (#1604)
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: gcanlin <canlinguosdu@gmail.com>
2026-03-04 14:17:15 +08:00
675c35db97 [Refactor] lora: reuse load_weights packed mapping (#991)
Signed-off-by: dongbo910220 <1275604947@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
2026-03-04 02:58:43 +00:00
tzhouam ae818f4b45 Fix Qwen3Omni VisionTransformer compatibility with updated vLLM Qwen2_5_VisionAttention
The upstream vLLM Qwen2_5_VisionAttention.forward() now requires a
`sequence_lengths` argument, but Qwen3_VisionBlock.forward() does not
pass it, causing a TypeError at init time.

Instead of patching the vLLM package directly, introduce a
Qwen3Omni_VisionTransformer subclass in vllm-omni that overrides
forward() to properly compute and propagate sequence_lengths via
MMEncoderAttention (following qwen3_vl.py's pattern). Both thinker
and talker now import the fixed subclass.

Also bump the vLLM wheel to e196028dbaa41b.

Made-with: Cursor

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-03-04 02:51:02 +00:00
Yueqian Lin a3240fd484 [Bugfix] Fix transformers 5.x compat issues in online TTS serving (#1536)
Signed-off-by: linyueqian <linyueqian@outlook.com>
2026-03-04 01:28:32 +08:00
tzhouam 4914466f0b Update Dockerfile.ci to install vLLM wheel version 3dd142adb7f7fa3b0b8345ee046ba36a12d9e530 and remove ExtractHiddenStatesProposer references from GPU model runners for cleaner code.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-03-03 15:09:26 +00:00
tzhouam 2398a7c1d7 Merge origin/main into dev/rebase-v0170
Resolve conflict in omni_generation_scheduler.py: keep both
num_cached_tokens tracking and additional_information caching.

Made-with: Cursor

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-03-03 14:31:05 +00:00
Junhong Liu 742d845b8e Make chunk_size and left_context_size configurable via YAML for async chunking (#1423)
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Signed-off-by: Junhong Liu <ljh_lbj@163.com>
2026-03-03 22:06:52 +08:00
tzhouam 16824aa295 Add max_query_len parameter to _LocalPredictorKVCache for improved query handling
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-03-03 14:04:07 +00:00
Yuanheng Zhao b13761ed26 [Bugfix] Fix filepath resolution for model with subdir and GLM-Image generation (#1609)
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
2026-03-03 23:40:53 +13:00
tzhouam 2da9d28f1b Enhance reproducibility in Ulysses attention tests by replacing current_omni_platform.seed_everything with torch's manual_seed and adding support for CUDA devices. Introduce get_supported_tasks method in AsyncOmni class to dynamically determine available tasks based on stage capabilities.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-03-03 09:45:51 +00:00
dengyunyang d275c0c993 [FEATURE] Tile/Patch parallelism refactor for easily support other models (#1366)
Signed-off-by: dengyunyang <584797741@qq.com>
2026-03-03 15:48:51 +08:00
tzhouam 98926707f5 rebase 2026/3/3
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-03-03 03:50:34 +00:00
zhanqiuhu 80c89e00cb [Bugfix] Fix Qwen3-TTS code predictor crash due to missing vLLM config context (#1619)
Signed-off-by: Zhanqiu <zh338@cornell.edu>
2026-03-03 11:02:13 +08:00
tzhouam 3e539fcb6a Merge remote-tracking branch 'origin/main' into dev/vllm-align
Made-with: Cursor

# Conflicts:
#	vllm_omni/entrypoints/openai/serving_chat.py
#	vllm_omni/worker/gpu_model_runner.py

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-03-03 02:57:27 +00:00
Alex BrooksandSYLAR a09b6499f8 [Bugfix][Model] Fix LongCat Image Config Handling / Layer Creation (#1485)
Signed-off-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
2026-03-03 00:50:08 +08:00
Samit 22a51a7cdb [Feature][Wan2.2] Speed up diffusion model startup by multi-thread weight loading (#1504)
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
2026-03-02 14:53:25 +00:00
rongfu.leng a4b2e48718 [Bugfix] Import InputPreprocessor into Renderer (#1566)
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
2026-03-02 21:54:15 +08:00
Baoyuan Qi f371f7bf05 [bugfix] Fix unexpected argument 'is_finished' in function llm2code2wav_async_chunk of mimo-audio (#1570)
Signed-off-by: Baoyuan Qi <qibaoyuan@126.com>
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
2026-03-02 21:49:06 +08:00
rein yang f082f6eedb [Bugfix] fix kernel error for qwen3-omni (#1602)
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
2026-03-02 18:12:30 +08:00
Ting FU 1ca198eaff [BugFix] Fix load_weights error when loading HunyuanImage3.0 (#1598)
Signed-off-by: Semmer2 <semmer@live.cn>
2026-03-02 08:40:48 +00:00
汪志鹏andCursor e37a89ffcb [Feature]: Support cfg kv-cache transfer in multi-stage (#1422)
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-03-02 05:06:28 +00:00
tzhouam 473d3ecd29 Update vLLM wheel version and adjust GPU memory utilization in CI configuration
- Updated the vLLM wheel commit hash in Dockerfile.ci to a60985b07eaf0fefa4c353ad2309b745707591ed.
- Modified the mock shared memory handle name in test_basic_connectors.py for clarity.
- Increased GPU memory utilization to 0.9 and adjusted max_num_batched_tokens to 4096 in qwen2_5_omni_ci.yaml to enable testing on H800.
- Refactored GPU model runner classes to remove deprecated activate_lora parameter and ensure compatibility with num_active_loras.
- Enhanced request handling in the OmniGPUModelRunner class to improve streaming request updates.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-03-02 03:08:45 +00:00
dsinghviandZhou Taichang 68764cc4f4 [Refactor]: Phase1 for rebasing_additional_info (#1394)
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk>
2026-03-02 10:19:41 +08:00
tzhouam 26744bcb35 Merge origin/main into dev/vllm-align
Resolved conflicts:
- vllm_omni/inputs/preprocess.py: took origin/main's elif mm_processor_kwargs branch, fixed undefined mm_uuids reference
- vllm_omni/model_executor/models/qwen3_tts/qwen3_tts_code_predictor_vllm.py: took origin/main's NPU support

Made-with: Cursor

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-03-02 02:10:32 +00:00
汪志鹏 3d9fa8d53f [BugFix]: fix a lot of bug (#1565)
Signed-off-by: princepride <wangzhipeng628@gmail.com>
2026-02-28 16:26:56 +08:00
wangyu cd2234a1bf [CI] Modify some CI test cases to run on L4 environment to reduce H100 resource usage. (#1543)
Signed-off-by: yenuo26 <410167048@qq.com>
Signed-off-by: wangyu <53896905+yenuo26@users.noreply.github.com>
2026-02-28 14:41:51 +08:00