SYLAR
f0cec3ecd4
Merge branch 'main' into lance-model-support
2026-05-22 04:13:58 +08:00
lishunyang12
7b885f9d13
[Lance] Precision matching, recipe, and README cleanup
...
- Precision matching across the Lance transformer and pipeline so vLLM-Omni
outputs match the upstream reference end-to-end (text2img / img2img /
text2video / video2video / image and video understanding).
- Add recipes/ByteDance/Lance.md covering all single-stage modalities for
offline + online; register it in recipes/README.md.
- Offline README: document --height/--width and update the default
resolution to 1024x1024 (matches the upstream inference_lance.sh).
- Gitignore lance_compare/ (local precision-matching dump scripts; not part
of the repo).
- Remove LANCE_PR.md; the PR description lives on GitHub.
Signed-off-by: lishunyang12 <lishunyang12@163.com >
2026-05-21 20:07:40 +00:00
Jialong Liu and Yueqian Lin
e949ccf087
[Bugfix] Fix MiMo-Audio voice instability: stochastic local_sampler + codec streaming context ( #3686 )
...
Signed-off-by: Galleons2029 <Galleons777@gmail.com >
Signed-off-by: Jialong Liu <88185941+Galleons2029@users.noreply.github.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-05-21 10:18:00 -04:00
Yuanheng Zhao
2bc9cb7f13
[Feat] Support online quantization (fp8/int8) for LTX-2 ( #3700 )
...
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com >
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com >
2026-05-21 19:52:21 +08:00
8297570c2b
[AutoRound] Support WAN2.2 W4A16 quantization model ( #3353 )
...
Signed-off-by: lvliang-intel <liang1.lv@intel.com >
Signed-off-by: hyh_hh <huyinghong1@huawei.com >
Co-authored-by: hxhhhlalala <hyh_hh@163.com >
Co-authored-by: hyh_hh <huyinghong1@huawei.com >
2026-05-21 17:54:41 +08:00
Siyuan Kong
7d9d8aee79
[Bugfix] Fix qwen2_5_omni weight loading ( #3598 )
...
Signed-off-by: ksiyuan <ksiyuan@umich.edu >
2026-05-21 16:43:33 +08:00
Samit
083b5e363f
[Diffusion] Support LoRA in step-wise execution ( #3639 )
...
Signed-off-by: samithuang <285365963@qq.com >
2026-05-20 22:26:18 +08:00
Canlin Guo
2aafe0bf94
[Perf] Optimize sampler D2H sync for HY-Image ( #3617 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2026-05-20 15:40:34 +08:00
bjf-frz and bjf-frz
5cf4605bf9
[Bugfix]Fix distributed stage0 multimodal cache routing ( #3740 )
...
Signed-off-by: bjf-frz <240110042+bjf-frz@users.noreply.github.com >
Co-authored-by: bjf-frz <240110042+bjf-frz@users.noreply.github.com >
2026-05-20 15:31:41 +08:00
Yueqian Lin
fc8486cf20
[Doc][TTS] CosyVoice3 online docs + residual TTS yaml cleanup + remove VoxCPM v1 ( #3748 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
2026-05-20 15:24:06 +08:00
yangsonglin13
1c530d984d
Yuanrong TransferEngine Connector for NPU ( #3180 )
...
Signed-off-by: yangsonglin13 <yangsonglin566@gmail.com >
2026-05-20 14:51:33 +08:00
hxhhhlalala and hyh_hh
9dd36e3afd
[NPU][Quant] Add W4A4 MXFP4 online & MXFP4 dual-scale online/offline quantization support for Wan2.2 T2V / I2V inference on Ascend NPU ( #3578 )
...
Signed-off-by: hyh_hh <huyinghong1@huawei.com >
Co-authored-by: hyh_hh <huyinghong1@huawei.com >
2026-05-20 06:21:29 +00:00
Lancer
109bba8cdf
[Bugfix] Treat kv_cache_dtype=auto as unset for ring attention ( #3622 )
...
Signed-off-by: Lancer <maruixiang6688@gmail.com >
2026-05-20 05:18:57 +00:00
Yueqian Lin and Hongsheng Liu
f99e8b2d3c
[Bench] Add audio-streaming continuity metric for TTS ( #3618 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-05-20 05:08:56 +00:00
Nick Cao
06ab723afc
[Perf][VoxCPM2][Ming-Flash-Omni] Use global CUDA graph pool ( #3361 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
2026-05-20 05:01:42 +00:00
Nick Cao and Claude
d4e90bbba8
[Bugfix][TTS] Only populate voice_name for uploaded voices without inline ref_audio ( #3523 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-05-20 04:36:59 +00:00
Y. Fisher and zzh
29179599fc
[Feat] Enable VAE parallel in HunyuanImage3 ( #3091 )
...
Signed-off-by: KexiongYu <yukexiong1@huawei.com >
Signed-off-by: zzh <943967662@qq.com >
Co-authored-by: zzh <943967662@qq.com >
2026-05-20 09:55:33 +08:00
WeiQing Chen
a3d4ed809d
[Quantization][tools] Add diffusion quantization output comparison tool ( #3175 )
...
Signed-off-by: david6666666 <530634352@qq.com >
Signed-off-by: David Chen <530634352@qq.com >
2026-05-20 00:14:08 +08:00
dengyunyang
3c58868c9a
[BugFix] fix mult cli timeout with get kv ( #3741 )
...
Signed-off-by: dengyunyang <584797741@qq.com >
2026-05-19 14:22:27 +00:00
Schatten
18186db216
[Cleanup] Remove unused build_base_engine_args after #1115 ( #3720 )
...
Signed-off-by: Schatten <czhengt@qq.com >
2026-05-19 19:23:45 +08:00
Lu and SYLAR
14e5baceaf
[Qwen-Image] Drop unused vision tower from text encoder ( #3608 )
...
Signed-off-by: lulugoodcoder <luludachiever@gmail.com >
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
2026-05-19 19:19:58 +08:00
chickeyton and herotai214
e277feacaf
[large-scale-serving] Integrate OmniCoordinator into stage engine pipeline ( #3569 )
...
Signed-off-by: chickeyton <ngton2014@gmail.com >
Signed-off-by: herotai214 <herotai214@gmail.com >
Co-authored-by: herotai214 <herotai214@gmail.com >
2026-05-19 12:08:20 +08:00
JiaHong
ca9fd0b71c
Reject non-positive Flux2 Klein inference steps ( #3717 )
...
Signed-off-by: MmMaiIIi <2360655509@qq.com >
2026-05-19 11:52:41 +08:00
JiaHong and WeiQing Chen
3ac739817f
[Bugfix] Reject empty prompts in Flux2 Klein diffusion pipeline ( #3711 )
...
Signed-off-by: MmMaiIIi <2360655509@qq.com >
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
2026-05-19 11:30:54 +08:00
fan2956
2c6b1bb0c0
[Bugfix] Fix hunyuanimage3 dit quant storageshape mismatch error ( #3694 )
...
Signed-off-by: fan2956 <zhoufan53@huawei.com >
2026-05-19 10:24:27 +08:00
Rustam Khadipash
89f8819525
[Feature] Add support for Pipeline Parallel and integrate it into Wan 2.2 ( #2322 )
...
Signed-off-by: Rustam Khadipash <16683750+hadipash@users.noreply.github.com >
2026-05-19 09:42:17 +08:00
Chendi.Xue
475a4002b0
[XPU] set flash_attn as default diffusion attn backend and fix k_len for cross_attn ( #3525 )
...
Signed-off-by: Chendi Xue <chendi.xue@intel.com >
2026-05-18 23:58:15 +00:00
lishunyang
69bcc6416d
[New Model] Lance (ByteDance) — unified AR+diffusion multimodal ( #3697 )
...
Adds Lance (bytedance-research/Lance), a 3B unified AR+diffusion
multimodal model on a Qwen2.5-VL backbone. Lance is BAGEL-lineage:
it reuses vLLM-Omni's BAGEL Mixture-of-Transformers core verbatim and
specializes only the ViT (Qwen2.5-VL vision), the VAE (Wan2.2) and the
checkpoint layout.
All six single-stage Lance modalities verified end-to-end on 1x B300:
t2i, t2v, image_edit, video_edit, x2t_image (image understanding) and
x2t_video (video understanding). 3-D mRoPE is wired through
BagelRotaryEmbedding for the x2t / video / edit paths. Includes the
upstream gap-analysis quality fixes (task-specific system prompts, full
Qwen chat template, upstream inference defaults and aspect-ratio
buckets).
Ships an offline example (examples/offline_inference/lance/), an
OpenAI-compatible online-serving example
(examples/online_serving/lance/) and an e2e test
(tests/e2e/online_serving/test_lance.py).
See LANCE_PR.md for the architecture rationale, per-modality status and
the two remaining follow-ups (two-stage AR+DiT, video_edit polish).
Closes #3697 .
Signed-off-by: lishunyang <lishunyang03@gmail.com >
2026-05-18 22:09:14 +00:00
Sy03 and amy-why-3459
ab59673f21
[Bugfix][Qwen3-Omni] Handle short Code2Wav chunk outputs ( #3687 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-05-18 18:59:23 +00:00
lyj-jjj and Cursor
821286794f
[HY-Imgae3.0] support hunyuan image3 dit fa-fp8 on npu ( #3540 )
...
Signed-off-by: lyj-jjj <liuyingjun5@huawei.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-05-18 09:35:30 -07:00
Ding Zuhao and nussejzz
f4115bd771
[Bugfix] Fix SenseNova U1 broken import after SupportsModuleOffload ( #3691 )
...
Signed-off-by: nussejzz <nussejzz@users.noreply.github.com >
Co-authored-by: nussejzz <nussejzz@users.noreply.github.com >
2026-05-18 21:12:46 +08:00
Yueqian Lin
990566aef1
[Bugfix][TTS] Drop meaningless TTFT from speech-endpoint benchmarks ( #3674 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
2026-05-18 17:18:18 +08:00
wuhang and Hongsheng Liu
4ba8e14981
Fix diffusion engine cleanup lifecycle ( #3494 )
...
Signed-off-by: wuhang <wuhang6@huawei.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-05-18 08:54:57 +08:00
c99df1ebd9
[TTS][Perf] Optimize Qwen3-TTS high-concurrency serving ( #3662 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
2026-05-17 16:58:59 -04:00
8cfc9179e6
Fix reasoning_parser crash: reconstruct StructuredOutputsConfig from dict ( #2845 )
...
Signed-off-by: Mike_Qiu <qiudayu.qdy@antgroup.com >
Co-authored-by: Mike_Qiu <qiudayu.qdy@antgroup.com >
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-05-17 23:30:24 +08:00
0da9ffdb0d
Fix output finish reason issue for audio chunk in stream mode ( #2849 )
...
Signed-off-by: Mike_Qiu <qiudayu.qdy@antgroup.com >
Co-authored-by: Mike_Qiu <qiudayu.qdy@antgroup.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-05-17 23:05:34 +08:00
4e88053750
[BugFix][HunyuanImage3] Set MRoPE dynamic_arg_dims so graph mode can compile ( #3630 )
...
Signed-off-by: TaffyOfficial <2324465096@qq.com >
Co-authored-by: TaffyOfficial <2324465096@qq.com >
Co-authored-by: Codex <codex@openai.com >
2026-05-17 22:42:59 +08:00
Reid and SYLAR
768943b879
[Frontend]Handle audio generate engine errors consistently ( #3316 )
...
Signed-off-by: reidliu41 <reid201711@gmail.com >
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
2026-05-17 14:10:26 +00:00
Yuanheng Zhao
220db62b3f
[Bugfix] Adapt LTX-2 connector arg with diffusers 0.38.0 ( #3661 )
...
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com >
2026-05-17 21:58:44 +08:00
9c5e35f485
[Config Refactor] Support Recursive Merging for Engine Args ( #3009 )
...
Signed-off-by: Alex Brooks <albrooks@redhat.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-05-17 07:33:03 +08:00
Yuanheng Zhao
a64ebf103b
[Refactor] Migrate and clean up TTS configs: CosyVoice3, OmniVoice, VoxCPM ( #3338 )
...
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com >
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com >
2026-05-17 07:32:34 +08:00
rongfu.leng
c5ac295e3c
[Feat] Add helios support cache dit ( #3470 )
...
Signed-off-by: rongfu.leng <lenronfu@gmail.com >
2026-05-16 21:11:34 +08:00
Zeng Chuang and Hongsheng Liu
ea35a0cc4a
[Bugfix]update process name for dit stage ( #3602 )
...
Signed-off-by: zengchuang <zengchuang3@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-05-16 20:51:31 +08:00
wuhang and Hongsheng Liu
0f4853ff86
[Bugfix] Support diffusion worker dead detect when use inline engine ( #3214 )
...
Signed-off-by: wuhang <wuhang6@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-05-16 20:50:29 +08:00
33220b1e39
[BugFix] Finish async_chunk requests without pad-token injection ( #3613 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-05-16 23:34:04 +12:00
+20
eb4e60ee64
[Rebase] Rebase to vllm v0.21.0 ( #3530 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Signed-off-by: NumberWan <wantszkin2003@gmail.com >
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com >
Signed-off-by: Dnoob <dxpouo@gmail.com >
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com >
Signed-off-by: rein yang <ruiruyang2@gmail.com >
Signed-off-by: Nick Cao <ncao@redhat.com >
Signed-off-by: zhumingjue <zhumingjue@huawei.com >
Signed-off-by: Ricardo Noriega De Soto <rnoriega@redhat.com >
Signed-off-by: lyj-jjj <liuyingjun5@huawei.com >
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Signed-off-by: wangyu <410167048@qq.com >
Signed-off-by: weizhoublue <weizhoublue@github.com >
Signed-off-by: weizhou.lan@daocloud.io <weizhou.lan@daocloud.io >
Signed-off-by: dengyunyang <584797741@qq.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
Signed-off-by: David Chen <530634352@qq.com >
Signed-off-by: Jie Liu <33612777+keeper-jie@users.noreply.github.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: natureofnature <wzliu@connect.hku.hk >
Signed-off-by: bjf-frz <frz123db@gmail.com >
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
Signed-off-by: KexiongYu <yukexiong1@huawei.com >
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com >
Signed-off-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com >
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com >
Co-authored-by: NumberWan <wantszkin2003@gmail.com >
Co-authored-by: dsinghvi <divyanshsinghvi@gmail.com >
Co-authored-by: Dnoob <dxpouo@gmail.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: knlnguyen1802 <knlnguyen1802@gmail.com >
Co-authored-by: Samit <285365963@qq.com >
Co-authored-by: rein yang <73573651+R2-Y@users.noreply.github.com >
Co-authored-by: Nick Cao <ncao@redhat.com >
Co-authored-by: zhumingjue138 <zhumingjue@huawei.com >
Co-authored-by: Ricardo Noriega <rnoriega@redhat.com >
Co-authored-by: lyj-jjj <liuyingjun5@huawei.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: gcanlin <canlinguosdu@gmail.com >
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com >
Co-authored-by: weizhoublue <45163302+weizhoublue@users.noreply.github.com >
Co-authored-by: weizhoublue <weizhoublue@github.com >
Co-authored-by: dengyunyang <584797741@qq.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
Co-authored-by: Jie Liu <33612777+keeper-jie@users.noreply.github.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: NATURE <wzliu@connect.hku.hk >
Co-authored-by: bjf-frz <frz123db@gmail.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
Co-authored-by: Y. Fisher <yukexiong1@huawei.com >
Co-authored-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com >
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com >
2026-05-16 18:18:43 +08:00
Zeyu Huang | 黃澤宇 and Copilot Autofix powered by AI
d18168ccbb
[bugfix] Fix diffusers backend input bug after #2913 ( #3644 )
...
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com >
Signed-off-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com >
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com >
2026-05-15 09:55:09 +00:00
Y. Fisher
779bf3118b
[Bugfix] fix compatibility of _hunyuan_image3_unpack_packed_topk between vllm / vllm ascend ( #3640 )
...
Signed-off-by: KexiongYu <yukexiong1@huawei.com >
2026-05-15 17:35:22 +08:00
amy-why-3459
e7ee5de09f
[BugFix] Fix the issue of thinker requests being preempted, causing shape mismatch. ( #3147 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-05-15 17:07:42 +08:00
bjf-frz
440c718d2a
[Bugfix]Fix multimodal cache routing for AR replicas ( #3605 )
...
Signed-off-by: bjf-frz <frz123db@gmail.com >
2026-05-15 16:42:18 +08:00