anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
1,692 Commits 235 Branches 31 Tags
f0cec3ecd4e1991bd1cf9c0a6bbad9d0fe80480b
Commit Graph
1692 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
SYLAR f0cec3ecd4 Merge branch 'main' into lance-model-support 2026-05-22 04:13:58 +08:00
lishunyang12 7b885f9d13 [Lance] Precision matching, recipe, and README cleanup 
- Precision matching across the Lance transformer and pipeline so vLLM-Omni
  outputs match the upstream reference end-to-end (text2img / img2img /
  text2video / video2video / image and video understanding).
- Add recipes/ByteDance/Lance.md covering all single-stage modalities for
  offline + online; register it in recipes/README.md.
- Offline README: document --height/--width and update the default
  resolution to 1024x1024 (matches the upstream inference_lance.sh).
- Gitignore lance_compare/ (local precision-matching dump scripts; not part
  of the repo).
- Remove LANCE_PR.md; the PR description lives on GitHub.

Signed-off-by: lishunyang12 <lishunyang12@163.com>
 2026-05-21 20:07:40 +00:00
Jialong LiuandYueqian Lin e949ccf087 [Bugfix] Fix MiMo-Audio voice instability: stochastic local_sampler + codec streaming context (#3686) 
Signed-off-by: Galleons2029 <Galleons777@gmail.com>
Signed-off-by: Jialong Liu <88185941+Galleons2029@users.noreply.github.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-05-21 10:18:00 -04:00
Hongsheng LiuandCursor bd1617ce2b Add new committers to governance page (#3749) 
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-05-21 22:06:01 +08:00
Yuanheng Zhao 2bc9cb7f13 [Feat] Support online quantization (fp8/int8) for LTX-2 (#3700) 
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
 2026-05-21 19:52:21 +08:00
Liang Lvhxhhhlalalahyh_hh
 		8297570c2b [AutoRound] Support WAN2.2 W4A16 quantization model (#3353) 
Signed-off-by: lvliang-intel <liang1.lv@intel.com>
Signed-off-by: hyh_hh <huyinghong1@huawei.com>
Co-authored-by: hxhhhlalala <hyh_hh@163.com>
Co-authored-by: hyh_hh <huyinghong1@huawei.com>
 2026-05-21 17:54:41 +08:00
NumberWan 474710dda9 [Benchmark] Route i2i/ti2i to POST /v1/images/edits in diffusion_benchmark_serving (#3693) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
 2026-05-21 17:18:02 +08:00
Siyuan Kong 7d9d8aee79 [Bugfix] Fix qwen2_5_omni weight loading (#3598) 
Signed-off-by: ksiyuan <ksiyuan@umich.edu>
 2026-05-21 16:43:33 +08:00
Samit 083b5e363f [Diffusion] Support LoRA in step-wise execution (#3639) 
Signed-off-by: samithuang <285365963@qq.com>
 2026-05-20 22:26:18 +08:00
WeiQing Chen f74cba84ac [Docs] Complete quantization nav and online guide (#3764) 
Signed-off-by: David Chen <530634352@qq.com>
 2026-05-20 15:52:40 +08:00
Canlin Guo 2aafe0bf94 [Perf] Optimize sampler D2H sync for HY-Image (#3617) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-05-20 15:40:34 +08:00
bjf-frzandbjf-frz 5cf4605bf9 [Bugfix]Fix distributed stage0 multimodal cache routing (#3740) 
Signed-off-by: bjf-frz <240110042+bjf-frz@users.noreply.github.com>
Co-authored-by: bjf-frz <240110042+bjf-frz@users.noreply.github.com>
 2026-05-20 15:31:41 +08:00
wangyu e1c6bc7597 [Test] add run_nightly_jobs.sh for local nightly pytest parity (#3670) 
Signed-off-by: wangyu <410167048@qq.com>
 2026-05-20 15:26:59 +08:00
Yueqian Lin fc8486cf20 [Doc][TTS] CosyVoice3 online docs + residual TTS yaml cleanup + remove VoxCPM v1 (#3748) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-05-20 15:24:06 +08:00
yangsonglin13 1c530d984d Yuanrong TransferEngine Connector for NPU (#3180) 
Signed-off-by: yangsonglin13 <yangsonglin566@gmail.com>
 2026-05-20 14:51:33 +08:00
hxhhhlalalaandhyh_hh 9dd36e3afd [NPU][Quant] Add W4A4 MXFP4 online & MXFP4 dual-scale online/offline quantization support for Wan2.2 T2V / I2V inference on Ascend NPU (#3578) 
Signed-off-by: hyh_hh <huyinghong1@huawei.com>
Co-authored-by: hyh_hh <huyinghong1@huawei.com>
 2026-05-20 06:21:29 +00:00
Lancer 109bba8cdf [Bugfix] Treat kv_cache_dtype=auto as unset for ring attention (#3622) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2026-05-20 05:18:57 +00:00
Yueqian LinandHongsheng Liu f99e8b2d3c [Bench] Add audio-streaming continuity metric for TTS (#3618) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-05-20 05:08:56 +00:00
Nick Cao 06ab723afc [Perf][VoxCPM2][Ming-Flash-Omni] Use global CUDA graph pool (#3361) 
Signed-off-by: Nick Cao <ncao@redhat.com>
 2026-05-20 05:01:42 +00:00
Chendi.Xue 12bcc32c3d [XPU][CI] fix test_qwen2_5_omni_expansion.py::test_mix_to_audio (#3761) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
 2026-05-20 12:40:08 +08:00
Nick CaoandClaude d4e90bbba8 [Bugfix][TTS] Only populate voice_name for uploaded voices without inline ref_audio (#3523) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-20 04:36:59 +00:00
Y. Fisherandzzh 29179599fc [Feat] Enable VAE parallel in HunyuanImage3 (#3091) 
Signed-off-by: KexiongYu <yukexiong1@huawei.com>
Signed-off-by: zzh <943967662@qq.com>
Co-authored-by: zzh <943967662@qq.com>
 2026-05-20 09:55:33 +08:00
wangyuandHongsheng Liu 4e82192fd6 [CI] optional --assert-baseline and update perf JSON baselines (#3695) 
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-05-20 09:15:53 +08:00
WeiQing Chen a3d4ed809d [Quantization][tools] Add diffusion quantization output comparison tool (#3175) 
Signed-off-by: david6666666 <530634352@qq.com>
Signed-off-by: David Chen <530634352@qq.com>
 2026-05-20 00:14:08 +08:00
dengyunyang 3c58868c9a [BugFix] fix mult cli timeout with get kv (#3741) 
Signed-off-by: dengyunyang <584797741@qq.com>
 2026-05-19 14:22:27 +00:00
Shin da53618793 [Recipe] Qwen/Qwen-Image-Edit (#3684) 
Signed-off-by: yixiaoer <shin@yixiaoer.sg>
 2026-05-19 19:56:38 +08:00
Schatten 18186db216 [Cleanup] Remove unused build_base_engine_args after #1115 (#3720) 
Signed-off-by: Schatten <czhengt@qq.com>
 2026-05-19 19:23:45 +08:00
LuandSYLAR 14e5baceaf [Qwen-Image] Drop unused vision tower from text encoder (#3608) 
Signed-off-by: lulugoodcoder <luludachiever@gmail.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-05-19 19:19:58 +08:00
wangyu 2af2a50e0e [CI] improve Buildkite testcase statistics reports (#3543) 
Signed-off-by: wangyu <410167048@qq.com>
 2026-05-19 15:22:02 +08:00
wangyu bd83ac9b4b [CI] invalid_param reliability suite and weekly http_invalid jobs (#3652) 
Signed-off-by: wangyu <410167048@qq.com>
 2026-05-19 14:28:47 +08:00
chickeytonandherotai214 e277feacaf [large-scale-serving] Integrate OmniCoordinator into stage engine pipeline (#3569) 
Signed-off-by: chickeyton <ngton2014@gmail.com>
Signed-off-by: herotai214 <herotai214@gmail.com>
Co-authored-by: herotai214 <herotai214@gmail.com>
 2026-05-19 12:08:20 +08:00
JiaHong ca9fd0b71c Reject non-positive Flux2 Klein inference steps (#3717) 
Signed-off-by: MmMaiIIi <2360655509@qq.com>
 2026-05-19 11:52:41 +08:00
JiaHongandWeiQing Chen 3ac739817f [Bugfix] Reject empty prompts in Flux2 Klein diffusion pipeline (#3711) 
Signed-off-by: MmMaiIIi <2360655509@qq.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
 2026-05-19 11:30:54 +08:00
bjf-frzandbjfwhite 1fa734419e [Refactor]Rename diffusion benchmark backend to endpoint (#3137) 
Signed-off-by: bjf-frz <frz123db@gmail.com>
Signed-off-by: bjfwhite <baijingfan1@huawei.com>
Co-authored-by: bjfwhite <baijingfan1@huawei.com>
 2026-05-19 11:30:33 +08:00
fan2956 2c6b1bb0c0 [Bugfix] Fix hunyuanimage3 dit quant storageshape mismatch error (#3694) 
Signed-off-by: fan2956 <zhoufan53@huawei.com>
 2026-05-19 10:24:27 +08:00
Daniel Huang e2ed1c4574 Disable sampler kernel for XPU test (#3718) 
Signed-off-by: Daniel Huang <daniel1.huang@intel.com>
 2026-05-19 02:19:16 +00:00
Rustam Khadipash 89f8819525 [Feature] Add support for Pipeline Parallel and integrate it into Wan 2.2 (#2322) 
Signed-off-by: Rustam Khadipash <16683750+hadipash@users.noreply.github.com>
 2026-05-19 09:42:17 +08:00
Chendi.Xue 475a4002b0 [XPU] set flash_attn as default diffusion attn backend and fix k_len for cross_attn (#3525) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
 2026-05-18 23:58:15 +00:00
lishunyang 69bcc6416d [New Model] Lance (ByteDance) — unified AR+diffusion multimodal (#3697) 
Adds Lance (bytedance-research/Lance), a 3B unified AR+diffusion
multimodal model on a Qwen2.5-VL backbone. Lance is BAGEL-lineage:
it reuses vLLM-Omni's BAGEL Mixture-of-Transformers core verbatim and
specializes only the ViT (Qwen2.5-VL vision), the VAE (Wan2.2) and the
checkpoint layout.

All six single-stage Lance modalities verified end-to-end on 1x B300:
t2i, t2v, image_edit, video_edit, x2t_image (image understanding) and
x2t_video (video understanding). 3-D mRoPE is wired through
BagelRotaryEmbedding for the x2t / video / edit paths. Includes the
upstream gap-analysis quality fixes (task-specific system prompts, full
Qwen chat template, upstream inference defaults and aspect-ratio
buckets).

Ships an offline example (examples/offline_inference/lance/), an
OpenAI-compatible online-serving example
(examples/online_serving/lance/) and an e2e test
(tests/e2e/online_serving/test_lance.py).

See LANCE_PR.md for the architecture rationale, per-modality status and
the two remaining follow-ups (two-stage AR+DiT, video_edit polish).

Closes #3697.

Signed-off-by: lishunyang <lishunyang03@gmail.com>
 2026-05-18 22:09:14 +00:00
Sy03andamy-why-3459 ab59673f21 [Bugfix][Qwen3-Omni] Handle short Code2Wav chunk outputs (#3687) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-05-18 18:59:23 +00:00
lyj-jjjandCursor 821286794f [HY-Imgae3.0] support hunyuan image3 dit fa-fp8 on npu (#3540) 
Signed-off-by: lyj-jjj <liuyingjun5@huawei.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-05-18 09:35:30 -07:00
amy-why-3459 309e5c38c6 [BugFix][CI]Fixing occasional CI failures (#3623) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-05-18 21:25:37 +08:00
Ding Zuhaoandnussejzz f4115bd771 [Bugfix] Fix SenseNova U1 broken import after SupportsModuleOffload (#3691) 
Signed-off-by: nussejzz <nussejzz@users.noreply.github.com>
Co-authored-by: nussejzz <nussejzz@users.noreply.github.com>
 2026-05-18 21:12:46 +08:00
Lancer dbc589dbca [Bugfix] fix diffusion quantization benchmarking for Omni outputs (#3653) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2026-05-18 17:23:40 +08:00
Yueqian Lin 990566aef1 [Bugfix][TTS] Drop meaningless TTFT from speech-endpoint benchmarks (#3674) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-05-18 17:18:18 +08:00
Chendi.Xue 6d37e77fb2 [XPU] update dockerfile and CI to 0.21.0 (#3675) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
 2026-05-18 03:06:56 +00:00
wuhangandHongsheng Liu 4ba8e14981 Fix diffusion engine cleanup lifecycle (#3494) 
Signed-off-by: wuhang <wuhang6@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-05-18 08:54:57 +08:00
Sy03Hongsheng LiuSYLAR
 		c99df1ebd9 [TTS][Perf] Optimize Qwen3-TTS high-concurrency serving (#3662) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-05-17 16:58:59 -04:00
bjf-frz 0a395f9de1 [SKILL]Add diffusion perf skill (#3461) 
Signed-off-by: bjf-frz <frz123db@gmail.com>
 2026-05-18 00:59:46 +08:00
Hongsheng Liuanddeepseek-v4-pro c0e132d973 [Doc] Reorganize available recipes into a table (#3671) 
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
Co-authored-by: deepseek-v4-pro <noreply@anthropic.com>
 2026-05-18 00:02:34 +08:00