anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
1,692 Commits 235 Branches 31 Tags
f0cec3ecd4e1991bd1cf9c0a6bbad9d0fe80480b
Commit Graph
465 Commits
Author SHA1 Message Date
SYLAR f0cec3ecd4 Merge branch 'main' into lance-model-support 2026-05-22 04:13:58 +08:00
lishunyang12 7b885f9d13 [Lance] Precision matching, recipe, and README cleanup 
- Precision matching across the Lance transformer and pipeline so vLLM-Omni
  outputs match the upstream reference end-to-end (text2img / img2img /
  text2video / video2video / image and video understanding).
- Add recipes/ByteDance/Lance.md covering all single-stage modalities for
  offline + online; register it in recipes/README.md.
- Offline README: document --height/--width and update the default
  resolution to 1024x1024 (matches the upstream inference_lance.sh).
- Gitignore lance_compare/ (local precision-matching dump scripts; not part
  of the repo).
- Remove LANCE_PR.md; the PR description lives on GitHub.

Signed-off-by: lishunyang12 <lishunyang12@163.com>
 2026-05-21 20:07:40 +00:00
Yuanheng Zhao 2bc9cb7f13 [Feat] Support online quantization (fp8/int8) for LTX-2 (#3700) 
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
 2026-05-21 19:52:21 +08:00
Liang Lvhxhhhlalalahyh_hh
 		8297570c2b [AutoRound] Support WAN2.2 W4A16 quantization model (#3353) 
Signed-off-by: lvliang-intel <liang1.lv@intel.com>
Signed-off-by: hyh_hh <huyinghong1@huawei.com>
Co-authored-by: hxhhhlalala <hyh_hh@163.com>
Co-authored-by: hyh_hh <huyinghong1@huawei.com>
 2026-05-21 17:54:41 +08:00
Samit 083b5e363f [Diffusion] Support LoRA in step-wise execution (#3639) 
Signed-off-by: samithuang <285365963@qq.com>
 2026-05-20 22:26:18 +08:00
hxhhhlalalaandhyh_hh 9dd36e3afd [NPU][Quant] Add W4A4 MXFP4 online & MXFP4 dual-scale online/offline quantization support for Wan2.2 T2V / I2V inference on Ascend NPU (#3578) 
Signed-off-by: hyh_hh <huyinghong1@huawei.com>
Co-authored-by: hyh_hh <huyinghong1@huawei.com>
 2026-05-20 06:21:29 +00:00
Lancer 109bba8cdf [Bugfix] Treat kv_cache_dtype=auto as unset for ring attention (#3622) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2026-05-20 05:18:57 +00:00
Y. Fisherandzzh 29179599fc [Feat] Enable VAE parallel in HunyuanImage3 (#3091) 
Signed-off-by: KexiongYu <yukexiong1@huawei.com>
Signed-off-by: zzh <943967662@qq.com>
Co-authored-by: zzh <943967662@qq.com>
 2026-05-20 09:55:33 +08:00
LuandSYLAR 14e5baceaf [Qwen-Image] Drop unused vision tower from text encoder (#3608) 
Signed-off-by: lulugoodcoder <luludachiever@gmail.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-05-19 19:19:58 +08:00
chickeytonandherotai214 e277feacaf [large-scale-serving] Integrate OmniCoordinator into stage engine pipeline (#3569) 
Signed-off-by: chickeyton <ngton2014@gmail.com>
Signed-off-by: herotai214 <herotai214@gmail.com>
Co-authored-by: herotai214 <herotai214@gmail.com>
 2026-05-19 12:08:20 +08:00
JiaHong ca9fd0b71c Reject non-positive Flux2 Klein inference steps (#3717) 
Signed-off-by: MmMaiIIi <2360655509@qq.com>
 2026-05-19 11:52:41 +08:00
JiaHongandWeiQing Chen 3ac739817f [Bugfix] Reject empty prompts in Flux2 Klein diffusion pipeline (#3711) 
Signed-off-by: MmMaiIIi <2360655509@qq.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
 2026-05-19 11:30:54 +08:00
Rustam Khadipash 89f8819525 [Feature] Add support for Pipeline Parallel and integrate it into Wan 2.2 (#2322) 
Signed-off-by: Rustam Khadipash <16683750+hadipash@users.noreply.github.com>
 2026-05-19 09:42:17 +08:00
Chendi.Xue 475a4002b0 [XPU] set flash_attn as default diffusion attn backend and fix k_len for cross_attn (#3525) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
 2026-05-18 23:58:15 +00:00
lishunyang 69bcc6416d [New Model] Lance (ByteDance) — unified AR+diffusion multimodal (#3697) 
Adds Lance (bytedance-research/Lance), a 3B unified AR+diffusion
multimodal model on a Qwen2.5-VL backbone. Lance is BAGEL-lineage:
it reuses vLLM-Omni's BAGEL Mixture-of-Transformers core verbatim and
specializes only the ViT (Qwen2.5-VL vision), the VAE (Wan2.2) and the
checkpoint layout.

All six single-stage Lance modalities verified end-to-end on 1x B300:
t2i, t2v, image_edit, video_edit, x2t_image (image understanding) and
x2t_video (video understanding). 3-D mRoPE is wired through
BagelRotaryEmbedding for the x2t / video / edit paths. Includes the
upstream gap-analysis quality fixes (task-specific system prompts, full
Qwen chat template, upstream inference defaults and aspect-ratio
buckets).

Ships an offline example (examples/offline_inference/lance/), an
OpenAI-compatible online-serving example
(examples/online_serving/lance/) and an e2e test
(tests/e2e/online_serving/test_lance.py).

See LANCE_PR.md for the architecture rationale, per-modality status and
the two remaining follow-ups (two-stage AR+DiT, video_edit polish).

Closes #3697.

Signed-off-by: lishunyang <lishunyang03@gmail.com>
 2026-05-18 22:09:14 +00:00
lyj-jjjandCursor 821286794f [HY-Imgae3.0] support hunyuan image3 dit fa-fp8 on npu (#3540) 
Signed-off-by: lyj-jjj <liuyingjun5@huawei.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-05-18 09:35:30 -07:00
Ding Zuhaoandnussejzz f4115bd771 [Bugfix] Fix SenseNova U1 broken import after SupportsModuleOffload (#3691) 
Signed-off-by: nussejzz <nussejzz@users.noreply.github.com>
Co-authored-by: nussejzz <nussejzz@users.noreply.github.com>
 2026-05-18 21:12:46 +08:00
wuhangandHongsheng Liu 4ba8e14981 Fix diffusion engine cleanup lifecycle (#3494) 
Signed-off-by: wuhang <wuhang6@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-05-18 08:54:57 +08:00
Yuanheng Zhao 220db62b3f [Bugfix] Adapt LTX-2 connector arg with diffusers 0.38.0 (#3661) 
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
 2026-05-17 21:58:44 +08:00
Yuanheng Zhao a64ebf103b [Refactor] Migrate and clean up TTS configs: CosyVoice3, OmniVoice, VoxCPM (#3338) 
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
 2026-05-17 07:32:34 +08:00
rongfu.leng c5ac295e3c [Feat] Add helios support cache dit (#3470) 
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
 2026-05-16 21:11:34 +08:00
Zeng ChuangandHongsheng Liu ea35a0cc4a [Bugfix]update process name for dit stage (#3602) 
Signed-off-by: zengchuang <zengchuang3@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-05-16 20:51:31 +08:00
wuhangandHongsheng Liu 0f4853ff86 [Bugfix] Support diffusion worker dead detect when use inline engine (#3214) 
Signed-off-by: wuhang <wuhang6@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-05-16 20:50:29 +08:00
+20
Zhou TaichangClaude Opus 4.7NumberWandsinghviDnoobYueqian LinHongsheng Liuknlnguyen1802Samitrein yangNick Caozhumingjue138Ricardo Noriegalyj-jjjCursorgcanlinwangyuweizhoublueweizhoubluedengyunyang汪志鹏WeiQing ChenJie LiuYueqian LinNATUREbjf-frzamy-why-3459Y. FisherZeyu Huang | 黃澤宇Copilot Autofix powered by AI
 		eb4e60ee64 [Rebase] Rebase to vllm v0.21.0 (#3530) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Signed-off-by: Dnoob <dxpouo@gmail.com>
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Signed-off-by: rein yang <ruiruyang2@gmail.com>
Signed-off-by: Nick Cao <ncao@redhat.com>
Signed-off-by: zhumingjue <zhumingjue@huawei.com>
Signed-off-by: Ricardo Noriega De Soto <rnoriega@redhat.com>
Signed-off-by: lyj-jjj <liuyingjun5@huawei.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: weizhoublue <weizhoublue@github.com>
Signed-off-by: weizhou.lan@daocloud.io <weizhou.lan@daocloud.io>
Signed-off-by: dengyunyang <584797741@qq.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: Jie Liu <33612777+keeper-jie@users.noreply.github.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
Signed-off-by: bjf-frz <frz123db@gmail.com>
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Signed-off-by: KexiongYu <yukexiong1@huawei.com>
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
Signed-off-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: NumberWan <wantszkin2003@gmail.com>
Co-authored-by: dsinghvi <divyanshsinghvi@gmail.com>
Co-authored-by: Dnoob <dxpouo@gmail.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: rein yang <73573651+R2-Y@users.noreply.github.com>
Co-authored-by: Nick Cao <ncao@redhat.com>
Co-authored-by: zhumingjue138 <zhumingjue@huawei.com>
Co-authored-by: Ricardo Noriega <rnoriega@redhat.com>
Co-authored-by: lyj-jjj <liuyingjun5@huawei.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Co-authored-by: weizhoublue <45163302+weizhoublue@users.noreply.github.com>
Co-authored-by: weizhoublue <weizhoublue@github.com>
Co-authored-by: dengyunyang <584797741@qq.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: Jie Liu <33612777+keeper-jie@users.noreply.github.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: NATURE <wzliu@connect.hku.hk>
Co-authored-by: bjf-frz <frz123db@gmail.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Y. Fisher <yukexiong1@huawei.com>
Co-authored-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
 2026-05-16 18:18:43 +08:00
Zeyu Huang | 黃澤宇andCopilot Autofix powered by AI d18168ccbb [bugfix] Fix diffusers backend input bug after #2913 (#3644) 
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
Signed-off-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
 2026-05-15 09:55:09 +00:00
lyj-jjjandCursor bbc00f9f86 [BugFix] fix(omni): isolate diffusion KV-cache dtype from vLLM --kv-cache-dtype #3585 (#3596) 
Signed-off-by: lyj-jjj <liuyingjun5@huawei.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-05-15 11:22:59 +08:00
TaffyOfficialTaffyOfficialTaffyOfficialskf1999
 		3f63aaf982 [Feature] HunyuanImage-3.0 IT2I: multi-image input + prompt API cleanup (#3444) 
Signed-off-by: TaffyOfficial <2324465096@qq.com>
Signed-off-by: TaffyOfficial <wu15922848573@outlook.com>
Signed-off-by: skf1999 <13234016272@163.com>
Signed-off-by: zuiho <2324465096@qq.com>
Signed-off-by: Claude Code <noreply@anthropic.com>
Signed-off-by: zuiho <wu15922848573@outlook.com>
Signed-off-by: TaffyOfficial <2587297563@qq.com>
Co-authored-by: TaffyOfficial <2324465096@qq.com>
Co-authored-by: TaffyOfficial <wu15922848573@outlook.com>
Co-authored-by: skf1999 <13234016272@163.com>
 2026-05-14 16:58:40 +08:00
wuhang 0d9d57acd9 [Entrypoint][Refactor] Make field type hint more concrete (#3139) 
Signed-off-by: wuhang <wuhang6@huawei.com>
 2026-05-14 11:25:55 +08:00
Alex Brooks 754d2e52fc [BugFix] Refresh TeaCache when num_inference_steps=None (#2240) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-05-13 20:20:18 +00:00
vraitiandClaude Opus 4.6 9de9d1f7b5 [Model] Add TP-aware MistralEncoder for FLUX.2-dev TP (#2465) 
Signed-off-by: vraiti <vraiti@redhat.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-05-13 12:33:55 -07:00
dengyunyang efd955674b [Bugfix][HunyuanImage3.0] Fix KV reuse compatibility in SP scenarios (#3546) 
Signed-off-by: dengyunyang <584797741@qq.com>
 2026-05-13 22:40:18 +08:00
Y. Fisher 4d3eed152a [Feat][Config] Support additional_config for diffusion worker (#3020) 
Signed-off-by: KexiongYu <yukexiong1@huawei.com>
Signed-off-by: Y. Fisher <yukexiong1@huawei.com>
 2026-05-13 14:22:43 +00:00
Zeng Chuang 16a84b29d5 [Bugfix] Add bot_task option of think_recaption for hunyuanimage3 it2i (#3551) 
Signed-off-by: zengchuang <zengchuang3@huawei.com>
 2026-05-13 22:10:35 +08:00
TJian b9cb57b631 [ROCm] Bugfix wan22 (#3463) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-05-13 20:50:57 +08:00
hxhhhlalalaandhyh_hh a715abd447 [NPU][Quant] Add W8A8 MXFP8 online/offline quantization support for Wan2.2 T2V / I2V / TI2V inference on Ascend NPU (#3140) 
Signed-off-by: hyh_hh <huyinghong1@huawei.com>
Co-authored-by: hyh_hh <huyinghong1@huawei.com>
 2026-05-13 17:14:42 +08:00
lyj-jjjCursorgcanlin
 		56ca7dd612 support online FP8 quantization for FA on NPU #2236 (#2640) 
Signed-off-by: lyj-jjj <liuyingjun5@huawei.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: gcanlin <canlinguosdu@gmail.com>
 2026-05-13 11:46:00 +08:00
knlnguyen1802andSamit aa1184d737 [bugfix, rl] Fix race condition bug on async running for diffusion model (#3379) 
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-05-12 14:55:30 +08:00
dengyunyangandHongsheng Liu 0d91fbbbb7 [Bugfix] Align the AR and DiT prompt formatting across both online and offline modes. (#3516) 
Signed-off-by: dengyunyang <584797741@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-05-12 12:13:30 +08:00
Lancer 955fcff828 [Chore] explicit .float() conversion in Helios's optimized_scale function (#3529) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2026-05-12 02:08:28 +00:00
Junhong Liu 6be59f7d19 [Fix] Fix RMSNorm inductor KeyError under HSDP + torch.compile (#3460) 
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
 2026-05-11 22:56:54 +08:00
Y. Fisher a33e2eb588 [Config] Add HunyuanImage3 deploy configs (#3172) 
Signed-off-by: KexiongYu <yukexiong1@huawei.com>
Signed-off-by: Y. Fisher <yukexiong1@huawei.com>
 2026-05-11 22:49:19 +08:00
汪志鹏 c9a8556c24 [New Model]: Add sensenova u1 support (#3319) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-05-11 22:19:00 +08:00
wuhang 3f27ffbd4d [Misc] Clean logs for image gen task (#3414) 
Signed-off-by: wuhang <wuhang6@huawei.com>
 2026-05-11 12:16:05 +00:00
Canlin Guo 5e263b6929 [BugFix] Rename attention_config to diffusion_attention_config (#3489) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-05-11 11:27:35 +08:00
dengyunyang 857356d5b7 [Feature] hunyuanimage support flash attn (#2981) 
Signed-off-by: dengyunyang <584797741@qq.com>
 2026-05-10 22:08:24 +08:00
Canlin GuoandHongsheng Liu 11c4c7f0ff [Diffusion][Attention] Support per-role attention backend via CLI (#2681) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-05-10 11:59:05 +08:00
baonudesifeizhaiandroG0d c4a0990044 （Phase 1）Add ModelOpt FP8 auto-detect support for diffusion checkpoints #2709 (#2913) 
Signed-off-by: roG0d <rodgarcas98@gmail.com>
Signed-off-by: roG0d <baonudesifeizhai@gmail.com>
Signed-off-by: baonudesifeizhai <85092850+baonudesifeizhai@users.noreply.github.com>
Co-authored-by: roG0d <rodgarcas98@gmail.com>
 2026-05-09 16:55:05 +08:00
Lancer de3a291710 [Bugfix] fix OmniGen2 offload and dtype mismatch (#2560) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
Signed-off-by: Lancer <402430575@qq.com>
 2026-05-09 09:13:37 +08:00
rongfu.leng b4ab37da22 [Bugfix] Qwen-Image use teachche serve will crash (#3450) 
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
 2026-05-09 06:41:59 +08:00
Yueqian Lin c2a624bec4 [Bugfix][StableAudio] Pass model_class_name to Omni() and declare audio class attrs (#3405) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-05-09 06:40:43 +08:00