Commit Graph
1807 Commits
Author SHA1 Message Date
lishunyang12 c15f51a2f6 Merge branch 'main' into lance-model-support; drop lance.yaml
Resolves conflicts in:
- vllm_omni/diffusion/registry.py: keep both LancePipeline (HEAD)
  and MingImagePipeline (main) entries.
- vllm_omni/diffusion/models/bagel/bagel_transformer.py: take main's
  simpler cfg_batched payload; HEAD referenced undefined locals.

Also strips vllm_omni/deploy/lance.yaml -- Lance is single-stage
diffusion, so its engine knobs (pipeline=lance, enforce_eager,
trust_remote_code, max_num_seqs=1, ...) flow directly to Omni(...)
/ vllm-omni serve instead of through a deploy YAML.
create_default_diffusion builds the stage config from those kwargs.

Touched: examples/offline_inference/lance/{end2end,gradio_demo}.py,
tests/e2e/online_serving/test_lance.py, tests/helpers/stage_config.py.

Signed-off-by: lishunyang12 <lishunyang12@163.com>
2026-05-31 11:50:21 +00:00
lishunyang12 7c934b6188 [Lance] Add Gradio demo and align inference with upstream
- Add ``examples/offline_inference/lance/gradio_demo.py`` — a unified 7-task UI
  (t2i / t2v / i2v / image_edit / video_edit / x2t_image / x2t_video) that
  loads ``Lance_3B`` and ``Lance_3B_Video`` on two GPUs so image and video
  tasks run concurrently.  Surfaces official Lance examples per task; aspect
  ratio is auto-snapped to one of upstream's 6 buckets from the input
  image/video.
- Fix x2t_image / x2t_video output format: route x2t_image through a new
  ``_forward_x2t_image`` handler (was falling through to the img2img path
  and emitting garbage), build the chat prefill in three segments
  (prefix → ViT → suffix) so the ``<|im_end|>`` markers line up with
  upstream's template, and start generation from a newline token instead of
  ``<|im_start|>`` so the assistant turn doesn't close itself with an
  immediate EOS.  Add a ``system_prompt`` override to ``render_lance_prompt``
  so per-example QA instructions surface ("Look at the image carefully and
  answer the question.") instead of the caption-style default.
- ``_forward_image_edit``: also stash the PIL image into ``custom_output``
  so callers reading ``outputs[0].output`` *or* ``custom_output["image"]``
  both work — the orchestrator IPC strips the bare ``output`` field across
  process boundaries.
- ``pipeline_registry.py``, ``model_executor/models/lance/*``, registry
  cleanup: remove the dead Lance-specific autoregressive shim
  (``lance.py``) and its stage-input processor; the diffusion pipeline owns
  the full flow now.
- Aspect-ratio buckets aligned with upstream's ``BucketResize`` (6 ratios:
  21:9 / 16:9 / 4:3 / 1:1 / 3:4 / 9:16, sized for ``resolution_vae`` ∈
  {480, 640, 768}).  3:2 / 2:3 dropped — upstream snaps those to 4:3 / 3:4.
- Gradio defaults match upstream's official scripts (cfg_text_scale=4.0,
  validation_num_timesteps=30, validation_timestep_shift=3.5, seed=42); only
  the demo *duration* is intentionally shorter than upstream's 121-frame
  default to keep the UI snappy (user can extend to 10s via slider for
  byte-identical reproduction).

Signed-off-by: lishunyang12 <lishunyang12@163.com>

Signed-off-by: WeiQing Chen <david6666666@users.noreply.github.com>
2026-05-31 11:40:40 +00:00
ChenWenjing 5dfdf584dd [Feat]audio streaming input for async chunk (#3614)
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
2026-05-31 17:39:25 +08:00
Hacoandzzhuoxin1508 74050cdacd [BugFix] Fix diffusion parallel_config YAML override and add deploy config field allowlist (#3483)
Signed-off-by: xiaohajiayou <923390377@qq.com>
Co-authored-by: zzhuoxin1508 <zzhuoxin1508@users.noreply.github.com>

Signed-off-by: WeiQing Chen <david6666666@users.noreply.github.com>
2026-05-31 08:58:52 +00:00
NumberWan 6f4bd3e45c [BugFix]Qwen-Image performance regression by using omni RMSNorm(RMSNorm backend) (#3933)
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
2026-05-31 08:06:18 +08:00
Canlin Guo 412bf42a75 [Refactor] Unify _talker_mtp_forward across GPU and NPU model runners (#3476)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
2026-05-31 07:58:00 +08:00
zzh ffd9771c54 Fix hunyuan resolve stop token ids (#3896)
Signed-off-by: zzh <943967662@qq.com>
2026-05-31 07:47:36 +08:00
DMAN 11c4fcedd3 [Recipe] add mistralai voxtral tts recipe (#3498)
Signed-off-by: Dmaner <2663515256@qq.com>
2026-05-31 00:18:52 +08:00
wuhang b0f3672aec [Bugfix] Fix DiffusionWorker crash on SIGINT: ensure NCCL/ZMQ cleanup on shutdown (#3872)
Signed-off-by: WU Hang <whlbx@hotmail.com>
2026-05-31 00:13:23 +08:00
WeiQing Chen 2289478e3a [CI] Temporarily skip failing Bagel connector tests (#4005)
Signed-off-by: David Chen <530634352@qq.com>
2026-05-30 17:46:29 +08:00
Alex BrooksandHongsheng Liu 8d13b54383 [CI/Bugfix] Async Request ID Aliasing (#3953)
Signed-off-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-05-30 16:59:42 +08:00
wuhangandAlicia 802a6242f0 [Bugfix][Tests] Remove unnecessary device map in tests init (#3958)
Signed-off-by: WU Hang <whlbx@hotmail.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
2026-05-30 16:18:48 +08:00
Yuanheng ZhaoandZheng Wengang ea6d811b58 [Config Refactor] Migrate Ming-flash-omni-2.0 Image-Gen deploy configs (#3975)
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Co-authored-by: Zheng Wengang <zwg0606@gmail.com>
2026-05-30 08:13:21 +00:00
WeiQing Chen 5994a0bbf5 docs: update WeChat QR code (#4003)
Signed-off-by: David Chen <530634352@qq.com>
2026-05-30 16:12:11 +08:00
Viacheslav KlimkovandYueqian Lin 914e92cf80 [Refactor][Qwen3-TTS] Extract reusable prompt-embeds builder and make tts_pad_embed a persistent buffer (#3992)
Signed-off-by: Viacheslav Klimkov <vklimkov@nvidia.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
2026-05-30 16:11:00 +08:00
dengyunyang 3b317285c4 [HunyuanImage][Perf] adapt to deploy config changes (#3996)
Signed-off-by: dengyunyang <584797741@qq.com>
2026-05-30 14:48:50 +08:00
WeiQing Chen a3456752c5 [Diffusion] Support USP and VAE patch parallel for HunyuanVideo 1.5 (#3979)
Signed-off-by: david6666666 <530634352@qq.com>
2026-05-30 14:40:42 +08:00
Chendi.Xue f43bb124bc [CI][XPU]Fix sage_attn hard-code import for cuda (#3994)
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
2026-05-30 09:43:14 +08:00
fywc 272e626100 [Feat] support cache-dit for SenseNova-U1 (#3906)
Signed-off-by: hanzheli <hanzheli@kuaishou.com>
2026-05-29 17:40:54 +00:00
knlnguyen1802andSamit 5377108438 [minor, fix] Allow passing class interface as custom pipeline argument (#2973)
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Samit <285365963@qq.com>
2026-05-29 23:39:38 +08:00
nperraud 9fdb3bcb89 [Bugfix][Model] Qwen3-TTS: don't collapse 2D ref_code list when estimating prompt length (#3940)
Signed-off-by: nperraud <nathanael.perraudin@gmail.com>
2026-05-29 09:18:25 -06:00
Chen ChengandClaude Opus 4.7 f1c2caa3c7 [Perf][Qwen3-TTS] Restore Code2Wav cross-request batching (RFC #3163 P0) (#3322)
Signed-off-by: ischencheng <cheng21@seas.upenn.edu>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-29 08:48:58 -06:00
b690a10e3a [Feat] Support MagCache (#1287)
Signed-off-by: Lancer <maruixiang6688@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
2026-05-29 22:26:23 +08:00
Aleksi VesantoandHongsheng Liu eba5617c95 [ROCm] Enable AITER backend with ring attention (#3511)
Signed-off-by: Aleksi Vesanto <avesanto@amd.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-05-29 22:20:29 +08:00
amy-why-3459andHongsheng Liu 2696751003 [Perf] Qwen3-Omni performance optimization (#3878)
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-05-29 22:15:23 +08:00
Alicia 8c4a42b16c Temp skip TEST - Entrypoint Test with H100 (#3989)
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
2026-05-29 22:37:09 +12:00
Huanyu Yang 3457758114 [Bugfix] Fix IndexError in DistributedVaeExecutor when vae_patch_parallel_size < world_size (#3928)
Signed-off-by: QingZhou-YangHY <3868850350@qq.com>
2026-05-29 03:17:34 -07:00
WeiQing Chen 67bb9b4a12 [CI][Bugfix] Fix LTX audio-video warmup output typing (#3964)
Signed-off-by: david6666666 <530634352@qq.com>
2026-05-29 07:46:02 +00:00
boatmanandsphinxkkkbc 4d9262a6eb [Feat]Support voice clone for omnivoice in online serving & add seed parameter for reproducible (#3668)
Signed-off-by: sphinxkkkbc <binchengkang8@gmail.com>
Co-authored-by: sphinxkkkbc <binchengkang8@gmail.com>
2026-05-29 07:42:08 +00:00
c69ed4037d [feat] Support prompt embedding caching for diffusion model (#2962)
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-05-29 15:36:03 +08:00
Alex Brooks 68a1d23e4d [Bugfix/Feature] Remove Hardcoded Flash Attention in Bagel & Support GQA in SDPA Backend (#3728)
Signed-off-by: Alex Brooks <albrooks@redhat.com>
2026-05-29 10:36:15 +08:00
Chen-Yo Sun 593c4c06aa [BugFix] Avoid Voxtral TTS loading error msg (#3951)
Signed-off-by: Chen-Yo Sun <chenyo.sun@mistral.ai>
2026-05-29 02:35:01 +00:00
NATURE 4987bec425 [bugfix] Solve Nightly / CI failed - tests/e2e/online_serving/test_bagel_expansion.py #3918 (#3936)
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
2026-05-29 02:05:07 +00:00
amy-why-3459 5dd1d0a772 [BugFix] Fix prefix-caching issue (#3726)
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
2026-05-29 01:51:54 +00:00
ChenWenjingandGao Han 834abdaf8b [ci][nightly] Voxcpm2 performance benchmark (#3864)
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
2026-05-29 09:44:29 +08:00
Jason 5dcf18a984 [Docs] Add recipe for Helios (#3114)
Signed-off-by: JasonJ2021 <jasonj@zju.edu.cn>
2026-05-29 09:34:37 +08:00
Yuxiang Liang 900b4187a1 [XPU] Add CosyVoice3Model support on Intel XPU (#2325)
Signed-off-by: Yuxiang Liang <yuliang@habana.ai>
Signed-off-by: Yuxiang Liang <yuxiang.liang@intel.com>
2026-05-29 00:59:45 +00:00
wuli666andwuli666 8935439428 [Bugfix] Fix Triton SnakeBeta kernel for bf16/fp16 inputs (#3472)
Signed-off-by: wuli666 <djjpro975@gmail.com>
Co-authored-by: wuli666 <djjpro975@gmail.com>
2026-05-28 17:10:36 +00:00
Dakai An bb9f21d0db Fix Qwen3-omni accuracy degradation from deepstack inputs under torch.compile (#3885)
Signed-off-by: Dakai An <dakaian108@gmail.com>
2026-05-28 23:57:56 +08:00
knlnguyen1802andSamit 5eac50f22c [bugfix, rl] Fix sleep do not release full memory in custom pipeline (#3818)
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Co-authored-by: Samit <285365963@qq.com>
2026-05-28 22:27:27 +08:00
126fb4b4a9 [Blackwell][1/N] Add SageAttention3 diffusion backend on blackwell(GB200/B200/RTX5090/PRO6000/DGX Spark available) (#3015)
Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: lishunyang <lishunyang12@163.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
2026-05-28 14:08:45 +00:00
zhumingjue138 8d9a4afed9 [Test] Add scenarios for L5 reliability test (#3729)
Signed-off-by: zhumingjue <zhumingjue@huawei.com>
2026-05-28 21:41:33 +08:00
WeiQing ChenandWeiQing Chen 1acd27c205 [CI][BugFix] Fix and Validate FP8 Z-Image quality gate (#3929)
Signed-off-by: WeiQing Chen <david6666666@users.noreply.github.com>
Co-authored-by: WeiQing Chen <david6666666@users.noreply.github.com>
2026-05-28 12:52:58 +00:00
nainiu258andHongsheng Liu 740ec608f4 [Docs]Add recipe for GLM-Image on 2x A800 GPUs and 1x A800 GPU (#2950)
Signed-off-by: nainiu258 <cperfect02@163.com>
Signed-off-by: nainiu258 <101917677+nainiu258@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-05-28 19:50:37 +08:00
Zeyu Huang | 黃澤宇 74258b18b7 [CI] Rectify L2~L4 Qwen Image Edit series tests (#3901) 2026-05-28 18:33:31 +08:00
rein yang 1c876f0cd1 [Perf] Trim Code2Wav CUDA Graph buckets for Qwen3-TTS single-GPU deploy (#3932)
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
2026-05-28 17:26:57 +08:00
mglyn ee212d12b1 [CI][Bugfix] Add request id to LTX2.3 CFG parallel test (#3934)
Signed-off-by: mglyn <1203789601@qq.com>
2026-05-28 17:26:16 +08:00
Nick CaoandClaude c20afebce4 [Refactor][Qwen3-TTS] Construct speech tokenizer encoder natively (#3360)
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-05-28 16:20:49 +08:00
WeiQing Chen 0e3bf4de8e [Doc] Reduce browser memory usage for docs (#3870)
Signed-off-by: David Chen <530634352@qq.com>
2026-05-28 15:42:09 +08:00
Liang Lv b72ffb82bb [AutoRound] Support GLM-Image W4A16 quantization model (#3059)
Signed-off-by: lvliang-intel <liang1.lv@intel.com>
2026-05-28 07:19:06 +00:00