anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
3,228 Commits 235 Branches 31 Tags
44ea27c8094095bbffd88fa3befdfaa55ba4bc50
Commit Graph
3228 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
wtz2333 9fbc36ab2d [AR-Diffusion] Bound KV metadata for long rollouts (#7498) 
Signed-off-by: wtz2333 <2955110911@qq.com>
 2026-09-18 12:06:02 +08:00
andyluo7haic0Cursor
 		1c7476ec19 [CI/Build][ROCm] Stabilize shared AMD test lanes (#7706) 
Signed-off-by: andyluo7 <andy.luo@amd.com>
Signed-off-by: haic0 <haic0@users.noreply.github.com>
Co-authored-by: haic0 <haic0@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-18 11:13:07 +08:00
Isotr0py 3affc7766d Enable VAE tiling for Qwen-Image-2.1 (#5) 
* init

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix

Signed-off-by: Isotr0py <mozf@inferact.ai>

* test(diffusion): add Qwen-Image-2.1 unit tests

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix(diffusion): Qwen-Image-2.1 VAE spatial ratio, tiling e2e verification

- AutoencoderKLQwenImage21: derive spatial_compression_ratio from the
  architecture (2 ** len(temperal_downsample) = 16x) instead of trusting
  the checkpoint's scale_factor_spatial=8, which corrupted tiled
  decode/encode geometry (output cropped to half size) and tiling gating.
- pipeline: _read_vae_scale_factor prefers the architecture-derived value
  for the same reason.
- models/qwen_image_21/__init__: lazy exports to break the circular import
  distributed VAE -> package __init__ -> pipeline -> distributed VAE.
- Add VAE unit tests: ratio derivation, tiled vs untiled decode/encode
  consistency, distributed tile_split/exec/merge == tiled_decode.
- e2e matrix (GB200, seed 42, 20 steps) all green; see e2e_out/qwen21_vae/.
- examples: qwen_image_21_edit.py gains --vae-use-tiling.
- docs: mark VAE patch-parallel verified for Qwen-Image-2.1; recipe notes
  the default-tile banding limitation at 2048px+.

* clean tests

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix cudnn determination

Signed-off-by: Isotr0py <mozf@inferact.ai>

---------

Signed-off-by: Isotr0py <mozf@inferact.ai>
 2026-09-18 10:02:50 +08:00
Isotr0py 4fc3cf8c60 Enable CUDA Graph for DiT decode layers (#4) 
* init

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix

Signed-off-by: Isotr0py <mozf@inferact.ai>

* test(diffusion): add Qwen-Image-2.1 unit tests

Signed-off-by: Isotr0py <mozf@inferact.ai>

* feat(qwen_image_21): opt-in CUDA graph capture for KV-cache decode steps

Decode steps (1..N) recompute only the target image's tokens at a fixed
shape, but the per-request dict KV cache made addresses unstable, so the
step could not be captured. Prefill now copies prefix K/V into persistent
fixed-address buffers (keyed by branch/batch/prefix_len/target_tokens/
dtype/device/backend) and re-points the caller's cache dicts at them;
decode replays a graph captured per key over the full 32-block forward.
Eager fallback (logged) for non-CUDA, SP/ring/TP>1, HSDP, compiled or
offload-hooked blocks, KV-cache quantization, dynamic LoRA wrappers,
padded text masks, unknown keys, and capture failure.

Off by default; opt in via OmniDiffusionConfig.enable_cuda_graph_decode
or the QwenImage21Transformer2DModel constructor kwarg.

E2E (GB200, T2I 1024x1024, 50 steps, CFG 4.0, CUDNN_ATTN):
171.2 -> 142.1 ms/step, 8.70s -> 7.24s total (1.20x), 44.9 dB PSNR
vs eager output.

* clean

Signed-off-by: Isotr0py <mozf@inferact.ai>

---------

Signed-off-by: Isotr0py <mozf@inferact.ai>
 2026-09-18 10:02:39 +08:00
Zhenqiang Xuandzqxu 242c3cddeb [Hardware][Ascend] Integrate MindIE dense and sparse quantized attention for Wan2.2 T2V (#7561) 
Signed-off-by: zqxu <xuzhenqiang3@huawei.com>
Co-authored-by: zqxu <xuzhenqiang3@huawei.com>
 2026-09-18 01:28:27 +00:00
Anjie Hou f8197e120f [5/N] Schedule chunk transfers as their own stage (#7406) 
Signed-off-by: specture724 <specture724@gmail.com>
 2026-09-18 09:27:56 +08:00
Alex Brooks 6bda549ac2 [Bugfix] Fix Parallel State Initialization For MoE + Diffusion (#7676) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-09-17 17:15:37 -06:00
Joshna-Medisetty 77d8de70da [XPU][HunyuanImage3] Fix HunyuanImage-3.0 text-to-image on XPU (#7674) 
Signed-off-by: Joshna Medisetty <joshna.medisetty@intel.com>
 2026-09-17 19:23:27 +00:00
Nick CaoandCodex b81f2c5d4b [Misc] treewide: remove shm_threshold_bytes from deploy config (#7522) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-17 19:22:14 +00:00
dongbo910220 a6570b6814 [Diffusion] Fuse ERNIE-Image Q/K RoPE for eager execution (#7502) 
Signed-off-by: dongbo910220 <1275604947@qq.com>
 2026-09-17 19:06:45 +00:00
BeatSeat 9cdc57d2e7 [Bugfix][MAGI-2] Fix layerwise offload OOM by pointing offload block attrs to 'block' (#7523) (#7540) 
Signed-off-by: BeatSeat <wendavid552@gmail.com>
 2026-09-17 19:02:08 +00:00
dongbo910220 97912de8d8 [Kernel] Fuse LongCat paired Q/K RoPE (#7500) 
Signed-off-by: dongbo910220 <1275604947@qq.com>
 2026-09-18 02:27:57 +08:00
Anjie HouClaude Opus 5Hongsheng Liu
 		2ab5d1701a [4/N] Unify worker-side MP4 encoding and wire Wan into it (#7048) 
Signed-off-by: specture724 <specture724@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-17 23:33:56 +08:00
amy-why-3459 882bf8bc95 [Frontend] Add a shared realtime web UI for MiniCPM-o and Qwen3-Omni (#7222) (#7585) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-09-17 23:32:31 +08:00
Canlin Guo eb1abbadf8 [Feature][TTS] MOSS-TTS: place the reference-audio encoder on the code2wav stage's GPU (#7724) 
Signed-off-by: Canlin Guo <canlinguosdu@gmail.com>
 2026-09-17 05:51:26 -07:00
howard.shan c61b30255b [Qwen3-Omni] Remove redundant text embedding in DeepStack path (#7573) 
Signed-off-by: howard-shan <shanwenhao668@gmail.com>
Signed-off-by: howard.shan <shanwenhao668@gmail.com>
 2026-09-17 05:47:10 -07:00
Tiago de Souza Fernandes 5b22f5562f fix(minicpmo45): deadline-align native duplex silence continuation (#7059) 
Signed-off-by: Tiagosf00 <tiagotsf2000@gmail.com>
 2026-09-17 05:26:34 -07:00
Chenchao Xu d0755b5716 [Model] Add π0.5 (Pi0.5) VLA model support (#6950) 
Signed-off-by: chenchaox <chenchaox@nvidia.com>
 2026-09-17 05:18:51 -07:00
ooooooye 873e9bff7c [Doc][NPU] Refresh MiniMax-H3 Ascend recipes for 0.28-era configurations (#7720) 
Signed-off-by: brandneway <gyuan4892@gmail.com>
 2026-09-17 11:05:15 +00:00
Clodagh Walsh bf74544a35 [Voxtral] Follow up to raise BadRequest for ref_audio (#7649) 
Signed-off-by: Clodagh Walsh <clodaghwalsh17@gmail.com>
 2026-09-17 03:56:58 -07:00
Tianyao WuandClaude f5e4f5fa42 [Bugfix][MiniCPM-o] Size the duplex HD slice reservation from the frame (#7654) 
Signed-off-by: Tianyao Wu <rayroy31@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-17 03:35:46 -07:00
psv666 e3be42e052 [Core] Give the OpenAI Realtime wire codec a reusable home (RFC #6592 P0a) (#7640) 
Signed-off-by: psv666 <2693925048@qq.com>
 2026-09-17 02:33:40 -07:00
MrDongslsandMr.Dong 825ab20b07 [Model][PersonaPlex] add per-row RingKV offsets and active rows (#7670) 
Signed-off-by: Mr.Dong <1042542469@qq.com>
Co-authored-by: Mr.Dong <1042542469@qq.com>
 2026-09-17 01:50:40 -07:00
WeiQing Chen cb439f3e84 [Diffusion][Model] Add direct FastH3 8-Step V2 checkpoint loading (#7610) 
Signed-off-by: david6666666 <530634352@qq.com>
 2026-09-17 08:41:52 +00:00
Alex Brooks 8c3d3407d1 [CI] Fix Structured Structured Diffusion Config Wan Test (#7707) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-09-17 00:01:01 -07:00
LOGO127 224152566a [Model][PersonaPlex] Add paced multi-session validation (#7428) 
Signed-off-by: luozijian <luozijian0924@gmail.com>
 2026-09-16 23:04:37 -07:00
Sy03 4200768f59 [Frontend] Scale Omni serving across API processes (#6923) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-16 22:29:31 -07:00
WallbreazzzCanlin GuoHongsheng Liu
 		a769d9357b [Bugfix][NPU] Fix talker_mtp ACL graph capture missing force_uniform_decode (#7008) 
Signed-off-by: Wallbreazzz <wallbreazzz@users.noreply.github.com>
Signed-off-by: Wallbreazzz <110282866+Wallbreazzz@users.noreply.github.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-16 21:50:38 -07:00
maithilijoshi20 f1a6e7ce2a [Core] Retain structured configs through runtime startup (#6849) 
Signed-off-by: maithilijoshi20 <97733343+maithilijoshi20@users.noreply.github.com>
 2026-09-16 20:47:38 -07:00
HAAZZZEEEE becfefd440 [Hardware][Ascend] Add Wan2.2 MXFP4 UOS, Smooth and W4A8 fallback (#7210) 
Signed-off-by: HAAZZZEEEE <wang-taicheng@qq.com>
 2026-09-17 03:11:40 +00:00
Linze Shi 4fab34c8f4 [Bugfix] Update ComfyUI serving-mode mock after helper rename (#7694) 
Signed-off-by: Linze-Shi <linzeshi0@gmail.com>
 2026-09-17 02:45:31 +00:00
WeiQing Chen 72cc012cfd [CI][bugfix] Fix MiniMax H3 FP8 quality test remote-code loading (#7617) 2026-09-16 18:52:23 -07:00
Bojiang Li d81777f297 [Doc] Scope Spark FP8 guidance to the validated revision (#7536) 
Signed-off-by: bojiang-li <327132355+bojiang-li@users.noreply.github.com>
 2026-09-17 08:30:47 +08:00
Linze Shi c8b3a17bd3 [Frontend] Allow explicit turn-based serving (#7675) 
Signed-off-by: Linze-Shi <linzeshi0@gmail.com>
 2026-09-16 17:29:57 -07:00
Aman c32aaeb2da [Bugfix] Fix Qwen3-TTS word timestamps with async chunking (#7544) 
Signed-off-by: bezdarnost <amanurumbekov@gmail.com>
 2026-09-16 15:07:52 -07:00
Weiming Liao 63032c91c6 [Frontend] Add ComfyUI Generate Music node with initial MiniMax Music 3 support (#7516) 
Signed-off-by: Weiming Liao <liaowm5@gmail.com>
 2026-09-16 13:43:43 -07:00
Yueqian Lin 5aadcbb0ab [Bugfix] Fix Breeze-TTS-2 repeated silence and audio EOS handling 2026-09-16 18:40:12 +00:00
akshatvishu a8434a60da [BugFix][TTS] Fix MOSS Realtime serving and generation (#5661) 
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
 2026-09-16 11:39:51 -07:00
nodeeeeee 04c55d0164 Preserve audio boundary embeddings in Qwen2.5-Omni interleaved input (#7510) 
Signed-off-by: nodeeeeee <zhangkai.nodeee@gmail.com>
 2026-09-16 11:39:13 -07:00
shiyichuanandmershi c97b5fc741 [Bugfix][Frontend] Route synthetic aborts through final stage (#7006) 
Signed-off-by: mershi <mershi@tencent.com>
Co-authored-by: mershi <mershi@tencent.com>
 2026-09-16 11:38:45 -07:00
Lei Ke 38c7eeee4c [BugFix][NPU] Fix Fish Speech S2 Pro NPU support (#7546) 
Signed-off-by: Lei Ke <1141466880@qq.com>
 2026-09-16 11:38:06 -07:00
Deep Shah 4392af5ce3 [Bugfix][Qwen3-Omni] Include deferred residual in captured Thinker states (#7304) 
Signed-off-by: Deep Shah <deep@socratic.co>
 2026-09-16 11:36:52 -07:00
THUqliu d4dba0ddb1 [Misc] Remove dead deploy keys from personaplex.yaml (#7454) 
Signed-off-by: THUqliu <3298452736@qq.com>
 2026-09-16 13:13:32 -04:00
hywandAlex Brooks 2a68471715 [Bugfix] Reject failed diffusion benchmark warmups (#7092) 
Signed-off-by: hyw <yuweih205@gmail.com>
Signed-off-by: HuangYuwei <yuweih205@gmail.com>
Signed-off-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: Alex Brooks <albrooks@redhat.com>
 2026-09-16 17:09:23 +00:00
y-null 3d75254a44 [CI][MiniCPM-o] Lower the Daily-Omni accuracy gate to 0.77 (#7657) 
Signed-off-by: y_null <y_null@qq.com>
 2026-09-16 16:00:45 +00:00
NumberWanandwangyu 81b5be274d [Bugfix][Qwen-Image] Restore RotaryEmbedding CUDA RoPE for Diffusers e2e (#7513) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com>
 2026-09-16 08:54:28 -07:00
psv666andamy-why-3459 1053f7b0bd [Bugfix] Preserve benchmark metric sample counts in results (#7624) 
Signed-off-by: psv666 <2693925048@qq.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-09-16 08:39:01 -07:00
Zhengyuan Su (苏政渊) 538eddd74d [Perf][CosyVoice3] Bounded-window incremental HiFT vocoder streaming (#7521) 
Signed-off-by: Zhengyuan Su <su.zhengyuan@u.nus.edu>
 2026-09-16 20:49:04 +08:00
Tianyao WuClaudeGao Hanamy-why-3459
 		fa639b889e [Bugfix] Fail a duplex session that outgrows max_model_len instead of killing the EngineCore (#7277) 
Signed-off-by: Tianyao Wu <rayroy31@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-09-16 20:08:25 +08:00
RyanYun09RyanYun09amy-why-3459
 		e6cebf8557 [Bugfix][Omni-DuplexEval] Load local Hugging Face dataset layouts in duplex eval loader (#7331) 
Signed-off-by: RyanYun09 <RyanYun09@users.noreply.github.com>
Co-authored-by: RyanYun09 <RyanYun09@users.noreply.github.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-09-16 20:05:52 +08:00