admin
4f5f99d8c6
probe(review): fp8 serve row + 3-ref fixture + fp8 quality-collapse diagnosis
...
Build Wheel / build (3.12) (pull_request) Failing after 50s
Build Wheel / build (3.11) (pull_request) Failing after 50s
pre-commit / pre-commit (pull_request) Failing after 54s
Review follow-up on PR #4 : run --quantization fp8 through the served
path on the 3090 (the flag issue 1 validated for capacity), add the
untested 3-reference fixture, and measure all counts.
Capacity: confirmed — all of 1/2/3/4 references fit and run (4-ref
44.1-44.4s, server peak 23,522MB, nvidia-smi max 23,983MiB; Marlin
W8A16 is faster than BF16, 3.3x the Diffusers nocache baseline).
Quality: every fp8 output is structurally broken on this commit and
card (~97% transparent RGBA, grey RGB, PSNR 7-9dB) — on the edit path
at every reference count and on T2I alike (issue 1's fp8 artifacts
reinspected: same collapse; that run validated rc+image, not output
structure). img_mlp exclusion does not fix it; transformer-only fp8
also collapses, pinning the culprit on the Marlin-quantized DiT.
3090 verdict stays not-built; upstream fix order is fp8 output
correctness before any capacity role for it.
2026-10-08 06:27:36 -04:00
admin
67eb7fdb48
probe: Qwen-Image conditioning-KV spill probe on RTX 3090 / RTX 5080 (issue 2)
...
Build Wheel / build (3.11) (pull_request) Failing after 33s
Build Wheel / build (3.12) (pull_request) Failing after 40s
pre-commit / pre-commit (pull_request) Failing after 41s
Evidence-only change: probe kit (serve wrapper, API client, PSNR
comparator, offline fp8_v driver), per-host run summaries, and the
findings report. No code changes to the stack.
Verdicts: not-built on both cards. On the 3090 the served reference
ceiling is two condition images with the conditioning cache on
(2.2x the Diffusers nocache baseline, pair PSNR mean 27.57 dB);
four references OOM on the resident BF16 working set, not the KV
(~0.3 GiB/reference, per-request). On the 5080 the server cannot
initialize under any flag: init_parameters materializes the
unquantized skeleton on-device before quantization/offload engage.
Also documents that the recipe's deploy-YAML extras route for
prefix_kv_cache_dtype does not reach the served transformer on
8484d85d
2026-10-08 00:58:29 -04:00
hurukawa
3e705fcdb1
[Model] Add JoyAI-Image-Edit pipeline support ( #4112 )
...
Build Wheel / build (3.12) (push) Failing after 36s
Build Wheel / build (3.11) (push) Failing after 36s
pre-commit / pre-commit (push) Failing after 44s
Signed-off-by: nagisa-kun <1434936049@qq.com >
Signed-off-by: nagisa <1434936049@qq.com >
Signed-off-by: hurukawa <61525444+nagisa-kunhah@users.noreply.github.com >
2026-10-08 08:40:55 +08:00
Yueqian Lin
6a70bda0f6
[Bugfix][Qwen3-Omni] Default the talker voice to Ethan ( #8568 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
2026-10-07 17:07:53 -07:00
Yueqian Lin
1c71a2c03a
[Bugfix] Report sample rate and channels for streamed speech PCM ( #8567 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
2026-10-07 17:07:49 -07:00
c32a26a845
Add XPU deploy overlays and mark Intel GPU support ( #8409 )
...
Signed-off-by: Joshna Medisetty <joshna.medisetty@intel.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Chendi.Xue <chendi.xue@intel.com >
2026-10-07 17:19:07 -05:00
Alex Brooks
8484d85d1f
[Bugfix] Fix Sender Address Propagation ( #8599 )
...
Build Wheel / build (3.12) (push) Failing after 1m16s
Build Wheel / build (3.11) (push) Failing after 1m17s
pre-commit / pre-commit (push) Failing after 59s
Signed-off-by: Alex Brooks <albrooks@redhat.com >
2026-10-07 15:04:01 -06:00
Nick Cao and Codex
744c202d86
[BugFix] Optimize turn-based /v1/realtime auto truncation ( #8566 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-07 15:57:09 -04:00
Alex Brooks
f3743ebdb6
[Bugfix] Fix In Place Compile Overwrites for Qwen3Omni Embeddings with TP > 1 ( #8532 )
...
Signed-off-by: Alex Brooks <albrooks@redhat.com >
2026-10-07 17:01:45 +00:00
amy-why-3459
3345fb7498
[CI/Build] Fix CPU groups in distributed VAE test mocks ( #8591 )
2026-10-07 10:24:15 -06:00
Weiming Liao
32ead8e5c8
[NPU] upgrade to v0.31.0 ( #8588 )
...
Signed-off-by: Weiming Liao <liaowm5@gmail.com >
2026-10-07 23:35:09 +08:00
haic0 and andyluo7
c8e69c08fb
[CI/Build][ROCm] Stabilize R2-01 platform selection ( #8561 )
...
Signed-off-by: haic0 <149741444+haic0@users.noreply.github.com >
Signed-off-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: andyluo7 <andy.luo@amd.com >
2026-10-07 08:03:03 -07:00
MaciejBalaNV
e54e185593
[feature] Reduce memory usage with SP ( #7728 )
...
Signed-off-by: Maciej Bala <mbala@nvidia.com >
2026-10-07 22:36:32 +08:00
3dc35694b3
[New model] Qwen-Image-2.1 support (rebase + review follow-ups) ( #8099 )
...
Signed-off-by: NumberWan <wantszkin2003@gmail.com >
Signed-off-by: SamitHuang <285365963@qq.com >
Signed-off-by: Samit <285365963@qq.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: SamitHuang <285365963@qq.com >
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk >
2026-10-07 21:23:59 +08:00
QianCyrus and Hongsheng Liu
8ef77be66e
[Model] Avoid repeated attention and MLP work in Klein TP ( #8553 )
...
Signed-off-by: QianCyrus <101633534+QianCyrus@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-10-07 21:21:05 +08:00
NATURE
b21df3bcb6
[Frontend] Share regular and duplex chat initialization ( #7646 )
...
Signed-off-by: natureofnature <wzliu@connect.hku.hk >
2026-10-07 10:49:34 +00:00
amy-why-3459
de253412b0
[Core] Defer generation payload host waits to output consumption ( #8540 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-10-07 17:20:54 +08:00
haic0 and Cursor
d5a3380103
[CI/Build][ROCm] Run diffusion CPU-offload coverage with SDPA ( #8570 )
...
Signed-off-by: haic0 <149741444+haic0@users.noreply.github.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-06 19:45:52 -07:00
amy-why-3459
9482bf1e8c
[Model] MiniCPM-o 4.5: enable fused CFM body by default on CUDA ( #8557 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-10-07 10:25:39 +08:00
712b90de9c
[Perf][Qwen3-Omni] Capture the thinker image encoder with CUDA graphs (part of #7452 ) ( #8215 )
...
Signed-off-by: MrlixiangWE <mrdanaer@gmail.com >
Co-authored-by: zijianc2 <157244773+zijianc2@users.noreply.github.com >
Co-authored-by: Yifan Tian <221685243+Sendoh-code@users.noreply.github.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-10-07 09:29:59 +08:00
Yueqian Lin
ed6bc5ae0c
[Perf][PersonaPlex] Batch live duplex appends and replay Mimi codec frames from CUDA graphs ( #8356 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
2026-10-07 00:05:08 +00:00
e61cd4290f
[CI/Build][ROCm] Move Pi0.5 CPU coverage to AMD nightly ( #8559 )
...
Signed-off-by: haic0 <149741444+haic0@users.noreply.github.com >
Signed-off-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: andyluo7 <andy.luo@amd.com >
2026-10-06 11:05:14 -07:00
a0d915254b
[CI][ROCm] Add and stabilize HunyuanImage3 nightly coverage ( #7934 )
...
Signed-off-by: haic0 <149741444+haic0@users.noreply.github.com >
Signed-off-by: akshatvishu <akshatnayak197@gmail.com >
Signed-off-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: akshatvishu <akshatnayak197@gmail.com >
Co-authored-by: andyluo7 <andy.luo@amd.com >
2026-10-06 09:54:27 -07:00
bad88bca25
[Frontend] MiniMax-H3: Add latent-mask editing to the ComfyUI extension ( #7575 )
...
Signed-off-by: chen hongwei <1792043268@qq.com >
Signed-off-by: Wang Zhipeng <wangzhipeng628@gmail.com >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: Wang Zhipeng <wangzhipeng628@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 16:32:40 +00:00
Sy03
7be49aa0ce
[Model] Default MOSS Local 1.5 to adaptive CUDA MRV2 with MPS ( #8438 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
2026-10-06 08:20:46 -07:00
Dong1017 and amy-why-3459
ee9ab9ea88
[Perf] Fuse AdaLayerNorm CUDA eager path with Triton ( #8231 )
...
Signed-off-by: GUOGUO <xwdong1998@163.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-10-06 15:11:45 +00:00
BeatSeat and amy-why-3459
182b77da4c
[Perf][MiniCPM-o] Stage 0 streaming audio/vision encoder CUDA graphs and candidate sampling ( #8430 )
...
Signed-off-by: BeatSeat <wendavid552@gmail.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-10-06 07:53:03 -07:00
Nick Cao and Codex
65699fe94c
[Core] Generalized turn-based /v1/realtime implementation ( #8339 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-06 22:07:31 +08:00
25fb257667
[Model] MiniMax-H3: accept raw video masks and resolve the latent grid server-side ( #7947 )
...
Signed-off-by: chen hongwei <1792043268@qq.com >
Signed-off-by: Wang Zhipeng <wangzhipeng628@gmail.com >
Co-authored-by: Wang Zhipeng <wangzhipeng628@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 12:46:35 +00:00
Nick Cao and Codex
0677064b87
[Bugfix] Migrate CreateAudio from deprecated class Config to ConfigDict ( #5245 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-06 08:42:09 -04:00
5f95115e70
Kandinsky6 ( #8537 )
...
Signed-off-by: leffff <levnovitskiy@gmail.com >
Signed-off-by: Artem Borisov <divotionchannel@gmail.com >
Co-authored-by: leffff <levnovitskiy@gmail.com >
Co-authored-by: Artem Borisov <divotionchannel@gmail.com >
2026-10-06 15:55:15 +08:00
haic0 and Cursor
1f63e97a40
[CI][ROCm] Add Cosmos3 nightly coverage ( #7933 )
...
Signed-off-by: haic0 <149741444+haic0@users.noreply.github.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-06 00:18:44 -07:00
QianCyrus
c2cd13c009
[Core] Add batch parallel VAE decode ( #8162 )
...
Signed-off-by: QianCyrus <101633534+QianCyrus@users.noreply.github.com >
2026-10-06 13:51:35 +08:00
haic0 and Cursor
2bcd1e449a
[CI][ROCm] Add LingBot nightly coverage ( #7935 )
...
Signed-off-by: haic0 <149741444+haic0@users.noreply.github.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-06 05:31:51 +00:00
Joshna-Medisetty
5e34e1c97d
[XPU][Docker] Align XPU image and CI with vLLM v0.31.0 ( #8528 )
...
Signed-off-by: Joshna-Medisetty <joshna.medisetty@intel.com >
2026-10-06 00:07:43 -05:00
0ad67f1d8f
[CI/Build][ROCm] Retry AMD Simple Diffusion shards on GPU hangs ( #8520 )
...
Signed-off-by: haic0 <149741444+haic0@users.noreply.github.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: andyluo7 <43718156+andyluo7@users.noreply.github.com >
2026-10-05 21:01:31 -07:00
QianCyrus
099f9b553d
[Model] Optimize Wan VAE first-frame decoding with Conv2d ( #8179 )
...
Signed-off-by: QianCyrus <101633534+QianCyrus@users.noreply.github.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-10-06 11:44:34 +08:00
haic0 and Cursor
e4a8b2e858
[CI/Build][ROCm] Reuse AITER cache for Mammoth attention tests ( #8527 )
...
Signed-off-by: haic0 <149741444+haic0@users.noreply.github.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-05 18:35:01 -07:00
Tianyao Wu and amy-why-3459
cc444b8875
[Bugfix][MiniCPM-o] Take the duplex vision tile from the image-processor config ( #8462 )
...
Signed-off-by: Tianyao Wu <rayroy31@gmail.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-10-06 09:04:21 +08:00
Linze Shi
aad3090877
[Model][PersonaPlex] Vectorize prefill embedding construction ( #7481 )
...
Signed-off-by: Linze-Shi <linzeshi0@gmail.com >
2026-10-05 18:52:41 +00:00
LOGO127
ae430dddc0
[Model][PersonaPlex] Encode only consumed Mimi codebooks ( #7399 )
...
Signed-off-by: luozijian <luozijian0924@gmail.com >
2026-10-05 18:49:33 +00:00
BeatSeat and amy-why-3459
79562e97ad
[Perf][MiniCPM-o] Code2Wav resident Euler slot pool, fused DiT body, and exact HiFT graphs ( #8443 )
...
Signed-off-by: BeatSeat <wendavid552@gmail.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-10-05 18:36:28 +00:00
Nick Cao and Codex
3aab12851f
[Bugfix] Configure logging in the Omni CLI/API Server ( #8518 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-05 13:53:14 -04:00
Wenbo Ji 嵇文博
539c533762
[Bugfix] Keep client error status in the single-GPU diffusion executor ( #8474 )
...
Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com >
2026-10-05 13:09:34 -04:00
amy-why-3459
9146284c16
[Perf] Add CUDA graphs for MiniCPM-o 4.5 input encoders ( #8332 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-10-05 15:01:07 +00:00
Yufeng He
a0e626caaf
[Bugfix] Validate sleep stage_ids before blocking admission ( #8501 )
...
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com >
2026-10-05 10:27:53 -04:00
Sy03
023716ecc4
[Bugfix] Do not assume a stream decoder in MRv2 eager-MTP preprocess ( #8484 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
2026-10-05 09:29:38 -04:00
5351cd6645
[Bugfix] Qwen3-Omni thinker: create deepstack buffers on the real device ( #8461 )
...
Signed-off-by: John Nunez <johnnynunez@users.noreply.github.com >
Co-authored-by: John Nunez <johnnynunez@users.noreply.github.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-10-05 21:00:04 +08:00
ztyan
091b256674
[Model][Perf] IndexTTS2: drop per-step hidden D2H in latent mode ( #8478 )
...
Signed-off-by: ztyan <191593482+vivivi-111@users.noreply.github.com >
2026-10-05 20:20:25 +08:00
Yueqian Lin
6dd0d1f931
[Bugfix] Update routed-expert tests for native auxiliary output ( #8500 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
2026-10-05 17:40:07 +08:00
admin
hurukawa