anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
2,199 Commits 235 Branches 31 Tags
f18bb69b7ddac5b2fe73e28c38c7fbe26a8da9cb
Commit Graph
986 Commits
Author SHA1 Message Date
tzhouamandClaude Opus 4.8 fe16565cf6 test(entrypoints): grade realtime audio output with whisper-large-v3 
The Qwen3-Omni /v1/realtime accuracy check transcribed the generated
audio with whisper-small, which mishears short Chinese TTS clips
(observed: 北京->韦京 plus a dropped leading sentence, sim=0.443) and
produced spurious sim<threshold failures under async_chunk codec
variability even though audio generation was correct. Switch the grader
to large-v3 so a failure here points at the model, not the ASR check.
Also parameterize the threshold per mode (0.65 async_chunk, 0.8 sync).

Verified on 2x H100 (GPU 0,1), both realtime tests pass:
  test_streaming_audio_input_pcm_output_async_chunk[async_chunk] PASSED
  test_streaming_audio_input_pcm_output[sync] PASSED

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-11 14:21:18 +08:00
tzhouamandClaude Opus 4.8 c898701d53 fix(engine): dedup ZMQ ports across stages to stop OmniMasterServer bind races 
Each (stage_id, replica_id) route allocated its 3 handshake/input/output
ports via a separate get_open_ports_list(count=3) call. That helper only
guarantees uniqueness *within* a single call, so across routes the OS could
hand back a just-closed ephemeral port. Two stages then drew the same port
and the second engine subprocess to bind() it died with
"zmq.error.ZMQError: Address already in use", surfacing as
"Orchestrator initialization failed" / "Stage N replica M exited before API
server became ready". This flaked multi-stage startup (e.g. Qwen3-Omni
thinker/talker/code2wav) — one CI shard passed while a parallel shard failed
on the same commit (build 2630, dev/vllm-align).

OmniMasterServer now tracks every port it hands out in self._allocated_ports
(seeded with the registration port and the coordinator ROUTER port) and a new
_alloc_unique_ports() redraws any collision with a bounded retry budget,
raising loudly on port exhaustion instead of spinning. Adds _port_from_zmq_address()
helper and two CPU-only regression tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-11 14:21:17 +08:00
tzhouamandClaude 9aed0e4eb8 test(accuracy): re-baseline HunyuanImage3 SSIM/PSNR for the vLLM rebase 
HunyuanImage-3 (expert-parallel MoE) accuracy fails deterministically on
dev/vllm-align (SSIM 0.9490 vs 0.97, PSNR 26.21 vs 30 dB) while passing on
main's nightly. The align branch changes no hunyuan code; the drift comes
from the newer upstream vLLM (65a7b4628409, 585 commits past the v0.24.0
tag main tracks) shifting MoE kernel numerics. The image still renders
correctly and the mean/p99 pixel-diff checks pass — only the structural
metrics dip below a strict bar.

Per owner decision, re-baseline to the new stack: relax SSIM 0.97->0.94
and PSNR 30.0->25.0 (below the deterministic 0.9490 / 26.21, with margin).
The HunyuanImage CI job uploads no image artifact, so the reference PNG
cannot be auto-regenerated here; relaxing thresholds is the equivalent
re-baseline. main keeps the stricter bar.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-07-11 14:21:17 +08:00
tzhouam d58beb13d9 rebase: align vllm-omni with vLLM 65a7b4628409 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-07-11 14:21:16 +08:00
tzhouam b73036f538 rebase: align vllm-omni with vLLM 740f379faefd 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-07-11 14:21:15 +08:00
tzhouam 7dcbe4e957 rebase: align vllm-omni with vLLM 2dfaae752b4d 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-07-11 14:21:14 +08:00
Kushanamkushanamkahmadian
 		ff6f0da271 [Core] Composable parallel: Phase 1 strategy overlay (TP/DP/stage-replica) (RFC #4084) (#4281) 
Signed-off-by: Kushan A. <kahmadian@nvidia.com>
Signed-off-by: kushanam <kushanam@users.noreply.github.com>
Signed-off-by: Kushan A <kahmadian@nvidia.com>
Signed-off-by: kahmadian <kahmadian@users.noreply.github.com>
Co-authored-by: kushanam <kushanam@users.noreply.github.com>
Co-authored-by: kahmadian <kahmadian@users.noreply.github.com>
 2026-07-10 21:29:24 -06:00
NumberWan fe478a95ab [BugFix] Restore vllm_c IR op priority and torch.nn.RMSNorm for Qwen-Image (#4964) (#5009) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
 2026-07-10 17:39:07 +00:00
amy-why-3459 cbd3939c3d [BugFix] Fix CI failures caused by stale test expectations after recent stage-payload / deploy-config changes. (#5015) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-07-10 18:02:17 +08:00
amy-why-3459 9a44e7e019 [Refactor][Phase 1]Remove redundant functions and logs (#4986) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-07-10 15:13:35 +08:00
LHXuuu 5c3923509d [BugFix][Ming-TTS] Use uploaded direct speaker embeddings by voice (#5005) 
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com>
 2026-07-10 07:06:18 +00:00
Zheng Wengang 9fe26cfacb [Bugfix][Model] Fix Qwen3-Omni video metadata handling for use_audio_in_video (#4959) 
Signed-off-by: ZhengWG <zwg0606@gmail.com>
 2026-07-10 14:24:18 +08:00
林鑫andyurain 020792e6be [Perf][Qwen3-TTS] Sample code predictor outputs via Gumbel-max (#4261) 
Signed-off-by: 林鑫 <77673148+l-wave@users.noreply.github.com>
Co-authored-by: yurain <yurain26@users.noreply.github.com>
 2026-07-10 05:41:54 +00:00
leo b3a4819694 Fix VoxCPM2 perf server trust remote code (#4984) 
Signed-off-by: leo <yaoliu548926@gmail.com>
 2026-07-10 04:01:10 +00:00
Alex Brooks 690864f277 [Tests] Tiny Diffusion Model Pattern (#4524) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-07-09 09:03:51 -06:00
Yueqian Lin da15a4cb1c [Bugfix] Drop out-of-vocabulary stop ids from min-tokens masking (qwen3-tts min_tokens engine crash) (#4971) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-07-08 11:24:31 -07:00
leo 267b49b880 Fix Voxtral TTS feedback and short ASR checks (#4954) 
Signed-off-by: leo <yaoliu548926@gmail.com>
 2026-07-08 16:52:48 +00:00
Alex Brooks b7d08fe5f6 [Core / Bugfix] Add Mechanism for Endpoint Rejection (#4762) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-07-08 09:00:47 -06:00
Samit b96bc5ac87 Support HuggingFace kernels package for native diffusion attention backends (#4926) 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
 2026-07-08 14:34:36 +08:00
Hyoseop SongloveysubyDidan Deng
 		17e27c80e3 [Test] Add image-to-video offline example docs and tests (#3573) 
Signed-off-by: loveysuby <loveysuby@users.noreply.github.com>
Signed-off-by: Hyoseop Song <crad_on25@naver.com>
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: loveysuby <loveysuby@users.noreply.github.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-07-08 11:20:30 +08:00
Yukim1andAdvik 460a5c6b48 [Refactor] Migrate MammothModa2 and OmniVoice to pipeline registry (#4647) 
Signed-off-by: Advik <scince5678@gmail.com>
Signed-off-by: Nightwing-77 <scince5678@gmail.com>
Signed-off-by: zwhzzz0821 <2831474076@qq.com>
Co-authored-by: Advik <scince5678@gmail.com>
 2026-07-08 00:44:33 +00:00
Ting FU 815abce2ba [BugFix] fix possible shape mismatch when using ROPE (#4655) 
Signed-off-by: Semmer <semmer@live.cn>
 2026-07-07 17:54:39 +08:00
Yuanheng Zhao d365d3a69e [Refactor] Migrate Ming-flash-omni-2.0 Image-gen examples with model_extras (#4835) 
Signed-off-by: Yuanheng <jonathan.zhaoyh@gmail.com>
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
 2026-07-07 08:09:12 +00:00
herotai214 d4a869fe5e [Bugfix] Fix tokenizer config for higgs-audio-v3 test error (#4918) 
Signed-off-by: herotai214 <herotai214@gmail.com>
 2026-07-06 10:53:24 +00:00
amy-why-3459 9ceca4ec85 [Bugfix] Fix full-payload mm splitting for dual hidden/scheduled batch axes (#4870 follow-up) (#4910) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-07-06 15:08:05 +08:00
Vensen ba7e9e6496 [CI][sleep mode] fix sleep pytest CI error (#4912) 
Signed-off-by: vensen <vensenmu@gmail.com>
 2026-07-06 15:07:01 +08:00
汪志鹏 14fad9835d [BugFix]: fix #3236 bug (#4877) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-07-05 17:26:09 +00:00
TaffyOfficialandzuiho 66ec3a2aff [Bugfix] Fix HunyuanImage3 MoE groups for vLLM 0.24 (#4893) 
Signed-off-by: zuiho <2324465096@qq.com>
Co-authored-by: zuiho <2324465096@qq.com>
 2026-07-05 15:37:42 +00:00
Vensen ddba6de2ab [Bugfix][sleep mode]: guard generation on partial wake and ensure wake idempotency (#4834) 
Signed-off-by: vensen <vensenmu@gmail.com>
 2026-07-05 14:50:08 +00:00
Yueqian Lin d7b365df64 [Bugfix][Qwen3-TTS] Keep seeded residual MTP sampling batched (#4889) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-07-05 21:47:06 +08:00
Zhou Taichang 86bdcaf3d5 [BDE] Phase 1: engine-level KV cache management for DreamZero (RFC #4366) (#4534) 2026-07-05 07:17:58 +08:00
Sy03 aa21412c01 [Fix][VoxCPM2] Follow up unified graph config gate (#4878) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-07-04 19:58:21 +00:00
Sy03andYueqian Lin 1b31f96240 [Bugfix] Fix Qwen3-TTS no-async-chunk payload splitting (#4870) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
 2026-07-04 19:57:55 +00:00
Sy03andYueqian Lin 0798b0bf03 [Bugfix][Qwen3-TTS] Seed residual MTP sampling (#4869) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
 2026-07-04 17:47:11 +00:00
boatman 29dcdec66d [Cleanup]Replace legacy logging branches with PreparedRequest.model_type (#4862) 
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com>
 2026-07-04 05:39:20 +00:00
Sy03andYueqian Lin 9eaca72382 [Perf][VoxCPM2] Add unified decode graph optimization (#4606) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-07-04 05:30:48 +00:00
CAICAII 51c14eeaed [BugFix] Preserve explicit deploy stage engine_extras (#4255) 
Signed-off-by: CAICAIIs <3360776475@qq.com>
 2026-07-04 04:16:21 +00:00
Haco f7cdc45977 [BugFix] deploy schema stage runtime overrides for CLI diffusion flags no longer works (#4101) 
Signed-off-by: xiaohajiayou <923390377@qq.com>
 2026-07-04 03:17:25 +00:00
Kotaro Kikuchi 46f10478dc [Bugfix] Qwen-Image-Layered: allow layers=2 to match upstream demo (#4777) 
Signed-off-by: Kotaro Kikuchi <ktrk115@gmail.com>
 2026-07-04 03:06:24 +00:00
Canlin Guo a8b2bf11d2 [Feature] Support MOSS-TTS-Local-v1.5 Streaming (#4804) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-07-03 16:00:30 +08:00
Aleksi VesantoandWeiQing Chen 0455bffc37 [ROCm][Perf] Add ROCm support for MXFP4 (w4a4) online quantization (#4104) 
Signed-off-by: Aleksi Vesanto <avesanto@amd.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
 2026-07-03 13:17:32 +08:00
akshatvishu 04f400d36c [Bugfix] Report stepwise Qwen-Image profiler metrics (#4517) 
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
 2026-07-03 04:32:22 +00:00
wangyu 0b59dfb94a [Bugfix] Fix pytest 9.1.0 duplicate parametrization in run_benchmark.py (#4784) 
Signed-off-by: wangyu <410167048@qq.com>
 2026-07-03 04:15:32 +00:00
akshatvishu d9cf85d5e2 [BugFix] Use KV cache scale mapper in custom loaders (#4810) 
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
 2026-07-03 04:11:28 +00:00
Mu GuanLin 3023bc40c2 [Feature][Refactor] LTX-2.3 I2V Support, Public Examples, and Shared Pipeline Refactor (#4739) 
Signed-off-by: mglyn <1203789601@qq.com>
 2026-07-03 11:29:22 +08:00
CAICAII b443504d61 [Diffusion] Add Diffusers quantization config bridge (#3837) 
Signed-off-by: CAICAIIs <3360776475@qq.com>
 2026-07-03 03:06:44 +00:00
Jiaqian Liu f00caf35c4 [image][bugfix] Fix HunyuanImage3 bridge request batching (#4849) 
Signed-off-by: Celeste-jq <591998922@qq.com>
 2026-07-03 02:58:13 +00:00
Sean Gillen 431a7c71ac [Model] Cosmos3: Skip unused action conditioning frames (#4789) 
Signed-off-by: Sean Gillen <sgillen@nvidia.com>
 2026-07-03 01:45:44 +00:00
Jun Hee YooandClaude Opus 4.8 9d8089095e [Bugfix] Support append/extend/count actions in TrackingArgumentParser (#4639) 
Signed-off-by: Junhee Yoo <junhee.yoo@navercorp.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-02 23:29:24 +00:00
Zeyu Huang | 黃澤宇andHongsheng Liu 1b0de21c55 [Chore]: refactor out unused/redundant params in diffusion pipelines (#1235) 
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-07-02 16:32:05 +02:00