anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
2,942 Commits 235 Branches 31 Tags
c4e61133fbb5587e0cf8972ce32ffac4e37fa739
Commit Graph
57 Commits
Author SHA1 Message Date
tzhouamandClaude Fable 5.1 37c0ea9a6b [Edge][WP8] Per-step overhead attribution, edge scheduling helpers, CPU thread sweep 
- vllm_omni/edge/step_stats.py: process-local counters enabled by
  VLLM_OMNI_STEP_STATS_DIR (scheduler schedule/update_from_output, engine-core
  step, chunk hop enqueue->put / put->get / get, orchestrator dispatch), dumped
  per process and merged into an attribution table (share of the 80 ms frame).
  Hooks: stage_engine_core_proc (instrument after construction, dump at exit),
  chunk_transfer_adapter (put/get timing, put->get via the shm lock-file mtime),
  orchestrator dispatch loop. Bench: stream_latency_bench.py --step-stats.
- vllm_omni/edge/scheduling.py: playback slack, prefill-throttle decision and
  the no-preemption KV admission rule (exact bytes/token from the talker KV
  geometry); adapt.derive_overrides now sizes max_num_seqs with it and
  auto_profile passes the geometry read from the model config.
- Engine re-reads parallel_stage_init after apply_deploy_profile so the edge
  profile's orchestrator default takes effect (was read before the profile).
- benchmarks/tts/cpu_thread_sweep.py: CPU-only 8/16/32-thread sweep (+inductor)
  with per-step attribution.
- Tests: tests/core/test_edge_scheduling.py, tests/edge/test_step_stats.py,
  tests/benchmarks/test_cpu_thread_sweep.py, profile default test.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-09 14:55:55 +08:00
NATUREandamy-why-3459 ff6e906a25 [Model] Preserve MiniCPM-o Talker full attention until capacity (#6799) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-09-06 22:25:35 +08:00
Sy03Nathan PriceEric
 		21dd4b33ac [Bugfix][MiniCPM-o] Stabilize native duplex streaming and Stage-1 handoffs (#6529) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Nathan Price <nathan@abridge.com>
Co-authored-by: Eric(Qun) <186020953+0xtoward@users.noreply.github.com>
 2026-08-29 13:05:24 -07:00
NATURE 87736a3cc5 [Bugfix][MiniCPM-o] Bound full-duplex Talker context with sliding recompute (#6626) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
 2026-08-30 00:49:42 +08:00
NATURE 54c1fc64b7 [Bugfix] Gate async-chunk segment resume by session mode (#6680) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
 2026-08-27 18:52:09 +08:00
Zhou TaichangandClaude Opus 5 651b03225b [Rebase] Rebase to vllm 0.28.0 (#6606) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-27 06:12:55 +00:00
Sy03andsy03 432b2c6ca3 [Model] Add native full-duplex Nemotron VoiceChat serving (#6089) 
Signed-off-by: Sysy <1370724210@qq.com>
Signed-off-by: sy03 <137072421@qq.com>
Co-authored-by: sy03 <137072421@qq.com>
 2026-08-26 13:19:20 -07:00
amy-why-3459 5de6f7c37a [Bugfix][MiniCPM-o] Serve native duplex from shipping YAMLs and fence… (#6619) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-08-26 10:56:51 +08:00
wy17003andHongsheng Liu b454e34966 [Refactor][2/N Scheduler]Replace _FULL_PAYLOAD_INPUT_STAGES with a resolved stage transport capability. (#6149) 
Signed-off-by: wy17003 <yaow17003@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-25 15:24:59 +08:00
NATURE 702a409664 [Feature][MiniCPM-o] Support configurable concurrent duplex sessions (#6021) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
 2026-08-25 02:27:30 +00:00
NATUREandamy-why-3459 4f7c08c0eb [Bugfix][MiniCPM-o] Fix async-chunk snapshot replacement and prompt cleanup (#6406) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-08-21 16:37:21 +08:00
NATURERuirui Yang | ReinHongsheng Liu
 		b57dc70ec4 [Bugfix] Reclaim resumable async-chunk requests on finish (#6360) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
Co-authored-by: Ruirui Yang | Rein <73573651+R2-Y@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-20 15:59:39 +08:00
Yueqian LinandHongsheng Liu 611c083587 [BugFix][TTS] R1.1-R1.4: turn silent async-chunk failures into visible ones (#6033) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-17 10:51:17 +08:00
Yuekai ZhangandClaude Fable 5 5be7541d01 [Model] Support NVIDIA-NemotronLabs-VoiceChat-11B offline speech-to-speech (#5842) 
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-12 16:25:48 +08:00
王金旭xuruiClaude Opus 4.8
 		9b0df3909a [Feature] timestamps for TTS via a forced-aligner pooling (#4795) 
Signed-off-by: wjinxu <1299461899@qq.com>
Co-authored-by: xurui <xurui1@modelbest.onaliyun.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-08-12 05:33:34 +00:00
Sy03andYueqian Lin 58007c0084 [New Model] Support IndexTTS 2.5 (#5957) 
Signed-off-by: Sysy <1370724210@qq.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-08-11 23:00:35 +00:00
Zhou TaichangandClaude Opus 5 2ce15ab7a0 [Rebase] Rebase to vllm 0.27.0 (#5976) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-11 15:23:46 +00:00
Yueqian LinandSy0307 e788ef6e76 [Feature] PersonaPlex (Moshi-based full-duplex S2S): native vLLM port + duplex serving (#4771) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Signed-off-by: Sy0307 <24841857+Sy0307@users.noreply.github.com>
Co-authored-by: Sy0307 <24841857+Sy0307@users.noreply.github.com>
 2026-08-04 20:10:33 -07:00
Shenglei FuandClaude f24a6165ff [Test] Prefix-cache passthrough test coverage prerequisite for W2 (#4855) (#5310) 
Signed-off-by: Shenglei Fu <sfu@confluent.io>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-04 20:03:57 -07:00
ruirui(rein) yang 5215e03a91 [Refactor][1/N Scheduler]Remove duplicated AR/generation scheduler plumbing and establish explicit shared lifecycle contracts. (#5461) 
Signed-off-by: R2-Y <ruiruyang2@gmail.com>
 2026-08-04 15:10:47 +08:00
SYLARandlishunyang12 6f98c807d2 [Bugfix] Restore BAGEL CFG KV transfers (#5620) 
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
 2026-08-01 15:06:05 +00:00
akshatvishu cd5daaea6a [BugFix][Metrics] Fix per-stage time and token stats in metrics API (#4974) 
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
 2026-08-01 03:54:17 +00:00
Shenglei FuandClaude 3b0718f22e [Bugfix] Fix active_stream_window silently stalling audio generation (#5349) (#5373) 
Signed-off-by: Shenglei Fu <sfu@confluent.io>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-07-30 14:56:18 +00:00
Yash JainandClaude Fable 5 daec3a6a49 [Bugfix] Trim AR sampled logprob rows with new_token_ids on mid-step stop (#5564) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-30 03:00:53 +00:00
wangyu 7c17111ff5 [Refactor][CI] Reorganize test tree, and move slow e2e to weekly (#4951) 
Signed-off-by: wangyu <410167048@qq.com>
 2026-07-29 23:53:03 +08:00
Sy03andHongsheng Liu 0feb92bdfa [Bugfix][Minicpm] Support full-duplex three-stage serving (#5380) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-07-29 01:58:34 +08:00
Zhou TaichangtzhouamClaude Fable 5Gao Han
 		c2b55de5b9 [Test] Give the logprobs scheduler doubles the vLLM 0.26 contracts (#5483) 
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: tzhouam <tzhouam@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-07-28 15:25:57 +00:00
wangyu cee2e40c33 [CI] Skip known-failing tests and filter NPU ready TTS by npu mark (#5489) 2026-07-28 15:26:36 +08:00
Yuekai ZhangClaude Fable 5Yueqian LinYueqian Lin
 		5014fea4a0 [New Model] support nemotron Audex (#4976) 
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-07-28 03:58:22 +00:00
WeiandSamit d3e0171cd7 [bugfix] validate AR sampled-token logprob alignment (#5273) 
Signed-off-by: hbhflw2000 <417911774@qq.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-07-28 10:34:19 +08:00
Zhou TaichangClaude Fable 5tzhouam
 		cbab1d4d8f [Rebase] Rebase to vllm 0.26.0 (#5443) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: tzhouam <tzhouam@gmail.com>
 2026-07-27 23:35:49 +08:00
ruirui(rein) yangandamy-why-3459 82172b112d [BugFix][MiniCPM-o] Restore audio output without async chunk (#5455) 
Signed-off-by: R2-Y <ruiruyang2@gmail.com>
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-07-27 22:19:46 +08:00
y-nullamy-why-3459Hongsheng Liu
 		b6c4578dac [BugFix]Add MiniCPM-o 4.5 end-to-end test suite (offline, online serving) (#5237) 
Signed-off-by: y_null <y_null@qq.com>
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-07-25 17:00:13 -07:00
Sy03 e043818abf [Bugfix][Frontend] Remove async-chunk guard from realtime API (#5383) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-07-25 09:20:47 +08:00
Viacheslav Klimkov 784c27f493 [Bugfix][Core] Fix async streaming segment-stop accounting (#5183) 
Signed-off-by: Viacheslav Klimkov <vklimkov@nvidia.com>
 2026-07-24 19:31:47 +08:00
rein yangandSy03 ad3a162eb8 [perf] Minicpm o4.5 optimization (#5228) 
Signed-off-by: R2-Y <ruiruyang2@gmail.com>
Co-authored-by: Sy03 <1370724210@qq.com>
 2026-07-24 17:20:39 +08:00
Sy03lishunyang12Yueqian LinStormMelodyGao HanHongsheng Liu
 		05b794f29b [Full Duplex] Feat: Support Full-Duplex realtime runtime & add MiniCPM-o 4.5 demo (#3907) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: StormMelody <22371184@zju.edu.cn>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-07-23 16:45:18 +08:00
rein yang 1984346bc7 [Refactor] [0/N] Scheduler clean dead code (#5312) 
Signed-off-by: R2-Y <ruiruyang2@gmail.com>
 2026-07-22 17:59:44 +08:00
Sy03 aa21412c01 [Fix][VoxCPM2] Follow up unified graph config gate (#4878) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-07-04 19:58:21 +00:00
Sy03andYueqian Lin 9eaca72382 [Perf][VoxCPM2] Add unified decode graph optimization (#4606) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-07-04 05:30:48 +00:00
BeatSeat 044240d8eb [Model] Add IndexTTS2 text-to-speech support (#3838) 
Signed-off-by: BeatSeat <wendavid552@gmail.com>
Signed-off-by: David <david@podcast.ai>
Signed-off-by: wendavid552 <wendavid552@gmail.com>
Signed-off-by: WenDavid <davidzhangcf@foxmail.com>
 2026-06-22 19:24:01 +00:00
longguoandnpuichigo 73df8326d7 [Bugfix] purge chunk-transfer zombies on every schedule tick to keep engine-core alive on aborts (fixes #3736) (#3774) 
Signed-off-by: abinggo <107740309+abinggo@users.noreply.github.com>
Co-authored-by: npuichigo <11533479+npuichigo@users.noreply.github.com>
 2026-06-10 09:00:23 +08:00
ChenWenjing 3842970a82 [bugfix] fix streaming input issue after rebase 0.22.0 (#4085) 
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
 2026-06-03 16:07:41 +08:00
TaffyOfficialandTaffyOfficial ee33601571 [Perf] Deduplicate AR prefix cache hidden-state CPU staging (#3734) 
Signed-off-by: TaffyOfficial <2324465096@qq.com>
Co-authored-by: TaffyOfficial <2324465096@qq.com>
 2026-06-01 09:35:07 +00:00
ChenWenjing 5dfdf584dd [Feat]audio streaming input for async chunk (#3614) 
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
 2026-05-31 17:39:25 +08:00
NATURE 3a347b10b0 [3/5][core]refactor communication layer: PR 3 of 5, all other models in non async mode (#3719) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
 2026-05-28 14:06:41 +08:00
LancerandAlicia 5c3c81920b [Bugfix] Ensure stage and diffusion subprocesses exit when parent dies unexpectedly (#3751) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-05-28 08:22:04 +08:00
Yueqian LinandRein Yang e996723bc9 [Bugfix] Fix Qwen3-TTS Stage 0 prefix-caching correctness (#3665) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>

the ci failure case is irrelavent to this PR and occurs rarely
 2026-05-25 23:09:38 +08:00
NATUREandHongsheng Liu 82a0b3a467 [2/5] [core]refactor communication layer: PR 2 of 5 Qwen3 Omni non async (#2677) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-05-15 14:14:08 +08:00
dsinghvi e40076e872 [Refactor] msgspec standardisation for data entry key names and improved type checks (#3149) 
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
 2026-05-12 14:30:13 +08:00