anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
2,942 Commits 235 Branches 31 Tags
c4e61133fbb5587e0cf8972ce32ffac4e37fa739
Commit Graph
140 Commits
Author SHA1 Message Date
tzhouamandClaude Fable 5.1 37c0ea9a6b [Edge][WP8] Per-step overhead attribution, edge scheduling helpers, CPU thread sweep 
- vllm_omni/edge/step_stats.py: process-local counters enabled by
  VLLM_OMNI_STEP_STATS_DIR (scheduler schedule/update_from_output, engine-core
  step, chunk hop enqueue->put / put->get / get, orchestrator dispatch), dumped
  per process and merged into an attribution table (share of the 80 ms frame).
  Hooks: stage_engine_core_proc (instrument after construction, dump at exit),
  chunk_transfer_adapter (put/get timing, put->get via the shm lock-file mtime),
  orchestrator dispatch loop. Bench: stream_latency_bench.py --step-stats.
- vllm_omni/edge/scheduling.py: playback slack, prefill-throttle decision and
  the no-preemption KV admission rule (exact bytes/token from the talker KV
  geometry); adapt.derive_overrides now sizes max_num_seqs with it and
  auto_profile passes the geometry read from the model config.
- Engine re-reads parallel_stage_init after apply_deploy_profile so the edge
  profile's orchestrator default takes effect (was read before the profile).
- benchmarks/tts/cpu_thread_sweep.py: CPU-only 8/16/32-thread sweep (+inductor)
  with per-step attribution.
- Tests: tests/core/test_edge_scheduling.py, tests/edge/test_step_stats.py,
  tests/benchmarks/test_cpu_thread_sweep.py, profile default test.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-09 14:55:55 +08:00
tzhouamandClaude Fable 5.1 8e912db6c8 [Edge][WP6] Slim the AR/TTS import path and add an edge dependency set 
* diffusers is no longer imported by vllm_omni.diffusion.data (annotation only);
  vllm_omni.diffusion.io_support (diffusion registry + distributed state) is
  imported lazily by config_factory and async_omni_engine, so constructing a
  Qwen3-TTS pipeline config / AsyncOmni no longer loads diffusers,
  x_transformers or whisper (guarded by tests/config/test_import_guard.py).
* requirements/common.txt = core.txt (AR/speech runtime) + media.txt
  (diffusion/video/extra audio); VLLM_OMNI_EDGE_BUILD=1 makes setup.py use
  requirements/<device>-edge.txt (core only); `edge` extra adds websockets and
  sounddevice for on-device clients.

Measured on this host (CUDA venv): `import vllm` 7.0-7.9 s, `import vllm_omni`
adds ~20 s of which ~17 s is transformers' own import (transformers.utils.import_utils)
pulled in by vllm_omni.patch / vllm_omni.version; that floor is not
vLLM-Omni's and is unchanged. Guard test: no diffusion module loaded on the TTS path.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-09 11:14:50 +08:00
tzhouamandClaude Fable 5.1 b22dcc59f7 [Edge][WP1] Add edge deploy profile and deploy_profile selector 
* vllm_omni/deploy/edge/qwen3_tts.yaml: single-device Qwen3-TTS profile with a
  ramped codec chunk schedule (2->4->8->16->25 frames) that removes the ~80 ms
  playback stall of the default 1->25 schedule, decode batch 1, max_num_seqs 4.
* deploy_profile="edge" (Omni/AsyncOmni kwarg, --deploy-profile CLI, headless
  serve) resolves vllm_omni/deploy/<profile>/<model_type>.yaml; mutually
  exclusive with deploy_config; explicit profiles never fall back silently.
* tests/helpers/stage_config.py: modify_stage_config anchors a relative
  base_config before writing the temp YAML so overlay-derived configs resolve.
* async_omni_engine.py also carries the init-timeline hooks used by WP2.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-09 10:27:33 +08:00
BruceLoveDecimal 5cd925631c [Feature][LingBot World]stepwise execution (#6844) 
Signed-off-by: BruceLoveDecimal <liuqihao970610@gmail.com>
Signed-off-by: liuqihao <liuqihao970610@gmail.com>
 2026-09-07 13:38:23 +08:00
SYLAR ea3cf99894 [diffusion][feature] Add component-selective offload policies (#5929) 
Signed-off-by: lishunyang12 <lishunyang12@163.com>
 2026-09-06 05:40:09 +08:00
Zhou TaichangandClaude Fable 5 142151fef5 Parallel stage initialization (admission + SH/EX device locks) (Worker refactor 1/N) (#5224) 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-09-05 22:22:41 +08:00
chickeytonClaude Fable 5Gao Han
 		5d7fdf9350 [Core] Graduate MiniCPM-o 4.5 and PersonaPlex full-duplex serving out of experimental (#6196) 
Signed-off-by: chickeyton <ngton2014@gmail.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-09-05 08:20:21 +00:00
Andy Zhou 2298359f67 [Bugfix][Diffusion] Preserve stage attention backend (#6645) 
Signed-off-by: AndyZhou952 <jzhoubc@connect.ust.hk>
 2026-09-01 20:18:44 +08:00
Yuxuan RhoandHongsheng Liu 7992c2213e [Config] Remove diffusion_batch_size and use max_num_seqs instead. (#6484) 
Signed-off-by: LyxWxj <yuxuanlou621@foxmail.com>
Signed-off-by: Yuxuan Rho <104440472+LyxWxj@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-01 01:02:54 +00:00
baonudesifeizhai 8be601ed7f [Diffusion] Add native SymmMem Fast Ulysses transport (#6340) 
Signed-off-by: roG0d <baonudesifeizhai@gmail.com>
 2026-08-30 04:14:21 +00:00
Nguyen Kha Nhat LongCursorSamit
 		bb5bf8fd0a [Bugfix] Fix colocate-async sleep admission race and deliver AR abort tokens (#6367) 
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Signed-off-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-08-29 09:11:39 +00:00
Bob Tian d934a5f7b8 [Bugfix] Report engine-queued requests as waiting in vllm_omni:num_requests gauges (#6549) 
Signed-off-by: Bob Tian <bobzetian@google.com>
 2026-08-29 03:58:26 +00:00
Allen WuTRAE CLITRAE CLI
 		d94061201d [Bugfix][Engine] Make abort idempotent during shutdown (#6327) 
Signed-off-by: Allen Wu <allenwu2795@gmail.com>
Co-authored-by: TRAE CLI <noreply@bytedance.com>
Co-authored-by: TRAE CLI <traecli@bytedance.com>
 2026-08-28 19:51:25 -07:00
Canlin Guolishunyang12yuanwu
 		b6a4baa659 [Feature] Support text encoder Disaggregation for H3 (#5885) 
Signed-off-by: Canlin Guo <canlinguosdu@gmail.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Signed-off-by: yuanwu <yuan.wu@intel.com>
Co-authored-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: yuanwu <yuan.wu@intel.com>
 2026-08-28 14:27:05 +08:00
Mu GuanLinandHongsheng Liu 5e4daaf197 [Diffusion] Add LTX-2.5 Diffusion VAE decoder support (#6189) 
Signed-off-by: mglyn <1203789601@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-27 23:54:40 +08:00
Yueqian LinandYueqian Lin 8cfede4309 [Perf][Engine] Event-driven orchestration loop (opt-in) — S1 of #4855 (#5221) 
Signed-off-by: Yueqian Lin <gao@yueqian.us>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <gao@yueqian.us>
 2026-08-26 22:15:25 +08:00
Honghan ZhuClaude Opus 4.7Gao Han
 		7ae15c74c8 text and image metric (#6150) 
Signed-off-by: zhuhh97 <zhuhonghan@huawei.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-08-26 02:54:45 +00:00
Hongsheng Liuandhsliuustc0106 c608910d20 [Diffusion] Register HWR mmap for direct DLO H2D (#6591) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
Co-authored-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
 2026-08-25 23:30:11 +08:00
Hongsheng Liuandhsliuustc0106 8360346a66 [Diffusion] Integrate no-AllGather DLO with Host Weight Runtime (#6486) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
Co-authored-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
 2026-08-25 08:03:53 +08:00
Nguyen Kha Nhat LongandCursor d15b7ca3e1 [Feature]: Support pause / resume and sleep / wake for AR stages in AsyncOmni (#6084) 
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-24 16:21:32 +08:00
bjf-frzandHongsheng Liu 072bfc02dd [FEAT]Add FastVideo VSA backend for Wan2.2 (#4820) 
Signed-off-by: bjf-frz <frz123db@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-24 14:48:31 +08:00
Rahul SteigerHongsheng LiuRahul Steiger
 		b39f599b09 [Core] Add UniProc diffusion executor for single-GPU (#6308) 
Signed-off-by: Rahul Steiger <rsteiger@nvidia.com>
Signed-off-by: Rahul Steiger <rsteiger@aws-cmh-slurm-1-vscode-04.cm.cluster>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Rahul Steiger <rsteiger@aws-cmh-slurm-1-vscode-04.cm.cluster>
 2026-08-22 21:58:21 +08:00
bjf-frzandHongsheng Liu 14bd8f8772 [REFACT]Refactor diffusion parallel state (#5531) 
Signed-off-by: bjf-frz <frz123db@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-19 11:23:10 +08:00
Yueqian Lin b283215c28 [Model] Add MiniMax Music 3 text-to-music (#6186) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-08-18 04:33:13 +00:00
Yukim1andHongsheng Liu baba7d1e2e [Refactor] Remove legacy stage_args YAML loader (#6200) 
Signed-off-by: zwhzzz0821 <2831474076@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-16 11:22:28 +08:00
Jack Chenzhangbin.327Hongsheng Liu
 		3875f8ded4 [Core] Add realtime AR-Diffusion tick sessions for LingBot World 2.0 (#5491) 
Signed-off-by: zhangbin.327 <zhangbin.327@bytedance.com>
Signed-off-by: Jack47 <scsvip@gmail.com>
Co-authored-by: zhangbin.327 <zhangbin.327@bytedance.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-16 09:11:20 +08:00
Yukim1andHongsheng Liu 4b66f62e10 [Refactor] Remove internal stage_configs_path plumbing (#5741) 
Signed-off-by: zwhzzz0821 <2831474076@qq.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-13 21:29:03 +08:00
Zhou TaichangandClaude Opus 5 2ce15ab7a0 [Rebase] Rebase to vllm 0.27.0 (#5976) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-11 15:23:46 +00:00
Jerry Song 89015d5cad [Feature]: Adds distilled LoRA support for diffusion models (#2783) 
Signed-off-by: Songrui625 <songrui625@gmail.com>
 2026-08-11 14:23:43 +00:00
Ruirui Yang | Reinpsv666Cursoramy-why-3459
 		322600bfb2 [CI][MiniCPM-o] Add MiniCPM-o 4.5 accuracy and performance coverage (#5524) 
Signed-off-by: R2-Y <ruiruyang2@gmail.com>
Signed-off-by: psv666 <2693925048@qq.com>
Co-authored-by: psv666 <2693925048@qq.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-08-11 14:53:49 +08:00
Nguyen Kha Nhat LongSamitGao HanHongsheng Liu
 		c588208cc6 [bugfix] Support lora request for non diffusion model (#5374) 
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-10 11:27:51 +08:00
NumberWan cfa6d66ed6 [BugFix][Nightly CI] Opt in FA deterministic for Qwen-Image accuracy (#5887) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
 2026-08-07 17:15:22 +08:00
Isotr0pylishunyang12SYLARDavid Chen
 		48298030d4 [Diffusion] Add Minimax-H3 modular pipeline support (#5720) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Signed-off-by: David Chen <530634352@qq.com>
Co-authored-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
Co-authored-by: David Chen <530634352@qq.com>
 2026-08-06 11:43:23 +00:00
SYLARandlishunyang12 1c2a81f6d8 feat(minimax-h3): enable 2x RTX 4090/5090 with DLO (#5764) 
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
 2026-08-06 09:02:29 +08:00
WeiQing Chen ae7bee0c43 [Feature] Align MiniMax H3 official input matrix (#5752) 
Signed-off-by: david6666666 <530634352@qq.com>
 2026-08-05 07:12:01 +00:00
Isotr0py 24741961eb [Model] Add MiniMax H3 diffusion support (#5691) 2026-08-03 14:10:40 +08:00
ChoHee 67c54777bb [Bugfix] Reject /v1/completions for thinker+talker models when --trust-remote-code is unset (#5525) 
Signed-off-by: ChoHee15 <cc5281@126.com>
 2026-08-01 15:49:12 +00:00
SYLARandlishunyang12 6f98c807d2 [Bugfix] Restore BAGEL CFG KV transfers (#5620) 
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
 2026-08-01 15:06:05 +00:00
akshatvishu cd5daaea6a [BugFix][Metrics] Fix per-stage time and token stats in metrics API (#4974) 
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
 2026-08-01 03:54:17 +00:00
Sy03 09c6ad4204 [CI/Build][MiniCPM-o] Allow auto-response before input commit (#5567) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-07-30 14:14:19 +00:00
evanchuenganddavid6666666 744c65b730 [Feature] Distributed layerwise offload with DP multi-concurrency + mmap weight… (#5397) 
Signed-off-by: evanchueng <evanchueng@outlook.com>
Signed-off-by: david6666666 <530634352@qq.com>
Co-authored-by: david6666666 <530634352@qq.com>
 2026-07-30 13:35:38 +00:00
Sy03andHongsheng Liu 0feb92bdfa [Bugfix][Minicpm] Support full-duplex three-stage serving (#5380) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-07-29 01:58:34 +08:00
Yuekai ZhangClaude Fable 5Yueqian LinYueqian Lin
 		5014fea4a0 [New Model] support nemotron Audex (#4976) 
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-07-28 03:58:22 +00:00
Zeyu Huang | 黃澤宇 eaf90d8eff [Frontend] Add midway prompt update for streaming video generation (#4652) 
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
 2026-07-27 14:07:43 +08:00
TaffyOfficialandTaffyOfficial 93786c9cd3 [perf] Add configurable diffusion compile granularity (#4603) 
Signed-off-by: TaffyOfficial <2324465096@qq.com>
Co-authored-by: TaffyOfficial <2324465096@qq.com>
 2026-07-24 15:22:06 +00:00
Honghan ZhuClaude Opus 4.7dengyunyang
 		84bbd57e4d [Feature] Add AllGather-KV sequence-parallel attention str… (#4968) 
Signed-off-by: zhuhh97 <zhuhonghan@huawei.com>
Signed-off-by: dph97 <nenbaying@163.com>
Signed-off-by: zhuhh97 <zhh@unknown>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: dengyunyang <584797741@qq.com>
 2026-07-24 14:45:26 +00:00
Sy03lishunyang12Yueqian LinStormMelodyGao HanHongsheng Liu
 		05b794f29b [Full Duplex] Feat: Support Full-Duplex realtime runtime & add MiniCPM-o 4.5 demo (#3907) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: StormMelody <22371184@zju.edu.cn>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-07-23 16:45:18 +08:00
汪志鹏 fb4dd41d79 [Refactor]: Remove unnecessary config getattr (#5199) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-07-22 03:48:05 -07:00
Yukim1andHongsheng Liu 793d3e9ab7 [Config] Return VllmOmniConfig from create_from_model (#4818) 
Signed-off-by: zwhzzz0821 <2831474076@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-07-18 09:21:07 +00:00
Zhou TaichangandClaude Opus 4.8 d3c47efc4f [Rebase] Rebase to vllm v0.25.0 (#5042) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-12 23:39:53 +08:00