anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
3,038 Commits 235 Branches 31 Tags
ab2f2a3d658db5be5ec96a950da55cb74b48e7aa
Commit Graph
136 Commits
Author SHA1 Message Date
tzhouam ab2f2a3d65 Audit all mobile and PC model pairings with archived evidence 2026-09-22 13:54:34 +08:00
tzhouam 1887d8294b Present support as mobile and PC by model matrix 2026-09-22 13:31:08 +08:00
tzhouam 5c75d5bb0a Document support as model and PC configuration pairings 2026-09-22 13:20:50 +08:00
tzhouam 723c550209 Integrate portable edge graph backends with Omni runtime 
Extract versioned stage contracts, model shared CPU/iGPU/NPU memory constraints, and add explicit graph routes, reservations, output acknowledgements, and worker retirement. Fix native Windows shared-buffer lifetime and host paths. Document qualified hardware paths and remaining rollout gates.
 2026-09-22 13:12:33 +08:00
tzhouamandClaude Fable 5.1 33de5d8629 Merge vLLM-Omni v0.29.0rc1 into edge-infer/omni-0.29.0 
The pair for vLLM v0.29.0: rc1's docs pin vllm==0.29.0 and a static scan of
rc1's vllm.* imports against the v0.29.0 tree finds nothing missing, while
our 0.28-based tree needs seven modules v0.29.0 no longer has
(vllm.inputs.preprocess, entrypoints.launcher, openai.cli_args,
openai.engine.protocol, serve.utils.server_utils, layers.batch_invariant,
models.hunyuan_v1).

Ten files conflicted. Upstream's structure is the base everywhere; the
fork's features are re-attached to it:
- deploy_profile / apply_deploy_profile run before rc1's resolve_omni_config
  pops deploy_config (async_omni_engine, cli/serve); the deploy-profile
  selector and the resolve_model_config_path chain it needs stay in
  entrypoints/utils.py; stage_overrides takes rc1's dict type.
- config_factory keeps the lazy diffusion output-type import; the diffusion
  registry import in rc1's new config/resolver.py is made lazy for the same
  WP6 reason (TTS path must not load diffusion).
- requirements: our core/media split kept; rc1's transformers>=5.13 floor
  and kernels==0.16.1 applied in core.txt / media.txt.
- TTS: rc1's split_granularity and bounded seed plus our output_mode=
  codec_tokens, _SessionInbox cancel handling and codec DELTA accumulation;
  our private _conditioning_cache_salt is superseded by rc1's public
  tts_adapters/base.py::conditioning_cache_salt.
- stage_runtime: rc1's single ExitStack G2 path (lock released before READY)
  replaces our two-path block, with the four worker_phase names the init
  profiler consumes still emitted; _cpu_placed / host admission kept.

Not yet run against vLLM 0.29.0: the tests listed in the W6 record.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-18 09:57:20 +08:00
tzhouamandClaude Fable 5.1 fa8b1a878e [Edge] Docs: chunk ramp / edge profile section and an edge recipe for Qwen3-TTS 
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-09 15:03:25 +08:00
Anjie HouandClaude Opus 5 b6f4df320d [Refactor][Diffusion] Resolve offload topology through one plan resolver (#7209) 
Signed-off-by: specture724 <specture724@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-09 08:17:38 +08:00
LHXuuu c9c25b33c3 [Frontend] Add server-side VAD for turn-based omni models (#6618) 
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com>
 2026-09-08 03:17:00 -07:00
Chenguang Zheng 087ac46318 [Refactor 1/N] Centralize Omni config resolution (#5140) 
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
 2026-09-08 10:30:26 +08:00
Zeyu Huang | 黃澤宇andAlicia 4b96c57730 [skip ci][Doc] Module design doc for diffusion runtime (#6440) 
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-09-07 21:52:24 +08:00
BruceLoveDecimal 5cd925631c [Feature][LingBot World]stepwise execution (#6844) 
Signed-off-by: BruceLoveDecimal <liuqihao970610@gmail.com>
Signed-off-by: liuqihao <liuqihao970610@gmail.com>
 2026-09-07 13:38:23 +08:00
Xiaobaixia 615f97b9c7 [Feature] Define the runner preprocess phase contract and regression coverage (#7136) 
Signed-off-by: xiaobaixia222 <289982752@qq.com>
 2026-09-06 17:20:04 +00:00
DMAN 06541d1f52 [Model] FP8 Support For Boogu-Image (#6925) 
Signed-off-by: Dmaner <2663515256@qq.com>
 2026-09-06 04:48:38 +00:00
SYLAR ea3cf99894 [diffusion][feature] Add component-selective offload policies (#5929) 
Signed-off-by: lishunyang12 <lishunyang12@163.com>
 2026-09-06 05:40:09 +08:00
chickeytonClaude Fable 5Gao Han
 		5d7fdf9350 [Core] Graduate MiniCPM-o 4.5 and PersonaPlex full-duplex serving out of experimental (#6196) 
Signed-off-by: chickeyton <ngton2014@gmail.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-09-05 08:20:21 +00:00
Hongsheng Liu d4096e01fe [Model] Add native MAGI-2 Preview diffusion support (#5918) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-09-05 15:32:30 +08:00
TriggerandGao Han ea14b04e9f [Bugfix][BAGEL] Restore original denoise step count (#7049) 
Signed-off-by: Trigger <Meng.Bo.Wang@outlook.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-09-04 17:05:05 +00:00
SYLARandhsliuustc0106 7493cbd3c2 [Bugfix] Fix Qwen3-Omni MoE backend selection (#7019) 
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
Co-authored-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-09-04 21:16:58 +08:00
Trigger e43c787544 [Model] Add BAGEL step execution and continuous batching (#6359) 2026-09-04 01:36:11 +00:00
Bo Li 5d20f6b900 [Doc] Refine TRTLLM attention guidance (#6724) 
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
 2026-09-02 19:42:37 +08:00
wangyu e338f0ece3 [CI]Split GPU jobs by cards_* and reject hand-written SKU marks (#6650) 
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: [Your Name] <your.email@example.com>
 2026-08-29 18:42:33 +08:00
ooooooye 5a5bc55c25 [Feature][Diffusion] Allow online INT8 quantization with DLO AllGather (#6573) 
Signed-off-by: brandneway <gyuan4892@gmail.com>
 2026-08-29 09:50:30 +00:00
Nguyen Kha Nhat LongCursorSamit
 		bb5bf8fd0a [Bugfix] Fix colocate-async sleep admission race and deliver AR abort tokens (#6367) 
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Signed-off-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-08-29 09:11:39 +00:00
Hongsheng Liu 6ec5ab11bd [Bugfix] Make HWR filesystem lifecycle transitions fail-closed (#6692) 
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-08-28 09:30:17 +08:00
Hongsheng Liuandhsliuustc0106 da4a08b65b [Diffusion] Add FLUX.2-klein BF16 HWR contract (#6651) 
Signed-off-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
 2026-08-27 12:51:15 +08:00
Yueqian LinandYueqian Lin 8cfede4309 [Perf][Engine] Event-driven orchestration loop (opt-in) — S1 of #4855 (#5221) 
Signed-off-by: Yueqian Lin <gao@yueqian.us>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <gao@yueqian.us>
 2026-08-26 22:15:25 +08:00
Honghan ZhuClaude Opus 4.7Gao Han
 		7ae15c74c8 text and image metric (#6150) 
Signed-off-by: zhuhh97 <zhuhonghan@huawei.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-08-26 02:54:45 +00:00
wangyuandCursor aeb348d8f8 [CI]Remove DreamID-Omni and MagiHuman support (#6357) 
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: [Your Name] <your.email@example.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-26 01:31:12 +00:00
Hongsheng Liuandhsliuustc0106 c608910d20 [Diffusion] Register HWR mmap for direct DLO H2D (#6591) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
Co-authored-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
 2026-08-25 23:30:11 +08:00
Hongsheng Liu 5a8b81ebeb [Model] Optimize MiniMax-H3 DLO component lifecycle (#6526) 
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-08-25 23:13:41 +08:00
Hongsheng Liuandhsliuustc0106 8360346a66 [Diffusion] Integrate no-AllGather DLO with Host Weight Runtime (#6486) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
Co-authored-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
 2026-08-25 08:03:53 +08:00
bjf-frzandHongsheng Liu 072bfc02dd [FEAT]Add FastVideo VSA backend for Wan2.2 (#4820) 
Signed-off-by: bjf-frz <frz123db@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-24 14:48:31 +08:00
ivanustoandHongsheng Liu f09bdcbc7d [Bugfix] Make the async-output wait bound configurable and default it higher (#6255) 
Signed-off-by: ivanusto <ivanusto@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-22 22:10:30 +08:00
Hongsheng Liu 493fc64397 [Diffusion] Add final-layout BF16 Host Weight Runtime artifacts (#6445) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-22 21:37:20 +08:00
Hongsheng Liu 15e38ac173 [Core] Add explicit post-load Host Weight Runtime publication (#6427) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-21 14:26:49 +08:00
Hongsheng Liu 73b623f2f7 [Core] Add Host Weight Runtime foundation (#6419) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-21 08:50:59 +08:00
Yukim1 6f55a06fab [Docs] Remove stale stage config references (#6270) 
Signed-off-by: zwhzzz0821 <2831474076@qq.com>
 2026-08-20 07:09:12 +08:00
Hongsheng Liu 284e05c88b [Core] Support online FP8 with DLO AllGather (#6279) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-18 23:48:07 +08:00
Wang fuyinandHongsheng Liu e1c5f59aab [Config] Reuse vLLM configs in VllmOmniConfig (#6050) 
Signed-off-by: Acerak01-fy <wfy2003324@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-18 21:27:58 +08:00
Hongsheng Liu 9d2bb23ff6 [Diffusion] Add loader-owned host-weight plans for DLO (TP=1) (#6213) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-16 11:02:38 +08:00
Jack Chenzhangbin.327Hongsheng Liu
 		3875f8ded4 [Core] Add realtime AR-Diffusion tick sessions for LingBot World 2.0 (#5491) 
Signed-off-by: zhangbin.327 <zhangbin.327@bytedance.com>
Signed-off-by: Jack47 <scsvip@gmail.com>
Co-authored-by: zhangbin.327 <zhangbin.327@bytedance.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-16 09:11:20 +08:00
SYLARandHongsheng Liu d1e230c95b [Diffusion][Quantization] Enable MiniMax-H3 global FP8 with DLO (#5910) 
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-15 23:01:26 +08:00
Samit 136cb27b05 [Bugfix] Fix lost async diffusion outputs in request-level batching (#6023) 
Signed-off-by: samithuang <285365963@qq.com>
 2026-08-15 18:27:51 +08:00
Hongsheng Liu 1260ceaae6 [Docs] Split diffusion attention and CPU offload guides (#6075) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-15 14:41:09 +08:00
vraitiandClaude Opus 4.6 c98517ce7f [Metrics] Add Diffusion Metrics (#4755) 
Signed-off-by: vraiti <vraiti@redhat.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-08-13 06:59:13 +00:00
Hongsheng Liu 1ff2753469 docs: align User Guide feature taxonomy (#6045) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-12 00:20:56 +08:00
Hongsheng Liuhsliu_ustcCopilot Autofix powered by AI
 		5e4b9f8c6e [Misc] Align CODEOWNERS with module/feature design docs (#5958) 
Signed-off-by: hsliu_ustc <hsliu_ustc@noreply.gitcode.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: hsliu_ustc <hsliu_ustc@noreply.gitcode.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
 2026-08-10 19:25:03 +08:00
Hongsheng Liu 9159cedbd7 docs: update vLLM-Omni architecture overview (#5914) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-09 09:16:18 +08:00
Hongsheng Liu fd0e92183a [Docs] [templates] Reorganize module design documentation (#5139) 
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-08 11:39:27 +08:00
Hongsheng Liu 62bf8d630d docs: document distributed layerwise offload compatibility (#5839) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-06 23:21:43 +08:00