anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
3,038 Commits 235 Branches 31 Tags
ab2f2a3d658db5be5ec96a950da55cb74b48e7aa
Commit Graph
54 Commits
Author SHA1 Message Date
tzhouamandClaude Fable 5.1 33de5d8629 Merge vLLM-Omni v0.29.0rc1 into edge-infer/omni-0.29.0 
The pair for vLLM v0.29.0: rc1's docs pin vllm==0.29.0 and a static scan of
rc1's vllm.* imports against the v0.29.0 tree finds nothing missing, while
our 0.28-based tree needs seven modules v0.29.0 no longer has
(vllm.inputs.preprocess, entrypoints.launcher, openai.cli_args,
openai.engine.protocol, serve.utils.server_utils, layers.batch_invariant,
models.hunyuan_v1).

Ten files conflicted. Upstream's structure is the base everywhere; the
fork's features are re-attached to it:
- deploy_profile / apply_deploy_profile run before rc1's resolve_omni_config
  pops deploy_config (async_omni_engine, cli/serve); the deploy-profile
  selector and the resolve_model_config_path chain it needs stay in
  entrypoints/utils.py; stage_overrides takes rc1's dict type.
- config_factory keeps the lazy diffusion output-type import; the diffusion
  registry import in rc1's new config/resolver.py is made lazy for the same
  WP6 reason (TTS path must not load diffusion).
- requirements: our core/media split kept; rc1's transformers>=5.13 floor
  and kernels==0.16.1 applied in core.txt / media.txt.
- TTS: rc1's split_granularity and bounded seed plus our output_mode=
  codec_tokens, _SessionInbox cancel handling and codec DELTA accumulation;
  our private _conditioning_cache_salt is superseded by rc1's public
  tts_adapters/base.py::conditioning_cache_salt.
- stage_runtime: rc1's single ExitStack G2 path (lock released before READY)
  replaces our two-path block, with the four worker_phase names the init
  profiler consumes still emitted; _cpu_placed / host admission kept.

Not yet run against vLLM 0.29.0: the tests listed in the W6 record.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-18 09:57:20 +08:00
tzhouamandClaude Opus 5 c4e61133fb [Edge][CPU] Fused norms off by default, and register the switches this branch added 
Two corrections and the paperwork they imply.

The fused-RMSNorm registration defaulted to *on*, which contradicts the
measurement it was committed with: those kernels are 3.6x and 4.5x faster in
isolation but measured 72.7 tok/s against 74.0 end to end, because inductor
already fuses the native norm into the residual add and an opaque call breaks
that fusion. The implementations stay registered, so the provider remains
selectable through `kernel_config.ir_op_priority` on a machine where the trade
lands differently; only the default ranking is left alone now, and the test
asserts both states rather than just the opt-in one.

The four switches this branch added (VLLM_OMNI_AOT_CACHE_SHIM,
_CPU_FUSED_NORMS, _CPU_RELEASE_MEMORY, _SPARK_GATE_REDUCTION) were not in the
reviewed environment-variable inventory, so
`test_statically_resolvable_environment_accesses_are_classified` failed. They
are classified now, the reviewed count moves 28 -> 32 with a note saying why,
and each has a row in docs/configuration/environment_variables.md carrying the
measurement that justifies its default.

`VLLM_OMNI_CPU_FAST_SAMPLER` and `VLLM_OMNI_STEP_STATS_SYNC` remain
unclassified and that test still fails on them. They arrived in 6ce8de1b, in
files this branch does not touch, so they are left to whoever owns that work
rather than documented from a guess.

Under `-m "core_model and cpu"`: 2593 passed before these fixes with 9
failures, of which two were mine. The rest are CUDA-only tests in a CPU venv
("Torch not compiled with CUDA enabled") plus test_qnn_export.py, which
another session is actively editing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-12 14:55:24 +08:00
Zhou TaichangandClaude Opus 5 aff7d64948 [Rebase] Rebase to vLLM 0.29.0 (#7230) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-10 12:08:28 +08:00
tzhouamandClaude Fable 5.1 536866b916 [Edge] Classify and document the edge env vars; fix the client decoder call 
- environment_variable_inventory: VLLM_OMNI_CPU_INDUCTOR / HW_PROFILE /
  INIT_TIMELINE / STEP_STATS_DIR (public), STAGE_ID / INIT_TIMELINE_DIR
  (internal); reference page rows added.
- edge_decoder_client.LocalDecoder: batched_chunked_decode takes a [B, Q, F]
  tensor, not a list (the WP5 e2e run failed with a TypeError); the decoder
  now also carries the ramp schedule attributes the Code2Wav stage sets.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-09 15:22:52 +08:00
汪志鹏 c59b48b47e [Feature][MiniMax-H3] Complete VSA and Ulysses support (#6909) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-09-02 07:19:21 -04:00
Hongsheng Liu d1c8beeb18 [Doc] Update README and docs for v0.28.0 (#6858) 
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-09-01 19:50:26 +08:00
Zheng Wengangandamy-why-3459 94b1546ce3 [Model] Qwen3-Omni: register thinker-only pipeline for Instruct serve (#6284) 
Signed-off-by: ZhengWG <zwg0606@gmail.com>
Signed-off-by: Zheng Wengang <zwg0606@gmail.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-08-27 08:21:03 +00:00
ChoHeeClaudeGao Han
 		c2723979c0 [Bugfix] Validate per-stage device layout before spawning workers (#5003) (#5742) 
Signed-off-by: ChoHee15 <cc5281@126.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-08-26 19:25:22 +00:00
Yueqian LinandYueqian Lin 8cfede4309 [Perf][Engine] Event-driven orchestration loop (opt-in) — S1 of #4855 (#5221) 
Signed-off-by: Yueqian Lin <gao@yueqian.us>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <gao@yueqian.us>
 2026-08-26 22:15:25 +08:00
amy-why-3459 5de6f7c37a [Bugfix][MiniCPM-o] Serve native duplex from shipping YAMLs and fence… (#6619) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-08-26 10:56:51 +08:00
Hongsheng Liu a708ae5b7a [Bugfix] Reconcile env-var inventory with post-#6217 main drift (#6631) 
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-08-26 06:13:28 +08:00
Hongsheng Liu 780e8d40aa [Doc] Add environment variables configuration reference (#6217) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-26 04:58:54 +08:00
Rahul SteigerHongsheng LiuRahul Steiger
 		b39f599b09 [Core] Add UniProc diffusion executor for single-GPU (#6308) 
Signed-off-by: Rahul Steiger <rsteiger@nvidia.com>
Signed-off-by: Rahul Steiger <rsteiger@aws-cmh-slurm-1-vscode-04.cm.cluster>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Rahul Steiger <rsteiger@aws-cmh-slurm-1-vscode-04.cm.cluster>
 2026-08-22 21:58:21 +08:00
Yukim1 6f55a06fab [Docs] Remove stale stage config references (#6270) 
Signed-off-by: zwhzzz0821 <2831474076@qq.com>
 2026-08-20 07:09:12 +08:00
Yukim1andzwhzzz0821 0a1846aa45 [Diffusion] Add native KV cache initialization and Scheduler-managed block allocation (#6094) 
Signed-off-by: zwhzzz0821 <zwhzzz0821@users.noreply.github.com>
Co-authored-by: zwhzzz0821 <zwhzzz0821@users.noreply.github.com>
 2026-08-18 07:00:24 +00:00
Hongsheng Liu 265d80b4dd [Docs] Classify PD disaggregation as an experimental feature (#6115) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-17 17:38:44 +08:00
bjf-frz 9f978923f2 [docs] consolidate diffusion execution modes (#5599) 
Signed-off-by: bjf-frz <frz123db@gmail.com>
 2026-08-05 22:41:18 +08:00
Hongsheng Liu 30cba84222 [Doc] Update README and installation docs for v0.26.0 (#5715) 
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-08-04 14:18:51 +08:00
Yukim1 7a2007cc17 [Refactor] Remove legacy --stage-configs-path from the serve CLI (#5647) 
Signed-off-by: zwhzzz0821 <2831474076@qq.com>
 2026-08-03 23:41:52 +08:00
bjf-frzandHongsheng Liu be667a8067 [Refact] Refactor diffusion output streaming (#5166) 
Signed-off-by: bjf-frz <frz123db@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-07-23 10:06:08 +08:00
Bingyu (Spencer) Liu dcf8b07d39 [Core] Cleanup Omni connector runtime and transfer backends (#5096) 
Signed-off-by: spencerr221 <liubingyu62@gmail.com>
 2026-07-22 09:44:15 +08:00
Yukim1zwhzzz0821SYLARHongsheng Liu
 		4fbbdfd250 [Refactor] Migrate Legacy Stage Configs to the Pipeline Registry (#5031) 
Signed-off-by: zwhzzz0821 <2831474076@qq.com>
Co-authored-by: zwhzzz0821 <zwhzzz0821@users.noreply.github.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-07-18 09:34:33 +00:00
Hongsheng Liuandhsliu_ustc 625892032b [Doc] Standardize serving names and identifiers (#5081) 
Signed-off-by: hsliu_ustc <hsliu_ustc@noreply.gitcode.com>
Co-authored-by: hsliu_ustc <hsliu_ustc@noreply.gitcode.com>
 2026-07-13 23:59:42 +08:00
Kushanamkushanamkahmadian
 		ff6f0da271 [Core] Composable parallel: Phase 1 strategy overlay (TP/DP/stage-replica) (RFC #4084) (#4281) 
Signed-off-by: Kushan A. <kahmadian@nvidia.com>
Signed-off-by: kushanam <kushanam@users.noreply.github.com>
Signed-off-by: Kushan A <kahmadian@nvidia.com>
Signed-off-by: kahmadian <kahmadian@users.noreply.github.com>
Co-authored-by: kushanam <kushanam@users.noreply.github.com>
Co-authored-by: kahmadian <kahmadian@users.noreply.github.com>
 2026-07-10 21:29:24 -06:00
amy-why-3459 9a44e7e019 [Refactor][Phase 1]Remove redundant functions and logs (#4986) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-07-10 15:13:35 +08:00
JiangJie ZhangandSamit c45ac74d43 [Core][Frontend] Support request-level batching for diffusion pipelines (#4079) 
Signed-off-by: jader <yjader@foxmail.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-06-29 18:09:19 +02:00
Isaac Hung e9d4931781 [Core] Remove dead custom_process_input_func hooks in stage input processors (#4531) 
Signed-off-by: Isaac Hung <manhei5001@gmail.com>
 2026-06-29 05:17:25 +02:00
amy-why-3459 0470e96771 [Docs] Modify Qwen3-Omni's recipe (#3109) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-04-27 21:41:13 +08:00
amy-why-3459 e375b12680 [Refactor] Remove the default value "mp" from distributed_executor_backend. (#3007) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-04-25 11:48:03 +08:00
wuhang 2e067bbc24 [Docs] CLI Docs updates (#2978) 
Signed-off-by: wuhang <wuhang6@huawei.com>
 2026-04-24 11:25:22 +08:00
SYLARreidliu41xiaohajiayouAlex BrooksHongsheng Liu
 		a3903810fe [Config Refactor][2/N] Pipeline + Deploy Config Schema (#2383) 
Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: xiaohajiayou <75477391+xiaohajiayou@users.noreply.github.com>
Co-authored-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-04-19 14:36:00 +12:00
Nick CaoandClaude f2edb81a59 [Cleanup] Remove dead runtime.defaults config parameters (#2343) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-17 23:36:41 +08:00
Didan Deng 911be0d426 [skip ci][Doc] Refine the Diffusion Features User Guide (#1928) 
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-03-30 14:28:04 +08:00
Zhou Taichang 05d8de813e [Bugfix] Remove duplicated config keyword max batch size (#1851) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2026-03-20 16:13:24 +08:00
Chenguang ZhengyinpePeiqi YinwuhangPeiqi Yinlinyueqianprincepridelishunyang
 		3158e9d28f [Entrypoint][Refactor] vLLM-Omni Entrypoint Refactoring (#1908) 
Signed-off-by: yinpe <11810305@mail.sustech.edu.cn>
Signed-off-by: Peiqi Yin <yinpeiqi809@gmail.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: wuhang <whlbx@hotmail.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
Signed-off-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: yinpe <11810305@mail.sustech.edu.cn>
Co-authored-by: Peiqi Yin <yinpeiqi809@gmail.com>
Co-authored-by: wuhang <whlbx@hotmail.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: lishunyang <lishunyang12@163.com>
 2026-03-18 17:33:53 +08:00
Jinheng 88caaf182b [Feature] Split #1303 Part 1: PD disaggregation scaffolding (#1863) 
Signed-off-by: Jinheng Li <ahengljh@gmail.com>
 2026-03-16 14:05:19 +08:00
汪志鹏 11d9ca8f41 [Feature]: Enable directly use OmniLLM init AR model (#1821) 2026-03-12 04:24:10 +00:00
Yuanheng Zhao 3b2b5012eb [Doc] Fix links in the configuration doc (#1615) 
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
 2026-03-03 04:52:51 +00:00
汪志鹏andCursor e37a89ffcb [Feature]: Support cfg kv-cache transfer in multi-stage (#1422) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-03-02 05:06:28 +00:00
Yanick Schraner 97f4c5b1e7 [Bugfix] Add TTS configuration options (#1177) 
Signed-off-by: Yanick Schraner <yanick.schraner@bs.ch>
 2026-02-26 00:14:44 +08:00
Canlin Guo c9392dff01 [Hardware] Support platforms and plugin system (#774) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-01-28 22:12:52 +08:00
+13
Zhou TaichangJaredforRealrootJustQJXU MingshiSihyeon JangJiangyun ZhucatcatZiming HuangYuhan LiuAliciaSamitYueqian LinWeiQing ChenPeiqi YinDidan DengHongsheng Liuwangyuwangyu31577汪志鹏John Liu BUAAD!NE$Hroot
 		776c3a794e Dev/rebase 0.14.0 (#847) 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: root <root@hk01dgx039.cm.cluster>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: TangPeng <85704592@qq.com>
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: iwzbi <wzbi@zju.edu.cn>
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Signed-off-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com>
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com>
Signed-off-by: Dinesh G <G.Dinesh@ibm.com>
Signed-off-by: gDINESH13 <dinesh13g@gmail.com>
Signed-off-by: root <root@hk01dgx028.cm.cluster>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: JaredforReal <w13431838023@gmail.com>
Co-authored-by: root <root@hk01dgx039.cm.cluster>
Co-authored-by: JustQJ <37905360+JustQJ@users.noreply.github.com>
Co-authored-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Co-authored-by: Sihyeon Jang <uneedsihyeon@gmail.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: catcat <108673086+iwzbi@users.noreply.github.com>
Co-authored-by: Ziming Huang <hzm414167@alibaba-inc.com>
Co-authored-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: John Liu BUAA <liukecheng97@gmail.com>
Co-authored-by: D!NE$H <67671800+gDINESH13@users.noreply.github.com>
Co-authored-by: root <root@hk01dgx028.cm.cluster>
 2026-01-21 00:25:25 +08:00
Jiangyun Zhu d7cd00e26c [Doc] refactor diffusion doc (#753) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-01-13 19:21:24 +08:00
Roger Wang aa5b628b6a [Doc] Update version 0.12.0 (#662) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2026-01-06 14:34:15 +08:00
rein yang c591ea124f [Bugfix] fix GPU VRAM calculation problem (#328) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
 2025-12-19 19:21:33 +08:00
Didan Deng 85f0c1177f [Diffusion]: Diffusion Ulysses-Sequence-Parallelism support (#189) 
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2025-12-17 15:41:31 +08:00
Lancer 31fedcfa2b [Misc] fix yaml config (#320) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2025-12-16 02:40:30 +00:00
Prajwal Azjy0516Samit
 		65ca13102d [Feature] teacache integration (#179) 
Signed-off-by: Prajwal A <prajwalanagani@gmail.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Samit <285365963@qq.com>
 2025-12-12 04:29:56 +00:00
rein yang baf1086d55 [CI] Add Qwen3-omni offline UT (#216) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
 2025-12-08 18:11:39 +08:00
Roger Wang c852267a1a [Misc] Update default stage config for qwen3-omni (#124) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-11-30 20:01:20 -08:00