anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
3,033 Commits 235 Branches 31 Tags
97d97b6503c4fa4e93907a7d15d52efa562fd75e
Commit Graph
128 Commits
Author SHA1 Message Date
tzhouamandClaude Fable 5.1 97d97b6503 [edge-infer W8] Omni serve CLI: uvloop optional, selector-loop asyncio.run on Windows 
uvloop ships no Windows wheels, so 'import uvloop' at module scope in
entrypoints/cli/serve.py killed vllm-omni serve on Windows at import
(measured in the serving e2e after the fcntl fix). _run_forever() uses
uvloop where present and asyncio.run on WindowsSelectorEventLoopPolicy
otherwise, which zmq.asyncio needs. The remaining serve-import blocker
on the Windows venv was a missing dependency, torchaudio (vLLM's own
requirement), installed into the venvs rather than patched.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-18 19:09:22 +08:00
tzhouamandClaude Fable 5.1 33de5d8629 Merge vLLM-Omni v0.29.0rc1 into edge-infer/omni-0.29.0 
The pair for vLLM v0.29.0: rc1's docs pin vllm==0.29.0 and a static scan of
rc1's vllm.* imports against the v0.29.0 tree finds nothing missing, while
our 0.28-based tree needs seven modules v0.29.0 no longer has
(vllm.inputs.preprocess, entrypoints.launcher, openai.cli_args,
openai.engine.protocol, serve.utils.server_utils, layers.batch_invariant,
models.hunyuan_v1).

Ten files conflicted. Upstream's structure is the base everywhere; the
fork's features are re-attached to it:
- deploy_profile / apply_deploy_profile run before rc1's resolve_omni_config
  pops deploy_config (async_omni_engine, cli/serve); the deploy-profile
  selector and the resolve_model_config_path chain it needs stay in
  entrypoints/utils.py; stage_overrides takes rc1's dict type.
- config_factory keeps the lazy diffusion output-type import; the diffusion
  registry import in rc1's new config/resolver.py is made lazy for the same
  WP6 reason (TTS path must not load diffusion).
- requirements: our core/media split kept; rc1's transformers>=5.13 floor
  and kernels==0.16.1 applied in core.txt / media.txt.
- TTS: rc1's split_granularity and bounded seed plus our output_mode=
  codec_tokens, _SessionInbox cancel handling and codec DELTA accumulation;
  our private _conditioning_cache_salt is superseded by rc1's public
  tts_adapters/base.py::conditioning_cache_salt.
- stage_runtime: rc1's single ExitStack G2 path (lock released before READY)
  replaces our two-path block, with the four worker_phase names the init
  profiler consumes still emitted; _cpu_placed / host admission kept.

Not yet run against vLLM 0.29.0: the tests listed in the W6 record.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-18 09:57:20 +08:00
Zhou TaichangandClaude Opus 5 aff7d64948 [Rebase] Rebase to vLLM 0.29.0 (#7230) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-10 12:08:28 +08:00
tzhouamandClaude Fable 5.1 b22dcc59f7 [Edge][WP1] Add edge deploy profile and deploy_profile selector 
* vllm_omni/deploy/edge/qwen3_tts.yaml: single-device Qwen3-TTS profile with a
  ramped codec chunk schedule (2->4->8->16->25 frames) that removes the ~80 ms
  playback stall of the default 1->25 schedule, decode batch 1, max_num_seqs 4.
* deploy_profile="edge" (Omni/AsyncOmni kwarg, --deploy-profile CLI, headless
  serve) resolves vllm_omni/deploy/<profile>/<model_type>.yaml; mutually
  exclusive with deploy_config; explicit profiles never fall back silently.
* tests/helpers/stage_config.py: modify_stage_config anchors a relative
  base_config before writing the temp YAML so overlay-derived configs resolve.
* async_omni_engine.py also carries the init-timeline hooks used by WP2.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-09 10:27:33 +08:00
tlysanhuo 7c2e30afd3 [Bugfix] Bound omni benchmark per-request timeout to 15 min by default (#7130) 
Signed-off-by: tly <2200895168@qq.com>
 2026-09-08 02:00:25 -07:00
Chenguang Zheng f3fabe4145 [Bugfix] Keep --disable-log-stats out of stage config resolution (#7237) 
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
 2026-09-08 16:19:07 +08:00
Chenguang Zheng 087ac46318 [Refactor 1/N] Centralize Omni config resolution (#5140) 
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
 2026-09-08 10:30:26 +08:00
ZacheryAU b1ec136bd7 [Frontend] Add omni benchmark support for image and video endpoints (#4728) 2026-09-08 09:28:21 +08:00
SYLAR ea3cf99894 [diffusion][feature] Add component-selective offload policies (#5929) 
Signed-off-by: lishunyang12 <lishunyang12@163.com>
 2026-09-06 05:40:09 +08:00
chickeytonClaude Fable 5Gao Han
 		5d7fdf9350 [Core] Graduate MiniCPM-o 4.5 and PersonaPlex full-duplex serving out of experimental (#6196) 
Signed-off-by: chickeyton <ngton2014@gmail.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-09-05 08:20:21 +00:00
baonudesifeizhai 8be601ed7f [Diffusion] Add native SymmMem Fast Ulysses transport (#6340) 
Signed-off-by: roG0d <baonudesifeizhai@gmail.com>
 2026-08-30 04:14:21 +00:00
Canlin Guolishunyang12yuanwu
 		b6a4baa659 [Feature] Support text encoder Disaggregation for H3 (#5885) 
Signed-off-by: Canlin Guo <canlinguosdu@gmail.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Signed-off-by: yuanwu <yuan.wu@intel.com>
Co-authored-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: yuanwu <yuan.wu@intel.com>
 2026-08-28 14:27:05 +08:00
Zhanshenzhuiandamy-why-3459 c6df39e9f7 [Benchmark][MiniCPM-o] Add Omni-DuplexEval support (#6634) 
Signed-off-by: Zhanshenzhui <99891002+zyforsure@users.noreply.github.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-08-27 16:31:50 +08:00
NATURE 497c537c6f [Benchmark] Add local OmniInteract realtime benchmark (#6522) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
 2026-08-25 15:32:58 +00:00
Hongsheng Liuandhsliuustc0106 c608910d20 [Diffusion] Register HWR mmap for direct DLO H2D (#6591) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
Co-authored-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
 2026-08-25 23:30:11 +08:00
Hongsheng Liuandhsliuustc0106 8360346a66 [Diffusion] Integrate no-AllGather DLO with Host Weight Runtime (#6486) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
Co-authored-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
 2026-08-25 08:03:53 +08:00
bjf-frzandHongsheng Liu 072bfc02dd [FEAT]Add FastVideo VSA backend for Wan2.2 (#4820) 
Signed-off-by: bjf-frz <frz123db@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-24 14:48:31 +08:00
Zhang BowenandHongsheng Liu 4882a36c41 [Model][Frontend] MiniMax-H3: Add opt-in planar video response encoding (#6288) 
Signed-off-by: MosCloud <bwzhang1991@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-22 04:04:14 +00:00
Yukim1 6f55a06fab [Docs] Remove stale stage config references (#6270) 
Signed-off-by: zwhzzz0821 <2831474076@qq.com>
 2026-08-20 07:09:12 +08:00
Yukim1andHongsheng Liu 4b66f62e10 [Refactor] Remove internal stage_configs_path plumbing (#5741) 
Signed-off-by: zwhzzz0821 <2831474076@qq.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-13 21:29:03 +08:00
王金旭xuruiClaude Opus 4.8
 		9b0df3909a [Feature] timestamps for TTS via a forced-aligner pooling (#4795) 
Signed-off-by: wjinxu <1299461899@qq.com>
Co-authored-by: xurui <xurui1@modelbest.onaliyun.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-08-12 05:33:34 +00:00
Jerry Song 89015d5cad [Feature]: Adds distilled LoRA support for diffusion models (#2783) 
Signed-off-by: Songrui625 <songrui625@gmail.com>
 2026-08-11 14:23:43 +00:00
Ruirui Yang | Reinpsv666Cursoramy-why-3459
 		322600bfb2 [CI][MiniCPM-o] Add MiniCPM-o 4.5 accuracy and performance coverage (#5524) 
Signed-off-by: R2-Y <ruiruyang2@gmail.com>
Signed-off-by: psv666 <2693925048@qq.com>
Co-authored-by: psv666 <2693925048@qq.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-08-11 14:53:49 +08:00
Zeyu Huang | 黃澤宇andHongsheng Liu 33dd9a5a3e [Core][Refactor][Diffusion] refactor request scheduler's admission wait policy (#5843) 
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-10 15:06:27 +08:00
NumberWan cfa6d66ed6 [BugFix][Nightly CI] Opt in FA deterministic for Qwen-Image accuracy (#5887) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
 2026-08-07 17:15:22 +08:00
Wang fuyin 3d7fc3b9ba [Config] Read engine args from VllmOmniConfig (#5678) 
Signed-off-by: Acerak01-fy <wfy2003324@163.com>
Signed-off-by: Wang  fuyin <wfy2003324@163.com>
 2026-08-07 11:17:33 +08:00
Isotr0pylishunyang12SYLARDavid Chen
 		48298030d4 [Diffusion] Add Minimax-H3 modular pipeline support (#5720) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Signed-off-by: David Chen <530634352@qq.com>
Co-authored-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
Co-authored-by: David Chen <530634352@qq.com>
 2026-08-06 11:43:23 +00:00
SYLARandlishunyang12 1c2a81f6d8 feat(minimax-h3): enable 2x RTX 4090/5090 with DLO (#5764) 
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
 2026-08-06 09:02:29 +08:00
Yukim1 7a2007cc17 [Refactor] Remove legacy --stage-configs-path from the serve CLI (#5647) 
Signed-off-by: zwhzzz0821 <2831474076@qq.com>
 2026-08-03 23:41:52 +08:00
Isotr0py 24741961eb [Model] Add MiniMax H3 diffusion support (#5691) 2026-08-03 14:10:40 +08:00
amy-why-3459andCursor 67aa6c55a4 [Bugfix][MiniCPM-o] Align Daily-Omni bench with MiniCPM interleaved A… (#5625) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-07-31 22:41:00 -07:00
evanchuenganddavid6666666 744c65b730 [Feature] Distributed layerwise offload with DP multi-concurrency + mmap weight… (#5397) 
Signed-off-by: evanchueng <evanchueng@outlook.com>
Signed-off-by: david6666666 <530634352@qq.com>
Co-authored-by: david6666666 <530634352@qq.com>
 2026-07-30 13:35:38 +00:00
Mu GuanLin 1b718f7b15 [Feat] [CI] [bugfix] Generalize LTX CFG parallelism to the complete guidance plan (#5547) 
Signed-off-by: mglyn <1203789601@qq.com>
 2026-07-29 17:40:40 +00:00
Sy03andHongsheng Liu 0feb92bdfa [Bugfix][Minicpm] Support full-duplex three-stage serving (#5380) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-07-29 01:58:34 +08:00
Mu GuanLin 12d75cf63a [Feature] Add official LTX multimodal guidance and precision guard (#5148) 
Signed-off-by: mglyn <1203789601@qq.com>
 2026-07-28 10:44:25 +00:00
Yuekai ZhangClaude Fable 5Yueqian LinYueqian Lin
 		5014fea4a0 [New Model] support nemotron Audex (#4976) 
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-07-28 03:58:22 +00:00
SYLARandlishunyang12 7762d664d8 [Attention] Add trtllm diffusion attention backend with Skip-Softmax (#5283) 
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
 2026-07-27 22:48:11 +08:00
TaffyOfficialandTaffyOfficial 93786c9cd3 [perf] Add configurable diffusion compile granularity (#4603) 
Signed-off-by: TaffyOfficial <2324465096@qq.com>
Co-authored-by: TaffyOfficial <2324465096@qq.com>
 2026-07-24 15:22:06 +00:00
Honghan ZhuClaude Opus 4.7dengyunyang
 		84bbd57e4d [Feature] Add AllGather-KV sequence-parallel attention str… (#4968) 
Signed-off-by: zhuhh97 <zhuhonghan@huawei.com>
Signed-off-by: dph97 <nenbaying@163.com>
Signed-off-by: zhuhh97 <zhh@unknown>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: dengyunyang <584797741@qq.com>
 2026-07-24 14:45:26 +00:00
rein yangandSy03 ad3a162eb8 [perf] Minicpm o4.5 optimization (#5228) 
Signed-off-by: R2-Y <ruiruyang2@gmail.com>
Co-authored-by: Sy03 <1370724210@qq.com>
 2026-07-24 17:20:39 +08:00
ZacheryAU 4c23a3164b [Refactor] Split vllm_omni/entrypoints/cli/benchmark/serve.py (#5206) 2026-07-23 16:00:32 +00:00
Yukim1andHongsheng Liu 793d3e9ab7 [Config] Return VllmOmniConfig from create_from_model (#4818) 
Signed-off-by: zwhzzz0821 <2831474076@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-07-18 09:21:07 +00:00
linzhenpl07 be517db18a examples: replica data-parallel recipe + benchmark for video DiT (#4707) (#5052) 
Signed-off-by: linzhenpl07 <linzhenpl07@gmail.com>
 2026-07-17 03:26:35 +00:00
Isotr0py b95905deb1 [Quantization] Migrate GGUF diffusion model support to OOT plugin (#4769) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-07-11 18:48:43 +00:00
Kushanamkushanamkahmadian
 		ff6f0da271 [Core] Composable parallel: Phase 1 strategy overlay (TP/DP/stage-replica) (RFC #4084) (#4281) 
Signed-off-by: Kushan A. <kahmadian@nvidia.com>
Signed-off-by: kushanam <kushanam@users.noreply.github.com>
Signed-off-by: Kushan A <kahmadian@nvidia.com>
Signed-off-by: kahmadian <kahmadian@users.noreply.github.com>
Co-authored-by: kushanam <kushanam@users.noreply.github.com>
Co-authored-by: kahmadian <kahmadian@users.noreply.github.com>
 2026-07-10 21:29:24 -06:00
JiangJie ZhangandSamit c45ac74d43 [Core][Frontend] Support request-level batching for diffusion pipelines (#4079) 
Signed-off-by: jader <yjader@foxmail.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-06-29 18:09:19 +02:00
Peiqi YinChenguang ZHENGCursorHongsheng LiuGao Han
 		504698dec8 [Core] Performance fix for orchestrator bottleneck: separates inter-stage from client outputs (#4527) 
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Co-authored-by: Chenguang ZHENG <645327136@qq.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-06-29 17:43:36 +02:00
Rahul SteigerandHongsheng Liu 327e9dcaf2 [Feature] Spatially-sharded (SP) decode for the Wan VAE (#4620) 
Signed-off-by: Rahul Steiger <rsteiger@nvidia.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-06-27 09:01:25 +08:00
Honghan ZhuClaude Opus 4.7zhuhh97
 		5db89b41c4 [Core] hunyuan-image prefetch kv (#4448) 
Signed-off-by: z00806815 <zhuhonghan@huawei.com>
Signed-off-by: zhuhh97 <zhuhonghan@huawei.com>
Signed-off-by: dph97 <nenbaying@163.com>
Signed-off-by: zhh <zhh@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: zhuhh97 <zhuhh97@users.noreply.github.com>
 2026-06-25 21:31:49 +08:00
Zeyu Huang | 黃澤宇Copilot Autofix powered by AICursor
 		3901ad8165 [feat] Streaming diffusion video generation output (#3737) 
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
Signed-off-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-06-23 09:57:53 +00:00