anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
3,038 Commits 235 Branches 31 Tags
ab2f2a3d658db5be5ec96a950da55cb74b48e7aa
Commit Graph
51 Commits
Author SHA1 Message Date
tzhouamandClaude Fable 5.1 33de5d8629 Merge vLLM-Omni v0.29.0rc1 into edge-infer/omni-0.29.0 
The pair for vLLM v0.29.0: rc1's docs pin vllm==0.29.0 and a static scan of
rc1's vllm.* imports against the v0.29.0 tree finds nothing missing, while
our 0.28-based tree needs seven modules v0.29.0 no longer has
(vllm.inputs.preprocess, entrypoints.launcher, openai.cli_args,
openai.engine.protocol, serve.utils.server_utils, layers.batch_invariant,
models.hunyuan_v1).

Ten files conflicted. Upstream's structure is the base everywhere; the
fork's features are re-attached to it:
- deploy_profile / apply_deploy_profile run before rc1's resolve_omni_config
  pops deploy_config (async_omni_engine, cli/serve); the deploy-profile
  selector and the resolve_model_config_path chain it needs stay in
  entrypoints/utils.py; stage_overrides takes rc1's dict type.
- config_factory keeps the lazy diffusion output-type import; the diffusion
  registry import in rc1's new config/resolver.py is made lazy for the same
  WP6 reason (TTS path must not load diffusion).
- requirements: our core/media split kept; rc1's transformers>=5.13 floor
  and kernels==0.16.1 applied in core.txt / media.txt.
- TTS: rc1's split_granularity and bounded seed plus our output_mode=
  codec_tokens, _SessionInbox cancel handling and codec DELTA accumulation;
  our private _conditioning_cache_salt is superseded by rc1's public
  tts_adapters/base.py::conditioning_cache_salt.
- stage_runtime: rc1's single ExitStack G2 path (lock released before READY)
  replaces our two-path block, with the four worker_phase names the init
  profiler consumes still emitted; _cpu_placed / host admission kept.

Not yet run against vLLM 0.29.0: the tests listed in the W6 record.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-18 09:57:20 +08:00
Sy03 b02862beee [Model] Optimize MOSS-TTS Local batched execution and streaming codec (#7202) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-09 13:02:12 -07:00
tzhouamandClaude Fable 5.1 073d31b9cd [Edge][WP2] Add initialization timeline and cold-start profiler 
* vllm_omni/engine/init_timeline.py: InitTimeline phase collector; engine
  phases (stages_init, g3_device_lock, g2_spawn/ready) and worker phases
  (proc_entry, engine_core_init, init_device, load_model/load_weights,
  talker_mtp_init, attn_prewarm, cudagraph_capture, talker_mtp_capture,
  code2wav_cudagraph, predictor_setup_compile/warmup/cudagraph) recorded via
  worker_phase/timed_phase; workers append JSONL under
  VLLM_OMNI_INIT_TIMELINE_DIR (inherited at spawn) and the engine merges them.
  Enabled by VLLM_OMNI_INIT_TIMELINE=<json> or log_stats; table logged at INFO,
  partial table on orchestrator startup timeout, suggested timeouts logged.
* omni_base.py: named deploy profiles default to stage_init_timeout=900 /
  init_timeout=1500 unless given explicitly.
* gpu_model_runner._init_talker_mtp tolerates max_cudagraph_capture_size=None;
  gpu_ar_model_runner.shutdown skips CUDA graph teardown on CPU.
* benchmarks/tts/init_profile.py: runs stream_latency_bench --init-only per
  configuration (default, cold_cache, eager_stage1, eager_both,
  parallel_stage_init, edge, ...) and tabulates phase durations.
* stream_latency_bench: --parallel-stage-init / --deploy-profile flags; the
  resource poller no longer shadows Thread._stop and skips GPU sampling on CPU.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-09 11:01:39 +08:00
Yuxuan RhoandHongsheng Liu 7992c2213e [Config] Remove diffusion_batch_size and use max_num_seqs instead. (#6484) 
Signed-off-by: LyxWxj <yuxuanlou621@foxmail.com>
Signed-off-by: Yuxuan Rho <104440472+LyxWxj@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-01 01:02:54 +00:00
Bob Tian d934a5f7b8 [Bugfix] Report engine-queued requests as waiting in vllm_omni:num_requests gauges (#6549) 
Signed-off-by: Bob Tian <bobzetian@google.com>
 2026-08-29 03:58:26 +00:00
Honghan ZhuClaude Opus 4.7Gao Han
 		7ae15c74c8 text and image metric (#6150) 
Signed-off-by: zhuhh97 <zhuhonghan@huawei.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-08-26 02:54:45 +00:00
maithilijoshi20andHongsheng Liu bd4f9acfd3 [Bugfix] Preserve pipeline sampling constraints (#6182) 
Signed-off-by: maithilijoshi20 <97733343+maithilijoshi20@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-22 14:57:09 +00:00
Ma1oneZhang 6f36fd3b78 [Bugfix] Preserve object storage URIs during model resolution (#5036) 
Signed-off-by: Ziyang Zhang <hafuhafu@qq.com>
 2026-08-15 14:11:48 +08:00
vraitiandClaude Opus 4.6 c98517ce7f [Metrics] Add Diffusion Metrics (#4755) 
Signed-off-by: vraiti <vraiti@redhat.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-08-13 06:59:13 +00:00
Shi Boao Bowie汪志鹏ruirui yangamy-why-3459Gao HanHongsheng Liu
 		fcf1085395 [Refactor][OutputProcessor 2/8]: OmniRequestOutput should inherit RequestOutput and no nested wrap-up (#5146) 
Signed-off-by: Boao Shi <aoibosh@connect.hku.hk>
Signed-off-by: bowieshi <aoibosh@connect.hku.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: R2-Y <ruiruyang2@gmail.com>
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Signed-off-by: Shi Boao Bowie <aoibosh@connect.hku.hk>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: ruirui(rein) yang <73573651+R2-Y@users.noreply.github.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-12 22:58:23 +08:00
Shenglei FuClaude Opus 4.8Yueqian Lin
 		ee118ab492 [Bugfix] Per-replica fault isolation: keep API server alive on single-stage death (#4285) (#4583) 
Signed-off-by: Shenglei Fu <sfu@confluent.io>
Signed-off-by: Shenglei Fu <117230642+ShengleiFu@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-08-10 23:00:18 -04:00
Isotr0pylishunyang12SYLARDavid Chen
 		48298030d4 [Diffusion] Add Minimax-H3 modular pipeline support (#5720) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Signed-off-by: David Chen <530634352@qq.com>
Co-authored-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
Co-authored-by: David Chen <530634352@qq.com>
 2026-08-06 11:43:23 +00:00
akshatvishu cd5daaea6a [BugFix][Metrics] Fix per-stage time and token stats in metrics API (#4974) 
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
 2026-08-01 03:54:17 +00:00
Yash Jain 911ea0c586 [Bugfix][Frontend] Parse VLLM_USE_MODELSCOPE via vllm.envs (#5428) 
Signed-off-by: Yash Jain <yashjainjain1704@gmail.com>
 2026-07-27 04:19:22 +00:00
Sy03lishunyang12Yueqian LinStormMelodyGao HanHongsheng Liu
 		05b794f29b [Full Duplex] Feat: Support Full-Duplex realtime runtime & add MiniCPM-o 4.5 demo (#3907) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: StormMelody <22371184@zju.edu.cn>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-07-23 16:45:18 +08:00
Yukim1andHongsheng Liu 793d3e9ab7 [Config] Return VllmOmniConfig from create_from_model (#4818) 
Signed-off-by: zwhzzz0821 <2831474076@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-07-18 09:21:07 +00:00
ZacheryAU b0f6b9de96 [Bugfix] clean per-stage audio RTF and align audio metrics (#4794) 2026-07-02 02:55:44 +00:00
Zeyu Huang | 黃澤宇Copilot Autofix powered by AICursor
 		3901ad8165 [feat] Streaming diffusion video generation output (#3737) 
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
Signed-off-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-06-23 09:57:53 +00:00
blondeandClaude Opus 4.6 8fa0394d29 [Bugfix] Wire log_stats to AsyncOmni and add missing token metrics for non-text requests (#4482) 
Signed-off-by: blonde <164845202+blondeCS@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-06-17 21:31:44 +08:00
MaciejBalaNVandHongsheng Liu 03cc888485 [refactor] Refactor guardrail error handling - add 400 error code (#4297) 
Signed-off-by: Maciej Bala <mbala@nvidia.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-06-11 21:09:24 +00:00
ZacheryAU bfc7cedcf1 Benchmark data statistics for each stage of omni models (#3628) 
Signed-off-by: ZacheryAU <70869627+ZacheryAU@users.noreply.github.com>
 2026-06-04 21:43:35 +08:00
Alex Brooks 12e75c6618 [Core] Integrate TrackingArgumentParser (#3369) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-06-02 21:56:24 +08:00
LHXuuuandvraiti 41a795b5d6 [Metrics] Add audio SLOs + cross-stage transfer families + per-(stage, replica) wrap for upstream vllm:* (#3576) 
Signed-off-by: vraiti <vraiti@redhat.com>
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com>
Co-authored-by: vraiti <vraiti@redhat.com>
 2026-06-01 20:20:48 +08:00
+1
Zhou TaichangClaude Opus 4.7skfZheng WengangHongsheng LiuWeiQing ChenSYLARYueqian Linnainiu258JasonYueqian Lin
 		f92d84fcf9 [Rebase] Rebase to vllm releases/v0.22.0 (#3891) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: ZhengWG <zwg0606@gmail.com>
Signed-off-by: david6666666 <530634352@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Signed-off-by: nainiu258 <cperfect02@163.com>
Signed-off-by: nainiu258 <101917677+nainiu258@users.noreply.github.com>
Signed-off-by: JasonJ2021 <jasonj@zju.edu.cn>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: skf <54565339+skf-1999@users.noreply.github.com>
Co-authored-by: Zheng Wengang <zwg0606@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: nainiu258 <101917677+nainiu258@users.noreply.github.com>
Co-authored-by: Jason <72191212+JasonJ2021@users.noreply.github.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-06-01 11:08:12 +08:00
Yueqian LinandRein Yang e996723bc9 [Bugfix] Fix Qwen3-TTS Stage 0 prefix-caching correctness (#3665) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>

the ci failure case is irrelavent to this PR and occurs rarely
 2026-05-25 23:09:38 +08:00
wuhangandHongsheng Liu 0f4853ff86 [Bugfix] Support diffusion worker dead detect when use inline engine (#3214) 
Signed-off-by: wuhang <wuhang6@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-05-16 20:50:29 +08:00
wuhang 0d9d57acd9 [Entrypoint][Refactor] Make field type hint more concrete (#3139) 
Signed-off-by: wuhang <wuhang6@huawei.com>
 2026-05-14 11:25:55 +08:00
汪志鹏 c9a8556c24 [New Model]: Add sensenova u1 support (#3319) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-05-11 22:19:00 +08:00
Haco 81ab2f98da [Config Refactor] Remove legacy Omni CLI arg helper and align tests with nullified parser defaults (#3144) 
Signed-off-by: xiaohajiayou <923390377@qq.com>
 2026-05-07 06:32:01 +08:00
HacoHongsheng LiuSYLARYueqian Lin
 		b076006c35 [BugFix] Fix Whitelist optimization CI failure (#3290) 
Signed-off-by: xiaohajiayou <923390377@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-05-06 14:25:20 +08:00
zhanqiuhu 1a93818e54 [Bugfix] Add GatedRepoError Report (#1616) 
Signed-off-by: Zhanqiu Hu <zhu@redhat.com>
 2026-05-05 08:13:30 +00:00
Gao Han 704b6bb6e6 [CI failed]Revert "[Config Refactor] Derive deploy override fields from stage config" (#3287) 
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-05-01 04:39:54 +01:00
Haco 01ebc0cdf5 [Config Refactor] Derive deploy override fields from stage config (#3162) 
Signed-off-by: xiaohajiayou <923390377@qq.com>
 2026-05-01 01:04:03 +08:00
Alex BrooksSy03Yueqian LinYueqian Lin
 		e03ab82dd2 [Refactor/Bugfix] Use Delta Messages for Streaming (#2911) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: Sy03 <1370724210@qq.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-04-26 15:11:53 +12:00
Jared WenandHongsheng Liu 6e35bfbbae Benchmark and Bugfix for GLM-Image (#3024) 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-04-25 20:49:19 +08:00
amy-why-3459 e375b12680 [Refactor] Remove the default value "mp" from distributed_executor_backend. (#3007) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-04-25 11:48:03 +08:00
Hacoandlishunyang 22ffb85063 [Config Refactor] sentinel default precedence (#3078) 
Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: xiaohajiayou <923390377@qq.com>
Co-authored-by: lishunyang <lishunyang12@163.com>
 2026-04-24 20:07:24 +08:00
wuhang fd9a0ee27f [Feature] Failure message shows more details (#2961) 
Signed-off-by: wuhang <wuhang6@huawei.com>
 2026-04-23 04:32:43 +08:00
Daniel HuangChenguang ZhengZhou TaichangHongsheng Liubjf-frz
 		52b53367ef [Enhancement] Engine runtime errors (#2426) 
Signed-off-by: Daniel Huang <daniel1.huang@intel.com>
Signed-off-by: Daniel Huang <pilotflyer824@gmail.com>
Signed-off-by: bjf-frz <frz123db@gmail.com>
Co-authored-by: Chenguang Zheng <645327136@qq.com>
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: bjf-frz <frz123db@gmail.com>
 2026-04-21 14:46:05 +08:00
Vensen e076378df1 [Feat][sleepmode] add omni sleepmode and ack protocol (#2022) 2026-04-21 08:19:53 +08:00
amy-why-3459 0393c58533 [BugFix] Fixed the issue where --no-async-chunk was not working. (#2934) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-04-20 13:50:17 +08:00
SYLARreidliu41xiaohajiayouAlex BrooksHongsheng Liu
 		a3903810fe [Config Refactor][2/N] Pipeline + Deploy Config Schema (#2383) 
Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: xiaohajiayou <75477391+xiaohajiayou@users.noreply.github.com>
Co-authored-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-04-19 14:36:00 +12:00
Bingyu (Spencer) Liu a5a4998ece [Feature] Support Prefill-Decode disaggregation via vLLM KV transfer (#2220) 
Signed-off-by: LiuBingyu <liubingyu62@gmail.com>
 2026-04-17 18:43:17 +08:00
Zhengyuan Su (苏政渊)ClaudeSamit
 		cb6a8739c1 [Feature] Add trajectory recording to BAGEL denoising loop (#2483) 
Signed-off-by: Zhengyuan Su <su.zhengyuan@u.nus.edu>
Signed-off-by: Zhengyuan Su (苏政渊) <su.zhengyuan@u.nus.edu>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-04-08 10:43:01 +08:00
dsinghvi e7e05a55cf [Feature] Speech batch entrypoint (#1701) 
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
 2026-03-24 10:52:01 -04:00
Canlin GuoSamitlishunyang lishunyang12@163.com
 		7217557950 [Refactor] Unify torch profiler for omni and diffusion models (#2099) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: lishunyang lishunyang12@163.com
 2026-03-24 15:25:52 +08:00
Didan Deng 1f5eca5686 [Perf] Qwen-Image Performance Nightly CI test (#1805) 
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-03-24 00:19:20 +08:00
knlnguyen1802 65d76a4127 [Feature][RL] Support batching for QwenImage in async mode (#1593) 
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
 2026-03-23 10:08:24 +00:00
Zhou Taichangandgcanlin a90a7690b8 [Rebase] Rebase to vllm v0.18.0 (#2037) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-21 20:37:38 +08:00
dengyunyangbjf-frzDidan Deng
 		fde4808723 [Performance] diffusion timing (#1757) 
Signed-off-by: dengyunyang <584797741@qq.com>
Signed-off-by: bjf-frz <frz123db@gmail.com>
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: bjf-frz <frz123db@gmail.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-03-19 12:26:12 +08:00