anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
2,978 Commits 235 Branches 31 Tags
fb94b87d4b0ec77f1aa04d57da3d32cc9dbc1ef4
Commit Graph
145 Commits
Author SHA1 Message Date
tzhouamandClaude Fable 5.1 fb94b87d4b refactor(inputs): build the Omni renderer as a subclass instead of a proxy 
OmniRenderer wrapped the upstream renderer, forwarded unknown attributes via
__getattr__, and re-bound eight BaseRenderer methods (render_cmpl, render_chat,
process_for_engine, _process_enc_dec and their async twins) onto the wrapper
so that they would reach Omni's _process_singleton. That allow-list hid any
concrete-renderer override of those methods and would silently skip the
Omni extras for any future BaseRenderer entry point not on the list.

Make OmniRenderer a mixin and create the renderer as a real subclass of the
upstream class instead:

- omni_renderer_cls(base_cls) builds (and caches) Omni<Base> = (OmniRenderer,
  base_cls); only _process_singleton / _process_singleton_async are extended,
  everything else is the upstream renderer unchanged.
- build_omni_renderer(vllm_config) resolves the class the way upstream
  renderer_from_config does (cached_tokenizer_from_config,
  tokenizer_args_from_config, RENDERER_REGISTRY.load_renderer_cls) and
  instantiates the Omni subclass; renderer_cls/tokenizer can be passed to
  skip the resolution.
- build_stage0_input_processor passes the renderer to InputProcessor up
  front instead of replacing it afterwards; the skip-tokenizer branch keeps
  _build_token_only_renderer and never resolves a tokenizer.

Rebinding an existing instance's __class__ was rejected: CPython refuses it
for HfRenderer ("object layout differs"), so the subclass is constructed.

Tests: tests/inputs/test_preprocess.py builds the renderer with
omni_renderer_cls and adds checks that the subclass is created once, that a
concrete renderer's render_cmpl override is no longer suppressed, and that
build_omni_renderer resolves like upstream or honours an explicit class;
tests/engine/test_async_omni_engine_stage_init.py asserts the stage-0
processor receives the built subclass instance and that the skip-tokenizer
branch never calls build_omni_renderer. Two pre-existing mypy complaints in
test_preprocess.py are fixed with annotations.

Validation: ruff, pre-commit (incl. mypy) on the four files; the modules
import and omni_renderer_cls(HfRenderer) builds; tests/inputs (32),
tests/engine/test_async_omni_engine_stage_init.py (55) and tests/entrypoints
(548) pass on CPU.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MpXe2LeRrUQGjZfKFaocMZ
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-09-10 00:41:53 +08:00
tzhouam efda9369a8 refactor: reduce copied vLLM 0.29 compatibility code 
Reuse upstream weight mapping and multimodal processing, consolidate Omni processor bridges, and trim unused Hunyuan internals. Route no-media kwargs through the active renderer and preserve GLM generation prompt formatting.

Validation: 453 CPU tests passed, 7 skipped; processor imports, Ruff, and diff checks passed. GPU and end-to-end CI not rerun.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-09-10 00:41:53 +08:00
kunkun 2f1845fa4e fix: reserve Qwen3-Omni NVFP4 backend fix (#7200) 
Signed-off-by: kunkunblueberry <1833921874@qq.com>
 2026-09-09 20:35:16 +08:00
Zheng Wengangandamy-why-3459 7c1b1e4189 [Bugfix][Engine] Group overlapping device stages into one sequential init component (#7328) 
Signed-off-by: ZhengWG <zwg0606@gmail.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-09-09 20:25:41 +08:00
Harry MellorHongsheng LiuClaude Opus 5
 		fdaf4e31bd Add vLLM-Omni library info to Hugging Face Hub requests (#5381) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-08 10:08:38 -06:00
Shuolei Wang 5378b77be4 [Model][JoyAI-VL-Interaction] Add native multi-stage pipeline (#5352) 
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
 2026-09-08 05:07:08 -07:00
LHXuuu c9c25b33c3 [Frontend] Add server-side VAD for turn-based omni models (#6618) 
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com>
 2026-09-08 03:17:00 -07:00
andyluo7andwangyu b58ff5cb8b [CI/Build] Wait for duplex reaper recovery in retry test (#7225) 
Signed-off-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com>
 2026-09-08 17:10:51 +08:00
Chenguang Zheng 087ac46318 [Refactor 1/N] Centralize Omni config resolution (#5140) 
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
 2026-09-08 10:30:26 +08:00
BruceLoveDecimal 5cd925631c [Feature][LingBot World]stepwise execution (#6844) 
Signed-off-by: BruceLoveDecimal <liuqihao970610@gmail.com>
Signed-off-by: liuqihao <liuqihao970610@gmail.com>
 2026-09-07 13:38:23 +08:00
汪志鹏andClaude Opus 5 ae70479619 [Bugfix] Drop the removed diffusion_batch_size kwarg from the stage-init test (#7101) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-05 17:05:33 +00:00
Zhou TaichangandClaude Fable 5 142151fef5 Parallel stage initialization (admission + SH/EX device locks) (Worker refactor 1/N) (#5224) 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-09-05 22:22:41 +08:00
chickeytonClaude Fable 5Gao Han
 		5d7fdf9350 [Core] Graduate MiniCPM-o 4.5 and PersonaPlex full-duplex serving out of experimental (#6196) 
Signed-off-by: chickeyton <ngton2014@gmail.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-09-05 08:20:21 +00:00
SYLARandhsliuustc0106 7493cbd3c2 [Bugfix] Fix Qwen3-Omni MoE backend selection (#7019) 
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
Co-authored-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-09-04 21:16:58 +08:00
Arifuzzaman Joy 3cddda9781 [Bugfix] Prevent build_engine_args_dict from mutating stage_config.engine_args (#6783) 
Signed-off-by: Arifuzzamanjoy <s1710374103@ru.ac.bd>
 2026-09-01 16:45:28 +00:00
Andy Zhou 2298359f67 [Bugfix][Diffusion] Preserve stage attention backend (#6645) 
Signed-off-by: AndyZhou952 <jzhoubc@connect.ust.hk>
 2026-09-01 20:18:44 +08:00
Yuxuan RhoandHongsheng Liu 7992c2213e [Config] Remove diffusion_batch_size and use max_num_seqs instead. (#6484) 
Signed-off-by: LyxWxj <yuxuanlou621@foxmail.com>
Signed-off-by: Yuxuan Rho <104440472+LyxWxj@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-01 01:02:54 +00:00
TJian ded8934626 [AMD] [CI] Fix tests for v0.28.0 (#6830) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-08-31 09:10:27 +08:00
Gao Han f7f48929db [Bugfix][Diffusion] Restore inline single-stage execution (#6813) 
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-08-30 21:40:47 +08:00
Yueqian Lin a6b559cf54 [Bugfix][MiniMax-Music3] Resolve stage subdirs against a real snapshot (#6640) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-08-30 06:51:24 +00:00
Sy03Nathan PriceEric
 		21dd4b33ac [Bugfix][MiniCPM-o] Stabilize native duplex streaming and Stage-1 handoffs (#6529) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Nathan Price <nathan@abridge.com>
Co-authored-by: Eric(Qun) <186020953+0xtoward@users.noreply.github.com>
 2026-08-29 13:05:24 -07:00
Sy03 3adee400a0 [Bugfix][Qwen3-TTS] Handle codec generations that exhaust their token budget (#6728) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-08-29 08:59:05 -07:00
Nguyen Kha Nhat LongCursorSamit
 		bb5bf8fd0a [Bugfix] Fix colocate-async sleep admission race and deliver AR abort tokens (#6367) 
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Signed-off-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-08-29 09:11:39 +00:00
Allen WuTRAE CLITRAE CLI
 		d94061201d [Bugfix][Engine] Make abort idempotent during shutdown (#6327) 
Signed-off-by: Allen Wu <allenwu2795@gmail.com>
Co-authored-by: TRAE CLI <noreply@bytedance.com>
Co-authored-by: TRAE CLI <traecli@bytedance.com>
 2026-08-28 19:51:25 -07:00
Sy03 009b992c5b [Bugfix][MiniCPM-o] Isolate duplex handoff failures and barge-in (#6170) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-08-28 15:46:41 +08:00
Canlin Guolishunyang12yuanwu
 		b6a4baa659 [Feature] Support text encoder Disaggregation for H3 (#5885) 
Signed-off-by: Canlin Guo <canlinguosdu@gmail.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Signed-off-by: yuanwu <yuan.wu@intel.com>
Co-authored-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: yuanwu <yuan.wu@intel.com>
 2026-08-28 14:27:05 +08:00
NATURE 54c1fc64b7 [Bugfix] Gate async-chunk segment resume by session mode (#6680) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
 2026-08-27 18:52:09 +08:00
Zhou TaichangandClaude Opus 5 651b03225b [Rebase] Rebase to vllm 0.28.0 (#6606) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-27 06:12:55 +00:00
ChoHeeClaudeGao Han
 		c2723979c0 [Bugfix] Validate per-stage device layout before spawning workers (#5003) (#5742) 
Signed-off-by: ChoHee15 <cc5281@126.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-08-26 19:25:22 +00:00
Yueqian LinandYueqian Lin 8cfede4309 [Perf][Engine] Event-driven orchestration loop (opt-in) — S1 of #4855 (#5221) 
Signed-off-by: Yueqian Lin <gao@yueqian.us>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <gao@yueqian.us>
 2026-08-26 22:15:25 +08:00
psv666andamy-why-3459 5ec576ab4c [Bugfix] Fix missing Realtime audio completion (#6564) 
Signed-off-by: psv666 <2693925048@qq.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-08-26 18:27:25 +08:00
Honghan ZhuClaude Opus 4.7Gao Han
 		7ae15c74c8 text and image metric (#6150) 
Signed-off-by: zhuhh97 <zhuhonghan@huawei.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-08-26 02:54:45 +00:00
Alex Brooks 0eb47b76d0 [Bugfix] Handle DoS Overflow Cases (#6598) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-08-25 16:09:03 +00:00
NumberWan bc8ea489ef [BugFix] Qwen-image performance regressed - Avoid mapping diffusion_batch_size onto scheduler max_num_seqs (#6525) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
 2026-08-25 03:36:11 -07:00
wy17003andHongsheng Liu b454e34966 [Refactor][2/N Scheduler]Replace _FULL_PAYLOAD_INPUT_STAGES with a resolved stage transport capability. (#6149) 
Signed-off-by: wy17003 <yaow17003@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-25 15:24:59 +08:00
WeiGao HanSamit
 		f0808c9793 [Bugfix] Fix remote replica membership lifecycle races (#5277) 
Signed-off-by: hbhflw2000 <417911774@qq.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Samit <285365963@qq.com>
 2026-08-25 10:34:59 +08:00
wangyuandamy-why-3459 4af0ddc1c2 [Test] Cleanup for helpers (#6523) 
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: [Your Name] <your.email@example.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-08-25 09:10:17 +08:00
Nguyen Kha Nhat LongandCursor d15b7ca3e1 [Feature]: Support pause / resume and sleep / wake for AR stages in AsyncOmni (#6084) 
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-24 16:21:32 +08:00
akshatvishu 39c16d75a9 [tts][bugfix] Fix CosyVoice3 sampling and stage handoff (#6424) 
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
 2026-08-22 21:20:57 -07:00
Yash Jain c6e38860e9 [Bugfix][Core] Fix replica device split when a stage omits devices (#5445) 
Signed-off-by: Yash Jain <yashjainjain1704@gmail.com>
 2026-08-21 00:44:03 +08:00
Inesh Reddy Chappidi cd3867c373 [Model] Suppress silence codec tokens for the first N Qwen3-TTS decod… (#5048) 
Signed-off-by: Inesh Reddy <ineshreddy249@gmail.com>
 2026-08-20 09:22:37 -07:00
amy-why-3459andCursor 9c61d5b67b [Bugfix][MiniCPM-o] Stop Talker mid-utterance truncation and load cod… (#6346) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-20 11:50:07 +08:00
Wang fuyinandHongsheng Liu e1c5f59aab [Config] Reuse vLLM configs in VllmOmniConfig (#6050) 
Signed-off-by: Acerak01-fy <wfy2003324@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-18 21:27:58 +08:00
Yueqian Lin b283215c28 [Model] Add MiniMax Music 3 text-to-music (#6186) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-08-18 04:33:13 +00:00
Yueqian LinandHongsheng Liu 611c083587 [BugFix][TTS] R1.1-R1.4: turn silent async-chunk failures into visible ones (#6033) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-17 10:51:17 +08:00
hurukawa 2a315e1a2d [Perf] Support request-level batching for Wan2.2 pipelines (#5676) 
Signed-off-by: nagisa-kun <1434936049@qq.com>
Signed-off-by: nagisa <1434936049@qq.com>
 2026-08-15 23:58:36 +08:00
m0g3r 09b6a1604a [Bugfix] Restore per-stage runtime env during launch (#6214) 
Signed-off-by: m0g3r <87276771+m0g3r@users.noreply.github.com>
 2026-08-15 13:00:27 +00:00
MrlixiangWE e57574ae29 [Bugfix] Carry ec_transfer_params and num_cache_creation_tokens on OmniRequestOutput (#6152) 
Signed-off-by: MrlixiangWE <mrdanaer@gmail.com>
 2026-08-15 08:37:27 +00:00
TJianandHongsheng Liu bc73546c06 [ROCm] [CI] Migrate CI to mi300x for v0.27.x (#5886) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Signed-off-by: TJian <tunjian.tan@embeddedllm.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-14 15:38:47 +08:00
Sy03 dbc0dd6d85 [Bugfix][Higgs-Audio-V3] Disable XQA decode (#6068) 
Signed-off-by: Sy03 <1370724210@qq.com>
Signed-off-by: Sysy <1370724210@qq.com>
 2026-08-13 10:00:50 -07:00