anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
2,959 Commits 235 Branches 31 Tags
c1e32fc01d55db4781062154cf2a5ccdce584094
Commit Graph
45 Commits
Author SHA1 Message Date
tzhouamandClaude Opus 5 c1e32fc01d rebase: close the remaining vLLM 0.29 drift found by CI 3009 
Five further removals/renames, each traced from a failing 3009 job:

- StageEngineCoreProcManager mirrors the parent __init__ rather than calling
  super(), and 0.29 added _request_shutdown_timeout there. The inherited
  shutdown() read it, raised AttributeError, and left the engine core
  subprocesses running, so the test process hung at interpreter exit until
  the 15 minute job timeout. That is the whole of the Engine&Model Executor
  exit-124: its pytest reported 567 passed, then sat idle for 663s. The
  failure surfaced only as a StagePool warning, never as a test failure.

- PromptUpdateDetails.select_text is gone; prompt updates are token oriented
  now. MiniCPM-o 4.5 kept three text-based image/video/audio replacements,
  which crashed stage-0 startup and account for the whole MiniCPM-o cluster.
  They encode the text and select the unk placeholder by token id instead.

- Qwen3-Omni: 0.29 marks audio_feature_lengths keep_on_cpu, while the audio
  tower derives device placement from feature_lens, so the mask comparison
  mixed cuda and cpu tensors. Moved explicitly, matching upstream
  Qwen3OmniMoeThinker._process_audio_input.

- AttentionBackend.get_kv_cache_shape is gone; the per-layer shape comes from
  the spec via compute_layer_kv_cache_shape_bytes, whose content dim is in
  bytes.

- WeightsMapper.get_unstacked_mapper is renamed to get_rename_mapper.

tests/engine/test_async_omni_engine_abort.py::test_abort fails here both with
and without these changes, so it is environmental rather than a regression.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146Xu5Sgj4vXxLxvLjV5YPj
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-09-10 00:41:53 +08:00
DMAN 06541d1f52 [Model] FP8 Support For Boogu-Image (#6925) 
Signed-off-by: Dmaner <2663515256@qq.com>
 2026-09-06 04:48:38 +00:00
Mu GuanLin 55a8bd6b80 [CI][Bugfix] Align the LTX-2 FP8 quality gate with the default recipe (#5831) 
Signed-off-by: mglyn <1203789601@qq.com>
 2026-08-29 02:24:09 +00:00
SYLARandHongsheng Liu 0e44fc7406 [diffusion][feature] Add offline SVDQuant W4A4 support (#6162) 
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-27 08:45:53 +08:00
jl9876andChendi.Xue 0d5d1115d6 [AutoRound] Add offline quantized MXFP4 model support (#5544) 
Signed-off-by: jl9876 <jia.li@intel.com>
Co-authored-by: Chendi.Xue <chendi.xue@intel.com>
 2026-08-25 14:44:38 +00:00
wangyuandamy-why-3459 4af0ddc1c2 [Test] Cleanup for helpers (#6523) 
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: [Your Name] <your.email@example.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-08-25 09:10:17 +08:00
SYLAR 1c917b2751 [Bugfix] Fix Qwen3-Omni AWQ quantization name mapping (#5687) 
Signed-off-by: lishunyang12 <lishunyang12@163.com>
 2026-08-23 15:13:36 +00:00
wangyuandamy-why-3459 0db262de2a [CI/Build] Move L1/E2E coverage to weekly and split scheduled L4/L5 pipelines (#6311) 
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: [Your Name] <your.email@example.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-08-23 11:49:37 +08:00
SYLARandHongsheng Liu d1e230c95b [Diffusion][Quantization] Enable MiniMax-H3 global FP8 with DLO (#5910) 
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-15 23:01:26 +08:00
baonudesifeizhai b300f66656 [Bugfix] Fix HSDP compatibility with the new online FP8 linear method (#5677) 
Signed-off-by: roG0d <baonudesifeizhai@gmail.com>
 2026-08-14 15:45:55 +08:00
Yukim1andHongsheng Liu 4b66f62e10 [Refactor] Remove internal stage_configs_path plumbing (#5741) 
Signed-off-by: zwhzzz0821 <2831474076@qq.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-13 21:29:03 +08:00
NATUREandAlicia d77a35a32b [Bugfix] Scope Bagel FP8 config to diffusion stage (#6085) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-08-13 11:23:47 +08:00
Shi Boao Bowie汪志鹏ruirui yangamy-why-3459Gao HanHongsheng Liu
 		fcf1085395 [Refactor][OutputProcessor 2/8]: OmniRequestOutput should inherit RequestOutput and no nested wrap-up (#5146) 
Signed-off-by: Boao Shi <aoibosh@connect.hku.hk>
Signed-off-by: bowieshi <aoibosh@connect.hku.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: R2-Y <ruiruyang2@gmail.com>
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Signed-off-by: Shi Boao Bowie <aoibosh@connect.hku.hk>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: ruirui(rein) yang <73573651+R2-Y@users.noreply.github.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-12 22:58:23 +08:00
Zhou TaichangandClaude Opus 5 2ce15ab7a0 [Rebase] Rebase to vllm 0.27.0 (#5976) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-11 15:23:46 +00:00
Zijun HuangandWeiQing Chen 93457f29ff [NPU][Quantization] Add RainFusion attention and INT8 online quantization for MiniMax H3 (#5706) 
Signed-off-by: Zijun Huang <zijunhua@usc.edu>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
 2026-08-06 14:15:21 +08:00
wangyu 7c17111ff5 [Refactor][CI] Reorganize test tree, and move slow e2e to weekly (#4951) 
Signed-off-by: wangyu <410167048@qq.com>
 2026-07-29 23:53:03 +08:00
Mu GuanLin 39e83e27f0 [Bug][CI] Pin the LTX FP8 quality gate to its established sigma schedule (#5302) 
Signed-off-by: mglyn <1203789601@qq.com>
 2026-07-22 14:36:32 +08:00
Diya Penganddiyapeng 6cd1ff7e64 [Quantization] Add FLUX.2-dev Mistral TE online FP8 support (#5136) 
Signed-off-by: diyapeng <50388704+diyapeng@users.noreply.github.com>
Co-authored-by: diyapeng <50388704+diyapeng@users.noreply.github.com>
 2026-07-20 16:16:31 +08:00
Diya Penganddiyapeng e0570a0859 [Quantization] Add BitsAndBytes W4 online quantization for diffusion transformer (#5037) 
Signed-off-by: diyapeng <50388704+diyapeng@users.noreply.github.com>
Co-authored-by: diyapeng <50388704+diyapeng@users.noreply.github.com>
 2026-07-14 15:48:48 +08:00
Isotr0py b95905deb1 [Quantization] Migrate GGUF diffusion model support to OOT plugin (#4769) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-07-11 18:48:43 +00:00
Aleksi VesantoandWeiQing Chen 0455bffc37 [ROCm][Perf] Add ROCm support for MXFP4 (w4a4) online quantization (#4104) 
Signed-off-by: Aleksi Vesanto <avesanto@amd.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
 2026-07-03 13:17:32 +08:00
Zhou Taichang a560ed184d [Rebase] Rebase to vllm v0.24.0 (#4709) 2026-06-30 19:41:06 +08:00
Kwokho TsuiandKwokho Tsui 7e152ca64b [Bugfix] Fix HSDP + FP8 online quantization compatibility (#4494) 
Signed-off-by: Kwokho Tsui <kh_tsui@163.com>
Co-authored-by: Kwokho Tsui <kh_tsui@163.com>
 2026-06-19 15:06:27 +08:00
Gao Han 963ba1ab7a [CI] Revert "[Feature] gen-only FP8 Quantization Support for SenseNova-U1" (#4196) 2026-06-06 08:39:10 +08:00
Guanzhe (Leo) Huang b63b36ded3 [Feature] gen-only FP8 Quantization Support for SenseNova-U1 (#3943) 
Signed-off-by: leohuang257 <masugchds@gmail.com>
 2026-06-05 02:51:11 +00:00
WeiQing ChenandWeiQing Chen 1acd27c205 [CI][BugFix] Fix and Validate FP8 Z-Image quality gate (#3929) 
Signed-off-by: WeiQing Chen <david6666666@users.noreply.github.com>
Co-authored-by: WeiQing Chen <david6666666@users.noreply.github.com>
 2026-05-28 12:52:58 +00:00
Chendi.Xue 1903cee56a [MXFP8][XPU] enable mxfp8 using vLLM main repo method (#3782) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
 2026-05-28 02:47:22 +00:00
Yuanheng Zhao 2bc9cb7f13 [Feat] Support online quantization (fp8/int8) for LTX-2 (#3700) 
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
 2026-05-21 19:52:21 +08:00
hxhhhlalalaandhyh_hh 9dd36e3afd [NPU][Quant] Add W4A4 MXFP4 online & MXFP4 dual-scale online/offline quantization support for Wan2.2 T2V / I2V inference on Ascend NPU (#3578) 
Signed-off-by: hyh_hh <huyinghong1@huawei.com>
Co-authored-by: hyh_hh <huyinghong1@huawei.com>
 2026-05-20 06:21:29 +00:00
WeiQing Chen a3d4ed809d [Quantization][tools] Add diffusion quantization output comparison tool (#3175) 
Signed-off-by: david6666666 <530634352@qq.com>
Signed-off-by: David Chen <530634352@qq.com>
 2026-05-20 00:14:08 +08:00
Lancer dbc589dbca [Bugfix] fix diffusion quantization benchmarking for Omni outputs (#3653) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2026-05-18 17:23:40 +08:00
wangyu b6f29ee614 [CI][Bugfix] skip fp8 Z-Image quality gate (#3531) and add torchdiffeq dev extra (#3563) 
Signed-off-by: wangyu <410167048@qq.com>
 2026-05-13 14:25:56 +08:00
baonudesifeizhaiandroG0d c4a0990044 （Phase 1）Add ModelOpt FP8 auto-detect support for diffusion checkpoints #2709 (#2913) 
Signed-off-by: roG0d <rodgarcas98@gmail.com>
Signed-off-by: roG0d <baonudesifeizhai@gmail.com>
Signed-off-by: baonudesifeizhai <85092850+baonudesifeizhai@users.noreply.github.com>
Co-authored-by: roG0d <rodgarcas98@gmail.com>
 2026-05-09 16:55:05 +08:00
wangyuandHuang, Zeyu 6f2ad7b403 [Test] Unify L2/L3 test layout, Buildkite steps, and test helpers (#2556) 
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
Signed-off-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Co-authored-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
 2026-05-08 14:12:23 +08:00
Nick Cao e6466cf0e6 [Refactor] Replace and ban a few torch.cuda functions in favor of torch.accelerator replacements. (#3365) 
Signed-off-by: Nick Cao <ncao@redhat.com>
 2026-05-07 23:23:59 +08:00
Liang LvandSYLAR ee15f39ea7 [AutoRound] Support Qwen Omni W4A16 quantization model (#2670) 
Signed-off-by: lvliang-intel <liang1.lv@intel.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-04-22 19:54:45 +08:00
wangyu 8a9add1c81 [CI] Restructure vLLM-Omni Test Layout, Fixture Scope, and Support Modules (#2620) 
Signed-off-by: wangyu <410167048@qq.com>
 2026-04-20 14:23:42 +08:00
Yuanheng Zhao e0cdbe9a5d [Misc] Cleanup: use consistent pytest-mock in unit tests (#2698) 
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
 2026-04-13 19:21:42 +08:00
zhumingjue138 e6f88f7c5a [CI] Reorganize the L1 L2 use cases and add markers (#2449) 
Signed-off-by: zhumingjue <zhumingjue@huawei.com>
 2026-04-09 10:31:21 +08:00
Zhou Taichang 2804a85e58 [Rebase] Rebase to vllm v0.19.0 (#2475) 2026-04-04 15:03:32 +08:00
Yi LiuandHongsheng Liu c1d2dcc3ac [AutoRound] Add offline quantized W4A16 model support (#1777) 
Signed-off-by: yiliu30 <yi4.liu@intel.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-04-02 13:57:53 +08:00
SYLAR 6b93459826 [Core] Unified quantization framework (#1764) 2026-03-24 18:56:55 +08:00
WeiQing Chen 602f831068 [Bugfix] fix gguf TypeError: GGUFConfig.get_name() missing 1 required positional argument: 'self', test: add diffusion gguf unit coverage (#1865) 
Signed-off-by: David Chen <530634352@qq.com>
 2026-03-23 14:56:15 +08:00
JuboYuAliciaWeiQing Chen
 		b766c4702c Int8 Quantization Support for DiT (Z-Image & Qwen-Image) (#1470) 
Signed-off-by: juboyu <767868009@qq.com>
Signed-off-by: JuboYu <767868009@qq.com>
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
 2026-03-19 22:58:29 +08:00
SYLAR 830d261f9d [Feature]: FP8 Quantization Support for DiT (#1034) 
Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-02-11 21:01:35 +08:00