anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
2,980 Commits 235 Branches 31 Tags
3ea9e605011fc2a0bfa9f3e8531286fb683068dd
Commit Graph
210 Commits
Author SHA1 Message Date
Tianyu Guo 9dabaf8b80 Default Qwen-Image 2.1 CFG scale to 1.0 (#7859) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2026-09-20 11:51:35 +08:00
Isotr0py 47e6acc1e3 Add FP8 KV cache support for Qwen-Image-2.1 DiT (#7797) 2026-09-19 00:06:58 +08:00
Isotr0py 2a92b71cb9 Add FP8 online qunatization support for Qwen-Image-2.1 (#7792) 
Signed-off-by: Isotr0py <mozf@inferact.ai>
 2026-09-18 22:33:16 +08:00
Tianyu Guo ab4cf24b3e Optimize Qwen-Image 2.1 attention and decode graph reuse (#7790) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2026-09-18 19:37:19 +08:00
Tianyu Guo b0a309e5e7 Isolate Qwen-Image 2.1 prefill and decode scheduling (#7770) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2026-09-18 14:11:22 +08:00
Isotr0py 3affc7766d Enable VAE tiling for Qwen-Image-2.1 (#5) 
* init

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix

Signed-off-by: Isotr0py <mozf@inferact.ai>

* test(diffusion): add Qwen-Image-2.1 unit tests

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix(diffusion): Qwen-Image-2.1 VAE spatial ratio, tiling e2e verification

- AutoencoderKLQwenImage21: derive spatial_compression_ratio from the
  architecture (2 ** len(temperal_downsample) = 16x) instead of trusting
  the checkpoint's scale_factor_spatial=8, which corrupted tiled
  decode/encode geometry (output cropped to half size) and tiling gating.
- pipeline: _read_vae_scale_factor prefers the architecture-derived value
  for the same reason.
- models/qwen_image_21/__init__: lazy exports to break the circular import
  distributed VAE -> package __init__ -> pipeline -> distributed VAE.
- Add VAE unit tests: ratio derivation, tiled vs untiled decode/encode
  consistency, distributed tile_split/exec/merge == tiled_decode.
- e2e matrix (GB200, seed 42, 20 steps) all green; see e2e_out/qwen21_vae/.
- examples: qwen_image_21_edit.py gains --vae-use-tiling.
- docs: mark VAE patch-parallel verified for Qwen-Image-2.1; recipe notes
  the default-tile banding limitation at 2048px+.

* clean tests

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix cudnn determination

Signed-off-by: Isotr0py <mozf@inferact.ai>

---------

Signed-off-by: Isotr0py <mozf@inferact.ai>
 2026-09-18 10:02:50 +08:00
Isotr0py 2912672b2e Initialize QMI pipeline implementation (#2) 
* init

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix

Signed-off-by: Isotr0py <mozf@inferact.ai>

---------

Signed-off-by: Isotr0py <mozf@inferact.ai>
 2026-09-16 15:14:23 +08:00
Sy03 333f14e97e [Bugfix][MiniCPM-o] Align structured chat content with native omni rendering (#7344) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-10 11:38:11 +08:00
Sy03 b02862beee [Model] Optimize MOSS-TTS Local batched execution and streaming codec (#7202) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-09 13:02:12 -07:00
bojiang-li 504e5a6a6e [Doc] Add dual DGX Spark MiniMax-H3 results (#7343) 
Signed-off-by: bojiang-li <327132355+bojiang-li@users.noreply.github.com>
 2026-09-09 18:48:41 +00:00
NumberWan 038c9b948f [Bugfix] Resolve #6931 hub FA3 on torch 2.13 via kernels 0.16.1 (#7185) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
 2026-09-09 10:20:34 +00:00
chi030303 c66df06c0f [Doc] Qwen3-TTS: add 0.6B on 1x A100 40GB (#7289) 
Signed-off-by: chi030303 <106855944+chi030303@users.noreply.github.com>
 2026-09-09 07:56:14 +00:00
Sparks 7712701578 [Doc] [Recipe] Add FLUX.1-schnell recipe for RTX 5090 32GB (#7299) 
Signed-off-by: Sparks-M <41097544+Sparks-M@users.noreply.github.com>
 2026-09-09 07:54:46 +00:00
Shuolei Wang 5378b77be4 [Model][JoyAI-VL-Interaction] Add native multi-stage pipeline (#5352) 
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
 2026-09-08 05:07:08 -07:00
Yueqian Lin e1c59edec6 [Refactor] fish_speech: vendor DAC codec modules, drop external fish-speech dependency (#4871) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-09-08 09:22:36 +08:00
Jierui (Jerry) Xu c3498fb869 [Model] Add Boogu-Image Turbo pipeline resolution (#6961) 
Signed-off-by: Jerry Xu <xjr2423@gmail.com>
 2026-09-08 08:59:23 +08:00
Mu GuanLin 3204a0b2ad [Model][Perf] Optimize LTX Ulysses sequence parallelism (#7079) 
Signed-off-by: mglyn <1203789601@qq.com>
 2026-09-07 10:48:26 +00:00
FredHuangNV 85b0bad2d3 [Frontend][Model] Add Cosmos3 control uploads to video API (#7027) 
Signed-off-by: Fred Huang <frhuang@nvidia.com>
 2026-09-07 01:38:56 -07:00
Lucas Ruan 34aa1d2afa [Doc] Add A100 recipe for Wan2.2-TI2V-5B (#7189) 
Signed-off-by: lucasruan1618 <huy40025071@gmail.com>
 2026-09-07 14:09:32 +08:00
BruceLoveDecimal 5cd925631c [Feature][LingBot World]stepwise execution (#6844) 
Signed-off-by: BruceLoveDecimal <liuqihao970610@gmail.com>
Signed-off-by: liuqihao <liuqihao970610@gmail.com>
 2026-09-07 13:38:23 +08:00
汪志鹏andClaude Opus 5 af74a5a155 [Feature][MiniMax-H3] Support the full LightX2V Turbo LoRA matrix (#7062) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-06 20:23:36 +08:00
DMAN 06541d1f52 [Model] FP8 Support For Boogu-Image (#6925) 
Signed-off-by: Dmaner <2663515256@qq.com>
 2026-09-06 04:48:38 +00:00
SYLAR ea3cf99894 [diffusion][feature] Add component-selective offload policies (#5929) 
Signed-off-by: lishunyang12 <lishunyang12@163.com>
 2026-09-06 05:40:09 +08:00
chickeytonClaude Fable 5Gao Han
 		5d7fdf9350 [Core] Graduate MiniCPM-o 4.5 and PersonaPlex full-duplex serving out of experimental (#6196) 
Signed-off-by: chickeyton <ngton2014@gmail.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-09-05 08:20:21 +00:00
Hongsheng Liu d4096e01fe [Model] Add native MAGI-2 Preview diffusion support (#5918) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-09-05 15:32:30 +08:00
vOv 13c617a140 [Model] Add sequence parallelism to SANA-Video 2B (#5940) 
Signed-off-by: cr-gao <gaochenrui@sjtu.edu.cn>
 2026-09-05 15:22:53 +08:00
boatman 348fd0fc07 [Feat][dots.tts] Add Online Serving Support (#6235) 
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com>
 2026-09-04 14:44:10 -07:00
TriggerandGao Han ea14b04e9f [Bugfix][BAGEL] Restore original denoise step count (#7049) 
Signed-off-by: Trigger <Meng.Bo.Wang@outlook.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-09-04 17:05:05 +00:00
Yuekai ZhangandClaude Fable 5 4179b0cae1 [Perf][NemotronVoiceChat] Fast defaults for all three deploy profiles: 2.8x offline, realtime full-duplex audio (#6354) 
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-09-03 18:49:19 -07:00
Trigger e43c787544 [Model] Add BAGEL step execution and continuous batching (#6359) 2026-09-04 01:36:11 +00:00
ZhanshenzhuiandSy03 48ae412a2f [model]Add experimental Mage-VL full-duplex adapter and offline integration (#6537) 
Signed-off-by: Zhanshenzhui <99891002+zyforsure@users.noreply.github.com>
Co-authored-by: Sy03 <1370724210@qq.com>
 2026-09-02 20:25:32 -07:00
hurukawa 8f284b343a [Feature][Diffusion]: CFG parallelism support for Boogu-Image (#6786) 
Signed-off-by: nagisa <1434936049@qq.com>
Signed-off-by: nagisa-kun <1434936049@qq.com>
 2026-09-03 10:37:21 +08:00
GXINandHongsheng Liu 3ab582c993 [Bugfix][Qwen3-TTS] Reject task mismatches before EngineCore dispatch (#6113) 
Signed-off-by: XIN GAO <1037396230@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-02 11:16:55 -07:00
vOvHongsheng LiuAlicia
 		3b6ce98421 [Model] Add Cache-DiT and CPU offload support for SANA-Video 2B (#5882) 
Signed-off-by: cr-gao <gaochenrui@sjtu.edu.cn>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-09-02 23:40:48 +08:00
Bo Li 5d20f6b900 [Doc] Refine TRTLLM attention guidance (#6724) 
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
 2026-09-02 19:42:37 +08:00
汪志鹏 c59b48b47e [Feature][MiniMax-H3] Complete VSA and Ulysses support (#6909) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-09-02 07:19:21 -04:00
vOv 57c08d34c3 [Model] Add TP and CFG parallelism to SANA-Video 2B (#5861) 
Signed-off-by: cr-gao <gaochenrui@sjtu.edu.cn>
 2026-09-02 17:44:41 +08:00
DanaerLeeandHongsheng Liu 0288c3f56e [Model] Support SenseNova-U1.5-8B-MoT and its distilled 8-step LoRA (#6516) 
Signed-off-by: MrlixiangWE <mrdanaer@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-01 22:53:53 +08:00
XRay2016 8e26dce1f4 [Diffusion] Add Boogu Edit-Turbo support docs and tests (#6701) 
Signed-off-by: xRay2016 <1150722393@qq.com>
 2026-09-01 08:41:23 +08:00
汪志鹏andHongsheng Liu 86b85c078b [Feature][MiniMax-H3] Fuse the FastH3 four-step adapter at load time (#6714) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-31 12:04:58 +08:00
mokekeandmokashliu 1a9d598d2b [Frontend] Encode interleaved video frames in parallel (#6776) 
Signed-off-by: mokashliu <mokashliu@tencent.com>
Co-authored-by: mokashliu <mokashliu@tencent.com>
 2026-08-31 07:04:27 +08:00
Sy03 3adee400a0 [Bugfix][Qwen3-TTS] Handle codec generations that exhaust their token budget (#6728) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-08-29 08:59:05 -07:00
Zijun HuangandWeiQing Chen 08cdee8dc4 [Feature] Support MiniMax-H3 FlashGen native LoRA with the legacy manager (#6666) 
Signed-off-by: Zijun Huang <zijunhua@usc.edu>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
 2026-08-29 06:52:19 +00:00
Canlin Guolishunyang12yuanwu
 		b6a4baa659 [Feature] Support text encoder Disaggregation for H3 (#5885) 
Signed-off-by: Canlin Guo <canlinguosdu@gmail.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Signed-off-by: yuanwu <yuan.wu@intel.com>
Co-authored-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: yuanwu <yuan.wu@intel.com>
 2026-08-28 14:27:05 +08:00
Hongsheng Liuandhsliuustc0106 4245fd6955 [Fix] Restore MammothModa2 support (#6694) 
Signed-off-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
Co-authored-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
 2026-08-28 08:32:32 +08:00
Mu GuanLinandHongsheng Liu 5e4daaf197 [Diffusion] Add LTX-2.5 Diffusion VAE decoder support (#6189) 
Signed-off-by: mglyn <1203789601@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-27 23:54:40 +08:00
Sy03andsy03 432b2c6ca3 [Model] Add native full-duplex Nemotron VoiceChat serving (#6089) 
Signed-off-by: Sysy <1370724210@qq.com>
Signed-off-by: sy03 <137072421@qq.com>
Co-authored-by: sy03 <137072421@qq.com>
 2026-08-26 13:19:20 -07:00
harley 926e7fbfa2 [Recipe] zai-org/GLM-TTS (2x non-standard RTX 4090 48GB) (#5769) 
Signed-off-by: harley <480497125@qq.com>
 2026-08-26 22:16:49 +08:00
Zhichao Zhang 816335cb46 [Doc] Add MiMo-Audio recipe for RTX 5090/5090D 32GB (#6559) 
Signed-off-by: chaosansui <zzc15560846421@163.com>
 2026-08-26 16:26:24 +08:00
Mu GuanLinandHongsheng Liu 59cf3add40 [Feature] Support MiniMax-H3 Turbo LoRA with DLO (#6550) 
Signed-off-by: mglyn <1203789601@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-26 16:15:47 +08:00