 tzhouamandClaude Fable 5.1
|
49a839ecf9
|
[Edge][WP5] Add codec-token streaming mode with on-device decoding
* qwen3_tts_talker_only pipeline (single terminal LLM_AR stage,
final_output_type "latent") + vllm_omni/deploy/qwen3_tts_talker_only.yaml;
a terminal talker publishes its per-step codes.audio rows to the client
(omni_client_multimodal_output_keys).
* /v1/audio/speech/stream: session.config.output_mode="codec_tokens" streams
codec.start (JSON) + binary frames (<IHB seq/n_frames/flags + int16 rows,
vllm_omni/entrypoints/openai/codec_stream.py) + codec.done; new input.cancel
message aborts an in-flight generation for both codec and PCM sessions
(single reader task + session inbox replaces direct socket reads); optional
session.config.seed; audio.done gains "cancelled" only when it happened.
* serving_speech._generate_codec_chunks aligns the cumulative code tensor to
output token ids (drops prefill placeholder/EOS rows) and emits deltas.
* TTS adapters: supports_codec_stream / codec_stream_spec (Qwen3-TTS: 16
codebooks, 2048 ids, 12.5 Hz, 24 kHz, decoder qwen3_tts_tokenizer_v2_12hz).
* examples/online_serving/text_to_speech/qwen3_tts/edge_decoder_client.py
decodes locally with Qwen3TTSTokenizerV2Decoder (ramp schedule, 72-frame
context, cancel/reset), codec_parity_check.py and
benchmarks/tts/codec_stream_e2e.py compare against the two-stage server.
* docs/serving/codec_token_streaming.md; CPU unit tests for the wire format,
alignment, session config and handler (frames, unsupported adapter, cancel);
L3 e2e tests/e2e/online_serving/test_qwen3_tts_codec_stream.py.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-09 11:10:13 +08:00 |
|
|
tzhouamandClaude Fable 5.1
|
35e43b9983
|
[Edge][WP3] Add CPU OmniPlatform (AR + generation stages on vLLM's CPU backend)
* vllm_omni/platforms/cpu/platform.py: CPUOmniPlatform(OmniPlatform, CpuPlatform)
selected when VLLM_TARGET_DEVICE=cpu or a +cpu vLLM wheel is installed
(builtin plugin "cpu"; explicit CPU target short-circuits accelerator probes
like vLLM does). NUMA-based memory queries, one logical device, no graph
wrapper (sentinel type for isinstance checks), inductor opt-in via
VLLM_OMNI_CPU_INDUCTOR=1, diffusion stages unsupported.
* vllm_omni/platforms/cpu/worker: CPUARWorker / CPUGenerationWorker compose
vLLM's CPUWorker with the Omni mixin (XPU pattern); CPUARModelRunner /
CPUGenerationModelRunner compose the Omni runners with CPUModelRunner
(no graphs, no async omni output; sleep/wake return error ACKs).
* patch.py skips the CuMem patch on CPU-only torch; Qwen3-TTS speaker encoder
and prompt-embeds dtype follow the model dtype (float32/float16 on CPU).
* vllm_omni/deploy/qwen3_tts.yaml: `platforms: cpu:` overlay (eager, no async
scheduling, absolute kv_cache_memory_bytes, disjoint OMP thread ranges).
* requirements/cpu.txt (+psutil, intel-openmp), docs/getting_started/installation/cpu.md.
Measured (this x86 Xeon 8480C host, vllm 0.28.0+cpu, bf16, 12+4 threads):
Qwen3-TTS 0.6B CustomVoice end-to-end on CPU, 12 prompts: TTFA p50 292 ms,
RTF ~1.7 (25-frame chunk ~4.6 s), init 185 s. Result JSON + init timeline
committed under benchmarks/tts/edge_results/.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-09 11:02:14 +08:00 |
|
|
Zeyu Huang | 黃澤宇andAlicia
|
4b96c57730
|
[skip ci][Doc] Module design doc for diffusion runtime (#6440)
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
|
2026-09-07 21:52:24 +08:00 |
|
|
Mu GuanLin
|
3204a0b2ad
|
[Model][Perf] Optimize LTX Ulysses sequence parallelism (#7079)
Signed-off-by: mglyn <1203789601@qq.com>
|
2026-09-07 10:48:26 +00:00 |
|
|
FredHuangNV
|
85b0bad2d3
|
[Frontend][Model] Add Cosmos3 control uploads to video API (#7027)
Signed-off-by: Fred Huang <frhuang@nvidia.com>
|
2026-09-07 01:38:56 -07:00 |
|
|
WeiQing Chen
|
b78c31eda6
|
[Doc] Update vLLM-Omni WeChat QR code (#7178)
Signed-off-by: David Chen <530634352@qq.com>
|
2026-09-07 06:28:30 +00:00 |
|
|
BruceLoveDecimal
|
5cd925631c
|
[Feature][LingBot World]stepwise execution (#6844)
Signed-off-by: BruceLoveDecimal <liuqihao970610@gmail.com>
Signed-off-by: liuqihao <liuqihao970610@gmail.com>
|
2026-09-07 13:38:23 +08:00 |
|
|
Zeyu Huang | 黃澤宇
|
2df95de421
|
[Core][Diffusion] Refactor prompt updates into a modality-ready interaction framework (#6294)
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
|
2026-09-07 11:40:04 +08:00 |
|
|
Xiaobaixia
|
615f97b9c7
|
[Feature] Define the runner preprocess phase contract and regression coverage (#7136)
Signed-off-by: xiaobaixia222 <289982752@qq.com>
|
2026-09-06 17:20:04 +00:00 |
|
|
汪志鹏andClaude Opus 5
|
af74a5a155
|
[Feature][MiniMax-H3] Support the full LightX2V Turbo LoRA matrix (#7062)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
2026-09-06 20:23:36 +08:00 |
|
|
DMAN
|
06541d1f52
|
[Model] FP8 Support For Boogu-Image (#6925)
Signed-off-by: Dmaner <2663515256@qq.com>
|
2026-09-06 04:48:38 +00:00 |
|
|
Max Freedom Pollard
|
946b843e61
|
[Doc] Fix markdownlint findings in the serving API reference (#7111)
Signed-off-by: Max Freedom Pollard <272618364+MaxFreedomPollard@users.noreply.github.com>
|
2026-09-06 08:40:10 +08:00 |
|
|
SYLAR
|
ea3cf99894
|
[diffusion][feature] Add component-selective offload policies (#5929)
Signed-off-by: lishunyang12 <lishunyang12@163.com>
|
2026-09-06 05:40:09 +08:00 |
|
|
|
5d7fdf9350
|
[Core] Graduate MiniCPM-o 4.5 and PersonaPlex full-duplex serving out of experimental (#6196)
Signed-off-by: chickeyton <ngton2014@gmail.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
|
2026-09-05 08:20:21 +00:00 |
|
|
Hongsheng Liu
|
d4096e01fe
|
[Model] Add native MAGI-2 Preview diffusion support (#5918)
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
|
2026-09-05 15:32:30 +08:00 |
|
|
vOv
|
13c617a140
|
[Model] Add sequence parallelism to SANA-Video 2B (#5940)
Signed-off-by: cr-gao <gaochenrui@sjtu.edu.cn>
|
2026-09-05 15:22:53 +08:00 |
|
|
boatman
|
348fd0fc07
|
[Feat][dots.tts] Add Online Serving Support (#6235)
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com>
|
2026-09-04 14:44:10 -07:00 |
|
|
TriggerandGao Han
|
ea14b04e9f
|
[Bugfix][BAGEL] Restore original denoise step count (#7049)
Signed-off-by: Trigger <Meng.Bo.Wang@outlook.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
|
2026-09-04 17:05:05 +00:00 |
|
|
SYLARandhsliuustc0106
|
22cefe04b4
|
[Bugfix] Update Hub kernel dependency floor (#7020)
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
Co-authored-by: hsliuustc0106 <liuhongsheng4@huawei.com>
|
2026-09-04 22:15:55 +08:00 |
|
|
SYLARandhsliuustc0106
|
7493cbd3c2
|
[Bugfix] Fix Qwen3-Omni MoE backend selection (#7019)
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
Co-authored-by: hsliuustc0106 <liuhongsheng4@huawei.com>
|
2026-09-04 21:16:58 +08:00 |
|
|
|
8d98cc0cb7
|
[Benchmark] Add local OmniInteract performance cases (#6817)
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
Co-authored-by: Ruirui Yang | Rein <73573651+R2-Y@users.noreply.github.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
|
2026-09-04 10:36:02 +08:00 |
|
|
Trigger
|
e43c787544
|
[Model] Add BAGEL step execution and continuous batching (#6359)
|
2026-09-04 01:36:11 +00:00 |
|
|
hurukawa
|
8f284b343a
|
[Feature][Diffusion]: CFG parallelism support for Boogu-Image (#6786)
Signed-off-by: nagisa <1434936049@qq.com>
Signed-off-by: nagisa-kun <1434936049@qq.com>
|
2026-09-03 10:37:21 +08:00 |
|
|
GXINandHongsheng Liu
|
3ab582c993
|
[Bugfix][Qwen3-TTS] Reject task mismatches before EngineCore dispatch (#6113)
Signed-off-by: XIN GAO <1037396230@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
|
2026-09-02 11:16:55 -07:00 |
|
|
|
95a9311696
|
[CI] Select mirror_hardwares presets via MIRROR_HW (H100/B200) (#5543)
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Signed-off-by: [Your Name] <your.email@example.com>
Signed-off-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
Co-authored-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Bo Li <22713281+bobboli@users.noreply.github.com>
|
2026-09-02 15:39:49 +00:00 |
|
|
Bo Li
|
5d20f6b900
|
[Doc] Refine TRTLLM attention guidance (#6724)
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
|
2026-09-02 19:42:37 +08:00 |
|
|
汪志鹏
|
c59b48b47e
|
[Feature][MiniMax-H3] Complete VSA and Ulysses support (#6909)
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
|
2026-09-02 07:19:21 -04:00 |
|
|
vOv
|
57c08d34c3
|
[Model] Add TP and CFG parallelism to SANA-Video 2B (#5861)
Signed-off-by: cr-gao <gaochenrui@sjtu.edu.cn>
|
2026-09-02 17:44:41 +08:00 |
|
|
wangyuandamy-why-3459
|
b6e72f3879
|
[CI] Migrate L4 jobs to l4-k8s and split by GPU count (#6890)
Signed-off-by: wangyu <410167048@qq.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
|
2026-09-02 17:09:59 +08:00 |
|
|
DanaerLeeandHongsheng Liu
|
0288c3f56e
|
[Model] Support SenseNova-U1.5-8B-MoT and its distilled 8-step LoRA (#6516)
Signed-off-by: MrlixiangWE <mrdanaer@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
|
2026-09-01 22:53:53 +08:00 |
|
|
Hongsheng Liuandhsliu_ustc
|
44c8d1c449
|
docs: add concise API server endpoint guide (#6114)
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: hsliu_ustc <hsliu_ustc@noreply.gitcode.com>
|
2026-09-01 20:34:09 +08:00 |
|
|
Hongsheng Liu
|
d1c8beeb18
|
[Doc] Update README and docs for v0.28.0 (#6858)
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
|
2026-09-01 19:50:26 +08:00 |
|
|
AuFlowandHongsheng Liu
|
49c6fc1540
|
[Frontend] Add configurable output sample rate to Speech API (#6553)
Signed-off-by: 郑倩倩 <2458983186@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
|
2026-09-01 17:25:09 +08:00 |
|
|
XRay2016
|
8e26dce1f4
|
[Diffusion] Add Boogu Edit-Turbo support docs and tests (#6701)
Signed-off-by: xRay2016 <1150722393@qq.com>
|
2026-09-01 08:41:23 +08:00 |
|
|
WeiQing Chen
|
4cca28e9bd
|
[Doc] Update vLLM-Omni WeChat QR code (#6832)
Signed-off-by: David Chen <530634352@qq.com>
|
2026-08-31 09:44:33 +08:00 |
|
|
wangyu
|
e338f0ece3
|
[CI]Split GPU jobs by cards_* and reject hand-written SKU marks (#6650)
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: [Your Name] <your.email@example.com>
|
2026-08-29 18:42:33 +08:00 |
|
|
HAAZZZEEEE
|
c50b7de167
|
[NPU][Diffusion] MiniMax-H3 RainFusion end_step tail fallback (#6037)
Signed-off-by: HAAZZZEEEE <wang-taicheng@qq.com>
|
2026-08-29 18:36:05 +08:00 |
|
|
ooooooye
|
5a5bc55c25
|
[Feature][Diffusion] Allow online INT8 quantization with DLO AllGather (#6573)
Signed-off-by: brandneway <gyuan4892@gmail.com>
|
2026-08-29 09:50:30 +00:00 |
|
|
|
bb5bf8fd0a
|
[Bugfix] Fix colocate-async sleep admission race and deliver AR abort tokens (#6367)
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Signed-off-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Samit <285365963@qq.com>
|
2026-08-29 09:11:39 +00:00 |
|
|
Zijun HuangandWeiQing Chen
|
08cdee8dc4
|
[Feature] Support MiniMax-H3 FlashGen native LoRA with the legacy manager (#6666)
Signed-off-by: Zijun Huang <zijunhua@usc.edu>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
|
2026-08-29 06:52:19 +00:00 |
|
|
Chendi.Xue
|
034af1f657
|
[CI][XPU]switch to use vLLM base docker for XPU (#6727)
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
|
2026-08-29 09:03:00 +08:00 |
|
|
Hongsheng Liu
|
6ec5ab11bd
|
[Bugfix] Make HWR filesystem lifecycle transitions fail-closed (#6692)
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
|
2026-08-28 09:30:17 +08:00 |
|
|
Hongsheng Liuandhsliuustc0106
|
4245fd6955
|
[Fix] Restore MammothModa2 support (#6694)
Signed-off-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
Co-authored-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
|
2026-08-28 08:32:32 +08:00 |
|
|
Zheng Wengangandamy-why-3459
|
94b1546ce3
|
[Model] Qwen3-Omni: register thinker-only pipeline for Instruct serve (#6284)
Signed-off-by: ZhengWG <zwg0606@gmail.com>
Signed-off-by: Zheng Wengang <zwg0606@gmail.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
|
2026-08-27 08:21:03 +00:00 |
|
|
Zhou TaichangandClaude Opus 5
|
651b03225b
|
[Rebase] Rebase to vllm 0.28.0 (#6606)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-08-27 06:12:55 +00:00 |
|
|
Hongsheng Liuandhsliuustc0106
|
da4a08b65b
|
[Diffusion] Add FLUX.2-klein BF16 HWR contract (#6651)
Signed-off-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
|
2026-08-27 12:51:15 +08:00 |
|
|
SYLARandHongsheng Liu
|
0e44fc7406
|
[diffusion][feature] Add offline SVDQuant W4A4 support (#6162)
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
|
2026-08-27 08:45:53 +08:00 |
|
|
|
c2723979c0
|
[Bugfix] Validate per-stage device layout before spawning workers (#5003) (#5742)
Signed-off-by: ChoHee15 <cc5281@126.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
|
2026-08-26 19:25:22 +00:00 |
|
|
Yueqian LinandYueqian Lin
|
8cfede4309
|
[Perf][Engine] Event-driven orchestration loop (opt-in) — S1 of #4855 (#5221)
Signed-off-by: Yueqian Lin <gao@yueqian.us>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <gao@yueqian.us>
|
2026-08-26 22:15:25 +08:00 |
|
|
amy-why-3459
|
5de6f7c37a
|
[Bugfix][MiniCPM-o] Serve native duplex from shipping YAMLs and fence… (#6619)
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
|
2026-08-26 10:56:51 +08:00 |
|