tzhouam and Claude Fable 5.1
536866b916
[Edge] Classify and document the edge env vars; fix the client decoder call
...
- environment_variable_inventory: VLLM_OMNI_CPU_INDUCTOR / HW_PROFILE /
INIT_TIMELINE / STEP_STATS_DIR (public), STAGE_ID / INIT_TIMELINE_DIR
(internal); reference page rows added.
- edge_decoder_client.LocalDecoder: batched_chunked_decode takes a [B, Q, F]
tensor, not a list (the WP5 e2e run failed with a TypeError); the decoder
now also carries the ramp schedule attributes the Code2Wav stage sets.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-09 15:22:52 +08:00
tzhouam and Claude Fable 5.1
8e912db6c8
[Edge][WP6] Slim the AR/TTS import path and add an edge dependency set
...
* diffusers is no longer imported by vllm_omni.diffusion.data (annotation only);
vllm_omni.diffusion.io_support (diffusion registry + distributed state) is
imported lazily by config_factory and async_omni_engine, so constructing a
Qwen3-TTS pipeline config / AsyncOmni no longer loads diffusers,
x_transformers or whisper (guarded by tests/config/test_import_guard.py).
* requirements/common.txt = core.txt (AR/speech runtime) + media.txt
(diffusion/video/extra audio); VLLM_OMNI_EDGE_BUILD=1 makes setup.py use
requirements/<device>-edge.txt (core only); `edge` extra adds websockets and
sounddevice for on-device clients.
Measured on this host (CUDA venv): `import vllm` 7.0-7.9 s, `import vllm_omni`
adds ~20 s of which ~17 s is transformers' own import (transformers.utils.import_utils)
pulled in by vllm_omni.patch / vllm_omni.version; that floor is not
vLLM-Omni's and is unchanged. Guard test: no diffusion module loaded on the TTS path.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-09 11:14:50 +08:00
tzhouam and Claude Fable 5.1
49a839ecf9
[Edge][WP5] Add codec-token streaming mode with on-device decoding
...
* qwen3_tts_talker_only pipeline (single terminal LLM_AR stage,
final_output_type "latent") + vllm_omni/deploy/qwen3_tts_talker_only.yaml;
a terminal talker publishes its per-step codes.audio rows to the client
(omni_client_multimodal_output_keys).
* /v1/audio/speech/stream: session.config.output_mode="codec_tokens" streams
codec.start (JSON) + binary frames (<IHB seq/n_frames/flags + int16 rows,
vllm_omni/entrypoints/openai/codec_stream.py) + codec.done; new input.cancel
message aborts an in-flight generation for both codec and PCM sessions
(single reader task + session inbox replaces direct socket reads); optional
session.config.seed; audio.done gains "cancelled" only when it happened.
* serving_speech._generate_codec_chunks aligns the cumulative code tensor to
output token ids (drops prefill placeholder/EOS rows) and emits deltas.
* TTS adapters: supports_codec_stream / codec_stream_spec (Qwen3-TTS: 16
codebooks, 2048 ids, 12.5 Hz, 24 kHz, decoder qwen3_tts_tokenizer_v2_12hz).
* examples/online_serving/text_to_speech/qwen3_tts/edge_decoder_client.py
decodes locally with Qwen3TTSTokenizerV2Decoder (ramp schedule, 72-frame
context, cancel/reset), codec_parity_check.py and
benchmarks/tts/codec_stream_e2e.py compare against the two-stage server.
* docs/serving/codec_token_streaming.md; CPU unit tests for the wire format,
alignment, session config and handler (frames, unsupported adapter, cancel);
L3 e2e tests/e2e/online_serving/test_qwen3_tts_codec_stream.py.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-09 11:10:13 +08:00
BruceLoveDecimal
5cd925631c
[Feature][LingBot World]stepwise execution ( #6844 )
...
Signed-off-by: BruceLoveDecimal <liuqihao970610@gmail.com >
Signed-off-by: liuqihao <liuqihao970610@gmail.com >
2026-09-07 13:38:23 +08:00
f26f9c3a7f
[Diffusion] Implement paged AR↔DiT KV connector ("v1", reusing vllm's native mooncake) ( #6310 )
...
Signed-off-by: asukaqaq-s <1311722138@qq.com >
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com >
Co-authored-by: asukaqaq-s <1311722138@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-09-06 05:41:54 +08:00
SYLAR
ea3cf99894
[diffusion][feature] Add component-selective offload policies ( #5929 )
...
Signed-off-by: lishunyang12 <lishunyang12@163.com >
2026-09-06 05:40:09 +08:00
Zhou Taichang and Claude Fable 5
142151fef5
Parallel stage initialization (admission + SH/EX device locks) (Worker refactor 1/N) ( #5224 )
...
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-09-05 22:22:41 +08:00
Hongsheng Liu
d4096e01fe
[Model] Add native MAGI-2 Preview diffusion support ( #5918 )
...
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com >
2026-09-05 15:32:30 +08:00
SYLAR and hsliuustc0106
7493cbd3c2
[Bugfix] Fix Qwen3-Omni MoE backend selection ( #7019 )
...
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com >
Co-authored-by: hsliuustc0106 <liuhongsheng4@huawei.com >
2026-09-04 21:16:58 +08:00
Yuekai Zhang and Claude Fable 5
4179b0cae1
[Perf][NemotronVoiceChat] Fast defaults for all three deploy profiles: 2.8x offline, realtime full-duplex audio ( #6354 )
...
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-09-03 18:49:19 -07:00
Trigger
e43c787544
[Model] Add BAGEL step execution and continuous batching ( #6359 )
2026-09-04 01:36:11 +00:00
maithilijoshi20 and maithilijoshi20
63ed0fdef4
[Bugfix] Fix vLLM serve error response imports ( #6707 )
...
Signed-off-by: maithilijoshi20 <maithilijoshi2003@gmail.com >
Signed-off-by: maithilijoshi20 <97733343+maithilijoshi20@users.noreply.github.com >
Co-authored-by: maithilijoshi20 <maithilijoshi2003@gmail.com >
2026-09-02 15:48:01 -07:00
汪志鹏
c59b48b47e
[Feature][MiniMax-H3] Complete VSA and Ulysses support ( #6909 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
2026-09-02 07:19:21 -04:00
DanaerLee and Hongsheng Liu
0288c3f56e
[Model] Support SenseNova-U1.5-8B-MoT and its distilled 8-step LoRA ( #6516 )
...
Signed-off-by: MrlixiangWE <mrdanaer@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-09-01 22:53:53 +08:00
Andy Zhou
2298359f67
[Bugfix][Diffusion] Preserve stage attention backend ( #6645 )
...
Signed-off-by: AndyZhou952 <jzhoubc@connect.ust.hk >
2026-09-01 20:18:44 +08:00
Alex Brooks
a65e89cbd4
[Core] Validate PipelineConfig in __post_init__ ( #6829 )
...
Signed-off-by: Alex Brooks <albrooks@redhat.com >
2026-09-01 11:17:57 +08:00
m0g3r
8fe913e461
[Core] Flag pipelines with no terminal output stage in PipelineConfig.validate() ( #6291 )
...
Signed-off-by: m0g3r <87276771+m0g3r@users.noreply.github.com >
2026-08-30 21:33:06 +00:00
baonudesifeizhai
8be601ed7f
[Diffusion] Add native SymmMem Fast Ulysses transport ( #6340 )
...
Signed-off-by: roG0d <baonudesifeizhai@gmail.com >
2026-08-30 04:14:21 +00:00
Yancy and Asthenia
73893e73ea
[Bugfix] Fix create_error_response import across vLLM versions ( #6773 )
...
Signed-off-by: Asthenia <asthenia0412@gmail.com >
Co-authored-by: Asthenia <asthenia0412@gmail.com >
2026-08-29 17:01:51 +00:00
b6a4baa659
[Feature] Support text encoder Disaggregation for H3 ( #5885 )
...
Signed-off-by: Canlin Guo <canlinguosdu@gmail.com >
Signed-off-by: lishunyang12 <lishunyang12@163.com >
Signed-off-by: yuanwu <yuan.wu@intel.com >
Co-authored-by: lishunyang12 <lishunyang12@163.com >
Co-authored-by: yuanwu <yuan.wu@intel.com >
2026-08-28 14:27:05 +08:00
Hongsheng Liu and hsliuustc0106
4245fd6955
[Fix] Restore MammothModa2 support ( #6694 )
...
Signed-off-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com >
Co-authored-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com >
2026-08-28 08:32:32 +08:00
NATURE
54c1fc64b7
[Bugfix] Gate async-chunk segment resume by session mode ( #6680 )
...
Signed-off-by: natureofnature <wzliu@connect.hku.hk >
2026-08-27 18:52:09 +08:00
Zheng Wengang and amy-why-3459
94b1546ce3
[Model] Qwen3-Omni: register thinker-only pipeline for Instruct serve ( #6284 )
...
Signed-off-by: ZhengWG <zwg0606@gmail.com >
Signed-off-by: Zheng Wengang <zwg0606@gmail.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-08-27 08:21:03 +00:00
Zhou Taichang and Claude Opus 5
651b03225b
[Rebase] Rebase to vllm 0.28.0 ( #6606 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-27 06:12:55 +00:00
Yueqian Lin and Yueqian Lin
8cfede4309
[Perf][Engine] Event-driven orchestration loop (opt-in) — S1 of #4855 ( #5221 )
...
Signed-off-by: Yueqian Lin <gao@yueqian.us >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Yueqian Lin <gao@yueqian.us >
2026-08-26 22:15:25 +08:00
Gao Han
7439ed41c2
Revert "[Bugfix] Restore name-based model detection for HF cache snapshot paths" ( #6642 )
2026-08-26 15:02:47 +08:00
Yueqian Lin
7ba5e8a146
[Bugfix] Restore name-based model detection for HF cache snapshot paths ( #6624 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
2026-08-25 23:27:35 -07:00
Hongsheng Liu
a708ae5b7a
[Bugfix] Reconcile env-var inventory with post-#6217 main drift ( #6631 )
...
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com >
2026-08-26 06:13:28 +08:00
Hongsheng Liu
780e8d40aa
[Doc] Add environment variables configuration reference ( #6217 )
...
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-26 04:58:54 +08:00
Hongsheng Liu and hsliuustc0106
c608910d20
[Diffusion] Register HWR mmap for direct DLO H2D ( #6591 )
...
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Signed-off-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com >
Co-authored-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com >
2026-08-25 23:30:11 +08:00
wy17003 and Hongsheng Liu
b454e34966
[Refactor][2/N Scheduler]Replace _FULL_PAYLOAD_INPUT_STAGES with a resolved stage transport capability. ( #6149 )
...
Signed-off-by: wy17003 <yaow17003@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-25 15:24:59 +08:00
Hongsheng Liu and hsliuustc0106
8360346a66
[Diffusion] Integrate no-AllGather DLO with Host Weight Runtime ( #6486 )
...
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Signed-off-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com >
Co-authored-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com >
2026-08-25 08:03:53 +08:00
bjf-frz and Hongsheng Liu
072bfc02dd
[FEAT]Add FastVideo VSA backend for Wan2.2 ( #4820 )
...
Signed-off-by: bjf-frz <frz123db@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-24 14:48:31 +08:00
wangyu
1e2a6446f7
[CI] Remove AudioX and MammothModa2 support ( #6353 )
...
Signed-off-by: wangyu <410167048@qq.com >
2026-08-23 04:22:05 +00:00
wangyu
2e096788a6
[CI]Remove SoulX-Singer support ( #6362 )
...
Signed-off-by: wangyu <410167048@qq.com >
2026-08-23 07:16:32 +08:00
maithilijoshi20 and Hongsheng Liu
bd4f9acfd3
[Bugfix] Preserve pipeline sampling constraints ( #6182 )
...
Signed-off-by: maithilijoshi20 <97733343+maithilijoshi20@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-22 14:57:09 +00:00
b39f599b09
[Core] Add UniProc diffusion executor for single-GPU ( #6308 )
...
Signed-off-by: Rahul Steiger <rsteiger@nvidia.com >
Signed-off-by: Rahul Steiger <rsteiger@aws-cmh-slurm-1-vscode-04.cm.cluster >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Rahul Steiger <rsteiger@aws-cmh-slurm-1-vscode-04.cm.cluster >
2026-08-22 21:58:21 +08:00
Wang fuyin and Hongsheng Liu
72ee535fcd
[Diffusion] Add native paged KV backend for diffusion workers ( #6102 )
...
Signed-off-by: Acerak01-fy <wfy2003324@163.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-22 21:45:45 +08:00
Inesh Reddy Chappidi
cd3867c373
[Model] Suppress silence codec tokens for the first N Qwen3-TTS decod… ( #5048 )
...
Signed-off-by: Inesh Reddy <ineshreddy249@gmail.com >
2026-08-20 09:22:37 -07:00
amy-why-3459 and Cursor
9c61d5b67b
[Bugfix][MiniCPM-o] Stop Talker mid-utterance truncation and load cod… ( #6346 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-08-20 11:50:07 +08:00
bjf-frz and Hongsheng Liu
14bd8f8772
[REFACT]Refactor diffusion parallel state ( #5531 )
...
Signed-off-by: bjf-frz <frz123db@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-19 11:23:10 +08:00
Hongsheng Liu
c3de04e127
[Bugfix] Avoid eager pi0 runtime import in pipeline registry ( #6322 )
...
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-18 23:42:03 +08:00
Wang fuyin and Hongsheng Liu
e1c5f59aab
[Config] Reuse vLLM configs in VllmOmniConfig ( #6050 )
...
Signed-off-by: Acerak01-fy <wfy2003324@163.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-18 21:27:58 +08:00
Yukim1 and zwhzzz0821
0a1846aa45
[Diffusion] Add native KV cache initialization and Scheduler-managed block allocation ( #6094 )
...
Signed-off-by: zwhzzz0821 <zwhzzz0821@users.noreply.github.com >
Co-authored-by: zwhzzz0821 <zwhzzz0821@users.noreply.github.com >
2026-08-18 07:00:24 +00:00
Yueqian Lin
b283215c28
[Model] Add MiniMax Music 3 text-to-music ( #6186 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
2026-08-18 04:33:13 +00:00
amy-why-3459
3c6648d8a5
[Bugfix][NPU] Break the pytest DiffusionOutput circular import ( #6293 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-08-18 10:11:10 +08:00
Yichen Wang and Didan Deng
2a9f6aaa8e
Add π0 VLA model support ( #4222 )
...
Signed-off-by: Yichen Wang <yichen.wang@bytedance.com >
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
2026-08-17 14:29:17 +08:00
Yukim1 and Hongsheng Liu
baba7d1e2e
[Refactor] Remove legacy stage_args YAML loader ( #6200 )
...
Signed-off-by: zwhzzz0821 <2831474076@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-16 11:22:28 +08:00
Ma1oneZhang
6f36fd3b78
[Bugfix] Preserve object storage URIs during model resolution ( #5036 )
...
Signed-off-by: Ziyang Zhang <hafuhafu@qq.com >
2026-08-15 14:11:48 +08:00
35cacf020c
[Model] Add Gepard-1.0 native-AR FSQ/NanoCodec TTS (offline inference) ( #5666 )
...
Signed-off-by: Mengjie Zhao <zmj0129@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-08-14 22:06:05 -07:00