tzhouam
8a4559c46c
Validate MiniCPM-o speech pipeline on native Windows GPU
2026-09-23 16:54:17 +08:00
tzhouam
c0deaa76ba
Repair MiniCPM-o and InternVLA scoped E2E paths
2026-09-23 15:15:07 +08:00
tzhouam
e0142926b6
Document MiniCPM-o checkpoint preflight and current deploy layout
2026-09-23 10:19:10 +08:00
tzhouam
723c550209
Integrate portable edge graph backends with Omni runtime
...
Extract versioned stage contracts, model shared CPU/iGPU/NPU memory constraints, and add explicit graph routes, reservations, output acknowledgements, and worker retirement. Fix native Windows shared-buffer lifetime and host paths. Document qualified hardware paths and remaining rollout gates.
2026-09-22 13:12:33 +08:00
tzhouam and Claude Fable 5.1
33de5d8629
Merge vLLM-Omni v0.29.0rc1 into edge-infer/omni-0.29.0
...
The pair for vLLM v0.29.0: rc1's docs pin vllm==0.29.0 and a static scan of
rc1's vllm.* imports against the v0.29.0 tree finds nothing missing, while
our 0.28-based tree needs seven modules v0.29.0 no longer has
(vllm.inputs.preprocess, entrypoints.launcher, openai.cli_args,
openai.engine.protocol, serve.utils.server_utils, layers.batch_invariant,
models.hunyuan_v1).
Ten files conflicted. Upstream's structure is the base everywhere; the
fork's features are re-attached to it:
- deploy_profile / apply_deploy_profile run before rc1's resolve_omni_config
pops deploy_config (async_omni_engine, cli/serve); the deploy-profile
selector and the resolve_model_config_path chain it needs stay in
entrypoints/utils.py; stage_overrides takes rc1's dict type.
- config_factory keeps the lazy diffusion output-type import; the diffusion
registry import in rc1's new config/resolver.py is made lazy for the same
WP6 reason (TTS path must not load diffusion).
- requirements: our core/media split kept; rc1's transformers>=5.13 floor
and kernels==0.16.1 applied in core.txt / media.txt.
- TTS: rc1's split_granularity and bounded seed plus our output_mode=
codec_tokens, _SessionInbox cancel handling and codec DELTA accumulation;
our private _conditioning_cache_salt is superseded by rc1's public
tts_adapters/base.py::conditioning_cache_salt.
- stage_runtime: rc1's single ExitStack G2 path (lock released before READY)
replaces our two-path block, with the four worker_phase names the init
profiler consumes still emitted; _cpu_placed / host admission kept.
Not yet run against vLLM 0.29.0: the tests listed in the W6 record.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-18 09:57:20 +08:00
Sy03
b02862beee
[Model] Optimize MOSS-TTS Local batched execution and streaming codec ( #7202 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
2026-09-09 13:02:12 -07:00
tzhouam and Claude Fable 5.1
b8d1e44ebe
[Edge][WP5] Codec stream diagnostics: per-request output/codes counters, client saves received codes, offline parity diag
...
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-09 16:02:02 +08:00
tzhouam and Claude Fable 5.1
536866b916
[Edge] Classify and document the edge env vars; fix the client decoder call
...
- environment_variable_inventory: VLLM_OMNI_CPU_INDUCTOR / HW_PROFILE /
INIT_TIMELINE / STEP_STATS_DIR (public), STAGE_ID / INIT_TIMELINE_DIR
(internal); reference page rows added.
- edge_decoder_client.LocalDecoder: batched_chunked_decode takes a [B, Q, F]
tensor, not a list (the WP5 e2e run failed with a TypeError); the decoder
now also carries the ramp schedule attributes the Code2Wav stage sets.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-09 15:22:52 +08:00
tzhouam and Claude Fable 5.1
580a201631
[Edge][WP4] Windowed Code2Wav decoder export (torch.export / ExecuTorch) + client backend; results summarizer
...
vllm_omni/edge/decoder_export.py exports a stateless windowed decoder (context
72 frames + chunk, one program per ramp chunk size) via torch.export (.pt2,
verified here on CPU for chunks 2 and 25 with the real 0.6B decoder: exact
match vs eager, 460 MB fp32 per program) or ExecuTorch XNNPACK (.pte; written
against the executorch 1.x API, not executed: package not installed, install
is permission gated; QNN partitioner export-only/unverified). The WP5 client
gains --decoder-backend executorch --pte-manifest for phone/NPU targets.
benchmarks/tts/edge_results/summarize.py renders all committed JSON as tables.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-09 15:02:19 +08:00
tzhouam and Claude Fable 5.1
49a839ecf9
[Edge][WP5] Add codec-token streaming mode with on-device decoding
...
* qwen3_tts_talker_only pipeline (single terminal LLM_AR stage,
final_output_type "latent") + vllm_omni/deploy/qwen3_tts_talker_only.yaml;
a terminal talker publishes its per-step codes.audio rows to the client
(omni_client_multimodal_output_keys).
* /v1/audio/speech/stream: session.config.output_mode="codec_tokens" streams
codec.start (JSON) + binary frames (<IHB seq/n_frames/flags + int16 rows,
vllm_omni/entrypoints/openai/codec_stream.py) + codec.done; new input.cancel
message aborts an in-flight generation for both codec and PCM sessions
(single reader task + session inbox replaces direct socket reads); optional
session.config.seed; audio.done gains "cancelled" only when it happened.
* serving_speech._generate_codec_chunks aligns the cumulative code tensor to
output token ids (drops prefill placeholder/EOS rows) and emits deltas.
* TTS adapters: supports_codec_stream / codec_stream_spec (Qwen3-TTS: 16
codebooks, 2048 ids, 12.5 Hz, 24 kHz, decoder qwen3_tts_tokenizer_v2_12hz).
* examples/online_serving/text_to_speech/qwen3_tts/edge_decoder_client.py
decodes locally with Qwen3TTSTokenizerV2Decoder (ramp schedule, 72-frame
context, cancel/reset), codec_parity_check.py and
benchmarks/tts/codec_stream_e2e.py compare against the two-stage server.
* docs/serving/codec_token_streaming.md; CPU unit tests for the wire format,
alignment, session config and handler (frames, unsupported adapter, cancel);
L3 e2e tests/e2e/online_serving/test_qwen3_tts_codec_stream.py.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-09 11:10:13 +08:00
2bae550c04
[Frontend] Opt-in WebSocket TTS split_granularity and session seed ( #7046 )
...
Signed-off-by: rk9595 <rakesh.kariya@somaiya.edu >
Signed-off-by: Rakesh Kariya <rakesh.kariya@somaiya.edu >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-09-08 19:15:59 -07:00
NancyFyong and NancyFyong
e74fe3c6e8
[Model] Add Audio8 TTS Preview 0.6B (DualAR, 44.1 kHz codec) ( #6157 )
...
Signed-off-by: NancyFyong <NancyFyong@users.noreply.github.com >
Co-authored-by: NancyFyong <NancyFyong@users.noreply.github.com >
2026-09-08 19:00:25 -07:00
Yancy and Asthenia
a31690041d
[Bugfix][Examples] Use --profiler-config flag in offline TTS examples ( #6763 )
...
Signed-off-by: Asthenia <asthenia0412@gmail.com >
Co-authored-by: Asthenia <asthenia0412@gmail.com >
2026-09-08 13:56:27 +00:00
LHXuuu
c9c25b33c3
[Frontend] Add server-side VAD for turn-based omni models ( #6618 )
...
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com >
2026-09-08 03:17:00 -07:00
Yueqian Lin
e1c59edec6
[Refactor] fish_speech: vendor DAC codec modules, drop external fish-speech dependency ( #4871 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
2026-09-08 09:22:36 +08:00
BruceLoveDecimal
5cd925631c
[Feature][LingBot World]stepwise execution ( #6844 )
...
Signed-off-by: BruceLoveDecimal <liuqihao970610@gmail.com >
Signed-off-by: liuqihao <liuqihao970610@gmail.com >
2026-09-07 13:38:23 +08:00
SYLAR
ea3cf99894
[diffusion][feature] Add component-selective offload policies ( #5929 )
...
Signed-off-by: lishunyang12 <lishunyang12@163.com >
2026-09-06 05:40:09 +08:00
Samit
44d3ae100a
[Bugfix][Diffusion] Fuse distilled LoRA weights before HSDP sharding ( #6948 )
...
Signed-off-by: samithuang <285365963@qq.com >
Signed-off-by: Samit <285365963@qq.com >
2026-09-06 01:29:59 +08:00
5d7fdf9350
[Core] Graduate MiniCPM-o 4.5 and PersonaPlex full-duplex serving out of experimental ( #6196 )
...
Signed-off-by: chickeyton <ngton2014@gmail.com >
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk >
2026-09-05 08:20:21 +00:00
Hongsheng Liu
d4096e01fe
[Model] Add native MAGI-2 Preview diffusion support ( #5918 )
...
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com >
2026-09-05 15:32:30 +08:00
LOGO127
b6b992ac7b
[Bugfix] Fix Higgs Audio v3 voice-clone token validation ( #7065 )
...
Signed-off-by: luozijian <luozijian0924@gmail.com >
2026-09-04 16:06:47 -07:00
boatman
348fd0fc07
[Feat][dots.tts] Add Online Serving Support ( #6235 )
...
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com >
2026-09-04 14:44:10 -07:00
Zhanshenzhui and Sy03
48ae412a2f
[model]Add experimental Mage-VL full-duplex adapter and offline integration ( #6537 )
...
Signed-off-by: Zhanshenzhui <99891002+zyforsure@users.noreply.github.com >
Co-authored-by: Sy03 <1370724210@qq.com >
2026-09-02 20:25:32 -07:00
GXIN and Hongsheng Liu
3ab582c993
[Bugfix][Qwen3-TTS] Reject task mismatches before EngineCore dispatch ( #6113 )
...
Signed-off-by: XIN GAO <1037396230@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-09-02 11:16:55 -07:00
Hongsheng Liu and hsliu_ustc
44c8d1c449
docs: add concise API server endpoint guide ( #6114 )
...
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: hsliu_ustc <hsliu_ustc@noreply.gitcode.com >
2026-09-01 20:34:09 +08:00
AuFlow and Hongsheng Liu
49c6fc1540
[Frontend] Add configurable output sample rate to Speech API ( #6553 )
...
Signed-off-by: 郑倩倩 <2458983186@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-09-01 17:25:09 +08:00
Gao Han
705e781b0e
[Bugfix][MiniCPM-o] Checkpoint playback before final input commit ( #6821 )
...
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk >
2026-08-31 15:18:29 +00:00
WeiQing Chen
55a226dcf1
[video][bugfix] Accept semantic video output in offline examples ( #6747 )
...
Signed-off-by: david6666666 <530634352@qq.com >
2026-08-29 06:26:28 +00:00
amy-why-3459
da94c3ebe1
[Bugfix][MiniCPM-o] Bind omni-duplex video frames to the unit they close ( #6404 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-08-28 16:58:20 +08:00
Sy03
009b992c5b
[Bugfix][MiniCPM-o] Isolate duplex handoff failures and barge-in ( #6170 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
2026-08-28 15:46:41 +08:00
Hongsheng Liu and hsliuustc0106
4245fd6955
[Fix] Restore MammothModa2 support ( #6694 )
...
Signed-off-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com >
Co-authored-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com >
2026-08-28 08:32:32 +08:00
Zhanshenzhui and amy-why-3459
c6df39e9f7
[Benchmark][MiniCPM-o] Add Omni-DuplexEval support ( #6634 )
...
Signed-off-by: Zhanshenzhui <99891002+zyforsure@users.noreply.github.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-08-27 16:31:50 +08:00
Zheng Wengang and amy-why-3459
94b1546ce3
[Model] Qwen3-Omni: register thinker-only pipeline for Instruct serve ( #6284 )
...
Signed-off-by: ZhengWG <zwg0606@gmail.com >
Signed-off-by: Zheng Wengang <zwg0606@gmail.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-08-27 08:21:03 +00:00
Zhou Taichang and Claude Opus 5
651b03225b
[Rebase] Rebase to vllm 0.28.0 ( #6606 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-27 06:12:55 +00:00
amy-why-3459
5de6f7c37a
[Bugfix][MiniCPM-o] Serve native duplex from shipping YAMLs and fence… ( #6619 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-08-26 10:56:51 +08:00
wangyu and Cursor
aeb348d8f8
[CI]Remove DreamID-Omni and MagiHuman support ( #6357 )
...
Signed-off-by: wangyu <410167048@qq.com >
Signed-off-by: [Your Name] <your.email@example.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-08-26 01:31:12 +00:00
NATURE
497c537c6f
[Benchmark] Add local OmniInteract realtime benchmark ( #6522 )
...
Signed-off-by: natureofnature <wzliu@connect.hku.hk >
2026-08-25 15:32:58 +00:00
Jae
72beeaee86
[Misc] move text-to-audio online examples to a unified folder ( #4807 )
...
Signed-off-by: zzehli <jaeli_ottawa@outlook.com >
2026-08-25 22:02:26 +08:00
leo and leo
50b8a150f9
[Model] Add native SANA-Video 2B T2V and I2V support ( #5508 )
...
Signed-off-by: leo <92263211+napleon-liu@users.noreply.github.com >
Signed-off-by: leo <92263211+liuyao0322@users.noreply.github.com >
Signed-off-by: liuyao0322 <yaoliu548926@gmail.com >
Co-authored-by: leo <92263211+napleon-liu@users.noreply.github.com >
2026-08-25 09:46:02 +08:00
wangyu
1e2a6446f7
[CI] Remove AudioX and MammothModa2 support ( #6353 )
...
Signed-off-by: wangyu <410167048@qq.com >
2026-08-23 04:22:05 +00:00
wangyu
2e096788a6
[CI]Remove SoulX-Singer support ( #6362 )
...
Signed-off-by: wangyu <410167048@qq.com >
2026-08-23 07:16:32 +08:00
Shin
70bdd49402
[Model] Add HiDream-O1-Image support ( #5194 )
...
Signed-off-by: yixiaoer <shin@yixiaoer.sg >
2026-08-21 14:15:37 +08:00
Yukim1
6f55a06fab
[Docs] Remove stale stage config references ( #6270 )
...
Signed-off-by: zwhzzz0821 <2831474076@qq.com >
2026-08-20 07:09:12 +08:00
Zeyu Huang | 黃澤宇 and Hongsheng Liu
8dc3d504d8
[skip ci][Doc] Remove obsolete DiffusionParallelConfig in many example scripts ( #6347 )
...
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-19 15:27:58 +08:00
Yuanheng Zhao
a630b0c2ec
[Ming] Cleanup Ming-family shared modules ( #6119 )
...
Signed-off-by: Yuanheng <jonathan.zhaoyh@gmail.com >
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com >
2026-08-19 14:13:40 +08:00
Yichen Wang and Didan Deng
2a9f6aaa8e
Add π0 VLA model support ( #4222 )
...
Signed-off-by: Yichen Wang <yichen.wang@bytedance.com >
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
2026-08-17 14:29:17 +08:00
Yukim1 and Hongsheng Liu
baba7d1e2e
[Refactor] Remove legacy stage_args YAML loader ( #6200 )
...
Signed-off-by: zwhzzz0821 <2831474076@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-16 11:22:28 +08:00
Hongsheng Liu
9d2bb23ff6
[Diffusion] Add loader-owned host-weight plans for DLO (TP=1) ( #6213 )
...
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-16 11:02:38 +08:00
3875f8ded4
[Core] Add realtime AR-Diffusion tick sessions for LingBot World 2.0 ( #5491 )
...
Signed-off-by: zhangbin.327 <zhangbin.327@bytedance.com >
Signed-off-by: Jack47 <scsvip@gmail.com >
Co-authored-by: zhangbin.327 <zhangbin.327@bytedance.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-16 09:11:20 +08:00
SYLAR
ce6ab0ca46
[Model] Add LTX-2.5 support ( #6070 )
...
Signed-off-by: lishunyang12 <lishunyang12@163.com >
2026-08-15 18:27:23 +08:00
tzhouam