anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
3,038 Commits 235 Branches 31 Tags
ab2f2a3d658db5be5ec96a950da55cb74b48e7aa
Commit Graph
525 Commits
Author SHA1 Message Date
tzhouamandClaude Fable 5.1 33de5d8629 Merge vLLM-Omni v0.29.0rc1 into edge-infer/omni-0.29.0 
The pair for vLLM v0.29.0: rc1's docs pin vllm==0.29.0 and a static scan of
rc1's vllm.* imports against the v0.29.0 tree finds nothing missing, while
our 0.28-based tree needs seven modules v0.29.0 no longer has
(vllm.inputs.preprocess, entrypoints.launcher, openai.cli_args,
openai.engine.protocol, serve.utils.server_utils, layers.batch_invariant,
models.hunyuan_v1).

Ten files conflicted. Upstream's structure is the base everywhere; the
fork's features are re-attached to it:
- deploy_profile / apply_deploy_profile run before rc1's resolve_omni_config
  pops deploy_config (async_omni_engine, cli/serve); the deploy-profile
  selector and the resolve_model_config_path chain it needs stay in
  entrypoints/utils.py; stage_overrides takes rc1's dict type.
- config_factory keeps the lazy diffusion output-type import; the diffusion
  registry import in rc1's new config/resolver.py is made lazy for the same
  WP6 reason (TTS path must not load diffusion).
- requirements: our core/media split kept; rc1's transformers>=5.13 floor
  and kernels==0.16.1 applied in core.txt / media.txt.
- TTS: rc1's split_granularity and bounded seed plus our output_mode=
  codec_tokens, _SessionInbox cancel handling and codec DELTA accumulation;
  our private _conditioning_cache_salt is superseded by rc1's public
  tts_adapters/base.py::conditioning_cache_salt.
- stage_runtime: rc1's single ExitStack G2 path (lock released before READY)
  replaces our two-path block, with the four worker_phase names the init
  profiler consumes still emitted; _cpu_placed / host admission kept.

Not yet run against vLLM 0.29.0: the tests listed in the W6 record.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-18 09:57:20 +08:00
Zhou TaichangandClaude Opus 5 aff7d64948 [Rebase] Rebase to vLLM 0.29.0 (#7230) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-10 12:08:28 +08:00
Sy03 333f14e97e [Bugfix][MiniCPM-o] Align structured chat content with native omni rendering (#7344) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-10 11:38:11 +08:00
kunkun 2f1845fa4e fix: reserve Qwen3-Omni NVFP4 backend fix (#7200) 
Signed-off-by: kunkunblueberry <1833921874@qq.com>
 2026-09-09 20:35:16 +08:00
NumberWan 038c9b948f [Bugfix] Resolve #6931 hub FA3 on torch 2.13 via kernels 0.16.1 (#7185) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
 2026-09-09 10:20:34 +00:00
tzhouamandClaude Fable 5.1 602e39647e [Edge][WP2] Finalize edge profile init knobs from init_profile; WP4 CPU emulation results 
init_profile 20260909-112200 (1.7B, L20X, warm): default 152.0 s, eager_stage1
149.9 s, eager_both 110.9 s (first TTFA 15 s, no graphs), parallel_stage_init
94.6 s, edge 145.7 s. Edge profile now carries `orchestrator: parallel_stage_init:
true`, applied by the deploy_profile selector (explicit values win); graphs stay on.

hw_emulation 20260909-110229 (CPU venv, 0.6B, deploy_profile=auto):
x86_cpu_avx2 (4c, fp32) TTFA 2.65 s RTF 11.9; x86_cpu_avx512 (8c) TTFA 0.68 s
RTF 2.45; x86_cpu_amx (16c) TTFA 0.51 s RTF 1.72; arm64 logic-only same as avx2.
Emulation rows now record the auto-detected hardware class.

Adds tests/e2e/offline_inference/test_qwen3_tts_edge_profile.py (L3, one GPU).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-09 14:40:13 +08:00
Mu GuanLin ab56148505 [Bugfix] Make LTX vocoder decoding deterministic (#7231) 
Signed-off-by: mglyn <1203789601@qq.com>
 2026-09-09 06:37:01 +00:00
tzhouamandClaude Fable 5.1 49a839ecf9 [Edge][WP5] Add codec-token streaming mode with on-device decoding 
* qwen3_tts_talker_only pipeline (single terminal LLM_AR stage,
  final_output_type "latent") + vllm_omni/deploy/qwen3_tts_talker_only.yaml;
  a terminal talker publishes its per-step codes.audio rows to the client
  (omni_client_multimodal_output_keys).
* /v1/audio/speech/stream: session.config.output_mode="codec_tokens" streams
  codec.start (JSON) + binary frames (<IHB seq/n_frames/flags + int16 rows,
  vllm_omni/entrypoints/openai/codec_stream.py) + codec.done; new input.cancel
  message aborts an in-flight generation for both codec and PCM sessions
  (single reader task + session inbox replaces direct socket reads); optional
  session.config.seed; audio.done gains "cancelled" only when it happened.
* serving_speech._generate_codec_chunks aligns the cumulative code tensor to
  output token ids (drops prefill placeholder/EOS rows) and emits deltas.
* TTS adapters: supports_codec_stream / codec_stream_spec (Qwen3-TTS: 16
  codebooks, 2048 ids, 12.5 Hz, 24 kHz, decoder qwen3_tts_tokenizer_v2_12hz).
* examples/online_serving/text_to_speech/qwen3_tts/edge_decoder_client.py
  decodes locally with Qwen3TTSTokenizerV2Decoder (ramp schedule, 72-frame
  context, cancel/reset), codec_parity_check.py and
  benchmarks/tts/codec_stream_e2e.py compare against the two-stage server.
* docs/serving/codec_token_streaming.md; CPU unit tests for the wire format,
  alignment, session config and handler (frames, unsupported adapter, cancel);
  L3 e2e tests/e2e/online_serving/test_qwen3_tts_codec_stream.py.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-09 11:10:13 +08:00
NancyFyongandNancyFyong e74fe3c6e8 [Model] Add Audio8 TTS Preview 0.6B (DualAR, 44.1 kHz codec) (#6157) 
Signed-off-by: NancyFyong <NancyFyong@users.noreply.github.com>
Co-authored-by: NancyFyong <NancyFyong@users.noreply.github.com>
 2026-09-08 19:00:25 -07:00
boatman cdf027874c [Feat][OmniVoice]Support Varlen Attn, Request-Batch and Step-Execution (#6408) 
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com>
 2026-09-08 18:58:38 -07:00
Harry MellorHongsheng LiuClaude Opus 5
 		fdaf4e31bd Add vLLM-Omni library info to Hugging Face Hub requests (#5381) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-08 10:08:38 -06:00
liangmenghuangandClaude Fable 5.1 7a80b79698 [Bugfix][Model] GR00T-N1.7: honor the per-request seed for flow-matching noise (#7253) 
Signed-off-by: liangmengh <liangmengh@nvidia.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-08 15:53:20 +00:00
LHXuuu c9c25b33c3 [Frontend] Add server-side VAD for turn-based omni models (#6618) 
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com>
 2026-09-08 03:17:00 -07:00
BruceLoveDecimal 5cd925631c [Feature][LingBot World]stepwise execution (#6844) 
Signed-off-by: BruceLoveDecimal <liuqihao970610@gmail.com>
Signed-off-by: liuqihao <liuqihao970610@gmail.com>
 2026-09-07 13:38:23 +08:00
Xiaobaixia 615f97b9c7 [Feature] Define the runner preprocess phase contract and regression coverage (#7136) 
Signed-off-by: xiaobaixia222 <289982752@qq.com>
 2026-09-06 17:20:04 +00:00
SYLAR ea3cf99894 [diffusion][feature] Add component-selective offload policies (#5929) 
Signed-off-by: lishunyang12 <lishunyang12@163.com>
 2026-09-06 05:40:09 +08:00
chickeytonClaude Fable 5Gao Han
 		5d7fdf9350 [Core] Graduate MiniCPM-o 4.5 and PersonaPlex full-duplex serving out of experimental (#6196) 
Signed-off-by: chickeyton <ngton2014@gmail.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-09-05 08:20:21 +00:00
Hongsheng Liu d4096e01fe [Model] Add native MAGI-2 Preview diffusion support (#5918) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-09-05 15:32:30 +08:00
boatman 348fd0fc07 [Feat][dots.tts] Add Online Serving Support (#6235) 
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com>
 2026-09-04 14:44:10 -07:00
Yuekai ZhangandClaude Fable 5 4179b0cae1 [Perf][NemotronVoiceChat] Fast defaults for all three deploy profiles: 2.8x offline, realtime full-duplex audio (#6354) 
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-09-03 18:49:19 -07:00
ZhanshenzhuiandSy03 48ae412a2f [model]Add experimental Mage-VL full-duplex adapter and offline integration (#6537) 
Signed-off-by: Zhanshenzhui <99891002+zyforsure@users.noreply.github.com>
Co-authored-by: Sy03 <1370724210@qq.com>
 2026-09-02 20:25:32 -07:00
Zeng Chuang b5e898f295 [Feat] Add support and testcase for HunyuanImage-3.0-Instruct-Distil (#4048) 
Signed-off-by: zengchuang <zengchuang3@huawei.com>
Signed-off-by: zengchuang-hw <zengchuang1005@gmail.com>
 2026-09-03 02:43:13 +00:00
hurukawa 8f284b343a [Feature][Diffusion]: CFG parallelism support for Boogu-Image (#6786) 
Signed-off-by: nagisa <1434936049@qq.com>
Signed-off-by: nagisa-kun <1434936049@qq.com>
 2026-09-03 10:37:21 +08:00
wangyulishunyang12CursorBo Li
 		95a9311696 [CI] Select mirror_hardwares presets via MIRROR_HW (H100/B200) (#5543) 
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Signed-off-by: [Your Name] <your.email@example.com>
Signed-off-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
Co-authored-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Bo Li <22713281+bobboli@users.noreply.github.com>
 2026-09-02 15:39:49 +00:00
andyluo7andHongsheng Liu 477083a103 [CI][ROCm] Fix deterministic AMD L3 blockers (#6884) 
Signed-off-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-01 22:53:00 +08:00
WeiQing Chen e2b83db683 [Test] Add MiniMax-H3 DLO DP2 variants and Turbo LoRA L3 test (#6556) 
Signed-off-by: david6666666 <530634352@qq.com>
 2026-09-01 10:48:52 +00:00
AuFlowandHongsheng Liu 49c6fc1540 [Frontend] Add configurable output sample rate to Speech API (#6553) 
Signed-off-by: 郑倩倩 <2458983186@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-01 17:25:09 +08:00
XRay2016 8e26dce1f4 [Diffusion] Add Boogu Edit-Turbo support docs and tests (#6701) 
Signed-off-by: xRay2016 <1150722393@qq.com>
 2026-09-01 08:41:23 +08:00
Gao Han b81aeb7b86 [CI][MiniCPM-o] Make audio consistency checks deterministic (#6828) 
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-08-31 13:08:49 +00:00
wangyu 3d1ad0c0f4 [CI][Bugfix]Drop dummy-weight Qwen3-TTS Base from Ready CI (#6861) 
Signed-off-by: wangyu <410167048@qq.com>
 2026-08-31 12:13:26 +00:00
Inesh Reddy Chappidi 5e06d2e73c [CI/Build] Align duplex stage-input deadline with the e2e client timeout (#6831) 
Signed-off-by: Inesh Reddy Chappidi <ineshreddy249@gmail.com>
 2026-08-31 02:55:36 +00:00
akshatvishu f321cdbe16 [Bugfix] Use Seed-TTS reference audio with MiniCPM-o 4.5 (#6628) 
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
 2026-08-31 10:27:34 +08:00
Yueqian Lin 1d75e63f7f [CI/Build] Add dots.tts weekly e2e coverage (#6174) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-08-30 11:37:08 +00:00
akshatvishu c41925689d [BugFix][Higgs-Audio-V3] Fix transcript and concurrent sampling tests (#6422) 
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
 2026-08-30 08:52:15 +00:00
y-null 99a2ae1fc8 [Bugfix][MiniCPM-o] Anchor duplex soft-interrupt listen sandwich before commit (#6772) 
Signed-off-by: y_null <y_null@qq.com>
 2026-08-30 03:25:43 +00:00
wangyu e338f0ece3 [CI]Split GPU jobs by cards_* and reject hand-written SKU marks (#6650) 
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: [Your Name] <your.email@example.com>
 2026-08-29 18:42:33 +08:00
psv666 a5b8c82788 [Bugfix][MiniCPM-o] Fix duplex camera frame fixture (#6757) 
Signed-off-by: psv666 <2693925048@qq.com>
 2026-08-29 09:57:53 +00:00
Nguyen Kha Nhat LongCursorSamit
 		bb5bf8fd0a [Bugfix] Fix colocate-async sleep admission race and deliver AR abort tokens (#6367) 
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Signed-off-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-08-29 09:11:39 +00:00
amy-why-3459 da94c3ebe1 [Bugfix][MiniCPM-o] Bind omni-duplex video frames to the unit they close (#6404) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-08-28 16:58:20 +08:00
Sy03 009b992c5b [Bugfix][MiniCPM-o] Isolate duplex handoff failures and barge-in (#6170) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-08-28 15:46:41 +08:00
zzh c201fd77c6 Add HYvideo1.5 benchmark (#6349) 
Signed-off-by: zzh <943967662@qq.com>
 2026-08-28 14:31:03 +08:00
Canlin Guolishunyang12yuanwu
 		b6a4baa659 [Feature] Support text encoder Disaggregation for H3 (#5885) 
Signed-off-by: Canlin Guo <canlinguosdu@gmail.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Signed-off-by: yuanwu <yuan.wu@intel.com>
Co-authored-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: yuanwu <yuan.wu@intel.com>
 2026-08-28 14:27:05 +08:00
WeiQing Chen ed837e350e [Test] Stabilize MiniMax-H3 reference accuracy compile (#6688) 
Signed-off-by: david6666666 <530634352@qq.com>
 2026-08-28 14:18:28 +08:00
Hongsheng Liuandhsliuustc0106 4245fd6955 [Fix] Restore MammothModa2 support (#6694) 
Signed-off-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
Co-authored-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
 2026-08-28 08:32:32 +08:00
Mu GuanLinandHongsheng Liu 5e4daaf197 [Diffusion] Add LTX-2.5 Diffusion VAE decoder support (#6189) 
Signed-off-by: mglyn <1203789601@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-27 23:54:40 +08:00
Deep Shahandamy-why-3459 069e6c5fbc [CI/Build][MiniCPM-o] Derive duplex admission-probe limit from the deploy config (#6678) 
Signed-off-by: Deep Shah <deep@socratic.co>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-08-27 19:20:01 +08:00
Zhanshenzhuiandamy-why-3459 c6df39e9f7 [Benchmark][MiniCPM-o] Add Omni-DuplexEval support (#6634) 
Signed-off-by: Zhanshenzhui <99891002+zyforsure@users.noreply.github.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-08-27 16:31:50 +08:00
Zhou TaichangandClaude Opus 5 651b03225b [Rebase] Rebase to vllm 0.28.0 (#6606) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-27 06:12:55 +00:00
Sy03andsy03 432b2c6ca3 [Model] Add native full-duplex Nemotron VoiceChat serving (#6089) 
Signed-off-by: Sysy <1370724210@qq.com>
Signed-off-by: sy03 <137072421@qq.com>
Co-authored-by: sy03 <137072421@qq.com>
 2026-08-26 13:19:20 -07:00
Anurag 91dd7fcaec [Bugfix] Reject instructions changes on locked native duplex sessions (#6318) 
Signed-off-by: ANURAG KANADE <anuragkanade6@gmail.com>
 2026-08-26 16:50:31 +00:00