anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
2,977 Commits 235 Branches 31 Tags
a58825ed6cc4ef814e5ec8afeffade476a7de431
Commit Graph
192 Commits
Author SHA1 Message Date
tzhouamandClaude Fable 5.1 a58825ed6c perf(omnivoice): tokenize the prompt once per request 
apply() already tokenized the prompt to build the engine prompt ids, but the
shared OmniMultiModalProcessor._apply_hf_processor_main then called
_call_hf_processor again, which rebuilt the control-token prompt and ran the
text tokenizer a second time before encoding the reference audio. Upstream's
_cached_apply_hf_processor invokes _apply_hf_processor_main for every
request, so this also happened for text-only requests and for requests whose
reference audio was already cached.

Split the old _call_hf_processor into _encode_prompt_text (used once, in
apply) and _encode_reference_audio, and override _apply_hf_processor_main to
encode the reference audio only, returning just the passthrough data when
there is no audio. _get_mm_fields_config registers only ref_audio_tokens and
ref_audio_len, so dropping input_ids from the media result changes no
consumer. The now-unreferenced _call_hf_processor is deleted; the prompt-text
kwarg transport is left as is.

Validation: ruff and pre-commit on both files; the module imports cleanly;
new tests/model_executor/models/omnivoice/test_processor_prompt_tokenization.py
covers text-only apply (one tokenizer call), the audio path (audio encoded,
no re-tokenization) and the empty-items path (no tokenizer, no encoder);
tests/model_executor/models/omnivoice (106), tests/inputs/test_mm_processor.py
and tests/entrypoints/openai_api/test_tts_{adapter,detection}.py (162) pass.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MpXe2LeRrUQGjZfKFaocMZ
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-09-10 00:41:53 +08:00
tzhouam efda9369a8 refactor: reduce copied vLLM 0.29 compatibility code 
Reuse upstream weight mapping and multimodal processing, consolidate Omni processor bridges, and trim unused Hunyuan internals. Route no-media kwargs through the active renderer and preserve GLM generation prompt formatting.

Validation: 453 CPU tests passed, 7 skipped; processor imports, Ruff, and diff checks passed. GPU and end-to-end CI not rerun.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-09-10 00:41:53 +08:00
tzhouam f9879239ba chore: satisfy changed-file policy 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-09-10 00:41:52 +08:00
tzhouam 87a1fd8da2 rebase: adapt to vLLM v0.29 APIs 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-09-10 00:41:52 +08:00
NancyFyongandNancyFyong e74fe3c6e8 [Model] Add Audio8 TTS Preview 0.6B (DualAR, 44.1 kHz codec) (#6157) 
Signed-off-by: NancyFyong <NancyFyong@users.noreply.github.com>
Co-authored-by: NancyFyong <NancyFyong@users.noreply.github.com>
 2026-09-08 19:00:25 -07:00
boatman cdf027874c [Feat][OmniVoice]Support Varlen Attn, Request-Batch and Step-Execution (#6408) 
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com>
 2026-09-08 18:58:38 -07:00
eval d7678e2bb3 [Bugfix][MiniCPM-o] Fix the audio_embeds input path (#5730) 
Signed-off-by: eval-dev <0xe5bca0@gmail.com>
Signed-off-by: eval <74645252+eval-dev@users.noreply.github.com>
 2026-09-08 18:41:42 -07:00
Zhichao Zhang 5078c7ad66 [Bugfix][MiMo-Audio] Align code2wav decode with tokenizer device (#6539) 
Signed-off-by: chaosansui <zzc15560846421@163.com>
Signed-off-by: Zhichao Zhang <60429419+smartDream-chao@users.noreply.github.com>
 2026-09-08 18:38:11 -07:00
Harry MellorHongsheng LiuClaude Opus 5
 		fdaf4e31bd Add vLLM-Omni library info to Hugging Face Hub requests (#5381) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-08 10:08:38 -06:00
Deep Shahandamy-why-3459 1413eaab7c [Bugfix][Qwen3-Omni] Read code-predictor RoPE theta from rope_parameters (#7228) 
Signed-off-by: Deep Shah <deep@socratic.co>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-09-08 10:25:27 +08:00
Yueqian Lin e1c59edec6 [Refactor] fish_speech: vendor DAC codec modules, drop external fish-speech dependency (#4871) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-09-08 09:22:36 +08:00
boatman 043f03d576 [Migrate]Move model-specific helpers out of serving_speech (#6697) 
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com>
 2026-09-08 08:59:45 +08:00
Pujitha PaladuguPujitha Paladuguamy-why-3459
 		cf36f5e474 [Bugfix][MiniCPM-o] Allow ragged audio_feature_lens across a batch (#7071) 
Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
Co-authored-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-09-07 23:51:13 +08:00
NATUREandamy-why-3459 ff6e906a25 [Model] Preserve MiniCPM-o Talker full attention until capacity (#6799) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-09-06 22:25:35 +08:00
Hongsheng Liu 715b8b8746 [Model] MiniMax-H3: validate versioned text-conditioning handoff (#6720) 
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-09-06 21:45:10 +08:00
SYLARandhsliuustc0106 14d2d00536 [Bugfix] Fix Hunyuan Image-3 expert mapping contract (#7021) 
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
Co-authored-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-09-06 21:43:37 +08:00
chickeytonClaude Fable 5Gao Han
 		5d7fdf9350 [Core] Graduate MiniCPM-o 4.5 and PersonaPlex full-duplex serving out of experimental (#6196) 
Signed-off-by: chickeyton <ngton2014@gmail.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-09-05 08:20:21 +00:00
boatman 66bbebd9d3 [Bugfix] Fix fallback loading for IndexTTS external models (#6137) 
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com>
 2026-09-04 19:58:33 -07:00
BruceLoveDecimalandClaude Opus 5 1e74807c35 [Feature][Moss-TTS] Reference Audio Preprocessing (#4982) 
Signed-off-by: BruceLoveDecimal <39156883+BruceLoveDecimal@users.noreply.github.com>
Signed-off-by: liuqihao <liuqihao970610@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 16:36:26 -07:00
LOGO127 b6b992ac7b [Bugfix] Fix Higgs Audio v3 voice-clone token validation (#7065) 
Signed-off-by: luozijian <luozijian0924@gmail.com>
 2026-09-04 16:06:47 -07:00
Yuekai ZhangandClaude Fable 5 4179b0cae1 [Perf][NemotronVoiceChat] Fast defaults for all three deploy profiles: 2.8x offline, realtime full-duplex audio (#6354) 
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-09-03 18:49:19 -07:00
Jim Ban 711daacabf [Bugfix][TTS] Make CosyVoice3 TRT plan publication concurrency-safe (#6955) 
Signed-off-by: BANANASJIM <bananasjim1@gmail.com>
 2026-09-03 06:44:42 +00:00
DanaerLee ca6b4e2112 [Perf][Bugfix][OmniVoice] Restore float16 serving, and fuse the generator hot loop (#6317) 
Signed-off-by: MrlixiangWE <mrdanaer@gmail.com>
 2026-09-02 19:54:55 +00:00
Jim Ban b9a88bf51e [Bugfix][TTS] Isolate concurrent StepAudio2 TRT builds (#6957) 
Signed-off-by: BANANASJIM <bananasjim1@gmail.com>
 2026-09-02 19:53:02 +00:00
GXINandHongsheng Liu 3ab582c993 [Bugfix][Qwen3-TTS] Reject task mismatches before EngineCore dispatch (#6113) 
Signed-off-by: XIN GAO <1037396230@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-02 11:16:55 -07:00
wangyulishunyang12CursorBo Li
 		95a9311696 [CI] Select mirror_hardwares presets via MIRROR_HW (H100/B200) (#5543) 
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Signed-off-by: [Your Name] <your.email@example.com>
Signed-off-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
Co-authored-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Bo Li <22713281+bobboli@users.noreply.github.com>
 2026-09-02 15:39:49 +00:00
Alex Brooks a65e89cbd4 [Core] Validate PipelineConfig in __post_init__ (#6829) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-09-01 11:17:57 +08:00
akshatvishu f321cdbe16 [Bugfix] Use Seed-TTS reference audio with MiniCPM-o 4.5 (#6628) 
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
 2026-08-31 10:27:34 +08:00
akshatvishu c41925689d [BugFix][Higgs-Audio-V3] Fix transcript and concurrent sampling tests (#6422) 
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
 2026-08-30 08:52:15 +00:00
Yueqian Lin a6b559cf54 [Bugfix][MiniMax-Music3] Resolve stage subdirs against a real snapshot (#6640) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-08-30 06:51:24 +00:00
NATURE 87736a3cc5 [Bugfix][MiniCPM-o] Bound full-duplex Talker context with sliding recompute (#6626) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
 2026-08-30 00:49:42 +08:00
wangyu e338f0ece3 [CI]Split GPU jobs by cards_* and reject hand-written SKU marks (#6650) 
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: [Your Name] <your.email@example.com>
 2026-08-29 18:42:33 +08:00
psv666andGao Han 5c47fd0bc8 [Bugfix][Qwen3-Omni] Handle missing packed modules mapping (#6748) 
Signed-off-by: psv666 <2693925048@qq.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-08-29 15:24:49 +08:00
ZacheryAUandamy-why-3459 457c6e2793 [bugfix][MiniCPM-o] Re-land empty full-payload and async_chunk=false processor tests (#6745) 
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-08-29 13:39:54 +08:00
boatman bfe8967ca2 [Bugfix][StepAudio2] Fix async-chunk metadata mismatch and boundary glitches (#5917) 
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com>
 2026-08-28 21:45:24 -07:00
Gao Han 6b9ce05feb Revert "[bugfix][MiniCPM-o] Fix offline_inference/test_minicpmo_4_5.py and online_serving/ one" (#6730) 2026-08-29 08:14:02 +08:00
ZacheryAUandamy-why-3459 6d84409a68 [bugfix][MiniCPM-o] Fix offline_inference/test_minicpmo_4_5.py and online_serving/ one (#5464) 
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-08-28 19:20:09 +00:00
Canlin Guolishunyang12yuanwu
 		b6a4baa659 [Feature] Support text encoder Disaggregation for H3 (#5885) 
Signed-off-by: Canlin Guo <canlinguosdu@gmail.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Signed-off-by: yuanwu <yuan.wu@intel.com>
Co-authored-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: yuanwu <yuan.wu@intel.com>
 2026-08-28 14:27:05 +08:00
BruceLoveDecimalandamy-why-3459 ab8d45d3cc [Bugfix][MiniCPM-o] Retire CFM DiT CUDA graphs a generation at a time (#6587) 
Signed-off-by: BruceLoveDecimal <liuqihao970610@gmail.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-08-28 14:07:57 +08:00
Hongsheng Liuandhsliuustc0106 4245fd6955 [Fix] Restore MammothModa2 support (#6694) 
Signed-off-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
Co-authored-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
 2026-08-28 08:32:32 +08:00
SYLARandHongsheng Liu 0e44fc7406 [diffusion][feature] Add offline SVDQuant W4A4 support (#6162) 
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-27 08:45:53 +08:00
Sy03andsy03 432b2c6ca3 [Model] Add native full-duplex Nemotron VoiceChat serving (#6089) 
Signed-off-by: Sysy <1370724210@qq.com>
Signed-off-by: sy03 <137072421@qq.com>
Co-authored-by: sy03 <137072421@qq.com>
 2026-08-26 13:19:20 -07:00
Allen WuTRAE CLISy03TRAE CLI
 		f90e992dc1 [Model] Optimize CosyVoice3 TensorRT stream handoff (#5673) 
Signed-off-by: Allen Wu <allenwu2795@gmail.com>
Co-authored-by: TRAE CLI <noreply@bytedance.com>
Co-authored-by: Sy03 <1370724210@qq.com>
Co-authored-by: TRAE CLI <traecli@bytedance.com>
 2026-08-25 22:51:46 -07:00
wy17003andHongsheng Liu b454e34966 [Refactor][2/N Scheduler]Replace _FULL_PAYLOAD_INPUT_STAGES with a resolved stage transport capability. (#6149) 
Signed-off-by: wy17003 <yaow17003@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-25 15:24:59 +08:00
NATURE 702a409664 [Feature][MiniCPM-o] Support configurable concurrent duplex sessions (#6021) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
 2026-08-25 02:27:30 +00:00
wangyu 1e2a6446f7 [CI] Remove AudioX and MammothModa2 support (#6353) 
Signed-off-by: wangyu <410167048@qq.com>
 2026-08-23 04:22:05 +00:00
akshatvishu 39c16d75a9 [tts][bugfix] Fix CosyVoice3 sampling and stage handoff (#6424) 
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
 2026-08-22 21:20:57 -07:00
wangyuandamy-why-3459 0db262de2a [CI/Build] Move L1/E2E coverage to weekly and split scheduled L4/L5 pipelines (#6311) 
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: [Your Name] <your.email@example.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-08-23 11:49:37 +08:00
amy-why-3459 42059f19d5 [Bugfix][MiniCPM-o] Cap offline Talker generation at remaining context (#6458) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-08-22 12:42:49 +08:00
NATUREandamy-why-3459 4f7c08c0eb [Bugfix][MiniCPM-o] Fix async-chunk snapshot replacement and prompt cleanup (#6406) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-08-21 16:37:21 +08:00