lishunyang12
c15f51a2f6
Merge branch 'main' into lance-model-support; drop lance.yaml
...
Resolves conflicts in:
- vllm_omni/diffusion/registry.py: keep both LancePipeline (HEAD)
and MingImagePipeline (main) entries.
- vllm_omni/diffusion/models/bagel/bagel_transformer.py: take main's
simpler cfg_batched payload; HEAD referenced undefined locals.
Also strips vllm_omni/deploy/lance.yaml -- Lance is single-stage
diffusion, so its engine knobs (pipeline=lance, enforce_eager,
trust_remote_code, max_num_seqs=1, ...) flow directly to Omni(...)
/ vllm-omni serve instead of through a deploy YAML.
create_default_diffusion builds the stage config from those kwargs.
Touched: examples/offline_inference/lance/{end2end,gradio_demo}.py,
tests/e2e/online_serving/test_lance.py, tests/helpers/stage_config.py.
Signed-off-by: lishunyang12 <lishunyang12@163.com >
2026-05-31 11:50:21 +00:00
ChenWenjing
5dfdf584dd
[Feat]audio streaming input for async chunk ( #3614 )
...
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn >
2026-05-31 17:39:25 +08:00
Yuanheng Zhao and Zheng Wengang
ea6d811b58
[Config Refactor] Migrate Ming-flash-omni-2.0 Image-Gen deploy configs ( #3975 )
...
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com >
Co-authored-by: Zheng Wengang <zwg0606@gmail.com >
2026-05-30 08:13:21 +00:00
fywc
272e626100
[Feat] support cache-dit for SenseNova-U1 ( #3906 )
...
Signed-off-by: hanzheli <hanzheli@kuaishou.com >
2026-05-29 17:40:54 +00:00
boatman and sphinxkkkbc
4d9262a6eb
[Feat]Support voice clone for omnivoice in online serving & add seed parameter for reproducible ( #3668 )
...
Signed-off-by: sphinxkkkbc <binchengkang8@gmail.com >
Co-authored-by: sphinxkkkbc <binchengkang8@gmail.com >
2026-05-29 07:42:08 +00:00
Sy03 and Yueqian Lin
d37912b071
[TTS][Perf] Optimize Fish Speech S2 Pro high-concurrency serving ( #3773 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-05-27 13:25:49 -05:00
a698269e47
[Model] Add Ming-flash-omni-2.0 Image Generation (Diffusion) Stage ( #2875 )
...
Signed-off-by: ZhengWG <zwg0606@gmail.com >
Signed-off-by: Zheng Wengang <zwg0606@gmail.com >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-authored-by: Yuanheng Zhao <54058983+yuanheng-zhao@users.noreply.github.com >
2026-05-27 23:21:20 +08:00
8f45e68bcd
[Higgs-Audio] bosonai/higgs-audio-v2-generation-3B-base TTS model support ( #3762 )
...
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com >
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-05-25 23:16:07 +08:00
hurukawa and Hongsheng Liu
b52a5ca401
[Bugfix][Example][OmniVoice] Drop hardcoded "voice": "default" from speech_client.py ( #3829 )
...
Signed-off-by: nagisa-kun <1434936049@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-05-25 09:38:24 +08:00
BeatSeat and Yueqian Lin
28ce618f0e
[Model] Add GLM-TTS text-to-speech model support ( #3141 )
...
Signed-off-by: BeatSeat <wendavid552@gmail.com >
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-05-24 05:42:26 +08:00
SYLAR
f0cec3ecd4
Merge branch 'main' into lance-model-support
2026-05-22 04:13:58 +08:00
Yueqian Lin
fc8486cf20
[Doc][TTS] CosyVoice3 online docs + residual TTS yaml cleanup + remove VoxCPM v1 ( #3748 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
2026-05-20 15:24:06 +08:00
lishunyang
69bcc6416d
[New Model] Lance (ByteDance) — unified AR+diffusion multimodal ( #3697 )
...
Adds Lance (bytedance-research/Lance), a 3B unified AR+diffusion
multimodal model on a Qwen2.5-VL backbone. Lance is BAGEL-lineage:
it reuses vLLM-Omni's BAGEL Mixture-of-Transformers core verbatim and
specializes only the ViT (Qwen2.5-VL vision), the VAE (Wan2.2) and the
checkpoint layout.
All six single-stage Lance modalities verified end-to-end on 1x B300:
t2i, t2v, image_edit, video_edit, x2t_image (image understanding) and
x2t_video (video understanding). 3-D mRoPE is wired through
BagelRotaryEmbedding for the x2t / video / edit paths. Includes the
upstream gap-analysis quality fixes (task-specific system prompts, full
Qwen chat template, upstream inference defaults and aspect-ratio
buckets).
Ships an offline example (examples/offline_inference/lance/), an
OpenAI-compatible online-serving example
(examples/online_serving/lance/) and an e2e test
(tests/e2e/online_serving/test_lance.py).
See LANCE_PR.md for the architecture rationale, per-modality status and
the two remaining follow-ups (two-stage AR+DiT, video_edit polish).
Closes #3697 .
Signed-off-by: lishunyang <lishunyang03@gmail.com >
2026-05-18 22:09:14 +00:00
Jie Liu and Yueqian Lin
90f5b3c3a1
Update streaming_speech_client.py to solve Qwen/Qwen3-TTS-12Hz-0.6B-CustomVoice voice problem ( #3380 )
...
Signed-off-by: Jie Liu <33612777+keeper-jie@users.noreply.github.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
2026-05-14 20:43:05 -07:00
d7ea5d5979
[New Model] Add support for tencent/Covo-Audio-Chat ( #2293 )
...
Signed-off-by: Dnoob <dxpouo@gmail.com >
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-05-12 14:39:58 +08:00
汪志鹏
c9a8556c24
[New Model]: Add sensenova u1 support ( #3319 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
2026-05-11 22:19:00 +08:00
lsyyyyy and 汪志鹏
32663f21d5
[Feat] support hsdp for Bagel ( #3150 )
...
Signed-off-by: siyuan.lei <siyuanlei37@gmail.com >
Signed-off-by: lsyyyyy <siyuanlei37@gmail.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
2026-05-08 15:20:42 +08:00
wangyu and Huang, Zeyu
6f2ad7b403
[Test] Unify L2/L3 test layout, Buildkite steps, and test helpers ( #2556 )
...
Signed-off-by: wangyu <410167048@qq.com >
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com >
Signed-off-by: wangyu <53896905+yenuo26@users.noreply.github.com >
Co-authored-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com >
2026-05-08 14:12:23 +08:00
1e5f288a91
[FEAT] support multi-stage deployment ( #2396 )
...
Signed-off-by: ZhengWG <zwg0606@gmail.com >
Signed-off-by: Zheng Wengang <zwg0606@gmail.com >
Signed-off-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com >
Signed-off-by: yinpe <11810305@mail.sustech.edu.cn >
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com >
Co-authored-by: yinpe <11810305@mail.sustech.edu.cn >
Co-authored-by: yinpeiqi <yinpeiqi809@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk >
Co-authored-by: Chenguang Zheng <645327136@qq.com >
2026-05-06 19:33:18 +08:00
Yueqian Lin
5cf3f7947b
[Docs] Consolidate moss_tts_nano + ming_flash_omni_tts into TTS hub ( #3358 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
2026-05-05 17:36:16 +00:00
Zeyu Huang | 黃澤宇 and Copilot Autofix powered by AI
f19891e4f1
[bugfix][CI] Diffusers backend update ( #3096 )
...
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com >
Signed-off-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com >
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com >
2026-05-05 20:54:39 +08:00
Yueqian Lin
703e31fc47
[Docs] Consolidate per-model TTS examples into a single hub ( #3234 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
2026-05-05 08:41:29 +12:00
c708aecb18
[Diffusion] [Model] Support AudioX ( #2077 )
...
Signed-off-by: Zhang Jian <jianmusings@gmail.com >
Signed-off-by: Zhang <jianmusings@gmail.com >
Signed-off-by: Zhang <zhang.jian@u.nus.edu >
Signed-off-by: Zhang Jian <e0322744@u.nus.edu >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: Zhang Jian <e0322744@u.nus.edu >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
2026-05-04 10:24:35 -04:00
Yueqian Lin and Hongsheng Liu
7491b67446
[Bugfix][MOSS-TTS-Nano] Drop fictional voice presets, require ref_audio ( #3192 )
...
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-05-03 16:35:35 +12:00
Yuanheng Zhao and Hongsheng Liu
d49356e4ca
[Config Refactor] Migrate Ming and Ming-TTS deploy/pipline configs ( #3154 )
...
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com >
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-05-03 14:43:49 +12:00
4c2bea782e
Fix "Add online serving to Stable Audio Diffusion and introduce v1/audio/generate endpoint" ( #1794 )
...
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
2026-04-28 18:06:57 +00:00
amy-why-3459
0470e96771
[Docs] Modify Qwen3-Omni's recipe ( #3109 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-04-27 21:41:13 +08:00
Yueqian Lin and Hongsheng Liu
afa2b09684
feat: add MOSS-TTS-Nano single-stage TTS support ( #2753 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-04-27 10:26:26 +08:00
7b876324c2
[Feature] Streaming video input with EVS frame filtering (RFC #2201 Phase 2-4) ( #2342 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Co-authored-by: shimei.jmh <shimei.jmh@taobao.com >
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
2026-04-24 22:54:35 +08:00
汪志鹏 and SYLAR
37e5954acf
[Config Refactor]: Remove bagel yaml ( #2936 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
2026-04-24 21:21:30 +08:00
wuhang
2e067bbc24
[Docs] CLI Docs updates ( #2978 )
...
Signed-off-by: wuhang <wuhang6@huawei.com >
2026-04-24 11:25:22 +08:00
3a32cd6780
[Model] Ming-flash-omni-2.0 Omni-Speech and TTS ( #2890 )
...
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com >
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com >
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com >
Co-authored-by: LHXuuu <xulianhao.xlh@antgroup.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-04-23 12:04:52 +08:00
Zeyu Huang | 黃澤宇
d8cc7a0409
[feat]: General diffusers adapter backend to run diffusion models ( #2724 )
...
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com >
2026-04-23 00:07:18 +08:00
Jared Wen and SYLAR
554233208f
[ConfigRefactor] GLM-Image ( #2977 )
...
Signed-off-by: JaredforReal <w13431838023@gmail.com >
Signed-off-by: Jared Wen <w13431838023@gmail.com >
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
2026-04-22 15:09:28 +08:00
Chen-Yo Sun and Yueqian Lin
8e1bbc9ab5
[Feature][Voxtral TTS] Support per-model extra sampling params & add cfg_alpha for Voxtral TTS ( #2338 )
...
Signed-off-by: Chen-Yo Sun <chenyo.sun@mistral.ai >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
2026-04-22 03:49:50 +00:00
Yueqian Lin and SYLAR
3f504b45d2
[Config Refactor] Migrate 5 TTS models (VoxCPM2 / CosyVoice3 / MiMo Audio / Voxtral TTS / Fish Speech S2 Pro) to Pipeline + Deploy schema ( #2958 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
2026-04-21 21:17:53 +08:00
a3903810fe
[Config Refactor][2/N] Pipeline + Deploy Config Schema ( #2383 )
...
Signed-off-by: lishunyang <lishunyang12@163.com >
Signed-off-by: reidliu41 <reid201711@gmail.com >
Signed-off-by: Alex Brooks <albrooks@redhat.com >
Co-authored-by: reidliu41 <reid201711@gmail.com >
Co-authored-by: xiaohajiayou <75477391+xiaohajiayou@users.noreply.github.com >
Co-authored-by: Alex Brooks <albrooks@redhat.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-04-19 14:36:00 +12:00
ChenWenjing
4124a1f858
[Feature]Support audio streaming input and output-phase2 ( #2581 )
...
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn >
2026-04-18 15:42:41 +08:00
Nick Cao
d2c23d7bd0
[Refactor] Remove resampy dependency ( #2891 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
2026-04-18 03:20:36 +00:00
Yuanheng Zhao
c0ccbb872f
[Model] Add Ming-flash-omni-2.0 Thinker Stage ( #1822 )
...
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com >
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com >
2026-04-17 19:10:02 +08:00
4bf4c63147
Add voxcpm model support. ( #2467 )
...
Signed-off-by: Celeste-jq <591998922@qq.com >
Signed-off-by: lyj-jjj <liuyingjun5@huawei.com >
Signed-off-by: IsleOfDawnlight <stellamou@qq.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Celeste-jq <591998922@qq.com >
Co-authored-by: lyj-jjj <liuyingjun5@huawei.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
2026-04-15 15:04:58 +08:00
Yueqian Lin
02e5dc747d
[Perf] VoxCPM2: streaming VAE + compile optimization (45% RTF reduction) ( #2758 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
2026-04-15 09:58:25 +08:00
iancarrasco-b10
48c30bc399
[Qwen3-TTS] Remove hardcoded distributed_executor_backend to improve single-GPU performance ( #2604 )
...
Signed-off-by: Ian Carrasco <ian.carrasco@baseten.co >
2026-04-14 14:49:13 +08:00
Yueqian Lin
d9e745ce2c
[Fix] VoxCPM2: support raw audio for voice cloning via OpenAI API ( #2720 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
2026-04-13 06:30:56 +00:00
Jiaqian Liu and Canlin Guo
8097747a5d
[Model] Adapt Wan2.2-I2V-A14B via LightX2V offline conversion path ( #2134 )
...
Signed-off-by: Celeste-jq <591998922@qq.com >
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com >
2026-04-13 14:20:04 +08:00
Samit
d1fef41266
[Doc] Add LTX-2 online serving deployment recipes with optimization benchmarks ( #1971 )
...
Signed-off-by: samithuang <285365963@qq.com >
Signed-off-by: Samit <285365963@qq.com >
2026-04-11 16:50:03 +08:00
Nick Cao and Claude
2bc183f6f0
[Refactor] Remove dependency on librosa ( #2273 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-04-11 02:11:31 +08:00
wuhang and Didan Deng
cb91cbe61e
[Entrypoint][Refactor]Stage CLI Refactor ( #2020 )
...
Signed-off-by: wuhang <wuhang6@huawei.com >
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
2026-04-10 14:40:37 +08:00
e2b0ee4b87
[Model] Add Dynin-omni model in vllm-omni ( #1759 )
...
Signed-off-by: kdg6245@snu.ac.kr <kdg6245@snu.ac.kr >
Signed-off-by: Yejoon Lee (IPAI) <leeyejoon@snu.ac.kr >
Signed-off-by: aidas (arpa-kt) <aidaslab@gmail.com >
Signed-off-by: Dogeun Kim <82812668+DOGEUNNKIM@users.noreply.github.com >
Co-authored-by: Yejoon Lee (IPAI) <leeyejoon@snu.ac.kr >
Co-authored-by: aidas (arpa-kt) <aidaslab@gmail.com >
2026-04-09 16:37:34 +08:00
Lancer
8609bc8ed9
[Feat] image2image for Z-Image ( #1580 )
...
Signed-off-by: Lancer <maruixiang6688@gmail.com >
Signed-off-by: Lancer <402430575@qq.com >
2026-04-08 14:41:24 +08:00
lishunyang12