anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
1,692 Commits 235 Branches 31 Tags
f0cec3ecd4e1991bd1cf9c0a6bbad9d0fe80480b
Commit Graph
147 Commits
Author SHA1 Message Date
SYLAR f0cec3ecd4 Merge branch 'main' into lance-model-support 2026-05-22 04:13:58 +08:00
Yueqian Lin fc8486cf20 [Doc][TTS] CosyVoice3 online docs + residual TTS yaml cleanup + remove VoxCPM v1 (#3748) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-05-20 15:24:06 +08:00
lishunyang 69bcc6416d [New Model] Lance (ByteDance) — unified AR+diffusion multimodal (#3697) 
Adds Lance (bytedance-research/Lance), a 3B unified AR+diffusion
multimodal model on a Qwen2.5-VL backbone. Lance is BAGEL-lineage:
it reuses vLLM-Omni's BAGEL Mixture-of-Transformers core verbatim and
specializes only the ViT (Qwen2.5-VL vision), the VAE (Wan2.2) and the
checkpoint layout.

All six single-stage Lance modalities verified end-to-end on 1x B300:
t2i, t2v, image_edit, video_edit, x2t_image (image understanding) and
x2t_video (video understanding). 3-D mRoPE is wired through
BagelRotaryEmbedding for the x2t / video / edit paths. Includes the
upstream gap-analysis quality fixes (task-specific system prompts, full
Qwen chat template, upstream inference defaults and aspect-ratio
buckets).

Ships an offline example (examples/offline_inference/lance/), an
OpenAI-compatible online-serving example
(examples/online_serving/lance/) and an e2e test
(tests/e2e/online_serving/test_lance.py).

See LANCE_PR.md for the architecture rationale, per-modality status and
the two remaining follow-ups (two-stage AR+DiT, video_edit polish).

Closes #3697.

Signed-off-by: lishunyang <lishunyang03@gmail.com>
 2026-05-18 22:09:14 +00:00
Jie LiuandYueqian Lin 90f5b3c3a1 Update streaming_speech_client.py to solve Qwen/Qwen3-TTS-12Hz-0.6B-CustomVoice voice problem (#3380) 
Signed-off-by: Jie Liu <33612777+keeper-jie@users.noreply.github.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
 2026-05-14 20:43:05 -07:00
DnoobYueqian LinHongsheng Liu
 		d7ea5d5979 [New Model] Add support for tencent/Covo-Audio-Chat (#2293) 
Signed-off-by: Dnoob <dxpouo@gmail.com>
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-05-12 14:39:58 +08:00
汪志鹏 c9a8556c24 [New Model]: Add sensenova u1 support (#3319) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-05-11 22:19:00 +08:00
lsyyyyyand汪志鹏 32663f21d5 [Feat] support hsdp for Bagel (#3150) 
Signed-off-by: siyuan.lei <siyuanlei37@gmail.com>
Signed-off-by: lsyyyyy <siyuanlei37@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-05-08 15:20:42 +08:00
wangyuandHuang, Zeyu 6f2ad7b403 [Test] Unify L2/L3 test layout, Buildkite steps, and test helpers (#2556) 
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
Signed-off-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Co-authored-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
 2026-05-08 14:12:23 +08:00
Zheng WengangPeiqi YinyinpeyinpeiqiHongsheng LiuGao HanChenguang Zheng
 		1e5f288a91 [FEAT] support multi-stage deployment (#2396) 
Signed-off-by: ZhengWG <zwg0606@gmail.com>
Signed-off-by: Zheng Wengang <zwg0606@gmail.com>
Signed-off-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Signed-off-by: yinpe <11810305@mail.sustech.edu.cn>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: yinpe <11810305@mail.sustech.edu.cn>
Co-authored-by: yinpeiqi <yinpeiqi809@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Chenguang Zheng <645327136@qq.com>
 2026-05-06 19:33:18 +08:00
Yueqian Lin 5cf3f7947b [Docs] Consolidate moss_tts_nano + ming_flash_omni_tts into TTS hub (#3358) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-05-05 17:36:16 +00:00
Zeyu Huang | 黃澤宇andCopilot Autofix powered by AI f19891e4f1 [bugfix][CI] Diffusers backend update (#3096) 
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
Signed-off-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
 2026-05-05 20:54:39 +08:00
Yueqian Lin 703e31fc47 [Docs] Consolidate per-model TTS examples into a single hub (#3234) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-05-05 08:41:29 +12:00
Zhang JianZhang Jian汪志鹏
 		c708aecb18 [Diffusion] [Model] Support AudioX (#2077) 
Signed-off-by: Zhang Jian <jianmusings@gmail.com>
Signed-off-by: Zhang <jianmusings@gmail.com>
Signed-off-by: Zhang <zhang.jian@u.nus.edu>
Signed-off-by: Zhang Jian <e0322744@u.nus.edu>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Zhang Jian <e0322744@u.nus.edu>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-05-04 10:24:35 -04:00
Yueqian LinandHongsheng Liu 7491b67446 [Bugfix][MOSS-TTS-Nano] Drop fictional voice presets, require ref_audio (#3192) 
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-05-03 16:35:35 +12:00
Yuanheng ZhaoandHongsheng Liu d49356e4ca [Config Refactor] Migrate Ming and Ming-TTS deploy/pipline configs (#3154) 
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-05-03 14:43:49 +12:00
Ekagra RanjanCopilotYueqian LinYueqian Lin
 		4c2bea782e Fix "Add online serving to Stable Audio Diffusion and introduce v1/audio/generate endpoint" (#1794) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
 2026-04-28 18:06:57 +00:00
amy-why-3459 0470e96771 [Docs] Modify Qwen3-Omni's recipe (#3109) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-04-27 21:41:13 +08:00
Yueqian LinandHongsheng Liu afa2b09684 feat: add MOSS-TTS-Nano single-stage TTS support (#2753) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-04-27 10:26:26 +08:00
Sy03shimei.jmhSYLAR
 		7b876324c2 [Feature] Streaming video input with EVS frame filtering (RFC #2201 Phase 2-4) (#2342) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: shimei.jmh <shimei.jmh@taobao.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-04-24 22:54:35 +08:00
汪志鹏andSYLAR 37e5954acf [Config Refactor]: Remove bagel yaml (#2936) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-04-24 21:21:30 +08:00
wuhang 2e067bbc24 [Docs] CLI Docs updates (#2978) 
Signed-off-by: wuhang <wuhang6@huawei.com>
 2026-04-24 11:25:22 +08:00
Yuanheng ZhaoLHXuuuHongsheng Liu
 		3a32cd6780 [Model] Ming-flash-omni-2.0 Omni-Speech and TTS (#2890) 
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com>
Co-authored-by: LHXuuu <xulianhao.xlh@antgroup.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-04-23 12:04:52 +08:00
Zeyu Huang | 黃澤宇 d8cc7a0409 [feat]: General diffusers adapter backend to run diffusion models (#2724) 
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
 2026-04-23 00:07:18 +08:00
Jared WenandSYLAR 554233208f [ConfigRefactor] GLM-Image (#2977) 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: Jared Wen <w13431838023@gmail.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-04-22 15:09:28 +08:00
Chen-Yo SunandYueqian Lin 8e1bbc9ab5 [Feature][Voxtral TTS] Support per-model extra sampling params & add cfg_alpha for Voxtral TTS (#2338) 
Signed-off-by: Chen-Yo Sun <chenyo.sun@mistral.ai>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
 2026-04-22 03:49:50 +00:00
Yueqian LinandSYLAR 3f504b45d2 [Config Refactor] Migrate 5 TTS models (VoxCPM2 / CosyVoice3 / MiMo Audio / Voxtral TTS / Fish Speech S2 Pro) to Pipeline + Deploy schema (#2958) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-04-21 21:17:53 +08:00
SYLARreidliu41xiaohajiayouAlex BrooksHongsheng Liu
 		a3903810fe [Config Refactor][2/N] Pipeline + Deploy Config Schema (#2383) 
Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: xiaohajiayou <75477391+xiaohajiayou@users.noreply.github.com>
Co-authored-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-04-19 14:36:00 +12:00
ChenWenjing 4124a1f858 [Feature]Support audio streaming input and output-phase2 (#2581) 
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
 2026-04-18 15:42:41 +08:00
Nick Cao d2c23d7bd0 [Refactor] Remove resampy dependency (#2891) 
Signed-off-by: Nick Cao <ncao@redhat.com>
 2026-04-18 03:20:36 +00:00
Yuanheng Zhao c0ccbb872f [Model] Add Ming-flash-omni-2.0 Thinker Stage (#1822) 
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
 2026-04-17 19:10:02 +08:00
IsleOfDawnlightCeleste-jqlyj-jjjYueqian Lin
 		4bf4c63147 Add voxcpm model support. (#2467) 
Signed-off-by: Celeste-jq <591998922@qq.com>
Signed-off-by: lyj-jjj <liuyingjun5@huawei.com>
Signed-off-by: IsleOfDawnlight <stellamou@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Celeste-jq <591998922@qq.com>
Co-authored-by: lyj-jjj <liuyingjun5@huawei.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
 2026-04-15 15:04:58 +08:00
Yueqian Lin 02e5dc747d [Perf] VoxCPM2: streaming VAE + compile optimization (45% RTF reduction) (#2758) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-04-15 09:58:25 +08:00
iancarrasco-b10 48c30bc399 [Qwen3-TTS] Remove hardcoded distributed_executor_backend to improve single-GPU performance (#2604) 
Signed-off-by: Ian Carrasco <ian.carrasco@baseten.co>
 2026-04-14 14:49:13 +08:00
Yueqian Lin d9e745ce2c [Fix] VoxCPM2: support raw audio for voice cloning via OpenAI API (#2720) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-04-13 06:30:56 +00:00
Jiaqian LiuandCanlin Guo 8097747a5d [Model] Adapt Wan2.2-I2V-A14B via LightX2V offline conversion path (#2134) 
Signed-off-by: Celeste-jq <591998922@qq.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
 2026-04-13 14:20:04 +08:00
Samit d1fef41266 [Doc] Add LTX-2 online serving deployment recipes with optimization benchmarks (#1971) 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
 2026-04-11 16:50:03 +08:00
Nick CaoandClaude 2bc183f6f0 [Refactor] Remove dependency on librosa (#2273) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-11 02:11:31 +08:00
wuhangandDidan Deng cb91cbe61e [Entrypoint][Refactor]Stage CLI Refactor (#2020) 
Signed-off-by: wuhang <wuhang6@huawei.com>
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-04-10 14:40:37 +08:00
Dogeun KimYejoon Leeaidas
 		e2b0ee4b87 [Model] Add Dynin-omni model in vllm-omni (#1759) 
Signed-off-by: kdg6245@snu.ac.kr <kdg6245@snu.ac.kr>
Signed-off-by: Yejoon Lee (IPAI) <leeyejoon@snu.ac.kr>
Signed-off-by: aidas (arpa-kt) <aidaslab@gmail.com>
Signed-off-by: Dogeun Kim <82812668+DOGEUNNKIM@users.noreply.github.com>
Co-authored-by: Yejoon Lee (IPAI) <leeyejoon@snu.ac.kr>
Co-authored-by: aidas (arpa-kt) <aidaslab@gmail.com>
 2026-04-09 16:37:34 +08:00
Lancer 8609bc8ed9 [Feat] image2image for Z-Image (#1580) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
Signed-off-by: Lancer <402430575@qq.com>
 2026-04-08 14:41:24 +08:00
Markus / Markandmarksverdhai feefdaee9f [Bugfix] Accept 'speaker' as alias for 'voice' in TTS speech API (#2424) 
Signed-off-by: marksverdhei <marksverdhei@hotmail.com>
Signed-off-by: marksverdhai <249650165+marksverdhai@users.noreply.github.com>
Co-authored-by: marksverdhai <249650165+marksverdhai@users.noreply.github.com>
 2026-04-07 15:07:01 +00:00
skf 9584dd6e69 [HunyuanImage3] Align system_prompt support with official implementation (#2270) 
Signed-off-by: skf1999 <13234016272@163.com>
 2026-04-07 15:37:17 +08:00
+11
Juan Pablo ZuluagaJuanPZuluagaYi LiuHongsheng LiuCanlin GuoBinh TangBinh TangDidan Dengrein yangzhumingjue138ChenWenjingvraiti汪志鹏Sy03chickeytonJerry SongLidang JiangClaude Opus 4.6Alex BrookslinyueqianYueqian Lin
 		f50c5a413f [Qwen3TTS] [TTS] [Feat] Refactor voice cache manager (#2108) 
Signed-off-by: JuanPZuluaga <juanz9312@gmal.com>
Signed-off-by: yiliu30 <yi4.liu@intel.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: Binh Tang <tangbinhna@gmail.com>
Signed-off-by: Binh Tang <binht@netflix.com>
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
Signed-off-by: vraiti <vraiti@redhat.com>
Signed-off-by: Songrui625 <songrui625@gmail.com>
Signed-off-by: Lidang Jiang <lidangjiang@gmail.com>
Signed-off-by: Lidang-Jiang <lidangjiang@gmail.com>
Signed-off-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: JuanPZuluaga <juanz9312@gmal.com>
Co-authored-by: Yi Liu <yi4.liu@intel.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
Co-authored-by: Binh Tang <tangbinhna@gmail.com>
Co-authored-by: Binh Tang <binht@netflix.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: rein yang <73573651+R2-Y@users.noreply.github.com>
Co-authored-by: zhumingjue138 <zhumingjue@huawei.com>
Co-authored-by: ChenWenjing <54166744+Shirley125@users.noreply.github.com>
Co-authored-by: vraiti <vraiti@redhat.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Sy03 <1370724210@qq.com>
Co-authored-by: chickeyton <ngton2014@gmail.com>
Co-authored-by: Jerry Song <46962917+Songrui625@users.noreply.github.com>
Co-authored-by: Lidang Jiang <119769478+Lidang-Jiang@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-04-03 16:51:58 -04:00
Yueqian Lin 0e83ebe1d4 [OmniVoice] Add two-stage TTS serving support (#2463) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-04-03 20:41:20 +00:00
ChenWenjingandHongsheng Liu 728cf6d023 [Feature] add session based audio streaming input (#2208) 
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-04-02 22:51:41 +08:00
Samit 7b965a7b6c [Feat][Benchmark] Add synchronous video generation endpoint POST /v1/videos/sync for benchmark test (#2049) 
Signed-off-by: samithuang <285365963@qq.com>
 2026-04-01 10:47:33 +08:00
汪志鹏 369f301a1b [BugFix] Add BAGEL single-stage diffusion config and fix multiple <im_start><im_end> bug (#2381) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-04-01 09:19:13 +08:00
noobHappylife 7cb7b3e8ad Fix Qwen3-TTS gradio demo (#2372) 
Signed-off-by: noobhappylife <aratar1991@hotmail.com>
 2026-03-31 11:42:09 +00:00
chickeyton fa0792a0aa [Doc] Add transformers version requirement in GLM-Image example doc (#2265) 
Signed-off-by: chickeyton <ngton2014@gmail.com>
 2026-03-29 15:56:51 +00:00
Lidang Jiang 168bea2440 [Misc] Make gradio an optional dependency and upgrade to >=6.7.0 (#2221) 
Signed-off-by: Lidang-Jiang <lidangjiang@gmail.com>
Signed-off-by: Lidang Jiang <lidangjiang@gmail.com>
 2026-03-28 12:33:55 -04:00