anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
Files
8619321b908321fdad9b03aaa9c1b27294c262c3
vllm-omni/docs/models/supported_models.md
T
LancerLancerHongsheng LiuGao Han
 9bf8f8ce79 [model] support FLUX.1-Kontext-dev (#561) 
Signed-off-by: Lancer <maruxiiang6688@gmail.com>
Signed-off-by: Lancer <maruixiang6688@gmail.com>
Signed-off-by: Lancer <402430575@qq.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Lancer <maruxiiang6688@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
2026-03-23 17:17:58 +00:00

5.9 KiB
Raw Blame History

Supported Models

vLLM-Omni supports unified multimodal comprehension and generation models across various tasks.

Model Implementation

If vLLM-Omni natively supports a model, its implementation can be found in gh-file:vllm_omni/model_executor/models and gh-file:vllm_omni/diffusion/models.

List of Supported Models for Nvidia GPU / AMD GPU

<style> th { white-space: nowrap; min-width: 0 !important; } </style>
Architecture Models Example HF Models
Qwen3OmniMoeForConditionalGeneration Qwen3-Omni Qwen/Qwen3-Omni-30B-A3B-Instruct
Qwen2_5OmniForConditionalGeneration Qwen2.5-Omni Qwen/Qwen2.5-Omni-7B, Qwen/Qwen2.5-Omni-3B
BagelForConditionalGeneration BAGEL (DiT-only) ByteDance-Seed/BAGEL-7B-MoT
HunyuanImage3ForCausalMM HunyuanImage3.0 (DiT-only) tencent/HunyuanImage-3.0, tencent/HunyuanImage-3.0-Instruct
QwenImagePipeline Qwen-Image Qwen/Qwen-Image
QwenImagePipeline Qwen-Image-2512 Qwen/Qwen-Image-2512
QwenImageEditPipeline Qwen-Image-Edit Qwen/Qwen-Image-Edit
QwenImageEditPlusPipeline Qwen-Image-Edit-2509 Qwen/Qwen-Image-Edit-2509
QwenImageLayeredPipeline Qwen-Image-Layered Qwen/Qwen-Image-Layered
GlmImagePipeline GLM-Image zai-org/GLM-Image
ZImagePipeline Z-Image Tongyi-MAI/Z-Image-Turbo
WanPipeline Wan2.1-T2V, Wan2.2-T2V, Wan2.2-TI2V Wan-AI/Wan2.1-T2V-1.3B-Diffusers, Wan-AI/Wan2.1-T2V-14B-Diffusers, Wan-AI/Wan2.2-T2V-A14B-Diffusers, Wan-AI/Wan2.2-TI2V-5B-Diffusers
WanImageToVideoPipeline Wan2.2-I2V Wan-AI/Wan2.2-I2V-A14B-Diffusers
LTX2Pipeline LTX-2-T2V Lightricks/LTX-2
LTX2ImageToVideoPipeline LTX-2-I2V Lightricks/LTX-2
HeliosPipeline, HeliosPyramidPipeline Helios BestWishYsh/Helios-Base, BestWishYsh/Helios-Mid, BestWishYsh/Helios-Distilled
OvisImagePipeline Ovis-Image OvisAI/Ovis-Image
LongcatImagePipeline LongCat-Image meituan-longcat/LongCat-Image
LongCatImageEditPipeline LongCat-Image-Edit meituan-longcat/LongCat-Image-Edit
StableDiffusion3Pipeline Stable-Diffusion-3 stabilityai/stable-diffusion-3.5-medium
CosyVoice3Model CosyVoice3 FunAudioLLM/Fun-CosyVoice3-0.5B-2512
MammothModa2ForConditionalGeneration MammothModa2-Preview bytedance-research/MammothModa2-Preview
Flux2KleinPipeline FLUX.2-klein black-forest-labs/FLUX.2-klein-4B, black-forest-labs/FLUX.2-klein-9B
FluxKontextPipeline FLUX.1-Kontext-dev black-forest-labs/FLUX.1-Kontext-dev
FluxPipeline FLUX.1-dev black-forest-labs/FLUX.1-dev
OmniGen2Pipeline OmniGen2 OmniGen2/OmniGen2
StableAudioPipeline Stable-Audio-Open stabilityai/stable-audio-open-1.0
Qwen3TTSForConditionalGeneration Qwen3-TTS-12Hz-1.7B-CustomVoice Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice
Qwen3TTSForConditionalGeneration Qwen3-TTS-12Hz-1.7B-VoiceDesign Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign
Qwen3TTSForConditionalGeneration Qwen3-TTS-12Hz-1.7B-Base Qwen/Qwen3-TTS-12Hz-0.6B-Base
GlmImageForConditionalGeneration GLM-Image zai-org/GLM-Image
NextStep11Pipeline NextStep-1.1 stepfun-ai/NextStep-1.1
MiMoAudioForConditionalGeneration MiMo-Audio-7B-Instruct XiaomiMiMo/MiMo-Audio-7B-Instruct
Flux2Pipeline FLUX.2-dev black-forest-labs/FLUX.2-dev
FishSpeechSlowARForConditionalGeneration Fish Speech S2 Pro fishaudio/s2-pro
DreamIDOmniPipeline DreamID-Omni XuGuo699/DreamID-Omni
HunyuanVideo15Pipeline HunyuanVideo-1.5-T2V hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-480p_t2v, hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-720p_t2v
HunyuanVideo15ImageToVideoPipeline HunyuanVideo-1.5-I2V hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-480p_i2v, hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-720p_i2v
VoxtralTTSForConditionalGeneration Voxtral TTS mistralai/Voxtral-4B-TTS-2603

List of Supported Models for NPU

<style> th { white-space: nowrap; min-width: 0 !important; } </style>
Architecture Models Example HF Models
Qwen3OmniMoeForConditionalGeneration Qwen3-Omni Qwen/Qwen3-Omni-30B-A3B-Instruct
Qwen2_5OmniForConditionalGeneration Qwen2.5-Omni Qwen/Qwen2.5-Omni-7B, Qwen/Qwen2.5-Omni-3B
HunyuanImage3ForCausalMM HunyuanImage3.0 (DiT-only) tencent/HunyuanImage-3.0, tencent/HunyuanImage-3.0-Instruct
QwenImagePipeline Qwen-Image Qwen/Qwen-Image
QwenImagePipeline Qwen-Image-2512 Qwen/Qwen-Image-2512
QwenImageEditPipeline Qwen-Image-Edit Qwen/Qwen-Image-Edit
QwenImageEditPlusPipeline Qwen-Image-Edit-2509 Qwen/Qwen-Image-Edit-2509
QwenImageLayeredPipeline Qwen-Image-Layered Qwen/Qwen-Image-Layered
QwenImageEditPlusPipeline Qwen-Image-Edit-2511 Qwen/Qwen-Image-Edit-2511
ZImagePipeline Z-Image Tongyi-MAI/Z-Image-Turbo
WanPipeline Wan2.1-T2V, Wan2.2-T2V, Wan2.2-TI2V Wan-AI/Wan2.1-T2V-1.3B-Diffusers, Wan-AI/Wan2.1-T2V-14B-Diffusers, Wan-AI/Wan2.2-T2V-A14B-Diffusers, Wan-AI/Wan2.2-TI2V-5B-Diffusers
WanImageToVideoPipeline Wan2.2-I2V Wan-AI/Wan2.2-I2V-A14B-Diffusers
LongcatImagePipeline LongCat-Image meituan-longcat/LongCat-Image
LongCatImageEditPipeline LongCat-Image-Edit meituan-longcat/LongCat-Image-Edit
Flux2KleinPipeline FLUX.2-klein black-forest-labs/FLUX.2-klein-4B, black-forest-labs/FLUX.2-klein-9B
Qwen3TTSForConditionalGeneration Qwen3-TTS-12Hz-1.7B-CustomVoice Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice
Qwen3TTSForConditionalGeneration Qwen3-TTS-12Hz-1.7B-VoiceDesign Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign
Qwen3TTSForConditionalGeneration Qwen3-TTS-12Hz-1.7B-Base Qwen/Qwen3-TTS-12Hz-0.6B-Base
Reference in new issue View Git Blame Copy Permalink