a7ab91fb85
Signed-off-by: suyanli220 <suyanli220@gmail.com> Signed-off-by: suyan.li <suyan.li@bytedance.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: suyan.li <suyan.li@bytedance.com> Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Community Recipes
This directory contains community-maintained recipes for answering a practical user question:
How do I run model X on hardware Y for task Z?
Add recipes for this repository under this in-repo recipes/ directory. To
keep naming and layout consistent, organize recipes by model vendor in a way
that is aligned with
vllm-project/recipes, but treat
that external repository as a reference for structure rather than the place to
add files for this repo. Use one Markdown file per model and hardware profile,
named <model>-<hardware>.md.
Note: The vLLM-Omni TTS recipes (12 models across 8 providers) are also published on the rendered recipes site at recipes.vllm.ai (see vllm-project/recipes#554). Individual pages resolve at
https://recipes.vllm.ai/<provider>/<model-id>, e.g.bosonai/higgs-audio-v3-tts-4b.
The Recipe column in the Supported Models documentation links to the
published recipe when available and otherwise to the corresponding recipe in
this directory. The recipe files remain the source of truth for commands and
validation details.
Example layout:
recipes/
Qwen/
Qwen3-Omni-H100.md
Qwen3-TTS-A100.md
Tencent/
Covo-Audio-Chat-L40S.md
Available Recipes
| Recipe | Task | Hardware |
|---|---|---|
Baidu/ERNIE-Image.md |
Text-to-image online serving (ERNIE-Image 8B) | 1x or 2x RTX 4090 24GB |
Bagel/BAGEL-7B-MoT.md |
Text-to-image with shared online/offline examples | 1x A100 80GB / 2x CUDA GPUs |
black-forest-labs/FLUX.1-schnell.md |
Offline text-to-image generation with CPU offload | 1x NVIDIA GeForce RTX 5090 32GB |
Boogu/Boogu-Image.md |
Text-to-image online serving (Boogu-Image-0.1-Base) | 1x A100/H100 40GB+ |
BosonAI/Higgs-Audio-V3-TTS.md |
Online + offline multilingual TTS with voice cloning | 1x H100 80GB |
ByteDance/Lance.md |
Unified AR+diffusion: text/img/video gen + understanding (Lance 3B) | 1x B300 / A100 80GB |
fishaudio/Fish-Speech-S2-Pro.md |
Online serving for TTS | 1x A800 80GB |
Helios/Helios.md |
Text-to-video, image-to-video, and video-to-video generation | 1x NVIDIA H20 |
HiDream-ai/HiDream-O1-Image.md |
Text-to-image with shared offline example | 1x or 2x H100 80GB |
inclusionAI/Ming-flash-omni-2.0.md |
Online serving for multimodal chat + standalone TTS | 4x H100 / 1x H100 80GB |
inclusionAI/Ming-omni-tts.md |
Offline + online dense Ming TTS/audio generation | 1x H100 80GB / 1x AMD MI300X (ROCm 7.2) |
k2-fsa/OmniVoice.md |
Offline and online multilingual text-to-speech | 1x AMD MI300X (ROCm 7.2) |
IndexTeam/IndexTTS-2.md |
Online serving for voice-cloned TTS with optional emotion control | 1x L4 24GB or larger CUDA GPU |
IndexTeam/IndexTTS-2_5.md |
Online + offline multilingual voice-cloned TTS with native speed and emotion control | 1x NVIDIA H20 96GB |
MiniMaxAI/MiniMax-H3.md |
T2VA, FL2VA, image+audio Ref2VA, and multi-video Ref2VA serving | 1x GPU with CPU offload / 4x B300 |
MiniMaxAI/MiniMax-H3-Spark-GB10.md |
MiniMax H3 T2VA and Ref2VA on DGX Spark | 1x or 2x DGX Spark (GB10) |
MiniMaxAI/MiniMax-H3-MUSA.md |
MiniMax H3 T2VA, FL2VA, and Ref2VA on MUSA | 4x MTT S5000 (TP4 Ref2VA profile) |
krea/Krea-2.md |
Text-to-image (Turbo + Raw), offline + online, with LoRA | 1x H100 80GB |
LTX/LTX-2.md |
LTX-2/LTX-2.3 text-to-video and image-to-video with synchronized audio | H200 141GB / 96GB-class GPU |
LTX/LTX-2.5.md |
LTX-2.5-Diffusers: Full/SFT one-stage and distilled two-stage T2V/I2V with synchronized audio | NVIDIA B300; cuDNN-qualified |
MammothModa2/MammothModa2.md |
Preview and Dev text-to-image (AR → DiT); Dev text/image understanding | Preview: 1x L40S 48GB / 1x ≥40GB GPU; Dev: 1x NVIDIA GPU with sufficient cache headroom |
NVIDIA/SANA-Video-2B.md |
Native and Diffusers-adapter 480p/720p T2V+I2V | 1x RTX 5090 32GB |
meituan-longcat/LongCat-Video-Avatar-1.5.md |
Audio-driven avatar video generation (AT2V / AI2V, single- and multi-speaker, AVC continuation) | 1x H100 80GB |
mistralai/Voxtral-TTS.md |
Online serving for TTS | 1x RTX 4090 24GB |
Robbyant/LingBot-Video.md |
Native dense and MoE text-to-video serving | Dense: 1x L20X; MoE: 1x L20X (~67.7 GiB peak) |
SandAI/MAGI-2-preview-L20X.md |
Native Preview-stage text/image-to-video-and-audio generation | 1x/4x NVIDIA L20X 140 GiB qualified |
Robbyant/LingBot-World-2.0.md |
Offline and experimental realtime interactive world generation | 1x H200/B200 or 2x B200 for TP=2 |
cosmos3/Cosmos3-Nano.md |
Text-to-image, text-to-video, image-to-video, video-to-video generation, text to video with sound, action policy | 1x H200 141GB / B300 |
cosmos3/Cosmos3-Edge.md |
T2I / T2V / I2V + action (Physical-AI) generation on the Nemotron-based Edge transformer | 1x GPU (~8 GiB) |
cosmos3/Cosmos3-Super.md |
64B T2I / T2V / I2V / V2V generation (+ optional audio) / Action policy | 8x H200/H100/A100 / 2x H200 / B300 |
OpenBMB/MiniCPM-o-4_5.md |
Online serving for omni multimodal chat (text / image / audio / video → text + 24 kHz speech) | 2x A100/H100 80GB / 3x mid-tier GPU / 8x RTX 4090 24GB |
OpenBMB/VoxCPM2.md |
Online + offline TTS with native AR pipeline (48 kHz, 30+ languages) | 1x RTX 4090 24GB |
OpenMOSS/MOSS-TTS.md |
Online + offline multilingual TTS (MOSS-TTS family, 8B) | 1x H100 80GB |
NVIDIA/Nemotron-Labs-Audex.md |
TTS / text-to-audio / audio understanding / cascaded S2S (Audex 2B + 30B-A3B) | 1x H100 80GB |
NVIDIA/NemotronLabs-VoiceChat.md |
Offline speech-to-speech voice chat (11B, frame-locked 12.5 Hz, 3-stage thinker/talker/code2wav) | 1x H100 80GB |
NVIDIA/SANA-WM.md |
First-frame image-to-video serving with camera control | 1x 24GB+ CUDA GPU; ~40GB disk |
Qwen/Qwen-Image.md |
Text-to-image serving with step-wise continuous batching replay and ModelOpt mixed FP8/NVFP4 | 1x A100 80GB / 2x B200 |
Qwen/Qwen-Image-2512.md |
Text-to-image serving with step-wise continuous batching replay and ModelOpt FP8 / mixed FP8/NVFP4 | 1x A800 80GB / 2x B200 |
Qwen/Qwen-Image-Edit.md |
Text-guided single-image editing | 1x or 2x H200 141GB |
Qwen/Qwen3-Omni.md |
Online serving for multimodal chat | 1x A100 80GB |
Qwen/Qwen3-TTS.md |
Text-to-speech serving (CustomVoice / VoiceDesign / Base) | 1x H100/A100 80GB / 1x AMD MI300X |
SenseNova/SenseNova-U1.md |
Unified image generation and understanding | 1x or 2x H200 / 1x AMD MI300X |
SenseNova/SenseNova-U1.5.md |
Unified image generation and understanding, with an 8-step distilled LoRA | 1x A800 80GB |
StabilityAI/Stable-Audio-Open.md |
Offline + online text-to-audio generation (Stable Audio Open) | 1x RTX 4090 24GB / 1x AMD MI300X |
Tencent/Covo-Audio-Chat.md |
Online serving for audio chat | 1x A100 80GB |
Tencent/HunyuanImage-3.0-Instruct.md |
DiT-only text-to-image serving and benchmark, including ModelOpt mixed FP8/NVFP4 | 4x H100/H800 80GB / 2x B200 |
Wan-AI/Wan2.2-T2V.md |
Text-to-video serving (Wan2.2 14B) with Skip-Softmax sparse attention | 1x datacenter Blackwell (B200/B300) |
Wan-AI/Wan2.2-TI2V-5B-A100.md |
Unified 720p text-to-video and image-to-video serving (Wan2.2 5B) | 1x A100-SXM4-80GB |
Wan-AI/Wan2.2-I2V.md |
Image-to-video serving (Wan2.2 14B) | 8x Ascend NPU (A2/A3) |
Wan-AI/Wan2.2-S2V.md |
Speech-to-video serving (Wan2.2 14B) | 2x A100/H100 80GB |
Wan-AI/Wan2.1-VACE.md |
Unified T2V, I2V, V2LF, FLF2V, inpaint, and R2V | 1x RTX 5090 (1.3B) / 1x L40S 48GB with layerwise offload (14B) |
XiaomiMiMo/MiMo-Audio.md |
Offline + online omni audio (TTS, ASR, dialogue) | 1x RTX 5090 / 5090D 32GB |
StabilityAI/Stable-Diffusion-3.5.md |
Text-to-image serving (SD 3.5-medium and SD 3.5-large) | 1x RTX A6000 48GB |
zai-org/GLM-TTS.md |
Online serving for Chinese/English zero-shot voice-cloned TTS | 1x A40 48GB / 2x non-standard RTX 4090 48GB (49,140 MiB each) |
GLM/GLM-Image.md |
Online serving for image generation | 1x A800 80GB / 2x A800 80GB |
JD/JoyAI-VL-Interaction.md |
Real-time streaming video-language interaction (proactive speak/silence/delegate) | 1x GPU 24GB+ |
Keep one accelerator model or platform per recipe file. Add a separately suffixed recipe and table row for another GPU, ROCm accelerator, or NPU.
See TEMPLATE.md for the recommended format.