aca4b7d65c
Signed-off-by: Chonghao Zhang <chzhang2021@gmail.com> Signed-off-by: chzhang2021 <chzhang2021@gmail.com> Signed-off-by: Chonghao Zhang <chonghaoz@meta.com> Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com> Co-authored-by: Chonghao Zhang <chonghaoz@meta.com>
2.4 KiB
Community Recipes
This directory contains community-maintained recipes for answering a practical user question:
How do I run model X on hardware Y for task Z?
Add recipes for this repository under this in-repo recipes/ directory. To
keep naming and layout consistent, organize recipes by model vendor in a way
that is aligned with
vllm-project/recipes, but treat
that external repository as a reference for structure rather than the place to
add files for this repo. Use one Markdown file per model family by default.
Example layout:
recipes/
Qwen/
Qwen3-Omni.md
Qwen3-TTS.md
Tencent-Hunyuan/
HunyuanVideo.md
Available Recipes
Qwen/Qwen-Image.md: text-to-image serving recipe for Qwen-Image on1x A100 80GB, including optional step-wise continuous batching replayQwen/Qwen3-Omni.md: online serving recipe for multimodal chat on1x A100 80GBQwen/Qwen3-TTS.md: text-to-speech serving recipe for Qwen3-TTS (CustomVoice / VoiceDesign / Base) on1x H100/A100 80GBLTX/LTX-2.md: text-to-video and image-to-video serving recipe for LTX-2 on1x H200 141GBWan-AI/Wan2.2-I2V.md: image-to-video serving recipe for Wan2.2 14B on8x Ascend NPU (A2/A3)Tencent-Hunyuan/HunyuanImage-3.0-Instruct.md: DiT-only text-to-image serving and benchmark recipe for HunyuanImage-3.0-Instruct on4x H100/H800 80GBinclusionAI/Ming-flash-omni-2.0.md: online serving recipe for multimodal chat (4x H100 80GB) and standalone TTS (1x H100 80GB)Baidu/ERNIE-Image.md: text-to-image serving online serving recipe for ERNIE-Image 8B on1x RTX 4090 24GBor2x RTX 4090 24GBfishaudio/Fish-Speech-S2-Pro.md: online serving recipe for TTS on1x A800 80GBaudiox/AudioX.md: offline + online recipe for AudioX unified text/video→audio diffusion on1x L4 24GB
Within a single recipe file, include different hardware support sections such
as GPU, ROCm, and NPU, and add concrete tested configurations like
1x A100 80GB or 2x L40S inside those sections when applicable.
See TEMPLATE.md for the recommended format.