diff --git a/recipes/README.md b/recipes/README.md new file mode 100644 index 000000000..293c73c48 --- /dev/null +++ b/recipes/README.md @@ -0,0 +1,27 @@ +# Community Recipes + +This directory contains community-maintained recipe templates for answering a +practical user question: + +> How do I run model X on hardware Y for task Z? + +We suggest organizing recipes by model vendor, aligned with +[`vllm-project/recipes`](https://github.com/vllm-project/recipes), and using +one Markdown file per model family by default. + +Example layout: + +```text +recipes/ + Qwen/ + Qwen3-Omni.md + Qwen3-TTS.md + Tencent-Hunyuan/ + HunyuanVideo.md +``` + +Within a single recipe file, include different hardware support sections such +as `GPU`, `ROCm`, and `NPU`, and add concrete tested configurations like +`1x A100 80GB` or `2x L40S` inside those sections when applicable. + +See [TEMPLATE.md](./TEMPLATE.md) for the recommended format. diff --git a/recipes/TEMPLATE.md b/recipes/TEMPLATE.md new file mode 100644 index 000000000..653ff5466 --- /dev/null +++ b/recipes/TEMPLATE.md @@ -0,0 +1,68 @@ +# Recipe Title + +> Example: Qwen3-Omni for speech chat on 1x A100 80GB + +## Summary + +- Vendor: +- Model: +- Task: +- Mode: +- Maintainer: + +## When to use this recipe + +Briefly describe the concrete scenario this recipe covers. + +## References + +- Upstream or canonical docs: +- Related example under `examples/`: +- Related issue or discussion: + +## Hardware Support + +Add one section per platform, such as `GPU`, `ROCm`, or `NPU`. Under each +platform section, document one or more tested hardware configurations. + +## GPU + +### 1x A100 80GB + +### Environment + +- OS: +- Python: +- Driver / runtime: +- vLLM version: +- vLLM-Omni version or commit: + +### Command + +```bash +# Add the exact command(s) here +``` + +### Verification + +```bash +# Add a quick validation command or expected output here +``` + +### Notes + +- Memory usage: +- Key flags: +- Known limitations: + +### 2x L40S + +Repeat the same structure for other hardware setups as needed. + +## ROCm + +Repeat the same structure for ROCm setups as needed. + +## NPU + +Repeat the same structure for NPU setups as needed.