draft vllm-omni recipes
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com> Signed-off-by: hsliu <liuhongsheng4@huawei.com>
This commit is contained in:
1 parent
956f53b278
commit
189824dfb9
2 files changed
+95
No files matched your search
@@ -0,0 +1,27 @@
|
||||
# Community Recipes
|
||||
|
||||
This directory contains community-maintained recipe templates for answering a
|
||||
practical user question:
|
||||
|
||||
> How do I run model X on hardware Y for task Z?
|
||||
|
||||
We suggest organizing recipes by model vendor, aligned with
|
||||
[`vllm-project/recipes`](https://github.com/vllm-project/recipes), and using
|
||||
one Markdown file per model family by default.
|
||||
|
||||
Example layout:
|
||||
|
||||
```text
|
||||
recipes/
|
||||
Qwen/
|
||||
Qwen3-Omni.md
|
||||
Qwen3-TTS.md
|
||||
Tencent-Hunyuan/
|
||||
HunyuanVideo.md
|
||||
```
|
||||
|
||||
Within a single recipe file, include different hardware support sections such
|
||||
as `GPU`, `ROCm`, and `NPU`, and add concrete tested configurations like
|
||||
`1x A100 80GB` or `2x L40S` inside those sections when applicable.
|
||||
|
||||
See [TEMPLATE.md](./TEMPLATE.md) for the recommended format.
|
||||
@@ -0,0 +1,68 @@
|
||||
# Recipe Title
|
||||
|
||||
> Example: Qwen3-Omni for speech chat on 1x A100 80GB
|
||||
|
||||
## Summary
|
||||
|
||||
- Vendor:
|
||||
- Model:
|
||||
- Task:
|
||||
- Mode:
|
||||
- Maintainer:
|
||||
|
||||
## When to use this recipe
|
||||
|
||||
Briefly describe the concrete scenario this recipe covers.
|
||||
|
||||
## References
|
||||
|
||||
- Upstream or canonical docs:
|
||||
- Related example under `examples/`:
|
||||
- Related issue or discussion:
|
||||
|
||||
## Hardware Support
|
||||
|
||||
Add one section per platform, such as `GPU`, `ROCm`, or `NPU`. Under each
|
||||
platform section, document one or more tested hardware configurations.
|
||||
|
||||
## GPU
|
||||
|
||||
### 1x A100 80GB
|
||||
|
||||
### Environment
|
||||
|
||||
- OS:
|
||||
- Python:
|
||||
- Driver / runtime:
|
||||
- vLLM version:
|
||||
- vLLM-Omni version or commit:
|
||||
|
||||
### Command
|
||||
|
||||
```bash
|
||||
# Add the exact command(s) here
|
||||
```
|
||||
|
||||
### Verification
|
||||
|
||||
```bash
|
||||
# Add a quick validation command or expected output here
|
||||
```
|
||||
|
||||
### Notes
|
||||
|
||||
- Memory usage:
|
||||
- Key flags:
|
||||
- Known limitations:
|
||||
|
||||
### 2x L40S
|
||||
|
||||
Repeat the same structure for other hardware setups as needed.
|
||||
|
||||
## ROCm
|
||||
|
||||
Repeat the same structure for ROCm setups as needed.
|
||||
|
||||
## NPU
|
||||
|
||||
Repeat the same structure for NPU setups as needed.
|
||||
Reference in new issue
Block a user