Roger WangandGao Han
a5800cd2c2
Signed-off-by: Roger Wang <hey@rogerw.io> Signed-off-by: Gao Han <gaohan19@huawei.com> Co-authored-by: Gao Han <gaohan19@huawei.com>
24 lines
535 B
Markdown
24 lines
535 B
Markdown
# Quickstart
|
|
|
|
This guide will help you quickly get started with vLLM-Omni to perform:
|
|
|
|
- Offline batched inference
|
|
- Online serving using OpenAI-compatible server
|
|
|
|
## Prerequisites
|
|
|
|
- OS: Linux
|
|
- Python: 3.12
|
|
|
|
## Installation
|
|
|
|
Please refer to [installation](installation/README.md)
|
|
|
|
## Offline Inference
|
|
|
|
Please refer to [offline inference](../user_guide/examples/offline_inference/qwen2_5_omni.md)
|
|
|
|
## Online Serving with OpenAI-Completions API
|
|
|
|
Please refer to [online serving](../user_guide/examples/online_serving/qwen2_5_omni.md)
|