25 lines
491 B
Markdown
25 lines
491 B
Markdown
# vLLM-Omni CLI Guide
|
|
|
|
The CLI for vLLM-Omni inherits from vllm with some additional arguments.
|
|
|
|
## serve
|
|
|
|
Starts the vLLM-Omni OpenAI Compatible API server.
|
|
|
|
Start with a model:
|
|
|
|
```bash
|
|
vllm serve Qwen/Qwen2.5-Omni-7B --omni
|
|
```
|
|
|
|
Specify the port:
|
|
|
|
```bash
|
|
vllm serve Qwen/Qwen2.5-Omni-7B --omni --port 8091
|
|
```
|
|
|
|
If you have custom stage configs file, launch the server with command below
|
|
```bash
|
|
vllm serve Qwen/Qwen2.5-Omni-7B --omni --stage-configs-path /path/to/stage_configs_file
|
|
```
|