[Docs] optimize and supplement docs system (#86)
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
This commit is contained in:
1 parent
38f58064eb
commit
db81ee9f69
75 files changed
+2452
-512
No files matched your search
@@ -0,0 +1,24 @@
|
||||
# vLLM-Omni CLI Guide
|
||||
|
||||
The CLI for vLLM-Omni inherits from vllm with some additional arguments.
|
||||
|
||||
## serve
|
||||
|
||||
Starts the vLLM-Omni OpenAI Compatible API server.
|
||||
|
||||
Start with a model:
|
||||
|
||||
```bash
|
||||
vllm serve Qwen/Qwen2.5-Omni-7B --omni
|
||||
```
|
||||
|
||||
Specify the port:
|
||||
|
||||
```bash
|
||||
vllm serve Qwen/Qwen2.5-Omni-7B --omni --port 8091
|
||||
```
|
||||
|
||||
If you have custom stage configs file, launch the server with command below
|
||||
```bash
|
||||
vllm serve Qwen/Qwen2.5-Omni-7B --omni --stage-configs-path /path/to/stage_configs_file
|
||||
```
|
||||
Reference in new issue
Block a user