docs: standardize API example ports
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
1 parent
8327f26216
commit
159ac2161a
5 files changed
+14
-14
No files matched your search
@@ -17,7 +17,7 @@ may also expose standard routes inherited from vLLM, such as
|
||||
Start a model with the shared serving command:
|
||||
|
||||
```bash
|
||||
vllm serve <model> --omni --port 8000
|
||||
vllm serve <model> --omni --port 8091
|
||||
```
|
||||
|
||||
Some models require additional flags or a deployment configuration. Use the
|
||||
@@ -26,13 +26,13 @@ model-specific guide when one is provided.
|
||||
After the server starts, check its health and served model name:
|
||||
|
||||
```bash
|
||||
export VLLM_OMNI_BASE_URL=http://localhost:8000
|
||||
export VLLM_OMNI_BASE_URL=http://localhost:8091
|
||||
|
||||
curl "$VLLM_OMNI_BASE_URL/health"
|
||||
curl "$VLLM_OMNI_BASE_URL/v1/models" | jq .
|
||||
```
|
||||
|
||||
Use `http://localhost:8000/v1` as the `base_url` for an OpenAI SDK client. If
|
||||
Use `http://localhost:8091/v1` as the `base_url` for an OpenAI SDK client. If
|
||||
the server was started with `--api-key`, include an `Authorization` header with
|
||||
`Bearer <api-key>` in requests.
|
||||
|
||||
|
||||
@@ -55,14 +55,14 @@ Start the duplex deployment:
|
||||
vllm serve openbmb/MiniCPM-o-4_5 --omni \
|
||||
--deploy-config vllm_omni/deploy/minicpmo_4_5_duplex.yaml \
|
||||
--trust-remote-code \
|
||||
--port 8099
|
||||
--port 8091
|
||||
```
|
||||
|
||||
Stream a mono, PCM16, 16 kHz WAV file with the provided client:
|
||||
|
||||
```bash
|
||||
python examples/online_serving/minicpmo/realtime_duplex_demo.py \
|
||||
--url 'ws://localhost:8099/v1/realtime?duplex=1' \
|
||||
--url 'ws://localhost:8091/v1/realtime?duplex=1' \
|
||||
--model openbmb/MiniCPM-o-4_5 \
|
||||
--input-wav input_16k_mono.wav \
|
||||
--ref-audio reference_voice.wav \
|
||||
|
||||
@@ -33,7 +33,7 @@ and closes.
|
||||
Start the bundled DreamZero deployment:
|
||||
|
||||
```bash
|
||||
vllm serve GEAR-Dreams/DreamZero-DROID --omni --port 8000 \
|
||||
vllm serve GEAR-Dreams/DreamZero-DROID --omni --port 8091 \
|
||||
--served-model-name dreamzero-droid \
|
||||
--deploy-config vllm_omni/deploy/dreamzero_tp1_cfg2.yaml \
|
||||
--enforce-eager --disable-log-stats
|
||||
@@ -51,7 +51,7 @@ hf download YangshenDeng/vllm-omni-dreamzero-assets \
|
||||
|
||||
python examples/online_serving/dreamzero/openpi_client.py \
|
||||
--host 127.0.0.1 \
|
||||
--port 8000 \
|
||||
--port 8091 \
|
||||
--video-dir outputs/dreamzero/assets
|
||||
```
|
||||
|
||||
|
||||
@@ -21,7 +21,7 @@ Start it with:
|
||||
```bash
|
||||
HF_TOKEN=... CUDA_VISIBLE_DEVICES=0 python -m \
|
||||
vllm_omni.experimental.fullduplex.personaplex.serving.server \
|
||||
--port 8124
|
||||
--port 8091
|
||||
```
|
||||
|
||||
| Route | Purpose |
|
||||
@@ -68,14 +68,14 @@ Start the model backend and orchestrator separately:
|
||||
```bash
|
||||
vllm serve jdopensource/JoyAI-VL-Interaction-Preview \
|
||||
--served-model-name JoyAI-VL-Interaction-Preview \
|
||||
--port 8061 \
|
||||
--port 8092 \
|
||||
--max-model-len 131072 \
|
||||
--enable-prefix-caching \
|
||||
--limit-mm-per-prompt '{"image":256,"video":1}'
|
||||
|
||||
python -m vllm_omni.experimental.fullduplex.joyvl.serving.server \
|
||||
--port 8070 \
|
||||
--main-backend-url http://127.0.0.1:8061/v1 \
|
||||
--port 8091 \
|
||||
--main-backend-url http://127.0.0.1:8092/v1 \
|
||||
--main-model JoyAI-VL-Interaction-Preview
|
||||
```
|
||||
|
||||
@@ -94,7 +94,7 @@ include an `interaction` block whose action is `silence`, `response`, or
|
||||
`delegate`.
|
||||
|
||||
```bash
|
||||
curl http://127.0.0.1:8070/v1/chat/completions \
|
||||
curl http://127.0.0.1:8091/v1/chat/completions \
|
||||
-H 'content-type: application/json' \
|
||||
-H 'x-session-id: demo' \
|
||||
-d '{
|
||||
|
||||
@@ -13,7 +13,7 @@ output:
|
||||
```bash
|
||||
vllm serve BestWishYsh/Helios-Distilled --omni \
|
||||
--diffusion-streaming-output \
|
||||
--port 8000
|
||||
--port 8091
|
||||
```
|
||||
|
||||
Install the client dependency and run the provided client:
|
||||
@@ -23,7 +23,7 @@ pip install websockets
|
||||
|
||||
python examples/online_serving/streaming_video_generation/streaming_video_client.py \
|
||||
--host 127.0.0.1 \
|
||||
--port 8000 \
|
||||
--port 8091 \
|
||||
--model BestWishYsh/Helios-Distilled \
|
||||
--prompt "A serene lakeside sunrise with mist over the water" \
|
||||
--output output.mp4
|
||||
|
||||
Reference in new issue
Block a user