anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

[Docs] CLI Docs updates (#2978)

Browse Source 
Signed-off-by: wuhang <wuhang6@huawei.com>
This commit is contained in:
wuhang authored and GitHub committed 2026-04-24 11:25:22 +08:00
1 parent 19980bb37c
commit 2e067bbc24
8 files changed
+312 -49
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/user_guide/examples/online_serving/bagel.md
+16 -8
View File
@@ -22,9 +22,16 @@ Or use the convenience script:
```bash
cd /workspace/vllm-omni/examples/online_serving/bagel
# Launch both stages in one session (legacy convenience flow)
bash run_server.sh
# Launch a single stage per terminal
bash run_server_stage_cli.sh --stage 0
bash run_server_stage_cli.sh --stage 1
```
If you have a custom stage configs file, launch the server with the command below:
```bash
vllm serve ByteDance-Seed/BAGEL-7B-MoT --omni --port 8091 --stage-configs-path /path/to/stage_configs_file
```
@@ -115,12 +122,13 @@ mooncake_master \
**2. Launch Stage 0 (Thinker / Orchestrator)** on the orchestrator node:
```bash
# API server port for client requests: 8000
vllm serve ByteDance-Seed/BAGEL-7B-MoT --omni \
--port 8000 \ # API server port for client requests
--port 8000 \
--stage-configs-path vllm_omni/model_executor/stage_configs/bagel_multiconnector.yaml \
--stage-id 0 \
-oma <ORCHESTRATOR_IP> \
-omp 8091
--omni-master-address <ORCHESTRATOR_IP> \
--omni-master-port 8091
```
**3. Launch Stage 1 (DiT)** on the remote node in headless mode:
@@ -130,8 +138,8 @@ vllm serve ByteDance-Seed/BAGEL-7B-MoT --omni \
--stage-configs-path vllm_omni/model_executor/stage_configs/bagel_multiconnector.yaml \
--stage-id 1 \
--headless \
-oma <ORCHESTRATOR_IP> \
-omp 8091
--omni-master-address <ORCHESTRATOR_IP> \
--omni-master-port 8091
```
**Mooncake Master arguments:**
@@ -150,8 +158,8 @@ vllm serve ByteDance-Seed/BAGEL-7B-MoT --omni \
| :------- | :---------- |
| `--stage-id` | Which stage this process runs (0 = Thinker, 1 = DiT) |
| `--headless` | Run without the API server (worker-only mode) |
| `-oma` | Orchestrator master address |
| `-omp` | Orchestrator master port for Stage 1 to connect to Stage 0 for task coordination |
| `--omni-master-address` | Orchestrator master address |
| `--omni-master-port` | Orchestrator master port for Stage 1 to connect to Stage 0 for task coordination |
> [!IMPORTANT]
> **Startup Order**: Stage 0 (orchestrator) must be launched **before** Stage 1 (headless).
@@ -165,7 +173,7 @@ All nodes must have network connectivity to each other. Ensure the following por
| :--- | :------- | :------ | :-------- |
| 50051 | TCP | Mooncake Master RPC | Worker → Orchestrator |
| 8080 | TCP | Mooncake HTTP Metadata Server | Worker → Orchestrator |
| 8091 | TCP | Orchestrator Master (`-omp`) | Worker → Orchestrator |
| 8091 | TCP | Orchestrator Master (`--omni-master-port`) | Worker → Orchestrator |
| 8000 | TCP | API Server (`--port`) | Client → Orchestrator |
| 9003 | TCP | Metrics (optional) | Monitoring → Orchestrator |