From ec4e12f379a9ec78fb8678988472f02fa671166b Mon Sep 17 00:00:00 2001 From: Hongsheng Liu Date: Thu, 6 Aug 2026 16:02:19 +0800 Subject: [PATCH] docs: remove Ray-based execution page Signed-off-by: Hongsheng Liu --- docs/.nav.yml | 1 - docs/design/feature/ray_based_execution.md | 59 ---------------------- docs/design/index.md | 1 - 3 files changed, 61 deletions(-) delete mode 100644 docs/design/feature/ray_based_execution.md diff --git a/docs/.nav.yml b/docs/.nav.yml index 3f99c1616..29de450e8 100644 --- a/docs/.nav.yml +++ b/docs/.nav.yml @@ -106,7 +106,6 @@ nav: - Feature Design: - Runtime and Stage Execution: - Disaggregated Inference: design/feature/disaggregated_inference.md - - Ray-based Execution: design/feature/ray_based_execution.md - Async Chunk: design/feature/async_chunk.md - Async Diffusion Output: design/feature/async_diffusion_output.md - Async Omni Output Materialization: design/feature/omni_async_output_materialization.md diff --git a/docs/design/feature/ray_based_execution.md b/docs/design/feature/ray_based_execution.md deleted file mode 100644 index 4f4caa642..000000000 --- a/docs/design/feature/ray_based_execution.md +++ /dev/null @@ -1,59 +0,0 @@ -# Distributed utils - -This directory (vllm_omni/distributed/ray_utils) contains utilities for distributed execution in vllm-omni, supporting both **Ray** and **Multiprocessing** backends. -## 1. Installation -```bash -pip install "ray[default]" -``` -## 2. Ray Utils - -The `ray_utils` module provides helper functions for managing Ray clusters and actors, which is essential for: -* **Multi-node deployment**: Running pipeline stages across different physical machines. -* **Resource management**: Efficient GPU/CPU allocation. - -### 2.1 Basic Usage - -To use the Ray backend, specify `worker_backend="ray"` when initializing the engine. - -**Command Line Example:** -```bash -vllm serve Qwen/Qwen2.5-Omni-7B \ - --omni \ - --port 8091 \ - --worker-backend ray \ - --ray-address auto -``` - -### 2.2 Cluster Setup - -**Step 1: Start Head Node** -Run this on your primary machine: -```bash -ray start --head --port=6399 -``` - -**Step 2: Connect Worker Nodes** -Run this on each worker machine: -```bash -ray start --address=:6399 -``` - -> **Tip**: For a complete cluster setup script, refer to the vLLM example: -> [run_cluster.sh](https://github.com/vllm-project/vllm/blob/main/examples/online_serving/run_cluster.sh) - -### 2.3 Distributed Connector Support - -When running on Ray, the system automatically adapts its communication strategy: - -* **Cross-Node**: Recommended to use `MooncakeTransferEngineConnector` (RDMA, fastest) or `MooncakeStoreConnector` (TCP fallback). -* **Same-Node**: Can still use `SharedMemoryConnector` for efficiency, or Ray's native object store (plasma). -* **SHM payload routing**: `SharedMemoryConnector` always stores payloads in `/dev/shm`. - -### 2.4 Internal Helpers - -* **`initialize_ray_cluster`**: Connects to an existing Ray cluster or starts a local one. - -## 3. Troubleshooting - -* **Connection Issues**: Ensure the Ray head node is accessible and ports (default 6399 in this example) are open. -* **Version Mismatch**: Ensure all nodes run the same version of Ray and Python. diff --git a/docs/design/index.md b/docs/design/index.md index 57fd9ba4c..43963ed1c 100644 --- a/docs/design/index.md +++ b/docs/design/index.md @@ -13,7 +13,6 @@ while the source paths remain stable so existing links continue to work. ### Runtime and stage execution - [Disaggregated Inference](feature/disaggregated_inference.md) -- [Ray-based Execution](feature/ray_based_execution.md) - [Async Chunk](feature/async_chunk.md) - [Async Diffusion Output](feature/async_diffusion_output.md) - [Async Omni Output Materialization](feature/omni_async_output_materialization.md)