docs: remove Ray-based execution page
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
1 parent
d272d57bef
commit
ec4e12f379
3 files changed
-61
No files matched your search
@@ -106,7 +106,6 @@ nav:
|
||||
- Feature Design:
|
||||
- Runtime and Stage Execution:
|
||||
- Disaggregated Inference: design/feature/disaggregated_inference.md
|
||||
- Ray-based Execution: design/feature/ray_based_execution.md
|
||||
- Async Chunk: design/feature/async_chunk.md
|
||||
- Async Diffusion Output: design/feature/async_diffusion_output.md
|
||||
- Async Omni Output Materialization: design/feature/omni_async_output_materialization.md
|
||||
|
||||
@@ -1,59 +0,0 @@
|
||||
# Distributed utils
|
||||
|
||||
This directory (vllm_omni/distributed/ray_utils) contains utilities for distributed execution in vllm-omni, supporting both **Ray** and **Multiprocessing** backends.
|
||||
## 1. Installation
|
||||
```bash
|
||||
pip install "ray[default]"
|
||||
```
|
||||
## 2. Ray Utils
|
||||
|
||||
The `ray_utils` module provides helper functions for managing Ray clusters and actors, which is essential for:
|
||||
* **Multi-node deployment**: Running pipeline stages across different physical machines.
|
||||
* **Resource management**: Efficient GPU/CPU allocation.
|
||||
|
||||
### 2.1 Basic Usage
|
||||
|
||||
To use the Ray backend, specify `worker_backend="ray"` when initializing the engine.
|
||||
|
||||
**Command Line Example:**
|
||||
```bash
|
||||
vllm serve Qwen/Qwen2.5-Omni-7B \
|
||||
--omni \
|
||||
--port 8091 \
|
||||
--worker-backend ray \
|
||||
--ray-address auto
|
||||
```
|
||||
|
||||
### 2.2 Cluster Setup
|
||||
|
||||
**Step 1: Start Head Node**
|
||||
Run this on your primary machine:
|
||||
```bash
|
||||
ray start --head --port=6399
|
||||
```
|
||||
|
||||
**Step 2: Connect Worker Nodes**
|
||||
Run this on each worker machine:
|
||||
```bash
|
||||
ray start --address=<HEAD_NODE_IP>:6399
|
||||
```
|
||||
|
||||
> **Tip**: For a complete cluster setup script, refer to the vLLM example:
|
||||
> [run_cluster.sh](https://github.com/vllm-project/vllm/blob/main/examples/online_serving/run_cluster.sh)
|
||||
|
||||
### 2.3 Distributed Connector Support
|
||||
|
||||
When running on Ray, the system automatically adapts its communication strategy:
|
||||
|
||||
* **Cross-Node**: Recommended to use `MooncakeTransferEngineConnector` (RDMA, fastest) or `MooncakeStoreConnector` (TCP fallback).
|
||||
* **Same-Node**: Can still use `SharedMemoryConnector` for efficiency, or Ray's native object store (plasma).
|
||||
* **SHM payload routing**: `SharedMemoryConnector` always stores payloads in `/dev/shm`.
|
||||
|
||||
### 2.4 Internal Helpers
|
||||
|
||||
* **`initialize_ray_cluster`**: Connects to an existing Ray cluster or starts a local one.
|
||||
|
||||
## 3. Troubleshooting
|
||||
|
||||
* **Connection Issues**: Ensure the Ray head node is accessible and ports (default 6399 in this example) are open.
|
||||
* **Version Mismatch**: Ensure all nodes run the same version of Ray and Python.
|
||||
@@ -13,7 +13,6 @@ while the source paths remain stable so existing links continue to work.
|
||||
### Runtime and stage execution
|
||||
|
||||
- [Disaggregated Inference](feature/disaggregated_inference.md)
|
||||
- [Ray-based Execution](feature/ray_based_execution.md)
|
||||
- [Async Chunk](feature/async_chunk.md)
|
||||
- [Async Diffusion Output](feature/async_diffusion_output.md)
|
||||
- [Async Omni Output Materialization](feature/omni_async_output_materialization.md)
|
||||
|
||||
Reference in new issue
Block a user