anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

[Feature] vLLM-Omni RDMA connector (#1019)

Browse Source 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
This commit is contained in:
NATURE authored and GitHub committed 2026-02-24 11:13:52 +08:00
1 parent 7b02b856a6
commit 1589931fa6
20 files changed
+3645 -63
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/design/feature/ray_based_execution.md
+1 -1
View File
@@ -45,7 +45,7 @@ ray start --address=<HEAD_NODE_IP>:6399
When running on Ray, the system automatically adapts its communication strategy:
* **Cross-Node**: Recommended to use `MooncakeConnector` (requires separate configuration).
* **Cross-Node**: Recommended to use `MooncakeTransferEngineConnector` (RDMA, fastest) or `MooncakeStoreConnector` (TCP fallback).
* **Same-Node**: Can still use `SharedMemoryConnector` for efficiency, or Ray's native object store (plasma).
* **SHM threshold default differs**: when `worker_backend="ray"`, the SharedMemoryConnector default threshold is set to `sys.maxsize`, which forces payloads to go inline (no SHM). Override `shm_threshold_bytes` in the connector config if you want SHM for Ray runs.