anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

[Core] Cleanup Omni connector runtime and transfer backends (#5096)

Browse Source 
Signed-off-by: spencerr221 <liubingyu62@gmail.com>
This commit is contained in:
Bingyu (Spencer) Liu authored and GitHub committed 2026-07-22 09:44:15 +08:00
1 parent 2cf5f57b3e
commit dcf8b07d39
53 files changed
+349 -448
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/design/feature/ray_based_execution.md
+1 -1
View File
@@ -47,7 +47,7 @@ When running on Ray, the system automatically adapts its communication strategy:
* **Cross-Node**: Recommended to use `MooncakeTransferEngineConnector` (RDMA, fastest) or `MooncakeStoreConnector` (TCP fallback).
* **Same-Node**: Can still use `SharedMemoryConnector` for efficiency, or Ray's native object store (plasma).
* **SHM threshold default differs**: when `worker_backend="ray"`, the SharedMemoryConnector default threshold is set to `sys.maxsize`, which forces payloads to go inline (no SHM). Override `shm_threshold_bytes` in the connector config if you want SHM for Ray runs.
* **SHM payload routing**: `SharedMemoryConnector` always stores payloads in `/dev/shm`.
### 2.4 Internal Helpers