anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

[Core] Add NIXL omni connector (#6093)

Browse Source 
Signed-off-by: yuanwu <yuan.wu@intel.com>
Signed-off-by: Yuan Wu <yuan.wu@intel.com>
This commit is contained in:
Yuan Wu authored and GitHub committed 2026-09-13 16:35:02 +08:00
1 parent 01a2f93256
commit 6753bbd18c
33 files changed
+3032 -66
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/design/feature/disaggregated_inference.md
+2
View File
@@ -9,6 +9,7 @@ Backend-specific setup lives in separate docs:
- [MooncakeStoreConnector](omni_connectors/mooncake_store_connector.md)
- [MooncakeTransferEngineConnector](omni_connectors/mooncake_transfer_engine_connector.md)
- [MoriTransferEngineConnector](omni_connectors/mori_transfer_engine_connector.md)
- [NixlConnector](omni_connectors/nixl_connector.md)
- [YuanrongConnector](omni_connectors/yuanrong_connector.md)
- [YuanrongTransferEngineConnector](omni_connectors/yuanrong_transfer_engine_connector.md)
@@ -25,6 +26,7 @@ Current connectors operate in D2H2D (device to host to device) mode.
| Multi node (Mooncake Store) | MooncakeStoreConnector | TCP-based, requires Mooncake Master + metadata server. |
| Multi node (Mooncake RDMA) | MooncakeTransferEngineConnector | RDMA/TCP direct transfer with managed memory pool. Fastest. |
| Multi node (Mori RDMA) | MoriTransferEngineConnector | RDMA direct transfer via Mori IOEngine. |
| Multi node (NIXL) | NixlConnector | NIXL `READ` over UCX; the only backend covering Intel XPU. |
| Multi node (Yuanrong) | YuanrongConnector | Requires Yuanrong Datasystem + etcd. |
| Ascend NPU P2P (Yuanrong TE) | YuanrongTransferEngineConnector | Uses Yuanrong TransferEngine directly. Configure NPU device IPv4 and `memory_pool_device: "npu"`. |