Files
vllm-omni/examples/edge
tzhouam 723c550209 Integrate portable edge graph backends with Omni runtime
Extract versioned stage contracts, model shared CPU/iGPU/NPU memory constraints, and add explicit graph routes, reservations, output acknowledgements, and worker retirement. Fix native Windows shared-buffer lifetime and host paths. Document qualified hardware paths and remaining rollout gates.
2026-09-22 13:12:33 +08:00
..

Local graph example

See the backend contract and deployment guide.

From a configured Omni environment:

python examples/edge/run_graph_stage.py /path/to/manifest.json \
  --python /path/to/ort-venv/bin/python --report /path/to/run.json

The worker environment needs numpy, onnx and onnxruntime. For Windows, use --windows-worker and the controller-visible path to python.exe. --ep vitisai or --ep dml selects a vendor provider; declare every permitted provider using --allowed-provider. No provider substitution is hidden.

The example reads fixed-bucket inputs from the bundle. It reports backend execution only; use your own model adapter and quality fixtures for a model claim.