vllm-omni/recipes/lerobot/Pi0.md

Pi0 (π0, Pi-Zero) VLA

Vision-Language-Action policy serving over the OpenPI realtime websocket

Summary

• Vendor: Physical Intelligence (checkpoint redistributed by LeRobot)
• Model: lerobot/pi0_base
• Task: Vision-Language-Action — multi-camera images + a language instruction + robot proprioceptive state → a continuous action chunk [action_horizon, action_dim] = [50, 32] via flow-matching denoising. π0 does not emit text tokens.
• Mode: Online serving via the OpenPI realtime robot API (/v1/realtime/robot/openpi websocket)
• Maintainer: Community

When to use this recipe

Use this recipe to deploy lerobot/pi0_base as a robot policy server: a robot client connects over the OpenPI websocket and, per control step, sends an observation (camera frames + state + instruction) and receives an action chunk.

A single pipeline class (Pi0Pipeline) serves the policy. π0 is stateless across calls (no KV reuse), so the protocol's session_id / reset are accepted but ignored.

Weights

π0 uses the LeRobot lerobot/pi0_base checkpoint (HF, ~13 GB). Either let the server download it (MODEL=lerobot/pi0_base) or point at a local copy.

Hardware Support

1x H200 141GB (Online serving)

Environment

• OS: Ubuntu 22.04+
• Python: 3.12+
• Driver / runtime: NVIDIA CUDA environment
• vLLM / vLLM-Omni: use the commit you are deploying from
• OpenPI client extras (for the example client): openpi-client (pip install -e packages/openpi-client from the openpi repo), websockets, msgpack, msgpack-numpy

Command

vllm serve lerobot/pi0_base \
  --omni \
  --host 0.0.0.0 --port 8000 \
  --served-model-name pi0 \
  --deploy-config vllm_omni/deploy/pi0.yaml \
  --enforce-eager --disable-log-stats

The deploy config (vllm_omni/deploy/pi0.yaml) declares a single diffusion stage (Pi0Pipeline), float32, max_num_seqs: 1, and the policy_server_config handshake metadata (3 cameras, joint_position, action horizon 50, action dim 32) that enables the OpenPI realtime serving layer.

Verification

Run the example client (sends a synthetic observation and prints the returned action chunk):

python examples/online_serving/pi0/openpi_client.py \
  --host 127.0.0.1 --port 8000 \
  --prompt "pick up the red block and place it in the bin"

It connects, prints the server metadata, sends robot observations, and prints the returned [action_horizon, action_dim] = [50, 32] action chunks. The observation is a flat dict per inference:

{
    "observation.images.base_0_rgb":        np.uint8[H, W, 3],
    "observation.images.left_wrist_0_rgb":  np.uint8[H, W, 3],
    "observation.images.right_wrist_0_rgb": np.uint8[H, W, 3],
    "state":   np.float32[state_dim],   # zero-padded to max_state_dim=32 server-side
    "prompt":  "pick up the red block",
    "session_id": "<uuid>",             # accepted but ignored (π0 is stateless)
}

Camera keys must match the server's image_feature_keys (the checkpoint's input_features order). If your robot uses different camera names, set model_config.image_key_map in pi0.yaml to map raw obs keys → feature keys.

Correctness

π0's flow-matching kernel is bit-for-bit matched to the LeRobot PI0Policy reference (max|Δ| = 7.15e-07, CPU/float32, fixed noise; see tests/diffusion/models/pi0/test_pi0_parity.py::test_pi0_vllm_omni_vs_lerobot).

References

• Model blog: https://www.physicalintelligence.company/blog/pi0
• Checkpoint: https://huggingface.co/lerobot/pi0_base
• Pipeline: vllm_omni/diffusion/models/pi0/pipeline_pi0.py
• Deploy config: vllm_omni/deploy/pi0.yaml
• Example client: examples/online_serving/pi0/
• Tests: CPU units + LeRobot parity in tests/diffusion/models/pi0/, OpenPI websocket e2e in tests/e2e/online_serving/test_pi0_expansion.py