---
title: Autoregressive Runtime
kind: module
status: draft
owners:
  - "@tzhouam"
  - "@yinpeiqi"
  - "@fake0fan"
  - "@Sy0307"
  - "@Gaohan123"
primary_code_paths:
  - vllm_omni/core/**
  - vllm_omni/worker/**
related_code_paths:
  - vllm_omni/model_executor/**
depends_on:
  - engine_orchestration.md
  - model_integration.md
  - input_output_modality_contracts.md
validation_paths:
  - tests/core/**
  - tests/worker/**
upstream_refs:
  - vllm.v1.core
  - vllm.v1.worker
last_reviewed: 2026-07-16
---

# Autoregressive runtime

The AR runtime extends vLLM scheduling and worker execution for omni-stage
inputs and outputs while preserving vLLM scheduling and cache semantics.

## Candidate invariants

### AR-INV-001: vLLM owns base scheduling semantics

**Rule:** Omni schedulers MUST preserve upstream request-state and cache
transitions unless an Omni-specific difference is documented and tested.

### AR-INV-002: Omni data crosses explicit adapters

**Rule:** Modality-specific stage data MUST be converted at an input or output
adapter, not injected through unrelated scheduler state.

### AR-INV-003: Workers execute assigned work

**Rule:** Workers and model runners MUST NOT implement cross-stage routing.

## Token-only output payloads

An AR stage whose downstream decoder consumes sampled tokens rather than
hidden states can set `omni_pooler_payload_include_hidden = False`. Hidden
states remain available for logits and sampling; prefill conditioning and
other required multimodal outputs still cross the stage boundary.

A chunk processor that needs IDs on steps without a tensor payload declares
`requires_token_updates = True`. The scheduler then enqueues nonempty sampled
IDs while preserving final-stage exclusions and the processor's EOF handling.
Processors without this declaration retain their tensor-payload trigger.

When hidden and multimodal payloads are absent, the shared runner skips empty
per-request payload construction. Prefix-cache merges and pending model
postprocess retain the normal path. This policy is independent of whether
output materialization runs inline or in the background.

Inline construction consumes query offsets, scheduled-token counts and
scheduler metadata before the next execution can reuse them. Background
construction snapshots those inputs. Both paths reuse the request IDs already
copied by upstream bookkeeping; connector collection stays on the main thread
because it can run a tensor-parallel collective.

## Safe-change guide

Test request lifecycle, abort, cache state, and every affected worker execution
mode against the supported upstream vLLM contract.