[MR V2] Migrate to Model Runner V2
Squashed migration rebased onto current main. Collapses the 46-commit dev/migrate-MR-v2 history (10 merge commits + superseded pre-rebase work) into a single commit, so the diff reflects only the genuine MR-V2 feature (worker_v2/ runners + model_states, omni schedulers, stage engine pipeline, pooling_output serialization bridge). Content already merged to main separately (step_audio2, tts_adapters, npu/_310p, etc.) drops out. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
cead615288
commit
9678ae58cf
74 files changed
+8244
-95
No files matched your search
@@ -121,6 +121,13 @@ class OmniEngineCoreOutput(EngineCoreOutput):
|
||||
# pooling_output is inherited from EngineCoreOutput as torch.Tensor | None
|
||||
# and retains its original vLLM semantics for pooling/embedding tasks.
|
||||
multimodal_output: dict[str, torch.Tensor] | None = None
|
||||
# Bytes-serialized carrier for a dict-shaped pooling_output (MR V2 cross-stage
|
||||
# hidden/codec handoff). vLLM 0.23 strictly decodes ``pooling_output`` as a
|
||||
# ``torch.Tensor``, so a per-request dict payload cannot ride that field. The
|
||||
# producer (OmniARScheduler) sets ``pooling_output=None`` and serializes the
|
||||
# dict here; the consumer (StagePool._poll_stage_raw) rehydrates it back into
|
||||
# ``pooling_output`` after decode. Tensor pooling outputs are unaffected.
|
||||
pooling_output_payload: AdditionalInformationPayload | None = None
|
||||
# Finished flag for streaming input segment
|
||||
is_segment_finished: bool | None = False
|
||||
# Streaming update prompt length
|
||||
|
||||
Reference in new issue
Block a user