[edge-infer] WIP snapshot: uncommitted M0/W1/W4/W5 edge work as of 2026-09-18

Verbatim copy of the edge/omni-edge working tree (tracked diffs + untracked
edge/local, tests/edge/local, docs/edge, pipeline files). Taken so that
vLLM-Omni v0.29.0rc1 can be merged on a branch without touching the
user's working tree. logs_edge/ and benchmark .npy outputs excluded.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
tzhouamandClaude Fable 5.1 committed 2026-09-18 09:47:29 +08:00
1 parent 3834de3105
commit 4e3e8a1ebc
62 files changed
+11043 -56

No files matched your search

@@ -0,0 +1,43 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""Spark-X2.5 as an Omni pipeline: one autoregressive text stage.
Spark's model definition has been in ``vllm_omni.model_executor.models`` for a
while, but only as a vLLM architecture -- there was no pipeline, so
``Omni(model=<a Spark checkpoint>)`` had nothing to resolve and the model was
reachable only through plain ``vllm.LLM``. That is the gap M0 closes: the text
mode has to run *through* the Omni stage runtime, or none of the stage
lifecycle, admission or cancellation work applies to it.
There is exactly one stage, and that is the design, not a simplification to be
undone later. The proposal's first decision is that an autoregressive session
stays on one backend by default: prefill and decode share the KV cache, and a
per-token boundary between devices buys nothing and costs a synchronization.
Spark's tokenizer, chat template and sampling are model-adapter concerns that
run in-process around this stage; they are not stages.
"""
from vllm_omni.config.stage_config import (
PipelineConfig,
StageExecutionType,
StagePipelineConfig,
)
SPARK2_5_PIPELINE = PipelineConfig(
model_type="spark2_5",
model_arch="Spark2_5ForCausalLM",
hf_architectures=("Spark2_5ForCausalLM",),
stages=(
StagePipelineConfig(
stage_id=0,
model_stage="spark2_5",
execution_type=StageExecutionType.LLM_AR,
input_sources=(),
final_output=True,
final_output_type="text",
owns_tokenizer=True,
engine_output_type="text",
model_arch="Spark2_5ForCausalLM",
),
),
)
@@ -494,9 +494,12 @@ class Spark2_5ForCausalLM(nn.Module, SupportsLoRA, SupportsPP):
inputs_embeds: torch.Tensor | None = None,
**kwargs,
) -> torch.Tensor | IntermediateTensors:
return self.model(
input_ids, positions, intermediate_tensors, inputs_embeds, **kwargs
)
# Omni's AR model runner hands every stage model ``sampling_metadata``,
# ``logits_index`` and ``sampler``; the omni-native models take them in
# ``**kwargs``. Spark samples through vLLM's own sampler, and the
# decoder stack reads none of them, so they stop here rather than
# widening ``Spark2_5Model.forward`` to a signature it ignores.
return self.model(input_ids, positions, intermediate_tensors, inputs_embeds)
def compute_logits(self, hidden_states: torch.Tensor) -> torch.Tensor | None:
return self.logits_processor(self.lm_head, hidden_states)