[Edge][Model] Spark-X2.5 (Spark2_5ForCausalLM) support

Spark-X2.5-1.7B is an Apache-2.0 on-device model built for agentic tool
use. vLLM 0.28 has no implementation for it, so it could not be served at
all; llama.cpp already ships one, which made it the only edge engine able
to run the model.

Three traits separate it from a Llama-style decoder:

* Hybrid attention -- layer_types interleaves three 512-token sliding
  layers with one full-attention layer, which is what makes the 1M-token
  context affordable on a phone.
* Per-layer-type RoPE -- sliding layers rotate all 256 head dims with
  theta 1e4, full layers only the first quarter with theta 5e6.
* A head-wise attention output gate -- g_proj reads the same post-norm
  input as QKV and scales each head's attention output by a sigmoid.

The checkpoint also stores QKV pre-fused as a single q_k_v_proj, so
load_weights splits it before vLLM's sharded QKVParallelLinear sees it.

Verified against the official remote-code implementation on transformers
4.57.1 (the version config.json targets; it does not import on the 5.x in
our venvs). In fp32 all three scenarios match exactly -- 32/32 greedy
tokens with a max top-20 logprob difference of 2.2e-05. In bf16 the short
and the 2465-token long-context prompts still match 32/32, which is what
exercises the sliding layers; the tool-calling prompt diverges at token 23
on a near-tie, and fp32 confirms that is rounding rather than a defect.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L1ax5wYLA6XbqEL1i8L9gZ
This commit is contained in:
tzhouamandClaude Opus 5 committed 2026-09-11 13:47:02 +08:00
1 parent 73cdc7395a
commit e4a565b3df
7 files changed
+678

No files matched your search

+4
View File
@@ -75,6 +75,9 @@ def _register_omni_hf_configs() -> None:
from vllm_omni.model_executor.models.qwen3_tts.configuration_qwen3_tts import (
Qwen3TTSConfig,
)
from vllm_omni.model_executor.models.spark2_5.configuration_spark2_5 import (
Spark2_5Config,
)
from vllm_omni.transformers_utils.configs.cosyvoice3 import CosyVoice3Config
from vllm_omni.transformers_utils.configs.glm_tts import GLMTTSConfig
from vllm_omni.transformers_utils.configs.omnivoice import OmniVoiceConfig
@@ -99,6 +102,7 @@ def _register_omni_hf_configs() -> None:
("moss_tts_local", MossTTSLocalConfig),
("moss_tts_realtime", MossTTSRealtimeConfig),
("qwen3_tts", Qwen3TTSConfig),
("spark2_5", Spark2_5Config),
("personaplex", PersonaPlexConfig),
("nemotron_voicechat", NemotronVoiceChatConfig),
("cosyvoice3", CosyVoice3Config),