618 MB of resident memory and 649 MB of peak, for one line -- the largest
single memory item found in this study, and unlike the others it costs nothing.
Spark advertises a 1,048,576-token context. This file passed
max_position=config.max_position_embeddings straight into get_rope, and vLLM
builds its cos/sin table with torch.arange(max_position), so a 2048-token
deployment carried tables for a million positions. Because the sliding and full
layers are trained with different theta and different partial rotary factors
there are two of them:
layers.0 (full, head_dim 256) [1048576, 256] 512 MB
layers.3 (sliding, partial 0.25) [1048576, 64] 128 MB
Both fully resident -- confirmed with mincore(2), not inferred. At
max_model_len 2048 they need 1.25 MB between them.
Found by memory_ledger.py, which attributes memory instead of subtracting
toward a remainder: 640 MB had been sitting in an unnamed "other" bucket across
114 tensors, and naming the largest storages showed what it was. After the cap
that bucket is 1.3 MB and unique tensor storage falls 638.8 MB, which agrees
independently with the end-to-end measurement.
vLLM's scheduler cannot produce a position beyond max_model_len, so covering
that is sufficient. Measured with VLLM_OMNI_SPARK_ROPE_FULL toggling the two in
identical code, 3 interleaved passes per arm on node-local cores:
1M tables settled 3891 MB peak 4019 MB init 13.5 s
capped settled 3273 MB peak 3370 MB init 13.7 s
The escape hatch restores the old behaviour and is covered by a test, as is the
cap itself.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
