edge-infer
Copyright 2026 tzhouam

This product is a derivative work of vLLM-Omni
(https://github.com/vllm-project/vllm-omni), licensed under the Apache License,
Version 2.0. Upstream code is Copyright the vLLM-Omni contributors, and vLLM-Omni
is itself a derivative of vLLM (https://github.com/vllm-project/vllm).

Modifications in this fork are confined to edge/CPU inference and are listed in
README.md; the substantive ones are:

  vllm_omni/model_executor/layers/quantization/cpu_int4.py
      4-bit weight-only quantization for the CPU backend
  vllm_omni/model_executor/models/spark2_5/spark2_5.py
      RoPE table sized to max_model_len rather than the advertised context
  vllm_omni/edge/
      device-class weight-path selection, KV budget sizing, memory release
  vllm_omni/platforms/cpu/, vllm_omni/patch.py
      CPU platform and CPU-backend patches
  benchmarks/edge_harness/
      measurement tooling and results (original to this fork)
  baselines/, docs/engines/, docs/analysis/
      benchmark results and written analyses (original to this fork)

vLLM itself is not modified and no vLLM source is redistributed here. This fork
changes vLLM's CPU behaviour through its own extension points at runtime
(vllm_omni/patch.py); the pinned upstream versions are recorded in
docs/edge/environment.md.

Baseline measurements reference llama.cpp (https://github.com/ggml-org/llama.cpp,
MIT). No llama.cpp source is redistributed; only benchmark output produced with
it, under baselines/llama_cpp/.

Upstream's README is preserved at docs/README.vllm-omni.md.