The pair for vLLM v0.29.0: rc1's docs pin vllm==0.29.0 and a static scan of
rc1's vllm.* imports against the v0.29.0 tree finds nothing missing, while
our 0.28-based tree needs seven modules v0.29.0 no longer has
(vllm.inputs.preprocess, entrypoints.launcher, openai.cli_args,
openai.engine.protocol, serve.utils.server_utils, layers.batch_invariant,
models.hunyuan_v1).
Ten files conflicted. Upstream's structure is the base everywhere; the
fork's features are re-attached to it:
- deploy_profile / apply_deploy_profile run before rc1's resolve_omni_config
pops deploy_config (async_omni_engine, cli/serve); the deploy-profile
selector and the resolve_model_config_path chain it needs stay in
entrypoints/utils.py; stage_overrides takes rc1's dict type.
- config_factory keeps the lazy diffusion output-type import; the diffusion
registry import in rc1's new config/resolver.py is made lazy for the same
WP6 reason (TTS path must not load diffusion).
- requirements: our core/media split kept; rc1's transformers>=5.13 floor
and kernels==0.16.1 applied in core.txt / media.txt.
- TTS: rc1's split_granularity and bounded seed plus our output_mode=
codec_tokens, _SessionInbox cancel handling and codec DELTA accumulation;
our private _conditioning_cache_salt is superseded by rc1's public
tts_adapters/base.py::conditioning_cache_salt.
- stage_runtime: rc1's single ExitStack G2 path (lock released before READY)
replaces our two-path block, with the four worker_phase names the init
profiler consumes still emitted; _cpu_placed / host admission kept.
Not yet run against vLLM 0.29.0: the tests listed in the W6 record.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Verbatim copy of the edge/omni-edge working tree (tracked diffs + untracked
edge/local, tests/edge/local, docs/edge, pipeline files). Taken so that
vLLM-Omni v0.29.0rc1 can be merged on a branch without touching the
user's working tree. logs_edge/ and benchmark .npy outputs excluded.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Renamed from vllm-embedding: nothing here is about embeddings, and the name had
come from the parent working directory. edge-infer says what it does. The
GitHub repository is renamed to match; nothing had been pushed yet.
The audit found the packaging was incomplete in two ways.
Scripts: 56 of them produced results that are published in this repo but were
not included, so the numbers could be read and not checked. Added under
benchmarks/edge_harness/experiments/ (one-off drivers: the AI Hub on-device
runs, the 4-bit GEMM microbenchmark, fidelity and parity checks, checkpoint
build and round-trip, activation capture, and the load generator used to prove
the contention guard actually refuses) and benchmarks/edge_harness/runs/ (the
24 shell drivers behind published grids). Both directories carry a README
saying plainly that they are a record of what was run rather than a supported
interface -- most hardcode absolute paths from the machine they ran on. The
supported entry points remain the tools one level up.
One broken relative link fixed (comparison.md is under docs/engines/, not
docs/analysis/). All other links and every script reference in the docs now
resolve inside the repo.
Secret scan: the token in ~/.qai_hub/client.ini is absent from the worktree and
from all 61 commits. Worth noting that an earlier version of my own check
reported a false positive, because `grep -rl ... | head` takes its exit status
from head, which always succeeds -- the same class of mistake as the measurement
errors this project keeps recording. The AI Hub scripts authenticate by reading
that file at runtime and contain no credential.
README reframed for the new name and the wider scope: the repository is the
engine changes, the harness, the raw results, the baselines and the analyses,
not just a patch set.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L1ax5wYLA6XbqEL1i8L9gZ
The previous commit said "vLLM itself is not modified" and described the
changes as going "through vLLM's own extension points". Challenged on it, and
the challenge was right: those are two different claims and only one of them
holds as written.
What is true: no file in the vLLM tree or in site-packages/vllm is edited, the
checkout is clean upstream and the wheel is stock 0.28.0+cpu, so there is no
patch series to apply and no vLLM fork to maintain.
What was wrong:
1. vllm_omni/patch.py does nine _patch_* functions performing ten direct
attribute replacements -- CpuPlatform.check_and_update_config,
CPUWorker.load_model, torch.compiler.load_compiled_function. That is
monkey-patching, not an extension point, and calling it one made the
arrangement sound more robust than it is. Only the registries are genuine
hooks: register_quantization_config, ir.ops.*.register_impl, platform
plugins, direct_register_custom_op.
2. "The changes survive a wheel upgrade rather than being lost on it" is true of
the registrations and overstated for the patches. Each patch is keyed to a
specific method on a specific vLLM class; an upstream rename or behaviour
change can break it or silently turn it into a no-op wrapper. The failure
mode is a missing optimisation, not a crash, which is worse -- so the doc now
says to re-run the harness after upgrading.
The section is rewritten to separate the two mechanisms, list what each patch
works around, and note that two of the three exist for upstream bugs and should
disappear when those are fixed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L1ax5wYLA6XbqEL1i8L9gZ
tzhouam