anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

Int8 weight-only compressed-tensors DiT artifact + serve-path fixes (issue 5) #6

Open
anodyine wants to merge 8 commits from ISSUE-5 into main
pull from: ISSUE-5
merge into: anodyine:main
Conversation 0 Commits 8 Files Changed 28
+4402 -12
8 Commits
Author SHA1 Message Date
admin e37c5aed40 probe(evidence): round two - W8A8 int8-dynamic passes the structure gate at BF16-parity PSNR; verdict flips to YES; upstream Marlin report draft staged
Build Wheel / build (3.12) (pull_request) Failing after 36s
Details
Build Wheel / build (3.11) (pull_request) Failing after 49s
Details
pre-commit / pre-commit (pull_request) Failing after 44s
Details
2026-10-08 09:06:49 -04:00
admin 704b8e29ae probe(round two): W8A8 int8-dynamic scheme — kernel isolation, raw-op probe, artifact builder support
pre-commit / pre-commit (pull_request) Failing after 53s
Details
Build Wheel / build (3.11) (pull_request) Failing after 42s
Details
Build Wheel / build (3.12) (pull_request) Failing after 42s
Details
2026-10-08 08:58:45 -04:00
admin bf65a41ceb probe(evidence): int8-wo capacity matrix, structure gate, PSNR, Marlin 8-bit NaN isolation (issue 5)
Build Wheel / build (3.11) (pull_request) Failing after 42s
Details
Build Wheel / build (3.12) (pull_request) Failing after 42s
Details
pre-commit / pre-commit (pull_request) Failing after 37s
Details
2026-10-08 08:00:51 -04:00
admin 74b596cd05 probe: artifact round-trip verifier (unpack+dequant vs sealed BF16) 2026-10-08 07:45:20 -04:00
admin 67e177860a probe: track issue-5 fixtures.json (ignore exception, matches issue-2 kit) 2026-10-08 07:39:55 -04:00
admin 321e8bb47f fix(quant): load offline-quantized compressed-tensors checkpoints directly on CPU under offload 
The diffusion loader treated any quant config without the MXFP8/
is_checkpoint_quantized markers as online quantization: it loaded every
component on the accelerator before offloading, so serving the int8
weight-only compressed-tensors artifact OOM'd at model construction
(encoder 17 GiB + VAE + packed DiT skeleton > 24 GiB). compressed-tensors
is always a serialized format with no on-load quantization pass — detect
it (through component wrappers too) and load on the host, round-tripping
each quantized module through the accelerator only for the post-load
Marlin repack, which has no CPU path.
 2026-10-08 07:36:01 -04:00
admin e778d21f91 probe: int8 weight-only compressed-tensors DiT artifact kit (issue 5) 
quantize_dit_int8wo.py builds the offline W8A16 artifact (per-channel
symmetric int8, pack-quantized via compressed-tensors' own packer,
per-component config wrapper so encoder/VAE stay BF16, digest manifest);
serve.sh/probe.py/gate.py/psnr.py adapt the issue-2 kit for the
structure-gated capacity matrix (1-4 refs x seeds 2301-2304).
 2026-10-08 07:01:55 -04:00
admin 2f8363c5fc fix(quant): resolve hyphenated registry methods and disk-dict from_config in quant factory 
_normalize_method_name canonicalizes dashes to underscores for _OVERRIDES
lookup, but vLLM's QUANTIZATION_METHODS keeps historical spellings
('compressed-tensors'), so both --quantization compressed-tensors and
checkpoint auto-detection raised 'Unknown quantization method'. Resolve
the normalized name back to the registry spelling, and fall back to
from_config for registry configs whose __init__ does not take the
serialized schema (compressed-tensors config_groups/format/ignore).
 2026-10-08 07:00:05 -04:00