tzhouam and Claude Fable 5.1
33de5d8629
Merge vLLM-Omni v0.29.0rc1 into edge-infer/omni-0.29.0
...
The pair for vLLM v0.29.0: rc1's docs pin vllm==0.29.0 and a static scan of
rc1's vllm.* imports against the v0.29.0 tree finds nothing missing, while
our 0.28-based tree needs seven modules v0.29.0 no longer has
(vllm.inputs.preprocess, entrypoints.launcher, openai.cli_args,
openai.engine.protocol, serve.utils.server_utils, layers.batch_invariant,
models.hunyuan_v1).
Ten files conflicted. Upstream's structure is the base everywhere; the
fork's features are re-attached to it:
- deploy_profile / apply_deploy_profile run before rc1's resolve_omni_config
pops deploy_config (async_omni_engine, cli/serve); the deploy-profile
selector and the resolve_model_config_path chain it needs stay in
entrypoints/utils.py; stage_overrides takes rc1's dict type.
- config_factory keeps the lazy diffusion output-type import; the diffusion
registry import in rc1's new config/resolver.py is made lazy for the same
WP6 reason (TTS path must not load diffusion).
- requirements: our core/media split kept; rc1's transformers>=5.13 floor
and kernels==0.16.1 applied in core.txt / media.txt.
- TTS: rc1's split_granularity and bounded seed plus our output_mode=
codec_tokens, _SessionInbox cancel handling and codec DELTA accumulation;
our private _conditioning_cache_salt is superseded by rc1's public
tts_adapters/base.py::conditioning_cache_salt.
- stage_runtime: rc1's single ExitStack G2 path (lock released before READY)
replaces our two-path block, with the four worker_phase names the init
profiler consumes still emitted; _cpu_placed / host admission kept.
Not yet run against vLLM 0.29.0: the tests listed in the W6 record.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-18 09:57:20 +08:00
Zhou Taichang and Claude Opus 5
aff7d64948
[Rebase] Rebase to vLLM 0.29.0 ( #7230 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-09-10 12:08:28 +08:00
zyz111222
6e79d582e3
[Bugfix][Ascend] fix npu 310/a5 bugs ( #6685 )
...
Signed-off-by: zouyizhou <zouyizhou@huawei.com >
2026-09-09 19:03:09 +08:00
tzhouam and Claude Fable 5.1
35e43b9983
[Edge][WP3] Add CPU OmniPlatform (AR + generation stages on vLLM's CPU backend)
...
* vllm_omni/platforms/cpu/platform.py: CPUOmniPlatform(OmniPlatform, CpuPlatform)
selected when VLLM_TARGET_DEVICE=cpu or a +cpu vLLM wheel is installed
(builtin plugin "cpu"; explicit CPU target short-circuits accelerator probes
like vLLM does). NUMA-based memory queries, one logical device, no graph
wrapper (sentinel type for isinstance checks), inductor opt-in via
VLLM_OMNI_CPU_INDUCTOR=1, diffusion stages unsupported.
* vllm_omni/platforms/cpu/worker: CPUARWorker / CPUGenerationWorker compose
vLLM's CPUWorker with the Omni mixin (XPU pattern); CPUARModelRunner /
CPUGenerationModelRunner compose the Omni runners with CPUModelRunner
(no graphs, no async omni output; sleep/wake return error ACKs).
* patch.py skips the CuMem patch on CPU-only torch; Qwen3-TTS speaker encoder
and prompt-embeds dtype follow the model dtype (float32/float16 on CPU).
* vllm_omni/deploy/qwen3_tts.yaml: `platforms: cpu:` overlay (eager, no async
scheduling, absolute kv_cache_memory_bytes, disjoint OMP thread ranges).
* requirements/cpu.txt (+psutil, intel-openmp), docs/getting_started/installation/cpu.md.
Measured (this x86 Xeon 8480C host, vllm 0.28.0+cpu, bf16, 12+4 threads):
Qwen3-TTS 0.6B CustomVoice end-to-end on CPU, 12 prompts: TTFA p50 292 ms,
RTF ~1.7 (25-frame chunk ~4.6 s), init 185 s. Result JSON + init timeline
committed under benchmarks/tts/edge_results/.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-09 11:02:14 +08:00
fe4a2a7477
[Diffusion] Add paged KV cache support for HunyuanImage3 DiT on NPU and GPU ( #6563 )
...
Signed-off-by: Acerak01-fy <wfy2003324@163.com >
Signed-off-by: zwhzzz0821 <zwhzzz0821@users.noreply.github.com >
Signed-off-by: zwhzzz0821 <2831474076@qq.com >
Co-authored-by: zwhzzz0821 <zwhzzz0821@users.noreply.github.com >
Co-authored-by: zwhzzz0821 <2831474076@qq.com >
2026-08-31 10:15:46 +00:00
19da238246
[Bugfix][NPU] Eager-import mindiesd for every diffusion attention backend ( #6054 )
...
Signed-off-by: brandneway <gyuan4892@gmail.com >
Co-authored-by: changetheway <guotaoyuan1@h-partners.com >
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
2026-08-27 14:41:36 +08:00
NATURE
702a409664
[Feature][MiniCPM-o] Support configurable concurrent duplex sessions ( #6021 )
...
Signed-off-by: natureofnature <wzliu@connect.hku.hk >
2026-08-25 02:27:30 +00:00
NATURE and amy-why-3459
4f7c08c0eb
[Bugfix][MiniCPM-o] Fix async-chunk snapshot replacement and prompt cleanup ( #6406 )
...
Signed-off-by: natureofnature <wzliu@connect.hku.hk >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-08-21 16:37:21 +08:00
37c11e6e4f
[Perf][Qwen3-TTS] Fuse QKV and gate_up projections in code predictor ( #5791 )
...
Signed-off-by: sheenlin <sheenlin@tencent.com >
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk >
Co-authored-by: sheenlin <sheenlin@tencent.com >
Co-authored-by: yurain <yurain26@users.noreply.github.com >
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk >
2026-08-20 19:22:40 +00:00
Zhou Taichang and Claude Fable 5
10906e3a6d
[Refactor] Worker/ModelRunner runner correctness fixes (G2/N) ( #5452 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-08-20 16:47:50 +08:00
Thanaji Rao Thakkalapelli and Chendi.Xue
efaf4276f9
[Bugfix] Record device-agnostic torch.Event on XPU to fix async D2H image corruption [XPU] ( #5571 )
...
Signed-off-by: Thanaji Rao Thakkalapelli <thanaji.rao.thakkalapelli@intel.com >
Co-authored-by: Chendi.Xue <chendi.xue@intel.com >
2026-08-19 16:30:38 -05:00
Zhou248
5d09cf27a9
[MiniCPM-o][NPU] Enable MiniCPM-o Code2Wav NPUGraph replay ( #5604 )
...
Signed-off-by: Zhou248 <62542847+Zhou248@users.noreply.github.com >
2026-08-17 15:09:51 +08:00
GXIN
6a604be610
[Bugfix][NPU] Fallback to BSND RoPE when Qwen3-TTS short-seq BNSD li… ( #5608 )
...
Signed-off-by: XIN GAO <1037396230@qq.com >
2026-08-05 13:02:47 +08:00
Gao Han
78c144f3a8
[CI failed] Revert "[Perf][Qwen3-TTS] Fuse QKV and gate_up projections in code predictor" ( #5777 )
2026-08-04 15:21:38 +00:00
6fdc4ca62d
[Perf][Qwen3-TTS] Fuse QKV and gate_up projections in code predictor ( #4958 )
...
Signed-off-by: sheenlin <sheenlin@tencent.com >
Co-authored-by: sheenlin <sheenlin@tencent.com >
Co-authored-by: yurain <yurain26@users.noreply.github.com >
2026-08-04 11:59:32 +00:00
zzh and dengyunyang
dba05c3517
Enable Diffusion Image models asynchronous output ( #4981 )
...
Signed-off-by: zzh <943967662@qq.com >
Co-authored-by: dengyunyang <584797741@qq.com >
2026-07-29 17:21:55 +00:00
fatSheep and amy-why-3459
5d44868e91
[Hardware][Ascend] Keep HiFT vocoder on NPU ( #5242 )
...
Signed-off-by: fatSheep <tzh2005t@163.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-07-21 14:30:30 +08:00
zyz111222 and amy-why-3459
2d58070f60
[Hardware][Ascend][Model] Optimize Qwen3-TTS on 310P ( #4841 )
...
Signed-off-by: zouyizhou <zouyizhou@huawei.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-07-15 08:57:44 +08:00
Chendi.Xue
161b005cfc
[XPU][CI] Fix xpu after rebasing for 0.25.0 ( #5066 )
...
Signed-off-by: Chendi Xue <chendi.xue@intel.com >
2026-07-13 15:34:27 +08:00
NumberWan
fe478a95ab
[BugFix] Restore vllm_c IR op priority and torch.nn.RMSNorm for Qwen-Image ( #4964 ) ( #5009 )
...
Signed-off-by: NumberWan <wantszkin2003@gmail.com >
2026-07-10 17:39:07 +00:00
zyz111222 and Canlin Guo
6de0ca36bb
[Hardware][Ascend] Adapt Qwen3 TTS for 310P ( #4283 )
...
Signed-off-by: zouyizhou <zouyizhou@huawei.com >
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com >
2026-06-12 09:39:21 +00:00
56ca7dd612
support online FP8 quantization for FA on NPU #2236 ( #2640 )
...
Signed-off-by: lyj-jjj <liuyingjun5@huawei.com >
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: gcanlin <canlinguosdu@gmail.com >
2026-05-13 11:46:00 +08:00
