[Model] Optimize Qwen3-Omni MRv2 handoff, code prediction and audio decoding (#8223)
Signed-off-by: Sy03 <1370724210@qq.com>
This commit is contained in:
35 files changed
+2363
-161
No files matched your search
@@ -116,4 +116,5 @@ The pre-#5137 pages are preserved in the
|
||||
[legacy module archive](module/archive/README.md) for historical reference and
|
||||
are not active design contracts.
|
||||
|
||||
- [Qwen3-Omni MRv2 performance](qwen3_omni_mrv2_performance.md)
|
||||
- [MiniCPM-o 4.5 turn-mode MRv2 performance](minicpm_o45_mrv2_performance.md)
|
||||
@@ -0,0 +1,22 @@
|
||||
# Qwen3-Omni MRv2 performance
|
||||
|
||||
This model adaptation builds on #8184's existing stage runtime and output
|
||||
contracts. It adds Talker CPU metadata parsing, sampled-token embedding
|
||||
handoff, an optional first-frame decoder, length-grouped Code2Wav graph
|
||||
buckets, and opt-in incremental short-KV code prediction. The H200 profile
|
||||
also enables conservative fused sampling and bounded cuDNN algorithm search.
|
||||
The shared predictor keeps the upstream re-prefill helper and its default
|
||||
sampling behavior; short KV is explicitly selected by the deployment profile.
|
||||
|
||||
Use `vllm_omni/deploy/qwen3_omni_moe_mrv2_h200.yaml` for the experimental H200
|
||||
profile. Preserve a control with `code_predictor_fused_sampling=false` and
|
||||
`codec_cudnn_benchmark=false` when C64 P99 is the primary requirement.
|
||||
cuDNN search restores process settings on exit and has startup cost. Kernel
|
||||
arithmetic is not bitwise-equivalent; the first-frame decoder uses additional
|
||||
model memory. Full-duplex performance is not established by turn-mode tests.
|
||||
|
||||
First-frame delivery requires `talker_first_audio: true` in connector extra,
|
||||
a one-frame initial codec chunk, CUDA, a local single-rank Talker and prefix caching disabled.
|
||||
Other deployments retain the regular Code2Wav path.
|
||||
`codec_fused_snake` explicitly opts CUDA profiles into fused decoder blocks;
|
||||
the base V1 profile retains its original blocks and indexed text cache.
|
||||
Reference in new issue
Block a user