[Model][Core] Avoid unused CosyVoice3 payloads and redundant AR output work (#8473)

Signed-off-by: Sorbett <sorbet111@163.com>
This commit is contained in:
Scott Ye authored and GitHub committed 2026-10-05 04:52:19 +00:00
1 parent 5ca23106fd
commit 1f47fa2f29
9 files changed
+525 -73

No files matched your search

+23
View File
@@ -47,6 +47,29 @@ adapter, not injected through unrelated scheduler state.
**Rule:** Workers and model runners MUST NOT implement cross-stage routing.
## Token-only output payloads
An AR stage whose downstream decoder consumes sampled tokens rather than
hidden states can set `omni_pooler_payload_include_hidden = False`. Hidden
states remain available for logits and sampling; prefill conditioning and
other required multimodal outputs still cross the stage boundary.
A chunk processor that needs IDs on steps without a tensor payload declares
`requires_token_updates = True`. The scheduler then enqueues nonempty sampled
IDs while preserving final-stage exclusions and the processor's EOF handling.
Processors without this declaration retain their tensor-payload trigger.
When hidden and multimodal payloads are absent, the shared runner skips empty
per-request payload construction. Prefix-cache merges and pending model
postprocess retain the normal path. This policy is independent of whether
output materialization runs inline or in the background.
Inline construction consumes query offsets, scheduled-token counts and
scheduler metadata before the next execution can reuse them. Background
construction snapshots those inputs. Both paths reuse the request IDs already
copied by upstream bookkeeping; connector collection stays on the main thread
because it can run a tensor-parallel collective.
## Safe-change guide
Test request lifecycle, abort, cache state, and every affected worker execution