[Model] Add BAGEL step execution and continuous batching (#6359)
This commit is contained in:
1 parent
fda3af8bbc
commit
e43c787544
24 files changed
+1965
-108
No files matched your search
@@ -150,8 +150,17 @@ that do not belong in the shared contract should be stored in `state.extra`.
|
||||
Queueing and lifecycle metadata remain in the scheduler's request state.
|
||||
|
||||
Current native pipelines that explicitly enable step execution include
|
||||
Qwen-Image, HunyuanImage3, and Helios. Step execution alone does not imply
|
||||
continuous-batching support: Qwen-Image accepts batched step states.
|
||||
Qwen-Image, BAGEL, HunyuanImage3, and Helios. Step execution alone does not
|
||||
imply continuous-batching support: Qwen-Image accepts batched step states.
|
||||
BAGEL accepts batched step states for image generation with its default,
|
||||
think-mode, and single-stage deploy configs. Only the diffusion stage uses the
|
||||
step protocol in two-stage deployments; explicit single-stage text output
|
||||
retains complete-request execution. BAGEL schedules exactly
|
||||
`num_inference_steps` denoising updates, including a one-step request. When a
|
||||
finished stepwise request is followed by an explicit text fallback wave, the
|
||||
runner retires its step state and paged diffusion-KV rows before the full
|
||||
forward dispatch. BAGEL's step path does not currently support sequence
|
||||
parallelism or a diffusion cache backend.
|
||||
HunyuanImage3 accepts batched step states only when its resolved self-attention
|
||||
backend is `TORCH_SDPA`; otherwise it rejects groups larger than one request.
|
||||
Configure `DIFFUSION_ATTENTION_BACKEND=TORCH_SDPA` or
|
||||
|
||||
Reference in new issue
Block a user