[Bugfix] Stop cancelled MiniMax-H3 requests at denoising boundaries (#7844)
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
This commit is contained in:
1 parent
b308e196c9
commit
69de153f6e
12 files changed
+702
-13
No files matched your search
@@ -120,7 +120,18 @@ request for execution.
|
||||
`DELETE /v1/videos/{video_id}` issues a bounded engine abort
|
||||
(`VLLM_OMNI_ABORT_TIMEOUT`, default 2s), then cancels the frontend
|
||||
task. Cancellation cleanup is also bounded and best-effort: it confirms
|
||||
the abort was queued, and the current request batch may still drain.
|
||||
the abort was submitted. In request execution mode, MiniMax-H3 checks
|
||||
cancellation at model boundaries,
|
||||
including before and after input preparation, after each denoising step, and
|
||||
before decode, and skips the remaining generation when cancelled. Input
|
||||
preparation or decoding already in progress may finish before the next boundary.
|
||||
Successful steps do not force device synchronization for cancellation.
|
||||
Parallel ranks agree before stopping; independent requests sharing a distributed
|
||||
AllGather offload wave can stop that wave early only when all its requests are
|
||||
cancelled, so a cancelled request cannot strand its live peers in a collective.
|
||||
In step execution mode, cancellation is handled by the scheduler between steps;
|
||||
the request-mode component-boundary checks do not apply within an active step.
|
||||
Other pipelines may still drain their current request batch.
|
||||
The job is then re-read so a completed save is not orphaned.
|
||||
|
||||
### Synchronous Response
|
||||
|
||||
Reference in new issue
Block a user