[Bugfix] Fix colocate-async sleep admission race and deliver AR abort tokens (#6367)
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com> Signed-off-by: Cursor <cursoragent@cursor.com> Signed-off-by: Samit <285365963@qq.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Samit <285365963@qq.com>
This commit is contained in:
23 files changed
+1229
-157
No files matched your search
@@ -43,6 +43,7 @@ depends_on:
|
||||
validation_paths:
|
||||
- tests/entrypoints/test_omni_entrypoints.py
|
||||
- tests/entrypoints/test_async_omni.py
|
||||
- tests/entrypoints/test_async_omni_pause_sleep_routing.py
|
||||
- tests/entrypoints/test_async_omni_duplex.py
|
||||
- tests/entrypoints/test_serve.py
|
||||
- tests/entrypoints/test_stream_finish_reason.py
|
||||
@@ -119,7 +120,13 @@ promotion.
|
||||
|
||||
Test request validation, protocol conversion, model-adapter routing,
|
||||
streaming, session identity, disconnect/cancellation, and error mapping for
|
||||
each affected offline or serving entrypoint.
|
||||
each affected offline or serving entrypoint. Sleep must wait for in-flight
|
||||
`generate()` admission before EngineCore offload; `wake_up` does not resume
|
||||
admission — callers must `resume_generation()`. Sleeping tags are tracked per
|
||||
stage so `wake_up(stage_ids=[0])` does not skip a later `wake_up(stage_ids=[1])`.
|
||||
Streaming input pumps take an admission slot immediately before each EngineCore
|
||||
ADD or update, not while waiting for the next client chunk. Frontend abort
|
||||
keeps `request_states` until `generate()` consumes the terminal output.
|
||||
|
||||
## Promotion gate
|
||||
|
||||
|
||||
Reference in new issue
Block a user