Merge main and preserve multi-API engine and duplex guards

Signed-off-by: Sy03 <1370724210@qq.com>
This commit is contained in:
Sy03 committed 2026-09-16 16:40:16 +08:00
commit daf91b4d7a
329 files changed
+36428 -35780

No files matched your search

@@ -76,6 +76,34 @@ preserve common `self` or `cross` policies. A model also decides whether its
path is compatible with automatic TRTLLM selection, because only the model
knows whether masking and packing satisfy the kernel contract.
### Model-owned implementation specializations
Use `Attention(..., impl_overrides={backend_name: implementation_class})`
when a model needs different computation within an existing backend contract.
The model defines the implementation in its own directory and passes it when
constructing the layer:
```python
# MyModelVSAImpl is a model-owned subclass of FastVideoVSAImpl.
self.attention = Attention(
num_heads=num_heads,
head_size=head_size,
softmax_scale=head_size**-0.5,
causal=False,
role="self",
impl_overrides={"FASTVIDEO_VSA": MyModelVSAImpl},
)
```
The override is applied after role/platform selection, keyed by the selected
backend's `get_name()`. It must subclass the **selected implementation**;
incompatible platform implementations raise `TypeError`. Unselected overrides
have no effect. Backend capabilities, constructor options, and shared
parallel dispatch remain unchanged and must be honored by the specialization.
Use a new backend for a new provider/capability contract. `custom_attention`
instead owns communication and requires `skip_sequence_parallel=True`.
## Adding or changing a backend
An implementation change is complete only when it:
@@ -91,8 +119,9 @@ An implementation change is complete only when it:
## FastVideo VSA model metadata
FastVideo VSA support is scoped to `FastVideo/FastWan2.2-TI2V-5B-Diffusers`, whose text-to-video and image-to-video modes both use `Wan22Pipeline`. It operates on a flattened DiT sequence but partitions tokens in
the original latent video grid. Wan integrations therefore attach the
FastWan's text-to-video and image-to-video modes both use `Wan22Pipeline`.
It operates on a flattened DiT sequence but partitions tokens in the original
latent video grid. Wan integrations therefore attach the
post-patch `(T, H, W)` shape as `vsa_dit_seq_shape` attention metadata. The
backend validates that `T * H * W` equals the sequence length, derives the
runtime block count, and selects the configured top-k key/value blocks for
@@ -104,3 +133,8 @@ the checkpoint contains those weights. Checkpoints without the projection do
not allocate or execute it. When top-k selects every block, native checkpoints
route to SDPA, while FastVideo DMD checkpoints preserve the VSA all-block path
to retain checkpoint semantics.
MiniMax-H3 supplies `MiniMaxH3VSAImpl` through `impl_overrides`.
`models/minimax_h3/attention/fastvideo_h3.py` owns its prefix layout, video tiling, and
learned compression gate. Pooling, block-map construction, and tile64 provider
calls are shared through `attention/ops/block_sparse.py`.
+359 -1056
View File
File diff suppressed because it is too large. Load diff
+9 -2
View File
@@ -16,6 +16,7 @@ required_reviewers:
- "@yenuo26"
- "@NickCao"
primary_code_paths:
- vllm_omni/engine/omni_engine_base.py
- vllm_omni/engine/async_omni_engine.py
- vllm_omni/engine/async_engine_utils.py
- vllm_omni/engine/orchestrator.py
@@ -28,8 +29,12 @@ related_code_paths:
- vllm_omni/engine/stage_pool.py
- vllm_omni/outputs/output_processor.py
- vllm_omni/distributed/omni_coordinator/**
# Duplex kernel design is owned by ../fullduplex.md; listed here because the
# orchestrator hosts the duplex control plane.
# The duplex siblings (DuplexOmniEngine / DuplexOrchestrator) and the
# engine-resident session framework are owned by ../fullduplex.md; listed
# here because they fill the template seams of OmniEngineBase and
# OrchestratorBase.
- vllm_omni/engine/duplex_omni_engine.py
- vllm_omni/engine/duplex_orchestrator.py
- vllm_omni/engine/duplex/**
depends_on:
- input_output_modality_contracts.md
@@ -48,6 +53,8 @@ validation_paths:
- tests/engine/test_cfg_companion_lifecycle.py
- tests/engine/test_rpc_result_router.py
- tests/engine/duplex/**
- tests/engine/test_duplex_orchestrator.py
- tests/engine/test_duplex_omni_engine.py
- tests/engine/test_duplex_import_boundary.py
upstream_refs:
- vllm.v1.engine.EngineCoreRequest
+8 -3
View File
@@ -19,7 +19,9 @@ required_reviewers:
primary_code_paths:
- vllm_omni/entrypoints/omni.py
- vllm_omni/entrypoints/async_omni.py
- vllm_omni/entrypoints/async_omni_base.py
- vllm_omni/entrypoints/omni_base.py
- vllm_omni/entrypoints/duplex_omni.py
- vllm_omni/entrypoints/cli/**
- vllm_omni/entrypoints/openai/**
- vllm_omni/entrypoints/openpi/**
@@ -27,14 +29,13 @@ primary_code_paths:
- vllm_omni/entrypoints/stage_utils.py
- vllm_omni/entrypoints/utils.py
- vllm_omni/entrypoints/duplex/**
- vllm_omni/entrypoints/duplex_request_client.py
- vllm_omni/clients/**
primary_path_exceptions:
- path: vllm_omni/entrypoints/openai/errors.py
owner: error_contracts.md
- path: vllm_omni/entrypoints/duplex/**
owner: ../fullduplex.md
- path: vllm_omni/entrypoints/duplex_request_client.py
- path: vllm_omni/entrypoints/duplex_omni.py
owner: ../fullduplex.md
- path: vllm_omni/clients/**
owner: ../fullduplex.md
@@ -42,7 +43,9 @@ related_code_paths:
- vllm_omni/errors.py
- vllm_omni/inputs/**
- vllm_omni/outputs/**
- vllm_omni/engine/omni_engine_base.py
- vllm_omni/engine/async_omni_engine.py
- vllm_omni/engine/duplex_omni_engine.py
- vllm_omni/config/**
- vllm_omni/deploy/**
depends_on:
@@ -53,7 +56,9 @@ validation_paths:
- tests/entrypoints/test_omni_entrypoints.py
- tests/entrypoints/test_async_omni.py
- tests/entrypoints/test_async_omni_pause_sleep_routing.py
- tests/entrypoints/test_async_omni_duplex.py
- tests/entrypoints/test_duplex_omni.py
- tests/entrypoints/duplex/**
- tests/entrypoints/openai_api/test_duplex_api_server.py
- tests/entrypoints/test_serve.py
- tests/entrypoints/test_stream_finish_reason.py
- tests/entrypoints/openai/**