[Docs] update documents (#2921)
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
This commit is contained in:
1 parent
78f237e603
commit
d435fe0707
6 files changed
+14
-14
No files matched your search
+1
-1
@@ -107,7 +107,7 @@ nav:
|
||||
- design/feature/hsdp.md
|
||||
- design/feature/cache_dit.md
|
||||
- design/feature/teacache.md
|
||||
- design/feature/async_chunk_design.md
|
||||
- design/feature/async_chunk.md
|
||||
- design/feature/vae_parallel.md
|
||||
- design/feature/diffusion_step_execution.md
|
||||
- Module Design:
|
||||
|
||||
@@ -408,18 +408,17 @@ Understanding the data structures is crucial for implementing stage transitions:
|
||||
|
||||
**Input to your function:**
|
||||
- `stage_list[source_stage_id].engine_outputs`: List of `EngineCoreOutput` objects
|
||||
- Each contains `outputs`: List of `RequestOutput` objects
|
||||
- Each `RequestOutput` has:
|
||||
- `token_ids`: Generated token IDs
|
||||
- `multimodal_output`: Dict with keys like `"code_predictor_codes"`, etc.
|
||||
- These are the hidden states or intermediate outputs from the model's forward pass
|
||||
- `prompt_token_ids`: Original prompt token IDs
|
||||
- - Each contains `outputs`: List of `RequestOutput` objects
|
||||
- Each `RequestOutput` has:
|
||||
- - - `token_ids`: Generated token IDs
|
||||
- `multimodal_output`: Dict with keys like `"code_predictor_codes"`, etc.These are the hidden states or intermediate outputs from the model's forward pass
|
||||
- `prompt_token_ids`: Original prompt token IDs
|
||||
|
||||
**Output from your function:**
|
||||
- Must return `list[OmniTokensPrompt]` where each `OmniTokensPrompt` contains:
|
||||
- `prompt_token_ids`: List[int] - Token IDs for the next stage
|
||||
- `additional_information`: Dict[str, Any] - Optional metadata (e.g., embeddings, hidden states)
|
||||
- `multi_modal_data`: Optional multimodal data if needed
|
||||
- - `prompt_token_ids`: List[int] - Token IDs for the next stage
|
||||
- `additional_information`: Dict[str, Any] - Optional metadata (e.g., embeddings, hidden states)
|
||||
- `multi_modal_data`: Optional multimodal data if needed
|
||||
|
||||
### How Model Outputs Are Stored
|
||||
|
||||
|
||||
@@ -28,7 +28,7 @@ and can be placed on different devices. Qwen3-TTS has two stages:
|
||||
|
||||
Each stage is a separate model class configured independently via YAML. The two stages
|
||||
are connected by the `async_chunk` framework, which enables inter-stage streaming for
|
||||
low first-packet latency (see [Async Chunk Design](../../design/feature/async_chunk_design.md)).
|
||||
low first-packet latency (see [Async Chunk Design](../../design/feature/async_chunk.md)).
|
||||
|
||||
### Without async_chunk (batch mode)
|
||||
|
||||
@@ -591,5 +591,5 @@ Adding a TTS model to vLLM-Omni involves:
|
||||
For more information, see:
|
||||
|
||||
- [Architecture Overview](../../design/architecture_overview.md)
|
||||
- [Async Chunk Design](../../design/feature/async_chunk_design.md)
|
||||
- [Async Chunk Design](../../design/feature/async_chunk.md)
|
||||
- [Stage Configuration Guide](../../configuration/stage_configs.md)
|
||||
@@ -1,4 +1,4 @@
|
||||
# Async Chunk Design
|
||||
# Async Chunk
|
||||
|
||||
## Table of Contents
|
||||
|
||||
@@ -88,8 +88,9 @@ The following diagram illustrates the **Async Chunk Architecture** for multi-sta
|
||||
</p>
|
||||
|
||||
**Diagram Legend:**
|
||||
|
||||
| Step | Stage Type | Description |
|
||||
|:------:|:-----------:|:------------|
|
||||
|------|-----------|------------|
|
||||
| `prefill` | Initialization | Context processing, KV cache initialization |
|
||||
| `decode` | Autoregressive | Token-by-token generation in AR stages |
|
||||
| `codes` | Audio Encoding | RVQ codec codes from Talker stage |
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 134 KiB After Width: | Height: | Size: 226 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 127 KiB After Width: | Height: | Size: 152 KiB |
Reference in new issue
Block a user