Merge remote-tracking branch 'origin/main' into qwen-image-2.1
# Conflicts: # tests/entrypoints/openai_api/test_image_server.py # vllm_omni/entrypoints/openai/api_server.py
This commit is contained in:
Isotr0py committed
commit
3e274a0024
220 files changed
+22342
-1079
No files matched your search
@@ -6,29 +6,29 @@
|
||||
ASR -> AURA -> Qwen3-TTS Talker -> Code2Wav
|
||||
```
|
||||
|
||||
Qwen3-TTS remains two engine stages so the pipeline reuses the existing native
|
||||
Talker and Code2Wav implementation.
|
||||
**Supported online path:** Realtime duplex at `/v1/realtime?duplex=1`.
|
||||
|
||||
The default deploy profile (`vllm_omni/deploy/aura_omni.yaml`) sets
|
||||
`session_mode: duplex`. The `aura_omni` pipeline declares `duplex_plugin`, and
|
||||
`DuplexOmniEngine` requires that mode (same pattern as MiniCPM-o duplex
|
||||
deploys). Turn-based `chat/completions` is not a supported AURA online mode
|
||||
against this profile.
|
||||
|
||||
`async_chunk` defaults to `false` in that file (safer if a turn-based
|
||||
`OmniOrchestrator` ever loads it). Duplex Realtime Stage2→3 still needs
|
||||
async chunks — use the smoke deploy below, or set `async_chunk: true`
|
||||
yourself.
|
||||
|
||||
```bash
|
||||
vllm serve aurateam/AURA \
|
||||
--omni \
|
||||
--deploy-config vllm_omni/deploy/aura_omni.yaml \
|
||||
--served-model-name aurateam/AURA \
|
||||
--trust-remote-code
|
||||
bash examples/online_serving/aura_omni/run_duplex_smoke_serve.sh
|
||||
python examples/online_serving/aura_omni/smoke_duplex_realtime_client.py
|
||||
```
|
||||
|
||||
Configure local checkpoints by editing per-stage `model` values in
|
||||
`vllm_omni/deploy/aura_omni.yaml`. The deploy file sets
|
||||
`pipeline: aura_omni`, so the four-stage topology is used even if the
|
||||
command-line `--model` points at one of the component checkpoints.
|
||||
Configure local checkpoints by editing per-stage `model` values in the deploy
|
||||
or smoke YAML. The file sets `pipeline: aura_omni`, so the four-stage topology
|
||||
is used even if the command-line `--model` points at one component checkpoint.
|
||||
|
||||
Send requests with `"model": "aurateam/AURA"`. The ASR, AURA, and Qwen3-TTS
|
||||
checkpoint paths are internal stage models from the deploy YAML, not the
|
||||
OpenAI-facing served model name.
|
||||
|
||||
The AURA stage can emit `<|silent|>`. Silent outputs are treated as a gate:
|
||||
they produce no Qwen3-TTS Talker input, so no audio is synthesized for that
|
||||
turn.
|
||||
Silent Stage1 outputs (`<|silent|>` / id `151669`) skip TTS for that turn.
|
||||
|
||||
## GPU Utilization Recommendation
|
||||
|
||||
@@ -40,26 +40,21 @@ VRAM each stage can reserve. Start with this split for a single GPU:
|
||||
- Stage 2 (Qwen3-TTS Talker): `0.20`
|
||||
- Stage 3 (Qwen3-TTS Code2Wav): `0.20`
|
||||
|
||||
## TTS Modes
|
||||
## TTS modes (stage extras)
|
||||
|
||||
`aura_omni` can pass AURA text to Qwen3-TTS in two task modes:
|
||||
When the duplex session supplies TTS extras, AURA text can feed Qwen3-TTS as:
|
||||
|
||||
- `Base`: voice clone from `tts_ref_audio` with ICL enabled in the AURA
|
||||
pipeline. Provide both `tts_ref_audio` and `tts_ref_text`. Set
|
||||
`tts_x_vector_only_mode=true` to disable ICL and use speaker embedding only.
|
||||
- `CustomVoice`: predefined speaker mode. Use a Qwen3-TTS CustomVoice
|
||||
checkpoint for stages 2 and 3 in `aura_omni.yaml`, then pass
|
||||
`tts_task_type=CustomVoice` and `tts_speaker`.
|
||||
- `Base`: voice clone from `tts_ref_audio` (optional x-vector-only mode)
|
||||
- `CustomVoice`: predefined speaker (`tts_speaker`) with a CustomVoice checkpoint
|
||||
on stages 2 and 3
|
||||
|
||||
By default, AURA responses are passed to Qwen3-TTS as text. Set
|
||||
`additional_information.tts_pass_token_ids=true` to pass AURA-generated
|
||||
assistant token ids directly instead. Even when token passthrough is disabled,
|
||||
the stage processor uses AURA token ids when available to estimate the Talker
|
||||
prefill length, so scheduling does not rely on raw character length.
|
||||
Optional `tts_pass_token_ids` passes AURA assistant token ids into Talker
|
||||
instead of detokenized text.
|
||||
|
||||
The example client exposes this as:
|
||||
## Old chat / Gradio / curl scripts (unsupported)
|
||||
|
||||
```bash
|
||||
python examples/online_serving/aura_omni/openai_chat_completion_client.py \
|
||||
--tts-pass-token-ids
|
||||
```
|
||||
`examples/online_serving/aura_omni/` still contains chat-completions, curl, and
|
||||
Gradio helpers from an older turn-based path. They do **not** work against the
|
||||
shipped duplex profile. That path was an incomplete stand-in before streaming
|
||||
I/O and session history were available; it is not a supported AURA mode. Use
|
||||
the Realtime duplex smoke client for online verification.
|
||||
@@ -247,12 +247,18 @@ curl http://localhost:8091/v1/chat/completions \
|
||||
-d @payload.json
|
||||
```
|
||||
|
||||
BAGEL img2img derives its generated image size from the input image. It
|
||||
preserves the input aspect ratio, aligns dimensions to the latent stride, and
|
||||
applies the checkpoint size limit; explicit `height` and `width` values do not
|
||||
override this model-specific resize policy. In step mode, the effective size is
|
||||
resolved before scheduler admission so requests with different resulting
|
||||
shapes are not combined.
|
||||
When no output size is requested (`size=auto` on `/v1/images/edits`, or no
|
||||
`height`/`width` in a chat request), BAGEL img2img derives the canvas from the
|
||||
input image: it preserves the input aspect ratio, aligns dimensions to the
|
||||
latent stride (16 px), and applies the checkpoint size limit. An explicit
|
||||
`size`, `height`, or `width` is honored instead, in both the single-stage and
|
||||
the two-stage deployment; the input image is still resized to a stride-aligned
|
||||
size for the VAE/ViT prefill. The latent grid needs multiples of 16, so a
|
||||
requested side that is not one is floored to the next multiple (1000x700
|
||||
generates 992x688) and the server logs a warning; the response reports the
|
||||
generated size. A side above the checkpoint limit is rejected with HTTP 400.
|
||||
In step mode, the effective size is resolved before scheduler admission so
|
||||
requests with different resulting shapes are not combined.
|
||||
|
||||
### Image to Text (img2text)
|
||||
|
||||
@@ -318,8 +324,8 @@ curl http://localhost:8091/v1/chat/completions \
|
||||
| `--server` / `-s` | `http://localhost:8091` | Server URL |
|
||||
| `--image-url` / `-i` | `None` | Input image URL or local path (img2img/img2text) |
|
||||
| `--modality` / `-m` | `text2img` | `text2img`, `img2img`, `img2text`, `text2text` |
|
||||
| `--height` | `512` | Text-to-image output height; BAGEL img2img derives its size from the input image |
|
||||
| `--width` | `512` | Text-to-image output width; BAGEL img2img derives its size from the input image |
|
||||
| `--height` | `512` | Output height, a multiple of 16; omit both `--height` and `--width` to let img2img derive the size from the input image |
|
||||
| `--width` | `512` | Output width, a multiple of 16; omit both `--height` and `--width` to let img2img derive the size from the input image |
|
||||
| `--steps` | `25` | Number of inference steps |
|
||||
| `--seed` | `42` | Random seed |
|
||||
| `--negative` | `None` | Negative prompt for CFG |
|
||||
|
||||
Reference in new issue
Block a user