[Bugfix][Qwen3-Omni] Fix synchronous Thinker pipeline parallelism (#7345)
Signed-off-by: Deep Shah <deep@socratic.co>
This commit is contained in:
1 parent
909b121e0a
commit
2f1a18e69f
7 files changed
+279
-9
No files matched your search
@@ -22,6 +22,10 @@ both stages on vLLM's Triton unquantized MoE path and avoids the unstable FlashI
|
||||
some GPUs.
|
||||
Asynchronous chunk streaming is **enabled by default** within the bundled configuration.
|
||||
|
||||
For Thinker pipeline parallelism, configure `pipeline_parallel_size` and `devices`
|
||||
on the Thinker stage and set `async_scheduling: false`. Keep Talker and Code2Wav
|
||||
at `pipeline_parallel_size: 1`. Audio chunk streaming can remain enabled.
|
||||
|
||||
To explicitly utilize a custom deployment YAML, specify the configuration path:
|
||||
```bash
|
||||
vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091 \
|
||||
|
||||
Reference in new issue
Block a user