Files
vllm-omni/tests
tzhouamandClaude Opus 4.8 fe16565cf6 test(entrypoints): grade realtime audio output with whisper-large-v3
The Qwen3-Omni /v1/realtime accuracy check transcribed the generated
audio with whisper-small, which mishears short Chinese TTS clips
(observed: 北京->韦京 plus a dropped leading sentence, sim=0.443) and
produced spurious sim<threshold failures under async_chunk codec
variability even though audio generation was correct. Switch the grader
to large-v3 so a failure here points at the model, not the ASR check.
Also parameterize the threshold per mode (0.65 async_chunk, 0.8 sync).

Verified on 2x H100 (GPU 0,1), both realtime tests pass:
  test_streaming_audio_input_pcm_output_async_chunk[async_chunk] PASSED
  test_streaming_audio_input_pcm_output[sync] PASSED

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 14:21:18 +08:00
..