[Refactor] Remove dependency on librosa (#2273)

Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
Nick CaoandClaude authored and GitHub committed 2026-04-11 02:11:31 +08:00
1 parent 687405c5f2
commit 2bc183f6f0
46 files changed
+229 -323

No files matched your search

@@ -250,13 +250,6 @@ For more details on the Mooncake connector and multi-node setup, see the [Moonca
## FAQ
- If you encounter an error about the backend of librosa, try to install ffmpeg with the command below.
```bash
sudo apt update
sudo apt install ffmpeg
```
- If you don’t know how much VRAM is needed for the model or encounter the OOM error, you can try to decrease the max_model_len.
| Stage | VRAM |
@@ -10,7 +10,7 @@ Install dependencies:
uv pip install -e .
```
> **Note:** This includes required libraries such as `librosa`, `soundfile`,
> **Note:** This includes required libraries such as `soundfile`,
> `onnxruntime`, `x-transformers`, and `einops` via
> `requirements/common.txt` and platform-specific requirements files.
@@ -189,29 +189,6 @@ Note: This task uses hardcoded message lists in the script.
## Troubleshooting
### Audio dependencies (soundfile, librosa)
This example depends on **soundfile** (read/write WAV) and **librosa** (load audio including MP3). Install the project requirements first:
```bash
pip install -r requirements/common.txt
# or at least: pip install soundfile>=0.13.1 librosa>=0.11.0
```
- **`soundfile` / libsndfile not found**
`soundfile` uses the C library **libsndfile**. On Linux, install the system package before pip:
- Debian/Ubuntu: `sudo apt-get install libsndfile1`
- For development builds: `sudo apt-get install libsndfile1-dev`
- Then: `pip install soundfile`
- **`librosa` fails to load MP3 or reports "No backend available"**
Loading MP3 (e.g. in `spoken_dialogue_sft_multiturn` with `.mp3` files) uses **ffmpeg** as the backend. Install ffmpeg:
- Debian/Ubuntu: `sudo apt-get install ffmpeg`
- macOS: `brew install ffmpeg`
- **`ImportError: No module named 'soundfile'` or `ModuleNotFoundError: ... librosa`**
Ensure you are in the same Python environment where vLLM Omni and the example dependencies are installed, and that `requirements/common.txt` (or the packages above) are installed.
### Tokenizer path
- **`MIMO_AUDIO_TOKENIZER_PATH` not set or model fails to find tokenizer**
@@ -64,14 +64,6 @@ If media file paths are not provided, the script will use default assets. Suppor
- `use_audio_in_video`: Extract audio from video
- `text`: Text-only query
### FAQ
If you encounter error about backend of librosa, try to install ffmpeg with command below.
```
sudo apt update
sudo apt install ffmpeg
```
## Example materials
??? abstract "end2end.py"
@@ -112,14 +112,6 @@ python end2end_async_chunk.py \
> async_chunk example when you need the stage-level concurrency semantics
> described in PR #962 / #1151.
### FAQ
If you encounter error about backend of librosa, try to install ffmpeg with command below.
```
sudo apt update
sudo apt install ffmpeg
```
## Example materials
??? abstract "end2end.py"
@@ -357,13 +357,6 @@ curl http://localhost:8091/v1/chat/completions \
## FAQ
- If you encounter an error about the backend of librosa, try to install ffmpeg with the command below.
```bash
sudo apt update
sudo apt install ffmpeg
```
- If you don’t know how much VRAM is needed for the model or encounter the OOM error, you can try to decrease the max_model_len.
| Stage | VRAM |
@@ -218,14 +218,6 @@ The gradio script supports the following arguments:
- `--port`: Port for Gradio server (default: 7861)
- `--share`: Share the Gradio demo publicly (creates a public link)
### FAQ
If you encounter error about backend of librosa, try to install ffmpeg with command below.
```
sudo apt update
sudo apt install ffmpeg
```
## Example materials
??? abstract "gradio_demo.py"
@@ -64,15 +64,6 @@ python openai_chat_completion_client_for_multimodal_generation.py \
bash run_curl_multimodal_generation.sh use_image
```
### FAQ
If you encounter error about backend of librosa, try to install ffmpeg with command below.
```
sudo apt update
sudo apt install ffmpeg
```
## Modality control
You can control output modalities to specify which types of output the model should generate. This is useful when you only need text output and want to skip audio generation stages for better performance.
@@ -211,14 +211,6 @@ with open("output.wav", "wb") as f:
f.write(response.content)
```
### FAQ
If you encounter error about backend of librosa, try to install ffmpeg with command below.
```
sudo apt update
sudo apt install ffmpeg
```
## API Reference
### Voices Endpoint