[Refactor] Migrate Legacy Stage Configs to the Pipeline Registry (#5031)
Signed-off-by: zwhzzz0821 <2831474076@qq.com> Co-authored-by: zwhzzz0821 <zwhzzz0821@users.noreply.github.com> Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com> Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
41 files changed
+520
-516
No files matched your search
@@ -51,7 +51,7 @@ Single request latency comparison between vLLM-Omni and official Step-Audio2 imp
|
||||
|
||||
**Benchmark environment:**
|
||||
- GPU: NVIDIA H100 80GB (single card)
|
||||
- Model: Step-Audio2-mini
|
||||
- Model: Step-Audio-2-mini
|
||||
- Warmup: 1 run, Measured: 3 runs (averaged)
|
||||
|
||||
### Async Chunk Streaming Performance
|
||||
@@ -73,7 +73,7 @@ Comparison between sequential (non-async) and async chunk modes via `/v1/audio/s
|
||||
|
||||
**Benchmark environment:**
|
||||
- GPU: 4x NVIDIA RTX 3090 24GB (TP=2 for Thinker, 1 GPU for Token2Wav)
|
||||
- Model: Step-Audio2-mini
|
||||
- Model: Step-Audio-2-mini
|
||||
- Endpoint: `/v1/audio/speech` (10 prompts, concurrency=1)
|
||||
- Measured via `bench_tts_serve.py`
|
||||
|
||||
@@ -91,23 +91,22 @@ pip install step-audio2
|
||||
|
||||
## Model Setup
|
||||
|
||||
Step-Audio2 ships a custom Transformers configuration, so this model-specific
|
||||
example enables remote code loading when it initializes the model.
|
||||
|
||||
### Option 1: Auto-download from HuggingFace (Recommended)
|
||||
|
||||
The script will **automatically download** the model on first run:
|
||||
The script will **automatically download** the model on first run when a
|
||||
HuggingFace model id is passed:
|
||||
|
||||
```bash
|
||||
# Just run without specifying --model, it will auto-download stepfun-ai/Step-Audio2-mini
|
||||
python end2end.py --query-type audio_to_text
|
||||
|
||||
# Or explicitly specify the HuggingFace model
|
||||
python end2end.py --query-type audio_to_text --model stepfun-ai/Step-Audio2-mini
|
||||
python end2end.py --query-type audio_to_text --model stepfun-ai/Step-Audio-2-mini
|
||||
```
|
||||
|
||||
Models will be cached in `~/.cache/huggingface/hub/` for future use.
|
||||
|
||||
**Available models**:
|
||||
- `stepfun-ai/Step-Audio2-mini` (smaller, faster)
|
||||
- `stepfun-ai/Step-Audio2-7B` (larger, better quality)
|
||||
**Supported model**:
|
||||
- `stepfun-ai/Step-Audio-2-mini`
|
||||
|
||||
### Option 2: Manual Download (for offline use)
|
||||
|
||||
@@ -115,15 +114,15 @@ Download and use locally:
|
||||
|
||||
```bash
|
||||
# Download from HuggingFace
|
||||
huggingface-cli download stepfun-ai/Step-Audio2-mini --local-dir ./models/Step-Audio2-mini
|
||||
hf download stepfun-ai/Step-Audio-2-mini --local-dir ./models/Step-Audio-2-mini
|
||||
|
||||
# Then use the local path
|
||||
python end2end.py --query-type audio_to_text --model ./models/Step-Audio2-mini
|
||||
python end2end.py --query-type audio_to_text --model ./models/Step-Audio-2-mini
|
||||
```
|
||||
|
||||
Ensure the model directory contains:
|
||||
```
|
||||
Step-Audio2-mini/
|
||||
Step-Audio-2-mini/
|
||||
├── config.json
|
||||
├── model.safetensors (or pytorch_model.bin)
|
||||
├── tokenizer.json
|
||||
@@ -144,20 +143,18 @@ Transcribe audio to text:
|
||||
|
||||
```bash
|
||||
# Quick start - Using default model and test audio
|
||||
python end2end.py --query-type audio_to_text
|
||||
python end2end.py --query-type audio_to_text \
|
||||
--model stepfun-ai/Step-Audio-2-mini
|
||||
|
||||
# Using your own audio file (model will auto-download)
|
||||
python end2end.py --query-type audio_to_text \
|
||||
--audio-path /path/to/input.wav
|
||||
|
||||
# With specific model
|
||||
python end2end.py --query-type audio_to_text \
|
||||
--audio-path /path/to/input.wav \
|
||||
--model stepfun-ai/Step-Audio2-7B
|
||||
--model stepfun-ai/Step-Audio-2-mini
|
||||
|
||||
# With custom question
|
||||
python end2end.py --query-type audio_to_text \
|
||||
--audio-path input.wav \
|
||||
--model stepfun-ai/Step-Audio-2-mini \
|
||||
--question "What is the speaker saying?"
|
||||
```
|
||||
|
||||
@@ -170,12 +167,8 @@ Convert text to speech:
|
||||
```bash
|
||||
# Basic TTS (model auto-downloads)
|
||||
python end2end.py --query-type text_to_audio \
|
||||
--text "Hello, this is a test of Step Audio 2 synthesis."
|
||||
|
||||
# With specific model
|
||||
python end2end.py --query-type text_to_audio \
|
||||
--text "Hello, this is a test." \
|
||||
--model stepfun-ai/Step-Audio2-7B
|
||||
--text "Hello, this is a test of Step Audio 2 synthesis." \
|
||||
--model stepfun-ai/Step-Audio-2-mini
|
||||
```
|
||||
|
||||
**Note**: Speaker voice is controlled by the `STEP_AUDIO2_DEFAULT_PROMPT_WAV` environment variable or the default prompt wav bundled with the model.
|
||||
@@ -191,12 +184,8 @@ Process input audio and generate output audio:
|
||||
```bash
|
||||
# Basic voice conversion (model auto-downloads)
|
||||
python end2end.py --query-type audio_to_audio \
|
||||
--audio-path /path/to/source_audio.wav
|
||||
|
||||
# With specific model
|
||||
python end2end.py --query-type audio_to_audio \
|
||||
--audio-path source.wav \
|
||||
--model stepfun-ai/Step-Audio2-7B
|
||||
--audio-path /path/to/source_audio.wav \
|
||||
--model stepfun-ai/Step-Audio-2-mini
|
||||
```
|
||||
|
||||
This mode:
|
||||
@@ -210,8 +199,14 @@ This mode:
|
||||
```bash
|
||||
# Use custom stage configuration
|
||||
python end2end.py --query-type audio_to_text \
|
||||
--model stepfun-ai/Step-Audio-2-mini \
|
||||
--stage-configs-path /path/to/custom_config.yaml
|
||||
|
||||
# Use custom deploy configuration
|
||||
python end2end.py --query-type audio_to_text \
|
||||
--model stepfun-ai/Step-Audio-2-mini \
|
||||
--deploy-config /path/to/step_audio_2_asr.yaml
|
||||
|
||||
# Multiple prompts (for batch testing)
|
||||
python end2end.py --query-type audio_to_text \
|
||||
--audio-path input.wav \
|
||||
@@ -246,14 +241,14 @@ python end2end.py --query-type text_to_audio \
|
||||
|
||||
## Configuration
|
||||
|
||||
### Stage Configuration
|
||||
### Deploy Configuration
|
||||
|
||||
The default configuration (`step_audio_2.yaml`) uses:
|
||||
The default configuration (`vllm_omni/deploy/step_audio_2.yaml`) uses:
|
||||
|
||||
- **Stage 0 (Thinker)**: GPU 0, 80% memory
|
||||
- **Stage 1 (Token2Wav)**: GPU 1, 30% memory
|
||||
- **Stage 0 (Thinker)**: GPUs 0-1 with tensor parallel size 2, 70% memory
|
||||
- **Stage 1 (Token2Wav)**: GPU 1, 20% memory
|
||||
|
||||
For **single GPU** setup, edit the config to use `devices: "0"` for both stages.
|
||||
For **single GPU** setup, edit a deploy config copy to use `devices: "0"` for both stages.
|
||||
|
||||
### Sampling Parameters
|
||||
|
||||
@@ -351,7 +346,7 @@ Complete example from audio to final output:
|
||||
# 1. ASR: Transcribe audio
|
||||
python end2end.py --query-type audio_to_text \
|
||||
--audio-path interview.wav \
|
||||
--model ./models/Step-Audio2-7B \
|
||||
--model ./models/Step-Audio-2-mini \
|
||||
--output-dir ./outputs
|
||||
|
||||
# 2. Check the transcription
|
||||
@@ -361,7 +356,7 @@ cat ./outputs/00000_text.txt
|
||||
STEP_AUDIO2_DEFAULT_PROMPT_WAV=./speaker_samples/female_voice.wav \
|
||||
python end2end.py --query-type text_to_audio \
|
||||
--text "The quick brown fox jumps over the lazy dog" \
|
||||
--model ./models/Step-Audio2-7B \
|
||||
--model ./models/Step-Audio-2-mini \
|
||||
--output-dir ./outputs
|
||||
|
||||
# 4. Listen to the result
|
||||
|
||||
Reference in new issue
Block a user