[MR V2] Migrate to Model Runner V2
Squashed migration rebased onto current main. Collapses the 46-commit dev/migrate-MR-v2 history (10 merge commits + superseded pre-rebase work) into a single commit, so the diff reflects only the genuine MR-V2 feature (worker_v2/ runners + model_states, omni schedulers, stage engine pipeline, pooling_output serialization bridge). Content already merged to main separately (step_audio2, tts_adapters, npu/_310p, etc.) drops out. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
cead615288
commit
9678ae58cf
74 files changed
+8244
-95
No files matched your search
@@ -32,6 +32,8 @@ python3 benchmarks/diffusion/diffusion_benchmark_serving.py \
|
||||
|
||||
- By default, image tasks talk to `http://<host>:<port>/v1/chat/completions`; video tasks talk to `/v1/videos`.
|
||||
- If you run the server on another host or port, pass `--base-url` accordingly.
|
||||
- To compare diffusion multi-replica serving, restart the server with different
|
||||
`runtime.num_replicas` values and keep the benchmark parameters unchanged.
|
||||
|
||||
## 2. Supported Datasets
|
||||
|
||||
@@ -93,6 +95,35 @@ You can point to your own trace using `--dataset-path`.
|
||||
|
||||
## 3. Benchmark Parameters
|
||||
|
||||
### Multi-replica sweep
|
||||
|
||||
For replica scaling, use the same model, prompt count, image/video size, and
|
||||
step count for every run. Change only the serving stage config:
|
||||
|
||||
```yaml
|
||||
runtime:
|
||||
devices: "0,1,2,3"
|
||||
num_replicas: 4
|
||||
```
|
||||
|
||||
Then run a fixed benchmark and write one JSON file per replica count:
|
||||
|
||||
```bash
|
||||
python3 benchmarks/diffusion/diffusion_benchmark_serving.py \
|
||||
--base-url http://localhost:8099 \
|
||||
--model Qwen/Qwen-Image \
|
||||
--task t2i \
|
||||
--dataset random \
|
||||
--num-prompts 32 \
|
||||
--max-concurrency 8 \
|
||||
--request-rate inf \
|
||||
--num-inference-steps 10 \
|
||||
--width 512 --height 512 \
|
||||
--output-file diffusion_replica_4.json
|
||||
```
|
||||
|
||||
Repeat for `num_replicas=1`, `2`, `3`, and `4`, using matching `runtime.devices`.
|
||||
|
||||
### Basic flags
|
||||
|
||||
- `--base-url`: Server address; `--endpoint` selects the path appended to this base URL.
|
||||
|
||||
Reference in new issue
Block a user