[Benchmark] Add LingBot-World realtime streaming benchmark (#7645)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
fa1d03ab8d
commit
856d19f42c
8 files changed
+2413
No files matched your search
@@ -41,6 +41,17 @@ LingBot-Video MoE transformer.
|
||||
- **Dense pipeline**: decoded-video MAE, MSE, PSNR, latency, and optional steady-state timings
|
||||
- **MoE transformer**: bitwise router, sparse-block, shared-expert, and full-transformer parity
|
||||
|
||||
### [LingBot-World](lingbot_world/README.md) — Realtime Streaming Cadence
|
||||
|
||||
Client-side benchmark for a `WS /v1/realtime/video` world-model session, where one
|
||||
request is a whole rollout and one AR block streams out as one video chunk. It
|
||||
measures a cadence rather than a throughput: there is exactly one session, and what
|
||||
matters is whether the next chunk arrives before the viewer finishes the last one.
|
||||
|
||||
- **Layout**: `lingbot_world/benchmark_lingbot_world_realtime.py` (WS client), `lingbot_world/workload.py` (rollout and metric math), `lingbot_world/configs/` (single-GPU eager and Ulysses-4 compiled deploy configs)
|
||||
- **Workload**: built-in image-conditioned rollout with a generated camera script, or a JSON spec with per-chunk actions and mid-rollout prompt updates
|
||||
- **Key metrics**: TTFC (time to first chunk), steady-state chunk inter-arrival percentiles, VIDEO_RTF (wall seconds per video second; lower is better), chunk-deadline attainment, and simulated playback underruns
|
||||
|
||||
### [Distributed](distributed/omni_connectors/README.md) — RDMA Connector Testing
|
||||
|
||||
RDMA environment setup and transfer tests for `MooncakeTransferEngineConnector`, including pytest-based single-node checks and manual cross-node benchmarks.
|
||||
|
||||
Reference in new issue
Block a user