[Benchmark] Add LingBot-World realtime streaming benchmark (#7645)

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Zhou TaichangandClaude Opus 5 authored and GitHub committed 2026-09-21 13:12:00 +00:00
1 parent fa1d03ab8d
commit 856d19f42c
8 files changed
+2413

No files matched your search

+11
View File
@@ -41,6 +41,17 @@ LingBot-Video MoE transformer.
- **Dense pipeline**: decoded-video MAE, MSE, PSNR, latency, and optional steady-state timings
- **MoE transformer**: bitwise router, sparse-block, shared-expert, and full-transformer parity
### [LingBot-World](lingbot_world/README.md) — Realtime Streaming Cadence
Client-side benchmark for a `WS /v1/realtime/video` world-model session, where one
request is a whole rollout and one AR block streams out as one video chunk. It
measures a cadence rather than a throughput: there is exactly one session, and what
matters is whether the next chunk arrives before the viewer finishes the last one.
- **Layout**: `lingbot_world/benchmark_lingbot_world_realtime.py` (WS client), `lingbot_world/workload.py` (rollout and metric math), `lingbot_world/configs/` (single-GPU eager and Ulysses-4 compiled deploy configs)
- **Workload**: built-in image-conditioned rollout with a generated camera script, or a JSON spec with per-chunk actions and mid-rollout prompt updates
- **Key metrics**: TTFC (time to first chunk), steady-state chunk inter-arrival percentiles, VIDEO_RTF (wall seconds per video second; lower is better), chunk-deadline attainment, and simulated playback underruns
### [Distributed](distributed/omni_connectors/README.md) — RDMA Connector Testing
RDMA environment setup and transfer tests for `MooncakeTransferEngineConnector`, including pytest-based single-node checks and manual cross-node benchmarks.