[Edge] Classify and document the edge env vars; fix the client decoder call
- environment_variable_inventory: VLLM_OMNI_CPU_INDUCTOR / HW_PROFILE / INIT_TIMELINE / STEP_STATS_DIR (public), STAGE_ID / INIT_TIMELINE_DIR (internal); reference page rows added. - edge_decoder_client.LocalDecoder: batched_chunked_decode takes a [B, Q, F] tensor, not a list (the WP5 e2e run failed with a TypeError); the decoder now also carries the ramp schedule attributes the Code2Wav stage sets. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
fa8b1a878e
commit
536866b916
4 files changed
+33
-9
No files matched your search
@@ -84,6 +84,10 @@ depends on the installed kernels and model path.
|
||||
| `SPEAKER_MAX_UPLOADED` | Integer; default `1000` | Speech server; read when speaker storage initializes | Environment-only setting. A non-integer logs a warning and uses `1000`; range is not otherwise validated. | Stable |
|
||||
| `VLLM_OMNI_ASYNC_OUTPUT_TIMEOUT` | Float seconds; default `600` | Diffusion engine async-output wait in `step_streaming`; resolved per call on the request path, not at import | Environment-only setting. A non-float or `<=0` value warns once and uses the default. | Experimental |
|
||||
| `VLLM_OMNI_EVENT_DRIVEN_ORCH` | `1`, `true`, `yes` or `on` enables; default `0` (off) | Orchestration loop and the serving-side final-output drain; read once when the `Orchestrator` is constructed | Environment-only setting. Values are stripped and case-normalized; any unrecognized value leaves the legacy poll loop selected. | Experimental |
|
||||
| `VLLM_OMNI_CPU_INDUCTOR` | `1` enables; default off | CPU OmniPlatform (`VLLM_TARGET_DEVICE=cpu`); read when the platform reports `supports_torch_inductor` | Environment-only setting. Turns on torch.compile/inductor for the CPU AR model runner (measured variant, see `benchmarks/tts/cpu_thread_sweep.py`). | Experimental |
|
||||
| `VLLM_OMNI_HW_PROFILE` | Filesystem path to a hardware-profile JSON | `deploy_profile="auto"` hardware adaptation (`vllm_omni/edge/hardware_probe.py`); read when the profile is loaded | Environment-only override of the live probe, used for emulation and tests; an unreadable file raises. | Experimental |
|
||||
| `VLLM_OMNI_INIT_TIMELINE` | Filesystem path (JSON) | Engine init timeline (`vllm_omni/engine/init_timeline.py`); read when `AsyncOmniEngine` starts | Environment-only setting. When set, init phases from the orchestrator and every stage worker are merged and written to this path; a partial timeline is written on init timeout. | Diagnostic |
|
||||
| `VLLM_OMNI_STEP_STATS_DIR` | Filesystem directory | Per-step overhead attribution (`vllm_omni/edge/step_stats.py`); read once per process on first use | Environment-only setting. Each process dumps `<role>_<pid>.json` (scheduler, engine step, chunk hop and dispatch timings); `stream_latency_bench.py --step-stats` merges them. Zero cost when unset. | Diagnostic |
|
||||
| `VLLM_OMNI_INPUT_WAIT_TIMEOUT_S` | Float seconds; default `600`; `<=0` disables | Full-payload input coordinator, not async-chunk transfer; read when the scheduler module imports in each worker | Environment-only setting. A non-float logs a warning and uses `600`. | Stable operational control |
|
||||
| `VLLM_OMNI_ORCH_MONITOR_PATH` | Filesystem path; default `<current-working-directory>/vllm_omni_orch_monitor_<timestamp>.json` | Orchestrator monitor enabled by `--enable-orch-monitor`; read when the monitor is created | Environment-only path override. Parent directories are created; write errors are logged. | Diagnostic |
|
||||
| `VLLM_OMNI_VIDEO_SYNC_TIMEOUT` | Float seconds; default `600` | Synchronous Videos API; read when the API server module imports | Environment-only setting. A non-float raises `ValueError` during import. | Experimental |
|
||||
|
||||
@@ -42,7 +42,13 @@ class LocalDecoder:
|
||||
"""Stateful chunked Qwen3-TTS decoder mirroring the Code2Wav stage."""
|
||||
|
||||
def __init__(
|
||||
self, model: str, device: str = "cpu", left_context: int = 72, chunk_frames: int = 25, dtype: str = "float32"
|
||||
self,
|
||||
model: str,
|
||||
device: str = "cpu",
|
||||
left_context: int = 72,
|
||||
chunk_frames: int = 25,
|
||||
dtype: str = "float32",
|
||||
ramp: list[int] | None = None,
|
||||
):
|
||||
from safetensors.torch import load_file
|
||||
|
||||
@@ -66,8 +72,12 @@ class LocalDecoder:
|
||||
self.decoder.eval().to(device=device, dtype=getattr(torch, dtype))
|
||||
if hasattr(self.decoder, "precompute_snake_caches"):
|
||||
self.decoder.precompute_snake_caches()
|
||||
if hasattr(self.decoder, "_incremental_chunk_frames"):
|
||||
self.decoder._incremental_chunk_frames = chunk_frames
|
||||
# Mirror the Code2Wav stage's chunk-schedule attributes so the decoder's
|
||||
# prefix/suffix transitions match the server's (qwen3_tts_code2wav.py).
|
||||
ramp = list(ramp or [])
|
||||
self.decoder._initial_codec_chunk_frames = int(ramp[0]) if ramp else 1
|
||||
self.decoder._incremental_chunk_frames = chunk_frames
|
||||
self.decoder._incremental_chunk_ramp = ramp
|
||||
self.device = device
|
||||
self.num_quantizers = int(cfg.decoder_config.num_quantizers)
|
||||
self.sample_rate = int(cfg.output_sample_rate)
|
||||
@@ -95,7 +105,7 @@ class LocalDecoder:
|
||||
return np.zeros(0, dtype=np.float32)
|
||||
qf = torch.as_tensor(codes.T, dtype=torch.long, device=self.device) # [Q, F]
|
||||
wavs = self.decoder.batched_chunked_decode(
|
||||
[qf],
|
||||
qf.unsqueeze(0), # [B=1, Q, F]; the decoder slices rows itself
|
||||
[int(qf.shape[-1])],
|
||||
caches=[self.state],
|
||||
chunk_size=self.chunk_frames,
|
||||
@@ -157,13 +167,15 @@ class ExecutorchWindowedDecoder:
|
||||
return wav[-n * self.hop :] if n < size else wav
|
||||
|
||||
|
||||
def build_decoder(args: argparse.Namespace, steady: int):
|
||||
def build_decoder(args: argparse.Namespace, steady: int, schedule: list[int] | None = None):
|
||||
backend = getattr(args, "decoder_backend", "torch")
|
||||
if backend == "executorch":
|
||||
if not getattr(args, "pte_manifest", None):
|
||||
raise SystemExit("--decoder-backend executorch needs --pte-manifest <export dir>/manifest.json")
|
||||
return ExecutorchWindowedDecoder(args.pte_manifest)
|
||||
return LocalDecoder(args.model, device=args.device, left_context=args.left_context, chunk_frames=steady)
|
||||
return LocalDecoder(
|
||||
args.model, device=args.device, left_context=args.left_context, chunk_frames=steady, ramp=list(schedule)
|
||||
)
|
||||
|
||||
|
||||
# ------------------------------------------------------------------- client
|
||||
@@ -173,7 +185,7 @@ async def run(args: argparse.Namespace) -> dict[str, Any]:
|
||||
ramp = args.ramp or DEFAULT_RAMP
|
||||
schedule = parse_chunk_ramp({"codec_chunk_ramp": ramp}, steady=ramp[-1]) or ramp
|
||||
steady = schedule[-1]
|
||||
decoder = build_decoder(args, steady)
|
||||
decoder = build_decoder(args, steady, list(schedule))
|
||||
url = f"ws://{args.host}:{args.port}/v1/audio/speech/stream"
|
||||
config: dict[str, Any] = {
|
||||
"type": "session.config",
|
||||
|
||||
@@ -164,12 +164,12 @@ def test_inventory_matches_reviewed_snapshot_counts():
|
||||
"""Make an inventory expansion an explicit review decision."""
|
||||
category_counts = Counter(item.category for item in ENVIRONMENT_VARIABLE_INVENTORY.values())
|
||||
assert category_counts == {
|
||||
EnvironmentVariableCategory.PUBLIC_OMNI: 24,
|
||||
EnvironmentVariableCategory.PUBLIC_OMNI: 28,
|
||||
EnvironmentVariableCategory.INHERITED_VLLM: 20,
|
||||
EnvironmentVariableCategory.PLATFORM_EXTERNAL: 27,
|
||||
EnvironmentVariableCategory.MODEL_SPECIFIC: 62,
|
||||
EnvironmentVariableCategory.BENCHMARK_TRANSITIONAL: 20,
|
||||
EnvironmentVariableCategory.INTERNAL: 2,
|
||||
EnvironmentVariableCategory.INTERNAL: 4,
|
||||
}
|
||||
|
||||
disposition_counts = Counter(
|
||||
|
||||
@@ -64,6 +64,11 @@ _PUBLIC_OMNI = (
|
||||
"SPEAKER_MAX_UPLOADED",
|
||||
"SPEAKER_SAMPLES_DIR",
|
||||
"VLLM_OMNI_ASYNC_OUTPUT_TIMEOUT",
|
||||
# edge branch (vllm_omni/edge, init timeline, CPU platform)
|
||||
"VLLM_OMNI_CPU_INDUCTOR",
|
||||
"VLLM_OMNI_HW_PROFILE",
|
||||
"VLLM_OMNI_INIT_TIMELINE",
|
||||
"VLLM_OMNI_STEP_STATS_DIR",
|
||||
"VLLM_OMNI_EVENT_DRIVEN_ORCH",
|
||||
"VLLM_OMNI_INPUT_WAIT_TIMEOUT_S",
|
||||
"VLLM_OMNI_ORCH_MONITOR_PATH",
|
||||
@@ -238,6 +243,9 @@ _BENCHMARK_TRANSITIONAL = (
|
||||
_INTERNAL = (
|
||||
"VLLM_OMNI_DLO_DP_WAVE_TIMEOUT",
|
||||
"VLLM_OMNI_REPLICA_ID",
|
||||
# edge branch: set by the orchestrator for its stage subprocesses
|
||||
"VLLM_OMNI_STAGE_ID",
|
||||
"VLLM_OMNI_INIT_TIMELINE_DIR",
|
||||
)
|
||||
|
||||
_REDACTED = frozenset(
|
||||
|
||||
Reference in new issue
Block a user