[CI] update daily omni min accuracy (#3536)
Signed-off-by: rein yang <ruiruyang2@gmail.com>
This commit is contained in:
1 parent
aa1184d737
commit
f4b28f2398
1 file changed
+3
-3
@@ -8,7 +8,7 @@ L4-style smoke runs against an already-running server. The pytest wrappers in
|
||||
H100 / MI325) because they launch the live Omni server inside the test.
|
||||
|
||||
1. **Daily-Omni** — MCQ accuracy fields in the saved JSON (``daily_omni_accuracy``, …); by default the
|
||||
run **fails** if accuracy is strictly below **0.69** (``--min-daily-omni-accuracy`` / ``ACC_BENCH_MIN_DAILY_OMNI_ACCURACY``).
|
||||
run **fails** if accuracy is strictly below **0.67** (``--min-daily-omni-accuracy`` / ``ACC_BENCH_MIN_DAILY_OMNI_ACCURACY``).
|
||||
2. **Seed-TTS** — ``seed-tts-eval``-style metrics when ``--seed-tts-wer-eval`` is used
|
||||
(WER / SIM / UTMOS keys from :func:`compute_seed_tts_wer_metrics`).
|
||||
|
||||
@@ -308,9 +308,9 @@ def build_arg_parser() -> argparse.ArgumentParser:
|
||||
p.add_argument(
|
||||
"--min-daily-omni-accuracy",
|
||||
type=float,
|
||||
default=float((os.environ.get("ACC_BENCH_MIN_DAILY_OMNI_ACCURACY") or "0.69").strip() or "0.69"),
|
||||
default=float((os.environ.get("ACC_BENCH_MIN_DAILY_OMNI_ACCURACY") or "0.67").strip() or "0.67"),
|
||||
help="Fail when daily_omni_accuracy is strictly below this threshold (0–1). "
|
||||
"Default baseline 0.69; override with env ACC_BENCH_MIN_DAILY_OMNI_ACCURACY or pass 0 to disable the floor.",
|
||||
"Default baseline 0.67; override with env ACC_BENCH_MIN_DAILY_OMNI_ACCURACY or pass 0 to disable the floor.",
|
||||
)
|
||||
|
||||
p.add_argument(
|
||||
|
||||
Reference in new issue
Block a user