BeatSeatandamy-why-3459
79562e97ad
Signed-off-by: BeatSeat <wendavid552@gmail.com> Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
26 lines
1.4 KiB
Markdown
26 lines
1.4 KiB
Markdown
# MiniCPM-o 4.5 MRv2 profiles
|
|
|
|
Turn mode uses the stage-level MRv2 contracts from #8184. Thinker emits live
|
|
latent metadata outside graph replay; Talker keeps codec history and EOS
|
|
control on device. Code2Wav reuses mainline Whole-Euler Flow graphs and shared
|
|
prompt state. The previous experimental block compilation, tiled attention,
|
|
channels-last and merged-CFM implementations are removed.
|
|
|
|
`minicpmo_4_5_turn_mrv2_h200.yaml` selects Talker capacity 16 and 4 GiB KV;
|
|
the generic MRv2 profile selects capacity 8 and 2 GiB KV. These configurations
|
|
require new end-to-end performance measurements after the mainline integration.
|
|
Previously reported numbers do not describe this revised codec backend.
|
|
|
|
The generic MRv2 profile and the opt-in native V1 duplex H200 profile enable
|
|
ordinary TF32 for CFM DiT GEMMs within Code2Wav forward/capture only
|
|
(`torch.backends.cuda.matmul.allow_tf32` on dense QKV/MLP, Triton
|
|
`input_precision="tf32"` on tiled attention). This is not compensated TF32x3.
|
|
The previous process matmul policy is restored afterwards. cuDNN's TF32 policy
|
|
is independent. HiFT stays IEEE FP32. TF32 changes rounding.
|
|
Duplex keeps the mainline V1 session path. Turn results do not establish duplex
|
|
performance or interruption correctness.
|
|
|
|
The shared asynchronous output snapshot and batched Talker preprocessing also
|
|
apply to V1 when async chunking/scheduling is enabled. Default V1 behavior must
|
|
therefore be included in end-to-end regression validation.
|