lin-0181

14.5 Controlled benchmark

A ten-anchor Markdown benchmark contains five productive ordered chains: schema before normalization, evidence before citation, abstention before formatting, tool contract before routing, and validator before final answer. Reversing a chain loses its benefit. The normalized workflow score is

\[ \text{progress}\times \text{faithfulness}. \]

Progress is the fraction of the five productive chains closed in the correct order; faithfulness checks whether the served evaluator reads the resulting skill state correctly. A bracket probe is a cheap deterministic read–write/dependency calculation. A validation probe sends a candidate artifact through the authoritative evaluator. The two columns therefore measure different resources and are never added as if they had equal cost.

Method

Bracket probes

Validation probes

Score

Greedy single edit

0

27

\(0.240\)

Random ordered pairs

0

10

\(0.266\) mean

Exhaustive ordered pairs

0

90

\(1.000\)

LASKO-prioritized

90

10

\(1.000\)

Table 14.2 The controlled benchmark isolates the value of an interaction screen; bracket probes are cheap, validation probes expensive.

Under a served-model validation contract, exhaustive search uses 91 calls and LASKO 11; both score \(1.000\). Greedy and random search score \(0.000\) and \(0.120\), respectively.