1.10 A controlled acquisition benchmark

A controlled benchmark can use a deterministic hand–eye world or an instruction-rich environment such as ALFWorld. An expert solves tasks while the learner receives progressively richer acquisition channels. The sequence is an evidence hierarchy: each condition tests what an additional channel contributes to the acquired sketch.

1. Trace imitation: expert state–action sequences only.

2. Explained imitation: traces plus subgoals, object roles, and action rationales.

3. Corrective apprenticeship: the learner acts and receives localized corrections from a teacher.

4. Active apprenticeship: the learner requests demonstrations, contrasts, or explanations that distinguish its remaining candidate sketches.

5. Acquisition plus DIAL: the acquired sketch initializes RELIC or another DIAL-X system, which must recognize and repair a withheld structural change.

Evaluation crosses task performance with structural recovery. Relevant measures include type accuracy, precondition and effect recovery, equation and invariant recovery, calibration of unsupported alternatives, teacher interventions required, compositional transfer, and preservation of earlier knowledge after accommodation. A held-out recombination is particularly important: it tests whether the learner acquired a theory that composes rather than a library of trajectories.

The tennis analogy suggests the expected outcome. Watching can supply a broad theory of the game. Coaching should be most valuable where the decisive variables are difficult to infer from expert behavior: grip, body position, timing, attention, and recovery. DIAL becomes relevant when the learner encounters a task variant for which the acquired ontology or skill grammar is insufficient, rather than merely executing an established stroke poorly.