13.5 The ALFWorld benchmark contract

The benchmark separates two questions. The competence question asks whether a policy equipped with structural state solves standard tasks and transports to unseen houses. The creativity question asks whether the system detects a withheld obligation, proposes a repair, admits it on disjoint evidence, and reuses it without sacrificing earlier structure.

Three channels must remain distinct:

Policy-visible.

The natural-language goal, current and past observations, executed actions, observation-derived outcomes, and admitted schemas with their provenance.

Evaluator-only.

PDDL facts, expert plans, game identity during action selection, and oracle success state before scoring.

Declared oracle control.

The official ALFWorld TextDQN receives the environment’s admissible-command list. It is therefore an oracle-action-space learned baseline, not a deployable counterpart to a policy that generates its own commands.

This qualification is central. Admissibility removes one source of failure but does not supply planning: a random admissible policy solved none of the 274 validation games. Conversely, comparing a generated-action RELIC policy with an admissible-command DQN without labeling the channel difference would overstate the empirical conclusion.

. RELIC evidence is partitioned into discovery, admission, confirmation, and transport. A game used to construct or rank a repair cannot certify that repair. Confirmation occurs only after the admitted structure and its thresholds have been frozen.