lin-0192

15.5 Natural GIRL

The advantage quotient identifies which policy distinctions matter, but it does not yet choose an intrinsic repair direction. Let \(\Pi \) be a smooth policy object, let \(\mathcal O_{\mathrm G}(\pi )\) denote the typed GIRL obstruction, and let \(F_\pi \) be a policy Fisher metric defined using a declared state-visitation law. The Natural LINCS construction of Chapter 4 specializes to

\begin{equation} \xi _\pi ^\star \in \operatorname *{arg\, min}_{\xi \in T_\pi \Pi } \frac12 F_\pi (\xi ,\xi ) \quad \text{subject to}\quad D\mathcal O_{\mathrm G,\pi }[\xi ] =-\mathcal O_{\mathrm G}(\pi ). \end{equation}

15.1

This is Natural GIRL: the Bellman, intervention, or advantage sketch defines what requires repair, while information geometry selects the least intrinsic policy change that cancels the linearized obstruction. For a scalar policy objective, the construction reduces in coordinates to the familiar natural policy-gradient direction. Natural actor–critic supplies one family of estimators for that direction; it does not replace the GIRL declaration or admission controller.

The visitation law in \(F_\pi \) is semantic data. On-policy, discounted, stationary, replay-weighted, and target-policy distributions generally define different geometries. In the off-policy setting of this chapter, the metric declaration must therefore be audited together with coverage and effective sample size. A Fisher matrix may also be singular when distinct parameter directions induce the same policy. Such directions should be quotiented as policy gauges before inversion, or handled by a registered damping or pseudoinverse rule.

Design principle

GIRL declares the decision-relevant obstruction; Natural GIRL chooses an intrinsic tangent proposal; the safety and recovery controller decides whether that proposal may alter the policy.