lin-0189

15.2 The tangent Bellman residual

For an admitted state direction \(v\in T_sX\),

\[ \delta _T(\theta ;s,v) = D_s\delta _B(\theta ;s)[v]. \]

This asks whether the two Bellman routes respond coherently to a local change of state representation. It is not the parameter gradient of the scalar TD loss.

Definition 15.1 GIRL architectural interface

A GIRL realization exposes:

  1. the base Bellman residual \(\delta _B\);

  2. the tangent residual \(\delta _T\) on declared probes;

  3. a cover of the transition, reward, bootstrap, and representation branches;

  4. a decision-relevant quotient; and

  5. an admission controller for cross-regime use.

For isotropic random probes with \(\mathbb E[vv^\top ]=I\), writing \(J_s=D_s\delta _B\) gives

\[ \mathbb E_v\lVert J_sv\rVert ^2 = \operatorname {tr}(J_s^\top J_s) = \lVert J_s\rVert _F^2. \]

JVPs therefore estimate the squared tangent residual without materializing a full Jacobian.