Exploration

Create an environment where two policies have similar returns but different tangent Bellman residuals under controlled transition perturbations. Compare GIRL with a return-matched RL baseline on adaptation and calibration. Does the tangent signal predict useful policy repair before reward reveals the change, and under what probe semantics does it become misleading?

Digital-edition research invitation; not part of the frozen dated PDF.

ch-girl

15 Infinitesimal Reinforcement Learning

GIRL

Temporal-difference learning turns Bellman inconsistency into a scalar error. GIRL retains the diagram that produced that error. It distinguishes the base Bellman obstruction, its tangent sensitivity, the action-common variation that should be quotiented away, the computational branch carrying the failure, and the evidence required to transport a repair across regimes.

¹1. GIRL abbreviates Gradient Infinitesimal Reinforcement Learning. It is an architectural layer above TD, not a replacement for the underlying policy-evaluation algorithm. ↩