15.9 Safe recovery

In a 12-state online actor–critic, the benign control contains no safety pathology: replay ESS remains near \(0.96\), no method makes a harmful update, and unconditional quotient GIRL achieves the best final return. Always-on gating is unnecessarily conservative. This negative result motivates routing: use the quotient alone while diagnostics are healthy.

Experiment: GIRL safe recovery under hidden regime change

In the adversarial study, reward preference and action-transition kernels change at epoch 10 while stale replay remains. The shift time is hidden from the methods. Unconditional GIRL makes harmful updates in roughly \(23\% \) of post-shift epochs. The recovery controller detects the shift in every seed, reduces harm to zero, and re-enters in \(29/30\) seeds after 4.14 epochs on average.

One seed remains abstinent through the finite horizon. The correct interpretation is unresolved uncertainty, not an unrecoverable environment.