lin-0168
13.3 The ALLORA objective
Let \(\mathcal T_i\) denote the task loss for adapter \(i\), and let \(\mathcal C\) be the set of pairs declared composable. A basic realization is
\[ \mathcal L_{\mathrm{ALLORA}} = \sum _i\mathcal T_i + \lambda _c \sum _{\ell }\sum _{(i,j)\in \mathcal C} \left\| [\Delta _i^\ell ,\Delta _j^\ell ] \right\| _F^2. \]
The task losses preserve independent usefulness; the bracket block reduces the selected interaction obstruction.
Definition
13.2
ALLORA admission profile
For a trained adapter family, the profile \(\Psi _{\mathrm A}\) retains individual task scores, commutator norms, hidden and output order errors, and worst-order capability and safety.
The factorization gauge leaves \(\Delta _i\) unchanged:
\[ (B_i,A_i)\sim (B_iG,G^{-1}A_i). \]
Measuring commutators of \(\Delta _i\), rather than raw factors, makes the principal observer descend through this presentation redundancy.