sec-transformer-challenge
4.8 Does existing machinery already solve ORACLE?
The strongest contemporary objection to this program is not that categorical structure is too abstract. It is that large Transformer systems already appear to have solved the problem. They converse fluently, construct programs, use tools, repair errors, and produce novel compositions over language, code, and images. Measurements of frontier agents show rapidly increasing competence on multi-step software tasks, even though success remains dependent on task duration, domain, scaffold, and evaluation protocol [ METR , 2026 , OpenAI , 2026b ] . Any theory of compositional learning that begins by discounting this evidence has protected itself from its most important empirical fact.
The appropriate response is to make ORACLE architecture-neutral. A Transformer, a recurrent network, a symbolic engine, or a hybrid agent may implement an ORACLE learner. The issue is not whether its internal tensors literally contain objects and arrows. The issue is whether the learning system satisfies the identification, invariance, persistence, and repair obligations that the categorical theory declares.
There is direct evidence that this distinction is substantive rather than philosophical. Liu et al. [ 2023 ] use Krohn–Rhodes theory to show that shallow Transformers can exploit the algebraic structure of finite automata, compiling recurrent transitions into parallel shortcut computations. This is a genuine compositional achievement. It also sharpens the open question: efficiently executing a machine whose structure can be decomposed is not yet the same as discovering that decomposition from an open-ended presentation or preserving it through later revision. We return to this representation–identification gap in Section 5.7.1.
Let \(P\) be a finite presentation and \(\mathcal Q_0\) a tested query family. A learner has compositional performance on \((P,\mathcal Q_0)\) when it returns acceptable answers to the queries in \(\mathcal Q_0\), including queries whose answers require composing fragments present in \(P\).
This definition deliberately includes the achievements of frontier models. It does not require the learner to expose a symbolic derivation, nor does it infer from success what representation the learner uses.
A learner compositionally identifies a presented world, relative to a doctrine \(\mathbb T\), observer family \(\mathcal O\), equivalence class \(\mathcal E\), and query language \(\mathcal Q\), when its limiting hypothesis determines the \(\mathcal Q\)-theory of the world up to \(\mathcal E\), remains invariant under admissible re-presentations, and transports settled answers through compatible continuations and declared repairs.
Compositional performance is therefore local to a presentation and a test. Compositional identification is a claim about the world class selected by an open-ended evidence process. Neither definition entails that the learner must recover a unique hidden implementation. Identification is only up to the equivalences and observers declared in advance.
Suppose two admissible worlds \(M,N\in \mathsf H_{\mathbb T}\) induce the same observer-visible answers on a finite presentation \(P\) and tested query family \(\mathcal Q_0\), but disagree on some admissible continuation or query \(q\in \mathcal Q\). Then success on \((P,\mathcal Q_0)\) cannot establish that a learner has identified \(M\) rather than \(N\).
The learner receives identical observer-visible evidence and is judged by identical answers in the two cases. Consequently every output distribution and every score determined only by \((P,\mathcal Q_0)\) is the same whether the underlying world is \(M\) or \(N\). The distinguishing query \(q\), or evidence from a continuation on which \(M\) and \(N\) separate, is absent. Finite test success therefore witnesses performance on the common observable fragment, not identification of either world.
The proposition is not a special criticism of Transformers. It is an information-theoretic warning about every learner, including a categorical one. It prevents the book from treating fluent output as proof of an identified categorical model, while equally preventing it from treating the lack of an explicit symbolic trace as proof that no such model has been acquired.