5.7 Language learning inside grammar and sequence doctrines

Grammar doctrine.

Gold’s identification-in-the-limit paradigm is an especially crisp UOCL specialization. The learner begins with a class of possible languages or grammars, receives a text or informant, and must eventually stabilize on the target language according to a declared equivalence [ Gold , 1967 ] . Composition enters through concatenation, parsing, syntactic substitution, translation, or coalgebraic generation, depending on the grammar doctrine. The alphabet, grammar formalism, hypothesis enumeration, presentation type, and criterion of limiting stabilization are all fixed before the learner begins. Gold’s impossibility results show precisely how identification changes when that doctrinal declaration changes.

Sequence-model doctrine.

A Transformer supplies a powerful realizer for a different, usually weaker, query doctrine. Given token prefixes, it estimates conditional continuation laws and generates completions [ Vaswani et al. , 2017 ] . We may therefore place its hypotheses in a category \(\mathbf{SeqMod}_{\Sigma }\) of parameterized sequence models and compare them by maps that preserve selected continuation behavior. Pretraining becomes online or batch assimilation of a large language presentation.

This doctrine assumes a tokenization, a category of finite prefixes and their extensions, an autoregressive factorization of sequence probability, a positional or equivariant treatment of order, and a shared parameterized conditional law. Likelihood optimization and attention provide the internal solution construction. Next-token and completion probabilities provide the dominant probes. These assumptions are extraordinarily broad compared with a finite grammar class, but they are still a doctrine rather than an absence of one.

This makes the user’s phrase “a category of possible languages” precise, but with an important caveat. The Transformer architecture does not by itself declare grammar morphisms, identify a unique language-generating coalgebra, or guarantee conservative persistence after parameter updates. It realizes a query-relative UOCL learner when those structures and its predictive quotient are supplied. Fluent generation demonstrates extraordinary competence on that quotient without settling whether the underlying compositional world has been explanatorily identified.

Doctrinal boundary.

Changing weights while preserving the token, prefix, and continuation interfaces is internal learning. Revising the tokenizer, memory structure, grammar ontology, multimodal grounding maps, or the claim that continuation prediction is the adequate query family changes the doctrine. Scaling a realizer within the old declaration does not by itself perform that repair.