22.1 Compositionality and compression

One influential tradition identifies learning with compression. In its minimum-description-length form, a learner selects a hypothesis \(H\) by balancing the code for the hypothesis against the code for the data given the hypothesis:

Algorithmic information theory idealizes this intuition through Kolmogorov complexity, while practical MDL makes the coding scheme and model class explicit [ Rissanen , 1978 , Grünwald , 2007 , Li and Vitányi , 2019 ] . This is a powerful account of regularity and model selection. It is not, however, the primitive adopted by LINCS.

LINCS begins with a sketch \(\mathbb S\), a realized diagram \(D\), and the typed obstruction

The obstruction records which declared path, cone, cocone, restriction, or factorization failed. It need not assign a total order to candidate models, and it need not live in a numerical space. Description length instead assigns a scalar after choosing a representation and code. The two constructions therefore operate at different logical levels: one diagnoses failure of a declared relationship; the other compares the economy of descriptions.

Neither property implies the other. A short program can generate data while systematically violating a safety constraint, a causal intervention law, or an argument’s source restrictions. Such a program compresses well but remains non-compositional relative to the relevant sketch. In the other direction, a compositional system may need a long description because its certificate retains local evidence, provenance, uncertainty, rebuttals, or exceptional contexts. Those fields are not waste merely because deleting them shortens the code.

There is nevertheless a close and useful interaction. Shared morphisms, repeated diagrams, reusable local sections, and stable restriction maps can all support shorter descriptions. Compositionality can therefore be a source of compression. But even perfect compression of the observed sample does not identify which compositions ought to hold, which presentation changes are null, or which repair preserves the intended semantics. Those choices belong to the declaration, quotient, and admission contract.

This distinction also clarifies the role of coding dependence. Kolmogorov complexity is invariant only up to a machine-dependent additive constant and is not computable in general; operational MDL replaces it with an explicit code and model family. LINCS faces an analogous but typed presentation issue: before comparing lengths, it must state which encodings represent the same decision-relevant object. A quotient \(\chi \) can remove declared presentation redundancy, after which a description-length functional

may compare candidate repairs \(R\) on the quotient. The quotient must precede the comparison; otherwise a coding convention can reward or punish variation that the application has already declared meaningless.

The Kimi K3 case study in Chapter 12 makes the distinction operational. Latent expert representations, sparse activation, recurrent sequence state, block summaries over depth, and low-precision weights all reduce some computational or representational burden. Yet each mechanism is admitted only relative to a different preservation question: routing balance, global retrieval, information flow across depth, task behavior, or train–serve consistency [ Kimi Team , 2026 ] . Compression is pervasive in the system, but the architecture specifies what each compression is allowed to forget.

This suggests a genuine research program rather than a rivalry between frameworks. One can ask when code lengths are compatible with restriction and gluing, when local compression bounds assemble globally, whether an admitted repair is minimal among repairs of the same type, and how much description is required by the certificate itself. In this role, compression is an observer of a LINCS system—and sometimes a valuable search principle—but not its ontology of learning.