0.3.1 Two small examples of compositional compression

0.3.1 Two small examples of compositional compression

Two elementary examples make this principle concrete before we turn to the larger developmental worlds.

LEGO: learn the construction theory, not every construction.

Suppose a child encounters a small collection of LEGO pieces. The visible configurations proliferate rapidly: two pieces can be attached in several ways, assemblies can be joined to other assemblies, and a structure can be taken apart and rebuilt along a different sequence. Memorizing every finished configuration would miss the reusable knowledge. A more compact learner discovers pieces and interfaces as types, attachment and separation as generating operations, and relations saying when distinct building histories produce the same assembly.

This is already an ORACLE problem. The learner receives only a growing presentation of particular pieces, attempted joins, successful separations, and observed equivalences. Its hypothesis is a generative theory whose composites describe assemblies not yet encountered. Joining is generally typed and partial—not every pair of pieces connects at every interface—so the lesson is richer than an unstructured binary operation. Assimilation expresses a new model using the existing operations. Accommodation adds an interface type, restricts a purported join, or revises a relation when a construction fails.

Diversity: learn tests, not hidden states.

Rivest and Schapire’s diversity representation gives the same compression principle a predictive form [ Rivest and Schapire , 1994 ] . For a finite-state system, an experiment consists of an action sequence followed by an observable test. Two experiments are identified when they return the same answer in every state. The learner can therefore represent the system by the resulting classes of tests and by the way actions transform those classes, rather than by enumerating or naming every hidden state.

Categorically, actions act on tests by precomposition, and observational equivalence forms a quotient of the generated test family. A compact, separating quotient answers every admitted predictive question even when the underlying state space is enormous. The qualification admitted is essential: tests sufficient for prediction may omit distinctions needed for control, causality, or safety. Chapter 5 develops the construction formally; here it supplies a simple moral. ORACLE should learn generators, transformation rules, and a query-relative quotient before it tries to reconstruct an exhaustive world state.