6.6 Novelty, information, and surprise

An information-theoretic tradition asks how unexpected an event or description is relative to a probability model. Palm broadens this perspective from Shannon’s partitions to repertoires and arbitrary covers, separating related notions of novelty, information, and surprise [ Palm , 2012 ] . This is valuable for creativity because real descriptions frequently overlap: an artifact may belong simultaneously to several styles, mechanisms, or explanatory classes.

Barto, Mirolli, and Baldassarre sharpen a distinction that is often blurred in computational and behavioral accounts [ Barto et al. , 2013 ] . Novelty is primarily relative to memory or a repertoire: has an item, representation, or relation been encountered before? Surprise is relative to a predictor: does an observation violate an expectation? The two may co-occur, but a familiar event can be surprising in a new context, while a novel event need not violate a registered prediction. Both can function as intrinsic motivations for exploration and learning.

This distinction also clarifies the Piagetian analysis. Surprise is a natural signal of disequilibrium, but not an automatic trigger for accommodation: the learner may assimilate the observation by repairing parameters or expectations within its current representation. Novelty may likewise enlarge a stored repertoire without changing its organizing concepts. A candidate extension is warranted only when persistent novelty or surprise exposes an inadequacy that cannot be repaired within the current representational language. The proposed change must still pass independent admission.

The connection to this book is richer than assigning a scalar novelty score. A cover resembles a family of partial observational contexts. One may ask not only whether a candidate is rare, but in which repertoires it is novel, which contexts distinguish it, and whether local descriptions agree on overlaps. This opens a path from information-theoretic novelty to sheaf-like contextual analysis.

Yet surprise and creativity must remain distinct. An outlier can be highly surprising and scientifically worthless; a powerful abstraction may appear simple once expressed in the right language. Moreover, the probability model used to score surprise is itself part of the inherited conceptual space. A transformational contribution may change the variables, descriptions, or sample space on which that score was defined.

. Information theory can quantify novelty or surprise relative to a registered repertoire. It does not, by itself, construct a new ontology, determine which structural promise failed, or certify that a proposed theory extension is explanatory. Those require typed representation and domain-specific admission.