sec-tangent-structure-language

0.11 Tangent structure and infinitesimal language

LINCS also asks how a compositional failure changes under small, admissible perturbations. On smooth spaces, a tangent vector \(v\in T_xX\) represents a first-order direction at \(x\). A smooth map \(f:X\to Y\) transports it by

\[ Tf(x,v)=\bigl(f(x),Df(x)[v]\bigr). \]

The chain rule makes \(T\) functorial. A tangent category abstracts this behavior beyond ordinary smooth manifolds [ Cockett and Cruttwell , 2014b ] .

Example 0.17 An equivariant encoder and its tangent

Let \(a_g:X\to X\) transform an input, let \(F_\theta :X\to Y\) be a learned encoder, and let \(b_g:Y\to Y\) be the corresponding action on its representation. Equivariance is the declaration that the square

Commutative diagram illustrating 0.11 Tangent structure and infinitesimal language.
\[ F_\theta \circ a_g=b_g\circ F_\theta \]

commutes. For an image encoder, \(a_g\) might rotate the image and \(b_g\) might rotate or reindex its feature representation. The sketch records this promise before any norm is chosen to measure its failure.

Applying \(T\) transports the entire declaration:

\[ TF_\theta \circ Ta_g = Tb_g\circ TF_\theta . \]

In ordinary coordinates, its directional part is

\[ DF_\theta (a_gx)\, Da_g(x)v = Db_g(F_\theta (x))\, DF_\theta (x)v. \]

The base square asks whether the two routes agree at \(x\). The tangent square asks whether their first-order responses agree around \(x\). A model can therefore fit sampled transformed inputs while remaining locally unstable to nearby perturbations.

An LLM encoder or hidden-state map can occupy the role of \(F_\theta \), but the transformation must be declared carefully. A self-attention block with no positional or causal structure is permutation equivariant; a standard causal LLM with positional information and a causal mask is not equivariant under arbitrary token permutations. Appropriate declarations may instead concern a controlled paraphrase, a formatting transformation, or a simultaneous reindexing of token representations and their structural data. The output action \(b_g\) may be the identity for a decision that should be invariant, or an explicit transport for token-level states. In either case, equivariance is a typed, testable promise, not an automatic property of the language model.

Design principle

A tangent category packages the chain rule so that LINCS can differentiate a compositional declaration, not merely the individual functions appearing in it.

Two constructions that look similar must be kept distinct. Write

\[ T_nX=\underbrace{TX\times _X\cdots \times _XTX}_{n\text{ factors}} \]

for the fiber power of tangent vectors based at the same point, whereas \(T^nX=T(T(\cdots T(X)))\) denotes iteration of the tangent functor. Thus \(T_2X=TX\times _XTX\) is the domain of fiberwise addition, while \(T^2X\) contains tangent vectors to \(TX\).

Definition 0.18 Tangent category

A tangent structure on a category \(\mathcal C\) is a tuple

\[ \mathbb T=(T,p,0,+,\ell ,c) \]

with the following data and axioms.

  1. \(T:\mathcal C\to \mathcal C\) is an endofunctor and \(p:T\Rightarrow 1_{\mathcal C}\) is a natural projection. Every finite fiber power \(T_nX\) of \(p_X:TX\to X\) exists, and every iterate \(T^m\) preserves these pullbacks.

  2. Natural transformations

    \[ 0:1_{\mathcal C}\Rightarrow T,\qquad +:T_2\Rightarrow T \]

    make \(p_X:TX\to X\) a commutative monoid object in the slice \(\mathcal C/X\). Hence only tangent vectors over the same base point may be added.

  3. The vertical lift \(\ell :T\Rightarrow T^2\), together with the zero section, is an additive-bundle morphism from \(p_X:TX\to X\) to \(T(p_X):T^2X\to TX\).

  4. The canonical flip \(c:T^2\Rightarrow T^2\) is an additive-bundle morphism from \(T(p_X):T^2X\to TX\) to \(p_{TX}:T^2X\to TX\).

  5. The lift and flip satisfy the tangent coherence axioms: in particular

    \[ c\circ c=1_{T^2},\qquad c\circ \ell =\ell . \]

    The standard iterated-lift and braid/interchange diagrams involving \(\ell ,c,T(\ell )\), and \(T(c)\) also commute.

  6. Let \(\pi _0,\pi _1:T_2X\rightrightarrows TX\) be the fiber-power projections and define the derived lift

    \[ \nu _X= T(+_X)\circ \left\langle \ell _X\circ \pi _0,\, 0_{TX}\circ \pi _1 \right\rangle :T_2X\longrightarrow T^2X. \]

    The vertical-lift universality axiom requires the square

    Commutative diagram illustrating 0.11 Tangent structure and infinitesimal language.

    to be a pullback. Equivalently, a second-order tangent that is vertical in the required sense factors uniquely through the derived lift.

A tangent category is a pair \((\mathcal C,\mathbb T)\) of a category and a chosen tangent structure on it.

The last two clauses abbreviate the standard coherence diagrams and universality square of Cockett and Cruttwell [ Cockett and Cruttwell , 2014b , Definition 2.3 ] ; they are axioms, not consequences of the first four pieces of data. In local coordinates on a smooth manifold, the structure has the familiar form

\[ \begin{aligned} p(x,v)& =x, & 0(x)& =(x,0),\\ +(x,v,w)& =(x,v+w),\qquad & \ell (x,v)& =(x,0;0,v),\\ c(x,v;\dot x,\dot v)& =(x,\dot x;v,\dot v).& & \end{aligned} \]

The flip exchanges the two first-order directions in a double tangent, while the vertical lift embeds a tangent vector as a vertical second-order direction.

Boundary

An endofunctor called \(T\), or the availability of automatic differentiation, does not by itself make a category tangent. The fiber pullbacks, additive bundles, lift, flip, coherence diagrams, and vertical universality property are all part of the declaration. A LINCS application using only some of this structure should state that weaker assumption rather than silently claiming a tangent category.

There is a second boundary that is just as important for DLINCS. The category of finite-dimensional smooth manifolds and smooth maps is a tangent category, but it is not Cartesian closed. It has products, so a smooth family such as a motion of a body can be written

\[ q:T\times B\longrightarrow E. \]

Cartesian closedness would additionally provide an exponential object \(E^T\) and the equivalent curried description

\[ \widehat q:B\longrightarrow E^T. \]

The smooth path space \(E^T\) is generally infinite-dimensional and is not an object of the ordinary category of finite-dimensional manifolds. Thus tangent structure supplies differentiation, but it does not by itself supply internal function spaces [ Lawvere , 1980 ] .

The Kock–Lawvere solution is to embed ordinary smooth spaces into a suitable well-adapted smooth topos \(\mathcal E\) [ Kock , 2006 , Moerdijk and Reyes , 1991 ] . Because a topos is Cartesian closed, it admits the natural correspondence

\[ \mathcal E(P\times X,Y) \; \cong \; \mathcal E(P,Y^X). \]

A parametrized model \(I:P\times X\to Y\) can therefore be treated internally as a map \(\widehat I:P\to Y^X\) into a genuine model-space object. In the same setting an infinitesimal object \(D\) represents tangent structure on the appropriate infinitesimally linear or microlinear objects by \(TX=X^D\). Cartesian closure and tangent structure remain logically distinct requirements; the smooth-topos setting is valuable because it supports them compatibly.

Design principle

Ordinary smooth manifolds motivate the tangent calculus of LINCS, but the internal higher-order semantics of DLINCS lives in a Cartesian-closed smooth topos. Products support parameterized evaluation; exponentials make spaces of models into objects of the theory.

This passage to a topos changes more than the available collection of smooth spaces. It also gives the theory an internal, generally intuitionistic logic. That logical consequence is the subject of Section 0.12.