lin-0074

5.2 Statistical tangent semantics

Let \(\mathcal M=\{ P_\theta :\theta \in \Theta \} \) be a regular \(q\)-dimensional statistical model. For

\[ a=\sum _{k=1}^{q}a^k\partial _k\in T_\theta \mathcal M, \]

the score representation is

\[ s_a(x)=\sum _{k=1}^{q}a^k\partial _k\log p_\theta (x). \]

When the Fisher information is positive definite, the score map identifies \(T_\theta \mathcal M\) with the finite-dimensional score span

\[ \mathcal T_\theta = \operatorname {span} \{ \partial _1\log p_\theta ,\ldots ,\partial _q\log p_\theta \} \subset L^2_0(P_\theta ). \]

It does not identify the tangent space with all of \(L^2_0(P_\theta )\).

A deterministic sufficient statistic \(t:X\to S\) provides a useful link to the Markov description. Because \(t\) is deterministic, it respects copying:

\[ \Delta _S t=(t\otimes t)\Delta _X. \]

For an exponential family,

\[ p_\theta (x) = h(x)\exp \{ \langle \theta ,T(x)\rangle -A(\theta )\} , \]

the sufficient statistic also generates the score directions,

\[ \partial _i\log p_\theta (x) = T_i(x)-\mathbb E_\theta [T_i(X)]. \]

Thus copyable summaries and statistical tangent directions are related, but they remain differently typed objects.

Example 5.1 Gaussian location

For \(P_\mu =\mathcal N(\mu ,\Sigma )\) with fixed positive-definite \(\Sigma \), the score of \(a\in T_\mu \mathbb R^d\) is

\[ s_a(x)=a^\top \Sigma ^{-1}(x-\mu ), \]

and the Fisher metric is \(g_F(a,b)=a^\top \Sigma ^{-1}b\). Translation protocols generate constant coordinate fields, so all their brackets vanish.

Example 5.2 Binomial family

For \(X\sim \operatorname {Binomial}(n,p)\), the statistic \(X\) is sufficient and

\[ \partial _p\log P_p(X) = \frac{X-np}{p(1-p)}. \]

This one-dimensional example has no nontrivial pairwise bracket test. It is a reminder that regular statistical tangent structure alone does not manufacture causal interactions.