In statistics, completeness is a property of a statistic for which the statistic optimally obtains information about the unknown parameters characterizing the distribution of the underlying data.
It is closely related to statistical sufficiency and often occurs in conjunction with it.
Mathematical definition
Suppose a random variable X (which may be a sequence (X1, …, Xn) of scalar-valued random variables), has a probability distribution belonging to a known family of probability distributions Pθ parametrized by θ. Let s(X) be any statistic based on X.
Then s(X) is a complete statistic if and only if
- E(g(s(X))) = 0 for all θ <math>\Rightarrow</math> g = 0 almost everywhere
and is boundedly complete if the implication holds for all bounded g.
Completeness of the family
It is not guaranteed that for a particular family of probabilities, a complete statistic will always exist. In contrast, a minimal sufficient statistic always exists.
In particular, if a complete statistic exists, then a statistic is complete if and only if it is minimal sufficient. Taking this fact into account, the family Pθ of distributions is called complete if and only if its minimal sufficient statistic is complete.
Heuristic interpretation
A complete statistic is similar to a sufficient statistic in that they both efficiently capture information about the parameter θ. Whereas a sufficient statistic captures as much information as possible from the data about θ, a complete statistic captures no additional information from the data which is not related to θ.
Examples
Sum of normals
Suppose (X1, X2) are independent, identically distributed random variables, normally distributed with expectation θ and variance 1.
The sum
- <math>s((X_1,\ X_2)) = X_1 + X_2\,\!</math>
is a complete statistic. To show this one demonstrates that there is no non-zero function <math>g</math> such that the expectation of
- <math>g(s(X_1,\ X_2)) = g(X_1+X_2)\,\!</math>
remains zero regardless of the value of θ.
That fact may be seen as follows. The probability distribution of X1 + X2 is normal with expectation 2θ and variance 2. Its probability density function in <math>x</math> is therefore proportional to
- <math>\exp\left(-(x-2\theta)^2/4\right).</math>
The expectation of g above would therefore be a constant times
- <math>\int_{-\infty}^\infty g(x)\exp\left(-(x-2\theta)^2/4\right)\,dx.</math>
A bit of algebra reduces this to
- <math>k(\theta) \int_{-\infty}^\infty h(x)e^{x\theta}\,dx\,\!</math>
where k(θ) is nowhere zero and
- <math>h(x)=g(x)e^{-x^2/4}.\,\!</math>
As a function of θ this is a two-sided Laplace transform of h(X), and cannot be identically zero unless h(x) is zero almost everywhere. The exponential is not zero, so this can only happen if g(x) is zero almost everywhere.
Counterexample 1
Again suppose (X1, X2) are independent, identically distributed random variables, normally distributed with expectation θ and variance 1.
Then
- <math>g((X_1,\ X_2)) = X_1 - X_2\,\!</math>
is an unbiased estimator of zero. Therefore the pair (X1, X2) itself is not a complete statistic (though it is a sufficient statistic).
Counterexample 2
Let U follow Uniform[-½,½]. Let X = U + θ, so that the distribution of X is parametrized by the mean θ = E(X).
Then if g(x) = sin(2πx), then E(g(X)) = 0 irrespective of θ. Therefore X itself is not a complete statistic for θ.
Utility
Lehmann-Scheffé theorem
The major importance of completeness is in the application of the Lehmann-Scheffé theorem,
which states that a statistic that is unbiased, complete and sufficient for some parameter θ is the best estimator for θ, i.e., the one that has a smaller expected loss for any convex loss function (in typical practice, a smaller mean squared error) among any estimators with the same expected value.
Basu’s theorem
Completeness is also a prerequisite for the applicability of Basu’s theorem: A statistic which is both complete and sufficient is independent of any ancillary statistic (one independent of the parameters θ).