Statistical Estimation

the aggregate of the methods used in mathematical statistics for the approximate determination of unknown probability distributions (or of quantities characterizing such distributions) from the results of observations.

In the most widely encountered case, the observations are independent, and their results form a sequence

(1) x₁, x₂,..., x_n,...

of independent random variables or vectors having the same unknown probability distribution with distribution function F(x). It frequently is assumed that the function F(x) depends in an unknown manner on one or more parameters and that only the values of these parameters are subject to determination. For example, an important part of the theory, especially in the multivariate case, was developed on the assumption that the unknown distribution is a normal distribution all or some of whose parameters are unknown (see).

The two main types of estimation procedures are point estimation and interval estimation. In point estimation, a function of the observation results is chosen to approximate the unknown parameter. In interval estimation, a range of values is indicated within which the unknown value of the parameter is likely to lie. In more general cases, such confidence intervals are replaced by more complicated confidence sets.

The estimation of the distribution function F(x) is discussed in NONPARAMETRIC METHODS, and the estimation of parameters is treated in ESTIMATOR.

Methods of statistical estimation have also been worked out for the case in which the observation results (1) are dependent and for the case where the subscript n is replaced by a continuously varying argument t—that is, for stochastic processes. In particular, extensive use is made of the estimation of quantities that characterize stochastic processes, for example, the correlation function and the spectral function. A new method of estimation known as stochastic approximation has been developed in connection with problems in regression analysis.

The classification and comparison of estimation methods are based on a number of principles, such as consistency, unbiased-ness, and invariance, that are considered in their most general form in the statistical decision theory.