Limit Theorems

in probability theory, a group of theorems that give the conditions governing the appearance of specific regularities as a result of the action of a large number of random factors.

Historically, the first limit theorems were Bernoulli’s theorem, which was set forth in 1713, and the Laplace theorem, which was published in 1812. These theorems are concerned with the distribution of the deviations of the frequency of occurrence of an event E in n independent trials from the event’s probability p, where 0 < p < 1. The frequency is defined as the ratio m/n, where m is the number of occurrences of E in n trials. In 1837, S. Poisson extended the theorems to the case where the probability pk of the occurrence of E in the kth trial may depend on k; he described the limiting behavior, as n → ∞, of the distribution of the deviations of the frequency m/n from the arithmetic mean p̄ of the probabilities pk (1 ≤ k ≤ n):

If X_k denotes a random variable whose value is equal to unity when E occurs in the kth trial and is equal to zero when E does not occur, then m can be represented as the sum

m = X₁ + X₂ + ··· + X_n

which permits the above theorems to be treated as special cases of general limit theorems dealing with sums of independent random variables—the law of large numbers and the central limit theorem.

Law of large numbers. Suppose

(*) X₁, X₂, …, X_n, …

is a sequence of independent random variables and s_n is the sum of the first n of the variables:

s_n = X₁ + X₂ + · · · + X_n

Let A_n and B_n² be respectively the mathematical expectation

A_n = E s_n = EX₁ + EX₂ + · · · + EX_n

and the variance

of the sum s_n. The sequence (*) is said to obey the law of large numbers if for any ∊ > 0 the probability of the inequality

approaches zero as n → ∞.

Broad conditions for the applicability of the law of large numbers were first obtained by P. L. Chebyshev in 1867. The conditions were subsequently generalized by A. A. Markov (the elder). The problem of the necessary and sufficient conditions for the applicability of the law of large numbers was definitively solved by A. N. Kolmogorov in 1928. When the X_n have a common distribution function, these conditions, as shown by A. Ia. Khinchin in 1929, reduce to a single condition: the X_n must have finite mathematical expectations.

Central limit theorem. The central limit theorem is applicable to the sequence (*) if for any z₁ and z₂ the limit of the probability of the inequality

Z₁B_n < s_n - A_n < z₂B_n

as n → ∞ is the quantity

Fairly general sufficient conditions for the applicability of the central limit theorem were given by Chebyshev in 1887. Gaps, however, were found in his proof that were filled only later by Markov, in 1898. An almost definitive solution of the problem was obtained by A. M. Liapunov in 1901. A precise formulation of the Liapunov theorem is as follows. Let

c_k = E ǀX_k – EX_kǀ ^2+δ δ > 0

C_n = c₁ + c₂ + ··· + c_n

If the ratio L_n = C_n/B_n^2+δ approaches zero as n → ∞, then the central limit theorem is applicable to the sequence (*). A definite solution of the problem of the conditions of applicability of the central limit theorem was obtained in its general outlines by S. N. Bernshtein in 1926 and completed by W. Feller in 1935.

Other trends in the study of limit theorems include the following.

(1) Investigations were begun by Markov and continued by Bernshtein and others on the conditions governing the applicability of the law of large numbers and the central limit theorem to sums of dependent variables.

(2) Even in the case of a sequence of identically distributed random variables simple examples can be given where the sums have a limiting distribution different from a normal distribution (nondegenerate distributions are meant here—that is, distributions that are not concentrated entirely at one point). In the papers of such mathematicians as the Soviets A. Ia. Khinchin and B. V. Gnedenko and the Frenchmen P. Lévy and W. Dublin, a complete study has been made of both the class of possible limiting distributions for sums of independent random variables and the conditions for convergence of the distributions of the sums to specific limiting distributions.

(3) Considerable attention has been devoted to local limit theorems. For example, let the variables X_n take on only integral values. Then the sums s_n also assume only integral values, and the question naturally arises of the limiting behavior of the probabilities P_n(m) that s_n = m, where m is an integer. A simple example of a local limit theorem is the Laplace theorem.

(4) Limit theorems in their classical formulation describe the behavior of individual sums s_n as n increases. Sufficiently general limit theorems for the probabilities of events that depend simultaneously on several sums were obtained for the first time by Kolmogorov in 1931. It follows from his results, for example, that under very broad conditions the limit of the probability of the inequality

is the quantity

(5) The above limit theorems are concerned with sums of random variables. Limit theorems of a different kind are exemplified by limit theorems for the terms of an ordered series. Such theorems have been studied in detail by the Soviet mathematicians B. V. Gnedenko and N. V. Smirnov.

(6) Finally, also classed with limit theorems are theorems that establish for sequences of random variables properties occurring with a probability equal to unity. An example is the law of iterated logarithm.