The Central Limit Theorem

A sequence of jointly distributed random variables with finite means and variances is said to obey the (classical) central limit theorem if the sequence , defined by

converges in distribution to a random variable that is normally distributed with mean 0 and variance 1. In terms of characteristic functions, the sequence obeys the central limit theorem if for every real number

The random variables are called the sequence of normalized consecutive sums of the sequence .

That the central limit theorem is true under fairly unrestrictive conditions on the random variables was already surmised by Laplace and Gauss in the early 1800’s. However, the first satisfactory conditions, backed by a rigorous proof, for the validity of the central limit theorem were given by Lyapunov in 1901. In the 1920’s and 1930’s the method of characteristic functions was used to extend the theorem in several directions and to obtain fairly unrestrictive necessary and sufficient conditions for its validity in the case in which the random variables are independent. More recent years have seen extensive work on extending the central limit theorem to the case of dependent random variables.

The reader is referred to the treatises of B. V. Gnedenko and A. N. Kolmogorov, Limit Distributions for Sums of Independent Random Variables , Addison-Wesley, Cambridge, Mass., 1954, and M. Loève, Probability Theory , Van Nostrand, New York, 1955, for a definitive treatment of the central limit theorem and its extensions.

From the point of view of the applications of probability theory, there are two main versions of the central limit theorem that one should have at his command. One should know conditions for the validity of the central limit theorem in the cases in which (i) the random variables are independent and identically distributed and (ii) the random variables are independent but not identically distributed.

Equation (4.4) is called Lyapunov’s condition for the validity of the central limit theorem for independent random variables .

We turn now to the proofs of theorems 4A and 4B. Consider first independent random variables , identically distributed as the random variable , with mean 0 and variance . Let be their normalized sum, given by (4.1). The characteristic function of may be written

Now tends to as tends to . Therefore, for each fixed exists (by lemma 3A) when . For as large as this, using the expansion given by (3.8),

Theorem 4A will be proved if we prove that

It is clear that to prove (4.7) will hold we need prove only that the integral in (4.6) tends to 0 as tends to infinity. Define . Then, for any

Now, for and for , in view of the inequalities . From these facts we may conclude that for any and real numbers and

Then

which tends to 0, as we let first tend to and then tend to . The proof of the central limit theorem for identically distributed independent random variables with finite variances is complete.

We next prove the central limit theorem under Lyapunov’s condition. For , let be a random variable with mean 0, finite variance , and th central moment . We have the following expansion of the logarithm of its characteristic function, for such that :

To prove (4.11), merely use in (3.8) the inequality , valid for any real number and .

Now, (4.4) and theoretical exercise 4.3 imply that

Then, for any fixed it holds for sufficiently large that for all . Therefore, exists and is given by

The first sum in (4.13) is equal to 1, whereas the second sum tends to 0 by Lyapunov’s condition, as does the third sum, since

The proof of the central limit theorem under Lyapunov’s condition is complete.

Theoretical exercises