Modes of Convergence of a Sequence of Random Variables

Consider a sequence of jointly distributed random variables , defined on the same probability space on which a probability function has been defined. Let be another random variable defined on the same probability space. The notion of the convergence of the sequence of random variables to the random variable can be defined in several ways.

We consider first the notion of convergence with probability one . We say that converges to with probability one if or, in words, if for almost all members of the probability space on which the random variables are defined . To prove that a sequence of random variables converges with probability one is often technically a difficult problem. Consequently, two other types of convergence of random variables, called, respectively, convergence in mean square and convergence in probability , have been introduced in probability theory. These modes of convergence are simpler to deal with than convergence with probability one and at the same time are conceptually similar to it.

The sequence is said to converge in mean square to the random variable , denoted if or, in words, if the mean square difference between and tends to 0.

The sequence is said to converge in probability to the random variable , denoted if for every positive number Equation (1.1) may be expressed in words: for any fixed difference the probability of the event that and differ by more than becomes arbitrarily close to 0 as tends to infinity.

Convergence in probability derives its importance from the fact that, like convergence with probability one, no moments need exist before it can be considered, as is the case with convergence in mean square. It is immediate that if convergence in mean square holds then so does convergence in probability; one need only consider the following form of Chebyshev’s inequality: for any

The relation that exists between convergence with probability one and convergence in probability is best understood by considering the following characterization of convergence with probability one, which we state without proof. Let be a sequence of jointly distributed random variables; converges to the random variable with probability one if and only if for every

On the other hand, the sequence converges to in probability if and only if for every (1.1) holds. Now, it is clear that if , then . Consequently,

and (1.3) implies (1.1). Thus, if converges to with probability one, it converges to in probability.

Convergence with probability one of the sequence to implies that one can make a probability statement simultaneously about all but a finite number of members of the sequence : given any positive numbers and , an integer exists such that

On the other hand, convergence in probability of the sequence to implies only that one can make simultaneous probability statements about each of all but a finite number of members of the sequence : given any positive numbers and an integer exists such that One thus sees that convergence in probability is implied by both convergence with probability one and by convergence in mean square. However, without additional conditions, convergence in probability implies neither convergence in mean square nor convergence with probability one. Further, convergence with probability one neither implies nor is implied by convergence in mean square.

The following theorem gives a condition under which convergence in mean square implies convergence with probability one.