Expectations of Sums of Random Variables

Random variables, which arise as, or may be represented as, sums of other random variables, play an important role in probability theory. In this section we obtain formulas for the mean, mean square, variance, and moment-generating function of a sum of random variables.

Let be jointly distributed random variables. Using the linearity properties of the expecration operation, we immediately obtain the following formulas for the mean, mean square, and variance of the sum:

Equations (4.2) and (4.3) follow from the facts

Equation (4.3) simplifies considerably if the random variables , are uncorrelated (by which is meant that for every ). Then the variance of the sum of the random variables is equal to the sum of the variances of the random variables; in symbols,

If the random variables are independent , then we may give a formula for the moment-generating function of their sum; for any real number

In words, the moment-generating function of the sum of independent random variables is equal to the product of their moment-generating functions . The importance of the moment-generating function in probability theory derives as much from the fact that (4.7) holds as from the fact that the moment-generating function may be used to compute moments. The proof of (4.7) follows immediately, once we rewrite (4.7) explicitly in terms of expectations:

Equations (4.1)-(4.3) are useful for finding the mean and variance of a random variable (without knowing the probability law of ) if one can represent as a sum of random variables , the mean, variances, and covariances of which are known.

Example 4A. A binomial random variable as a sum . The number of successes in independent repeated Bernoulli trials with probability of success at each trial is a random variable. Let us denote it by . It has been shown that obeys a binomial probability law with parameters and . Consequently,

We now show that (4.8) is an immediate consequence of (4.1), (4.6), and (4.7). Define random variables by or 0 , depending on whether the outcome of the th trial is a success or a failure. One may verify that (i) ; (ii) are independent random variables; (iii) for is a Bernoulli random variable, with mean , variance , and moment-generating function . The desired conclusion may now be inferred.

Example 4B. A hypergeometric random variable as a sum. The number of white balls drawn in a sample of size drawn without replacement from an urn containing balls, of which are white, is a random variable. Let us denote it by . It has been shown that obeys a hypergeometric probability law. Consequently,

We now show that (4.9) can be derived by means of (4.1) and (4.3), without knowing the probability law of . Define random variables or 0, depending on whether a white ball is or is not drawn on the th draw. Verify that (i) ; (ii) for is a Bernoulli random variable, with mean and . However, the random variables are not independent, and we need to compute their product moments and covariances for any . Now, , so that is equal to the probability that the balls drawn on the th and th draws are both white, which is equal to . Therefore,

Consequently, The desired conclusions may now be inferred.

Example 4C. The number of occupied urns as a sum. If distinguishable balls are distributed into distinguishable urns in such a way that each ball is equally likely to go into any urn, what is the expected number of occupied urns?

Solution

For let or 0, depending on whether the th urn is or is not occupied. Then is the number of occupied urns, and the expected number of occupied urns. The probability that a given urn will be occupied is equal to . Therefore, and .

Theoretical Exercises

4.1. Waiting times in coupon collecting . Assume that each pack of cigarettes of a certain brand contains one of a set of cards and that these cards are distributed among the packs at random (assume that the number of packsavailable is infinite). Let be the minimum number of packs that must be purchased in order to obtain a complete set of cards. Show that , which may be evaluated by using the formula (see . Cramér, Mathematical Methods of Statistics , Princeton University Press, 1946, p. 125)

in which . Verify that if . Hint : For let be the number of packs that must be purchased after distinct cards have been collected in order to collect the st distinct card. Show that by using the fact that has a geometric distribution.

4.2 . Continuation of (4.1) . For let be the minimum number of packs that must be purchased in order to obtain different cards. Show that

Show that approximately (for large )

Show further that the moment-generating function of is given by

4.3. Continuation of (4.1) . For preassigned cards let be the minimum number of packs that must be purchased in order to obtain all cards. Show that

4.4 . The mean and variance of the number of matches . Let be the number of matches obtained by distributing, 1 to an urn, balls, numbered 1 to , among urns, numbered 1 to . It was shown in theoretical exercise 3.3 of Chapter 5 that and . Show this, using the fact that , in which or 0, depending on whether the th urn does or does not contain ball number . Hint: Show that or , depending on whether or .

4.5 . Show that if are independent random variables with zero means and finite fourth moments, then the third and fourth moments of the sum are given by

If the random variables are independent and identically distributed as a random variable , then

4.6 . Let be a random sample of a random variable . Define the sample mean and the sample variance by

(i) Show that , in which . Hint : show that

(ii) Show that for .

Exercises

4.1 . Let , and be independent normally distributed random variables, each with mean 1 and variance 3. Find .

Answer

0.8413.

4.2 . Consider a sequence of independent repeated Bernoulli trials in which the probability of success on any trial is .

(i) Let be the number of trials required to achieve the th success. Find and .

Hint : Write as a sum, , in which is the number of trials between the st and th successes. The random variables are independent and identically distributed.

(ii) Let be the number of failures encountered before the th success is achieved. Find and .

4.3 . A fair coin is tossed times. Let be the number of times in the tosses that a tail is followed by a head. Show that and . Find Var .

4.4 . A man with keys wants to open his door. He tries the keys independently and at random. Let be the number of trials required to open the door. Find and if (i) unsuccessful keys are not eliminated from further selections, (ii) if they are. Assume that exactly one of the keys can open the door.

In exercises 4.5 and 4.6 consider an item of equipment that is composed by assembling in a straight line 4 components of lengths , and , respectively. Let .

4.5 . Assume for .

(i) Find the mean and variance of the length of the item if , and are uncorrelated.

(ii) Find the mean and variance of if for .

Answer

. (i) ; (ii) .

4.6 . Assume that for . Find the ratio , called the measurement signal-to-noise ratio of the length (see section 6), for both cases considered in exercise 4.5.

Expectations of Sums of Random Variables

Theoretical Exercises

Exercises

Quick Links

Social Media