Expectation, Mean, and Variance of Random Variable

Given the random variable , we define the expectation of the random variable, denoted by , as the mean of the probability law of ; in symbols, depending on whether is specified by its distribution function , its probability density function , or its probability mass function .

Given a random variable , which arises as a Borel function of a random variable so that for some Borel function , the expectation , in view of (1.1), is given by On the other hand, given the Borel function and the random variable , we can form the expectation of with respect to the probability law of , denoted by and defined by depending on whether is specified by its distribution function , its probability density function , or its probability mass function .

It is a striking fact, of great importance in probability theory, that for any random variable and Borel function

if either of these expectations exists . In words, (1.5) says that the expectation of the random variable is equal to the expectation of the function with respect to the random variable .

The validity of (1.5) is a direct consequence of the fact that the integrals used to define expectations are required to be absolutely convergent. ¹Some idea of the proof of (1.5), in the case that is continuous, can be gained. Partition the -axis in Fig. 1A into subintervals by points . Then approximately To each point on the -axis, there is a number of points , at which is equal to . Form the set of all such points on the -axis that correspond to the points . Arrange these points in increasing order, . These points divide the -axis into sub-intervals. Further, it is clear upon reflection that the last sum in (1.6) is equal to which completes our intuitive proof of (1.5). A rigorous proof of (1.5) cannot be attempted here, since a more careful treatment of the integration process does not lie within the scope of this book.

Figure 2.4.1 — **Fig. 1A.** With the aid of this graph of a possible function , one can see that (1.5) holds.

Given a random variable and a function , we thus find two distinct notions, represented by and , which nevertheless, are always numerically equal. It has become customary always to use the notation , since this notation is the most convenient for technical manipulation. However, the reader should be aware that although we write the concept in which we are really very often interested is , the expectation of the function with respect to the random variable . Thus, for example, the th moment of a random variable (for any integer ) is often defined as , the expectation of the th power of . From the point of view of the intuitive meaning of the th moment, however, it should be defined as the expectation of the function with respect to the probability law of the random variable . We shall define the moments of a random variable in terms of the notation of the expectation of a random variable. However, it should be borne in mind that we could define as well the moments of a random variable as the corresponding moments of the probability law of the random variable.

Given a random variable , we denote its mean by , its mean square by , its square mean by , its th moment about the point by , and its th central moment (that is, th moment about its mean) by . In particular, the variance of a random variable, denoted by Var , is defined as its second central moment, so that The standard deviation of a random variable, denoted by , is defined as the positive square root of its variance, so that The moment generating function of a random variable, denoted by , is defined for every real number by

It is shown in section 5 that if constitute a random sample of the random variable then the arithmetic mean is, for large , approximately equal to the mean . This fact has led early writers on probability theory to call the expected value of the random variable ; this terminology, however, is somewhat misleading, for if is the expected value of any random variable it is the expected value of the arithmetic mean of a random sample of the random variable.

Example 1A . he mean duration of the game of “odd man out.” The game of “odd man out” was described in example 3D of Chapter 3. On each independent play of the game, players independently toss fair coins. The game concludes when there is an odd man; that is, the game concludes the first time that exactly one of the coins falls heads or exactly one of the coins falls tails. Let be the number of plays required to conclude the game; more briefly, is called the duration of the game. Find the mean and standard deviation of .

Solution

It has been shown that the random variable obeys a geometric probability law with parameter . The mean of is then equal to the mean of the geometric probability law, so that . Similarly, . Thus, if , , and . The mean duration has the following interpretation; if are the durations of independent games of “odd man out”, then the average duration of the games is approximately equal to if the number of games is large. Note that in a game with five players the mean duration is not equal to an integer. Consequently, one will never observe a game whose duration is equal to the mean duration; nevertheless, the arithmetic mean of a large number of observed durations can be expected to be equal to the mean duration.

To find the mean and variance of the random variable , in the foregoing example we found the mean and variance of the probability law of . If a random variable can be represented as a Borel function of a random variable , one can find the mean and variance of without actually finding the probability law of . To do this, we make use of an extension of (1.5).

Let and be random variables such that for some Borel function . Then for any Borel function

in the sense that if either of these expectations exists then so does the other, and the two are equal.

To prove (1.11) we must prove that

The proof of (1.12) is beyond the scope of this book.

To illustrate the meaning of (1.11), we write it for the case in which the random variable is continuous and . Using the formula for the probability density function of , given by (8.8) of Chapter 7 , we have for any continuous function whereas One may verify directly that the integrals on the right-hand sides of (1.13) and (1.14) are equal, as asserted by (1.11) .

As one immediate consequence of (1.11), we have the following formula for the variance of a random variable , which arises as a function of another random variable:

Example 1B . The square of a normal random variable . Let be a normally distributed random variable with mean 0 and variance . Let . Then the mean and variance of are given by , .

If a random variable is known to be normally distributed with mean and variance , then for brevity one often writes is .

Example 1C . The logarithmic normal distribution . A random variable is said to have a logarithmic normal distribution if its logarithm is normally distributed. One may find the mean and variance of by finding the mean and variance of , in which is . Now is the value at of the moment-generating function of . Similarly . Since , it follows that and .

Example 1D shows how the mean (or the expectation) of a random variable is interpreted.

Example 1D . Disadvantageous or unfair bets. Roulette is played by spinning a ball on a circular wheel, which has been divided into thirty-seven arcs of equal length, bearing numbers from 0 to 36. ²Let denote the number of the arc on which the ball comes to rest. Assume each arc is equally likely to occur, so that the probability mass function of is given by for . Suppose that one is given even odds on a bet that the observed value of is an odd number; that is, on a 1 dollar bet one is paid 2 dollars (including one’s stake) if is odd, and one is paid nothing (so that one loses one’s stake) if is not odd. How much can one expect to win at roulette by consistently betting on an odd outcome?

Solution

Define a random variable as equal to the amount won by betting 1 dollar on an odd outcome at a play of the game of roulette. Then if is odd and if is not odd. Consequently, and . The mean of the random variable is then given by

The amount one can expect to win at roulette by betting on an odd outcome may be regarded as equal to the mean in the following sense. Let be one’s winnings in a succession of plays of roulette at which one has bet on an odd outcome. It is shown in section 5 that the average winnings in plays tends, as the number of plays becomes infinite, to . The fact that is equal to a negative number implies that betting on an odd outcome at roulette is disadvantageous (or unfair) for the bettor, since after a long series of plays he can expect to have lost money at a rate of 2.7 cents per dollar bet. Many games of chance are disadvantageous for the bettor in the sense that the mean winnings is negative. However, the mean (or expected) winnings describe just one aspect of what will occur in a long series of plays. For a gambler who is interested only in a modest increase in his fortune it is more important to know the probability that as a result of a series of bets on an odd outcome in roulette the size of his 1000-dollar fortune will increase to 1200 dollars before it decreases to zero. A home owner insures his home against destruction by fire, even though he is making a disadvantageous bet (in the sense that his expected money winnings are negative) because he is more concerned with making equal to zero the probability of a large loss.

Most random variables encountered in applications of probability theory have finite means and variances. However, random variables without finite means have long been encountered by physicists in connection with problems of return to equilibrium. The following example illustrates a random variable of this type that has infinite mean.

Example 1E . On long leads in fair games . Consider two players engaged in a friendly game of matching pennies with fair coins. The game is played as follows. One player tosses a coin, while the other player guesses the outcome, winning one cent if he guesses correctly and losing one cent if he guesses incorrectly. The two friends agree to stop playing the moment neither is winning. Let be the duration of the game; that is, is equal to the number of times coins are tossed before the players are even. Find , the mean duration of the game.

Solution

It is clear that the game of matching pennies with fair coins is not disadvantageous to either player in the sense that if is the winnings of a given player on any play of the game then . From this fact one may be led to the conclusion that the total winnings of a given player in plays will be equal to 0 in half the plays, over a very large number of plays. However, no such inference can be made. Indeed, consider the random variable , which represents the first trial at which . We now show that ; in words, the mean duration of the game of matching pennies is infinite. Note that this does not imply that the duration is infinite; it may be shown that there is probability one that in a finite number of plays the fortunes of the two players will equalize. To compute , we must compute its probability law. The duration of the game cannot be equal to an odd integer, since the fortunes will equalize if and only if each player has won on exactly half the tosses. We omit the computation of the probability that , for an even integer, and quote here the result (see W. Feller, An Introduction to Probability Theory and its Applications, second edition , Wiley, New York, 1957, p. 75):

The mean duration of the game is then given by

It may be shown, using Stirling’s formula, that

the sign indicating that the ratio of the two sides in (1.19) tends to 1 as tends to infinity. Consequently, for some constant . Therefore, the infinite series in (1.18) diverges, and .

To conclude this section, let us justify the fact that the integrals defining expectations are required to be absolutely convergent by showing, by example, that if the expectation of a continuous random variable is defined by

then it is not necessarily true that for any constant

Let be a random variable whose probability density function is an even function, that is, . Then, under the definition given by (1.20), the mean exists and equals 0, since for every . Now

Assuming , and letting , we may write

The first of these integrals vanishes, and the last tends to 1 as tends to . Consequently, to prove that if is defined by (1.20) one can find a random variable and a constant such that , it suffices to prove that one can find an even probability density function and a constant such that

An example of a continuous even probability density function satisfying (1.22) is the following. Letting , define

In words, vanishes, except for points , which lie within a distance 1 from a point that in absolute value is a perfect square . That is a probability density function follows from the fact that

That (1.22) holds for follows from the fact that for

Theoretical Exercises

1.1 . The mean and variance of a linear function of a random variable . Let be a random variable with finite mean and variance. Let and be real numbers. Show that

1.2 . Chebyshev’s inequality for random variables . Let be a random variable with finite mean and variance. Show that for any and any

Hint : if is continuous at these points.

1.3 . Continuation of example 1E. Using (1.17), show that .

Exercises

1.1 . Consider a gambler who is to win 1 dollar if a 6 appears when a fair die is tossed; otherwise he wins nothing. Find the mean and variance of his winnings.

Answer

Mean, ; variance, .

1.2 . Suppose that 0.008 is the probability of death within a year of a man aged 35. Find the mean and variance of the number of deaths within a year among 20,000 men of this age.

1.3 . Consider a man who buys a lottery ticket in a lottery that sells 100 tickets and that gives 4 prizes of 200 dollars, 10 prizes of 100 dollars, and 20 prizes of 10 dollars. How much should the man be willing to pay for a ticket in this lottery?

Answer

Mean winnings, 20 dollars.

1.4 . Would you pay 1 dollar to buy a ticket in a lottery that sells tickets and gives 1 prize of 100,000 dollars, 10 prizes of 10,000 dollars, and 100 prizes of 1000 dollars?

1.5 . Nine dimes and a silver dollar are in a red purse, and 10 dimes are in a black purse. Five coins are selected without replacement from the red purse and placed in the black purse. Then 5 coins are selected without replacement from the black purse and placed in the red purse. The amount of money in the red purse at the end of this experiment is a random variable. What is its mean and variance?

Answer

Mean, 1 dollar 60 cents; variance, 1800 cents .

1.6 . St.Petersburg problem (or paradox?) . How much would you be willing to pay to play the following game of chance. A fair coin is tossed by the player until heads appears. If heads appears on the first toss, the bank pays the player 1 dollar. If heads appears for the first time on the second throw the bank pays the player 2 dollars. If heads appears for the first time on the third throw the player receives dollars. In general, if heads appears for the first time on the th throw, the player receives dollars. The amount of money the player will win in this game is a random variable; find its mean. Would you be willing to pay this amount to play the game? (For a discussion of this problem and why it is sometimes called a paradox see T. C. Fry, Probability and Its Engineering Uses , Van Nostrand, New York, 1928, pp. 194–199.)

1.7 . The output of a certain manufacturer (it may be radio tubes, textiles, canned goods, etc.) is graded into 5 grades, labeled , and (in decreasing order of quality). The manufacturer’s profit, denoted by , on an item depends on the grade of the item, as indicated in the table. The grade of an item is random; however, the proportions of the manufacturer’s output in the various grades is known and is given in the table below. Find the mean and variance of , in which denotes the manufacturer’s profit on an item selected randomly from his production.

Grade of an Item	Profit on an Item of This Grade	Probability that an Item Is of This Grade

Answer

Mean, 58.75 cents; variance, 26 cents .

1.8 . Consider a person who commutes to the city from a suburb by train. He is accustomed to leaving his home between 7:30 and 8:00 A.M. The drive to the railroad station takes between 20 and 30 minutes. Assume that the departure time and length of trip are independent random variables, each uniformly distributed over their respective intervals. There are 3 trains that he can take, which leave the station and arrive in the city precisely on time. The first train leaves at 8:05 A.M. and arrives at 8:40 A.M., the second leaves at 8:25 A.M. and arrives at 8:55 A.M., the third leaves at 9:00 A.M. and arrives at 9:43 A.M.

(i) Find the mean and variance of his time of arrival in the city.

(ii) Find the mean and yariance of his time of arrival under the assumption that he leaves his home between 7:30 and 7:55 A.M.

1.9 . Two athletic teams play a series of games; the first team to win 4 games is the winner. Suppose that one of the teams is stronger than the other and has probability [equal to (i) 0.5, (ii) ] of winning each game, independent of the outcomes of any other game. Assume that a game cannot end in a tie. Find the mean and variance of the number of games required to conclude the series. (Use exercise 3.26 of Chapter 3.)

Answer

(i) Mean, 5.81, variance, 1.03; (ii) mean, 5.50, variance, 1.11.

1.10 . Consider an experiment that consists of players independently tossing fair coins. Let be the event that there is an “odd” man (that is, either exactly one of the coins falls heads or exactly one of the coins falls tails). For let be the number of times the experiment is repeated until the event occurs for the th time.

(i) Find the mean and variance of .

(ii) Evaluate and for and .

1.11 . Let an urn contain 5 balls, numbered 1 to 5. Let a sample of size 3 be drawn with replacement (without replacement) from the urn and let be the largest number in the sample. Find the mean and variance of .

Answer

With replacement, mean, 4.19, variance, 0.92; without replacement, mean, 4.5, variance 0.45.

1.12 . Let be . Find the mean and variance of (i) , (ii) where (a) is a given constant, (b) .

1.13 . Let and be independent random variables, each . Find the mean and variance of .

Answer

Mean, ; variance, .

1.14 . Find the mean and variance of a random variable that obeys the probability law of Laplace, specified by the probability density function, for some constants and :

1.15 . The velocity of a molecule with mass in a gas at absolute temperature is a random variable obeying the Maxwell Boltzmann law:

in which Boltzmann’s constant. Find the mean and variance of (i) the velocity of a molecule, (ii) the kinetic energy of a molecule.

Answer

At the end of the section we give an example that shows that (1.5) does not hold if the integrals used to define expectations are not required to converge absolutely. ↩︎
The roulette table described is the one traditionally in use in most European casinos. The roulette tables in many American casinos have wheels that are divided into 38 arcs, bearing numbers . ↩︎

Expectation, Mean, and Variance of a Random Variable

Theoretical Exercises

Exercises

Quick Links

Social Media