In section 4 of Chapter 2 the notion of conditional probability was discussed for events defined on a sample description space on which a probability function was defined. However, an important use of the notion of conditional probability is to set up a probability function on the subsets of a sample description space
As in section 2, for
Now, as shown in section 2, any event
Consequently, to know the value of
\begin{align} & P\left[A_{1}\right] \\ & P\left[A_{2} \mid A_{1}\right] \\ & P\left[A_{3} \mid A_{1}, A_{2}\right] \tag{4.2} \\ & \cdot \\ & \cdot \\ & P\left[A_{n} \mid A_{1}, A_{2}, \ldots, A_{n-1}\right] \end{align}
for any events
Example 4A . Consider an urn containing
Further illustrations of the specification of a probability function on the subsets of a space of
Example 4B . Consider two urns; urn I contains five white and three black balls, urn II, three white and seven black balls. One of the urns is selected at random, and a ball is drawn from it. Find the probability that the ball drawn will be white.
Solution
The sample description space of the experiment described consists of 2-tuples
Example 4C . A case of hemophilia . 1 The first child born to a certain woman was a boy who had hemophilia. The woman, who had a long family history devoid of hemophilia, was perturbed about having a second child. She reassured herself by reasoning as follows. “My son obviously did not inherit his hemophilia from me. Consequently, he is a mutant. The probability that my second child will have hemophilia, if he is a boy, is consequently the probability that he will be a mutant, which is a very small number
Solution
Let us write a 3-tuple
In making these assumptions (4.6) we have used the fact that the woman has no family history of hemophilia. A boy usually carries an
We are seeking
To compute
\begin{align} P\left[A_{2} A_{3}\right]= & P\left[A_{1} A_{2} A_{3}\right]+P\left[A_{1}^{c} A_{2} A_{3}\right] \tag{4.8} \\ = & P\left[A_{1}\right] P\left[A_{2} \mid A_{1}\right] P\left[A_{3} \mid A_{2}, A_{1}\right] \\ & +P\left[A_{1}^{c}\right] P\left[A_{2} \mid A_{1}^{c}\right] P\left[A_{3} \mid A_{2}, A_{1}^{c}\right] \\ = & 2 m\left(\frac{1}{2}\right)^{\frac{1}{2}}+(1-2 m) m m \\ = & \frac{1}{2} m \end{align}
since we may consider
\begin{align} P\left[A_{2}\right] & =P\left[A_{2} \mid A_{1}\right] P\left[A_{1}\right]+P\left[A_{2} \mid A_{1}^{c}\right] P\left[A_{1}^{c}\right] \tag{4.9} \\ & =\frac{1}{2} 2 m+m(1-2 m) \\ & \doteq 2 m. \end{align}
Consequently,
Thus the conditional probability that the second son of a woman with no family history of hemophilia will have hemophilia, given that her first son has hemophilia, is approximately
A very important use of the notion of conditional probability derives from the following extension of (4.5) . Let
Example 4D . On drawing a sample from a sample. Consider a box containing five radio tubes selected at random from the output of a machine, which is known to be
(i) Find the probability that a tube selected from the box will be defective.
(ii) Suppose that a tube selected at random from the box is defective; what is the probability that a second tube selected at random from the box will be defective?
Solution
To describe the results of the experiment that consists in selecting five tubes from the output of the machine and then selecting one tube from among the five previously selected, we write a 6-tuple
Assuming that the selections were independent,
To evaluate the sum in (4.13), we write it as
Let us next consider part (ii) of example 4D. To describe the results of the experiment that consists in selecting five tubes from the output of the machine and then selecting two tubes from among the five previously selected, we write a 7 -tuple
Consequently,
Bayes’s Theorem. There is an interesting consequence to (4.11) , which has led to much philosophical speculation and has been the source of much controversy. Let
The relation expressed by (4.16) is called “Bayes’s theorem” or “Bayes’s formula”, after the English philosopher Thomas Bayes. 2 If the events
Example 4E . Cancer diagnosis. Suppose, contrary to fact, there were a diagnostic test for cancer with the properties that
Let us assume that the probability that a person taking the test actually has cancer is given by
One should carefully consider the meaning of this result. On the one hand, the cancer diagnostic test is highly reliable, since it will detect cancer in
Example 4F . Prior and posterior probability. Consider an urn that contains a large number of coins: Not all of the coins are necessarily fair. Let a coin be chosen randomly from the urn and tossed independently 100 times. Suppose that in the 100 tosses heads appear 55 times. What is the probability that the coin selected is a fair coin (that is, the probability that the coin will fall heads at each toss is equal to
Solution
To describe the results of the experiment we write a 101-tuple
The probabilities
Let us next assume that
The probability
Our next example illustrates a controversial use of Bayes’s theorem.
Example 4G . Laplace’s rule of succession. Consider a coin that in
Solution
To describe the results of our observations, we write an
whereas
Let us now assume that
The sums in (4.24) may be approximately evaluated in the case that
\begin{align} \frac{1}{N} \sum_{j=1}^{N}\left(\frac{j}{N}\right)^{n+n^{\prime}} & \doteq \int_{0}^{1} x^{n+n^{\prime}} d x=\frac{1}{n+n^{\prime}+1} \\ \frac{1}{N} \sum_{j=1}^{N}\left(\frac{j}{N}\right)^{n} & \doteq \int_{0}^{1} x^{n} d x=\frac{1}{n+1} . \tag{4.25} \end{align}
Consequently, given that the first
Equation (4.26) is known as Laplace’s general rule of succession. If we take
Equation (4.27) is known as Laplace’s special rule of succession.
Equation (4.27) has been interpreted by some writers on probability theory to imply that if a theory has been verified in
Consider a tourist in a foreign city who scarcely understands the language. With trepidation, he selects a restaurant in which to eat. After ten meals taken there he has felt no ill effects. Consequently, he goes quite confidently to the restaurant the eleventh time in the knowledge that, according to the rule of succession, the probability is
However, it is easy to exhibit applications of the rule that lead to absurd answers. A boy is 10 years old today. The rule says that, having lived ten years, he has probability
Laplace gave the following often-quoted application of the special rule of succession. “Assume”, he says, “that history goes back 5000 years, that is,
It is to be emphasized that Baye’s formula and Laplace’s rule of succession are true theorems, of mathematical probability theory. The foregoing examples do not in any way cast doubt on the validity of these theorems. Rather they serve to illustrate what may be called the fundamental principle of applied probability theory: before applying a theorem, one must carefully ponder whether the hypotheses of the theorem may be assumed to be satisfied.
Theoretical Exercises
4.1 . An urn contains
4.2 . Consider a box containing
(i) Let
(ii) Suppose that
(iii) Suppose that
4.3 . Consider an urn containing
4.4 . An application of Bayes’s theorem. Suppose that in answering a question on a multiple choice test an examinee either knows the answer or he guesses. Let
4.5 . Solution of a difference equation. The difference equation
in which
Exercises
4.1 . Urn I contains 5 white and 7 black balls. Urn II contains 4 white and 2 black balls. Find the probability of drawing a white ball if (i) 1 urn is selected at random, and a ball is drawn from it, (ii) the 2 urns are emptied into a third urn from which 1 ball is drawn.
Answer
(i)
4.2 . Un I contains 5 white and 7 black balls. Un II contains 4 white and 2 black balls. An urn is selected at random, and a ball is drawn from it. Given that the ball drawn is white, what is the probability that urn I was chosen?
4.3 . A man draws a ball from an urn containing 4 white and 2 red balls. If the ball is white, he does not return it to the urn; if the ball is red, he does return it. He draws another ball. Let
Answer
(i)
4.4 . From an urn containing 6 white and 4 black balls, 5 balls are transferred into an empty second urn. From it 3 balls are transferred into an empty box. One ball is drawn from the box; it turns out to be white. What is the probability that exactly 4 of the balls transferred from the first to the second urn will be white?
4.5 . Consider an urn containing 12 balls, of which 8 are white. Let a sample of size 4 be drawn with replacement (without replacement). Next, let a ball be selected randomly from the sample of size 4. Find the probability that it will be white.
Answer
4.6 . Urn I contains 6 white and 4 black balls. Urn II contains 2 white and 2 black balls. From urn I 2 balls are transferred to urn II. A sample of size 2 is then drawn without replacement from urn II. What is the probability that the sample will contain exactly 1 white ball?
4.7 . Consider a box containing 5 radio tubes selected at random from the output of a machine, which is known to be
Answer
4.8 . Let the events
4.9 . In a certain college the geographical distribution of men students is as follows:
Answer
Let the event that a student wears a tie, comes from the East, comes from the Midwest, or comes from the Far West be denoted, respectively by
4.10 . Consider an urn containing 10 balls, of which 4 are white. Choose an integer
4.11 . Each of 3 boxes, identical in appearance, has 2 drawers. Box A contains a gold coin in each drawer; box
(i) What is the probability that the other drawer contains a silver coin? Write out the probability space of the experiment. Why is it fallacious to reason that the probability is
(ii) What is the probability that the box chosen was box
Answer
(i)
4.12 . Three prisoners, whom we may call
4.13 . A male rat is either doubly dominant
Answer
4.14 . Consider an urn that contains 5 white and 7 black balls. A ball is drawn and its color is noted. It is then replaced; in addition, 3 balls of the color drawn are added to the urn. A ball is then drawn from the urn. Find the probability that (i) the second ball drawn will be black, (ii) both balls drawn will be black.
4.15 . Consider a sample of size 3 drawn in the following manner. One starts with an urn containing 5 white and 7 red balls. At each trial a ball is drawn and its color is noted. The ball drawn is then returned to the urn, together with an additional ball of the same color. Find the probability that the sample will contain exactly (i) 0 white balls, (ii) 1 white ball, (iii) 3 white balls.
Answer
(i)
4.16 . A certain kind of nuclear particle splits into 0,1, or 2 new particles (which we call offsprings) with probabilities
(i) Find the probability that
(ii) Find the conditional probability that
(iii) Find the probability that
4.17 . A number, denoted by
(i) For each integer
(ii) Find the probability that
(iii) Find the conditional probability that
Answer
(i)
- I am indebted to my esteemed colleague Lincoln E. Moses for the idea of this example. ↩︎
- A reprint of Bayes’s original essay may be found in Biometrika, Vol. 46 (1958), pp. 293–315. ↩︎
- The use of Bayes’s formula to evaluate probabilities during the course of play of a bridge game is illustrated in Dan F. Waugh and Frederick V. Waugh, “On Probabilities in Bridge”, Journal of the American Statistical Association , Vol. 48 (1953), pp. 79–87. ↩︎