Independent Events and Families of Events

The notions of independent and dependent events play a central role in probability theory. Certain relations, which recur again and again in probability problems, may be given a general formulation in terms of these notions. If the events and have the property that the conditional probability of , given , is equal to the unconditional probability of , one intuitively feels that the event is statistically independent of , in the sense that the probability of having occurred is not affected by the knowledge that has occurred. We are thus led to the following formal definition.

Now suppose that both and have positive probability. Then both and are well defined, and from (4.6) of Chapter 2 it follows that  

If is independent of , it then follows that is independent of , since from (1.1) and (1.2) it follows that . It further follows from (1.1) and (1.2) that  

By means of (1.3) , a definition may be given of two events being independent, in which the two events play a symmetrical role.

Two events that do not satisfy (1.3) are said to be dependent (although a more precise terminology would be nonindependent ). Clearly, to say that two events are dependent is not very informative, for two events, and , are dependent if and only if . However, it is possible to classify dependent events to a certain extent, and this is done later. (See section 5.)

It should be noted that two mutually exclusive events, and , are independent if and only if , which is so if and only if either or has probability zero.

The notions of independent events and of conditional probability may be extended to more than two events. Suppose one has three events , and defined on a probability space. What are we to mean by the conditional probability of the event , given that the events and have occurred, denoted by ? From the point of view of the frequency interpretation of probability, by we mean the fraction of occurrences of both and on which also occurs. Consequently, we make the formal definition that if ; is undefined if .

Next, what do we mean by the statement that the event is independent of the events and ? It would seem that we should mean that the conditional probability of , given either or or the intersection , is equal to the unconditional probability of . We therefore make the following formal definition.

If (1.5) and (1.6) hold, it then follows that (assuming that the events , have positive probability, so that the conditional probabilities written below are well defined) \begin{align} & P[A \mid B, C]=P[A \mid B]=P[A \mid C]=P[A] \\ & P[B \mid A, C]=P[B \mid A]=P[B \mid C]=P[B] \tag{1.7} \\ & P[C \mid A, B]=P[C \mid A]=P[C \mid B]=P[C]. \end{align} 

Conversely, if all the relations in (1.7) hold, then all the relations in (1.5) and (1.6) hold.

It is to be emphasized that (1.5) does not imply (1.6) , so that three events, , and , which are pairwise independent [in the sense that (1.5) holds], are not necessarily independent. To see this, consider the following example.

We next define the notions of independence and of conditional probability for events .

We define the conditional probability of , given that the events , have occurred, denoted by ; \begin{align} P\left[A_{n} \mid A_{1}, A_{2}, \ldots, A_{n-1}\right] & =P\left[A_{n} \mid A_{1} A_{2} \cdots A_{n-1}\right] \tag{1.8} \\ & =\frac{P\left[A_{1} A_{2} \cdots A_{n}\right]}{P\left[A_{1} A_{2} \cdots A_{n-1}\right]} \end{align} if .

We define the events as independent (or statistically independent) if for every choice of integers from 1 to  

Equation (1.9) implies that for any choice of integers from 1 to (for which the following conditional probability is defined) and for any integer from 1 to not equal to one has  

We next consider families of independent events , for independent events never occur alone. Let and be two families of events; that is, and are sets whose members are events on some sample description space .

Two families of events and are said to be independent if any two events and , selected from and , respectively, are independent. More generally, families of events are said to be independent if any set of events (where is selected from is selected from , and so on, until is selected from ) is independent , in the sense that it satisfies the relation  

As an illustration of the fact that independent events occur in families, let us consider two independent events, and , which are defined on a sample description space . Define the families and by so that consists of , its complement , the certain event , and the impossible event , and, similarly, consists of , and .

We now show that if the events and are independent then the families of events and defined by (1.12) are independent. In order to prove this assertion, we must verify the validity of (1.11) with for each pair of events, one from each family, that may be chosen. Since each family has four members, there are sixteen such pairs. We verify (1.11) for only four of these pairs, namely , and , and leave to the reader the verification of (1.11) for the remaining twelve pairs. We have that and satisfy (1.11) by hypothesis. Next, we show that and satisfy (1.11) . By (5.2) of Chapter . Since, by hypothesis, , it follows that for by (5.3) of Chapter 1 . Next, and satisfy (1.11) , since and , so that . Next, and satisfy (1.11) , since and , so that .

More generally, by the same considerations, we may prove the following important theorem, which expresses (1.9) in a very concise form.

Theoretical Exercises

Exercises