Definition of Probability | Probability Theory by E. Parzen

The mathematical notions are now at hand with which one may state the postulates of a mathematical model of a random phenomenon. Let us recall that in our heuristic discussion of the notion of a random phenomenon in section 1 we accepted the so-called “frequency” interpretation of probability, according to which the probability of an event is a number (which we denote by ). This number can be known to us only by experience as the result of a very long series of observations of independent trials of the event . (By a trial of is meant an occurrence of the phenomenon on which is defined.) Having observed a long series of trials, the probability of represents the fraction of trials whose outcome has a description that is a member of . In view of the frequency interpretation of , it follows that a mathematical definition of the probability of an event cannot tell us the value of for any particular event . Rather a mathematical theory of probability must be concerned with the properties of the probability of an event considered as a function defined on all events. With these considerations in mind, we now give the following definition of probability.

The definition of probability as a function of events on the subsets of a sample description space of a random phenomenon:

Given a random situation, which is described by a sample description space , probability is a function ¹that to every event assigns a nonnegative real number, denoted by and called the probability of the event . The probability function must satisfy three axioms:

Axiom 1. for every event ,

Axiom 2. for the certain event ,

Axiom 3. , if , or in words, the probability of the union of two mutually exclusive events is the sum of their probabilities.

It should be clear that the properties stated by the foregoing axioms do constitute a formal statement of some of the properties of the numbers and , interpreted to represent the relative frequency of occurrence of the events and in a large number of occurrences of the random phenomenon on which they are defined. For any event, , let be the number of occurrences of in the occurrences of the phenomenon. Then, by the frequency interpretation of probability, . Clearly, . Next, , since, by the construction of , it occurs on every occurrence of the random phenomenon. Therefore, . Finally, for two mutually exclusive events, and . Thus axiom 3 is satisfied.

It therefore follows that any property of probabilities that can be shown to be logical consequences of axioms 1 to 3 will hold for probabilities interpreted as relative frequencies. We shall see that for many purposes axioms 1 to 3 constitute a sufficient basis from which to derive the properties of probabilities. In advanced studies of probability theory, in which more delicate questions concerning probability are investigated, it is found necessary to strengthen the axioms somewhat. At the end of this section we indicate briefly the two most important modifications required.

We now show how one can derive from axioms 1 to 3 some of the important properties that probability possesses. In particular, we show how axiom 3 suffices to enable us to compute the probabilities of events constructed by means of complementations and unions of other events in terms of the probabilities of these other events.

In order to be able to state briefly the hypotheses of the theorems subsequently proved, we need some terminology. It is to be emphasized that one can speak of the probability of an event only if the event is a subset of a definite sample description space , on whose subsets a probability function has been defined. Consequently, the hypothesis of a theorem concerning events should begin, “Let be a sample description space on the subsets of which a probability function has been defined. Let and be any two events on ”. For the sake of brevity, we write instead “Let and be any two events on a probability space”; by a probability space we mean a sample description space on which a probability function (satisfying axioms 1,2, and 3) has been defined.

Formula for the Probability of the Impossible Event .

Proof

By (4.4) it follows that the certain event and the impossible event are mutually exclusive; further, their union . Consequently, , from which it follows that .

Formula for the Probability of a Difference of Two Events And . For any two events, and , on a probability space

Proof

The events and are mutually exclusive, and their union is [compare (4.5) ]. Then, by axiom , from which (5.2) follows immediately.

Formula for the Probability of the Complement of an Event . For any event on a probability space

Proof

Let in (5.2). Since , and , we have obtained (5.3).

Formula for the Probability of a Union of Two Events And . For any two events, and , on a probability space

Proof

We use the fact that the event may be written as the union of the two mutually exclusive events, and . Then, by axiom 3, . By evaluating by (5.2), one obtains (5.4).

Note that (5.4) extends axiom 3 to the case in which the events whose union is being formed are not necessarily mutually exclusive.

We next obtain a basic property of the probability function, namely, that if an event is a subevent of another event , then the probability that will occur is less than or equal to the probability that will occur.

Inequality for the Probability of a Subevent . Let and be events on a probability space such that (that is, is a subevent of ). Then $Multiple \tagP\left[E F^{c}\right]=P[E]-P[F] \quad \text { if } F \subset E \tag{5.5}\]$

Proof

By (5.2), . Now, since , it follows that, as in (4.6), . Therefore, , which proves (5.5). Next, , by axiom 1. Therefore, , from which it follows that , which proves (5.6).

From the preceding inequality we may derive the basic fact that probabilities are numbers between 0 and 1: This is proved as follows. By axiom . Next, any event is a subevent of the certain event. Therefore, by (5.6), . However, by axiom , and the proof of the assertion is completed.

Formula for the Probability of the Union of a Finite Number of Mutually Exclusive Events . For any positive integer the probability of the union of mutually exclusive events is equal to the sum of the probabilities of the events; in symbols, if, for every two integers and which are not equal and which are between 1 and , inclusive, .

Proof

To prove (5.8), we make use of the principle of mathematical induction, which states that a proposition , which depends on an integer , is true for , if one shows that (i) it is true for , and (ii) it satisfies the implication: implies . Now, for any positive integer let be the proposition that for any set of mutually exclusive events, holds. That is true is obvious, since in the case that states that . Next, let be a definite integer, and let us assume that is true. Let us show that from the assumption that is true it follows that is true. Let , be mutually exclusive events. Since the events and are then mutually exclusive, it follows, by axiom 3, that From (5.9), and the assumption that is true, it follows that . We have thus shown that implies . By the principle of mathematical induction, it holds that the proposition applies to any positive integer . The proof of (5.8) is now complete.

The foregoing axioms are completely adequate for the study of random phenomena whose sample description spaces are finite. For the study of infinite sample description spaces, however, it is necessary to modify axiom 3. We may wish to consider an infinite sequence of mutually exclusive events, That the probability of the union of an infinite number of mutually exclusive events is equal to the sum of the probabilities of the events cannot be proved by axiom 3 but must be postulated separately. Consequently, in advanced studies of probability theory, instead of axiom 3, the following axiom is adopted.

Axiom . For any infinite sequence of mutually exclusive events, ,

A somewhat more esoteric modification in the foregoing axioms becomes necessary when we consider a random phenomenon whose sample description space is non-countably infinite. It may then turn out that there are subsets of that are nonprobabilizable, in the sense that it is not possible to assign a probability to these sets in a manner consistent with the axioms. If such is the case, then only probabilizable subsets of are defined as events. Since it may be proved that the union, intersection, and complements of events are events, this restriction of the notion of event causes no difficulty in application and renders the mathematical theory rigorous.

Exercises

5.1. Boole’s inequality. For a finite set of events, , Prove this assertion by means of the principle of mathematical induction.

5.2. Formula for the probability that exactly 1 of 2 events will occur. Show that for any 2 events, and , on a probability space The event is the event that exactly 1 of the events, and , will occur. Contrast (5.12) with (5.4), which could be called the formula for the probability that at least 1 of 2 events will occur.

5.3. Show that for any 3 events, , and , defined on a probability space, the probability of the event that at least 1 of the events will occur is given by

5.4. Let and be 2 events on a probability space. Show that

5.5. Let and be 2 events on a probability space. In terms of , and , express (i) for [exactly of the events, and , occur], (ii) for [at least of the events, and , occur], (iii) for [at most of the events, and , occur], (iv) occurs and does not occur].

Answer

exactly 0 exactly 1 . [exactly 2] [at least 0 . [at least 1 . [at least 2 [at most 0 . at most 1 at most 2 .

5.6. Let , and be 3 events on a probability space. In terms of , , and express for (i) [exactly of the events, , occur], (ii) [at least of the events, , occur], (iii) [at most of the events, , occur].

5.7. Evaluate the probabilities asked for in exercise 5.5 in the case that
(i) ,
(iii) .
(ii)

5.8. Evaluate the probabilities asked for in exercise 5.6 in the case that

(i) ,

(ii) .

The size of sets: The various formulas that have been developed for probabilities continue to hold true if one replaces by and for any set define as the number of elements in, or the size of, the set . Further, replace 1 by .

5.9. Suppose that a study of 900 college graduates 25 years after graduation revealed that 300 were “successes”, 300 had studied probability theory in college, and 100 were both “successes” and students of probability theory. Find, for , the number of persons in the group who had done of these two things: (i) exactly , (ii) at least , (iii) at most .

Answer

[exactly 0 ] [exactly 1 [exactly 2 .

[at least 0 ] [at least 1 ] [at least 2 .

at most 0 at most 1 at most 2 .

5.10. In a very hotly fought battle in a small war 270 men fought. Of these, 90 lost an eye, 90 lost an arm, and 90 lost a leg: 30 lost both an eye and an arm, 30 lost both an arm and a leg, and 30 lost both a leg and an eye; 10 lost all three. Find, for , the number of men who suffered of these injuries: (i) exactly , (ii) at least , (iii) no more than .

5.11. Certain data obtained from a study of a group of 1000 subscribers to a certain magazine relating to their sex, marital status, and education were reported as follows: 312 males, 470 married, 525 college graduates, 42 male college graduates, 147 married college graduates, 86 married males, and 25 married male college graduates. Show that the numbers reported in the various groups are not consistent.

Answer

Let , and denote, respectively, a set of college graduates, males and married persons. Show .

Definition: A function is a rule that assigns a real number to each element of a set of objects (called the domain of the function). Here the domain of the probability function is the set of all events on . ↩︎

The Definition of Probability as a Function of Events on a Sample Description Space

Exercises

Quick Links

Social Media