Distribution Functions

To describe completely a numerical-valued random phenomenon, one needs only to state its probability function. The probability function is a function of sets and for this reason is somewhat unwieldy to treat analytically. It would be preferable if there were a function of points (that is, a function of real numbers ), which would suffice to determine completely the probability function. In the case of a probability function, specified by a probability density function or by a probability mass function, the density and mass functions provide a point function that determines the probability function. Now it may be shown that for any numerical valued random phenomenon whatsoever there exists a point function, called the distribution function, which suffices to determine the probability function in the sense that the probability function may be reconstructed from the distribution function. The distribution function (often referred to as the cumulative distribution function or CDF ) thus provides a point function that contains all the information necessary to describe the probability properties of the random phenomenon. Consequently, to study the general properties of numerical valued random phenomena without restricting ourselves to those whose probability functions are specified by either a probability density function or by a probability mass function, it suffices to study the general properties of distribution functions.

Before discussing the general properties of distribution functions, let us consider the distribution functions of numerical valued random phenomena, whose probability functions are specified by either a probability mass function or a probability density function. If the probability function is specified by a probability mass function , then the corresponding distribution function for any real number is given by  

Equation (3.2) follows immediately from (3.1) and (2.7) . If the probability function is specified by a probability density function , then the corresponding distribution function for any real number is given by  

Equation (3.3) follows immediately from (3.1) and (2.1) .

We may classify numerical valued random phenomena by classifying their distribution functions . To begin with, consider a random phenomenon whose probability function is specified by its probability mass function, so that its distribution function is given by (3.2) . The graph then appears as it is shown in (Fig. 3A) ; it consists of a sequence of horizontal line segments, each one higher than its predecessor. The points at which one moves from one line to the next are called the jump points of the distribution function ; they occur at all points at which the probability mass function is positive. We define a discrete distribution function as one that is given by a formula of the form of (3.2) , in terms of a probability mass function , or equivalently as one whose graph (Fig. 3A) consists only of jumps and level stretches. The term “discrete” connotes the fact that the numerical valued random phenomenon corresponding to a discrete distribution function could be assigned, as its sample description space, the set consisting of the (at most countably infinite number of) points at which the graph of the distribution function jumps.

Let us next consider a numerical valued random phenomenon whose probability function is specified by a probability density function, so that its distribution function is given by (3.3) . The graph then appears (Fig. 3B) as an unbroken curve. The function is continuous. However, even more is true; the derivative exists at all points (except perhaps for a finite number of points) and is given by  

We define a continuous distribution function as one that is given by a formula of the form of (3.3) in terms of a probability density function.

Most of the distribution functions arising in practice are either discrete or continuous. Nevertheless, it is important to realize that there are distribution functions, such as the one whose graph is shown in (Fig. 3C) , that are neither discrete nor continuous. Such distribution functions are called mixed . A distribution function is called mixed if it can be written as a linear combination of two distribution functions, denoted by and , which are discrete and continuous, respectively, in the following way: for any real number in which and are constants between 0 and 1, whose sum is one. The distribution function , graphed in Fig. 3C , is mixed, since , in which and are the distribution functions graphed in Fig. 3A and 3B, respectively.

Any numerical valued random phenomenon possesses a probability mass function defined as follows: for any real number  

Thus represents the probability that the random phenomenon will have an observed value equal to . In terms of the representation of the probability function as a distribution of a unit mass over the real line, represents the mass (if any) concentrated at the point . It may be shown that represents the size of the jump at in the graph of the distribution function of the numerical valued random phenomenon. Consequently, for all if and only if is continuous.

We now introduce the following notation. Given a numerical valued random phenomenon, we write to denote the observed value of the random phenomenon. For any real numbers and we write to mean the probability that an observed value of the numerical valued random phenomenon lies in the interval to . It is important to keep in mind that represents an informal notation for .

Some writers on probability theory call a number determined by the outcome of a random experiment (as is the observed value of a numerical valued random phenomenon) a random variable. In Chapter 7 we give a rigorous definition of the notion of random variable in terms of the notion of function, and show that the observed value of a numerical valued random phenomenon can be regarded as a random variable. For the present we have the following definition:

Given an observed value of a numerical valued random phenomenon with distribution function and probability mass function , we have the following formulas for any real numbers and (in which ): \begin{align} & P[a<X \leq b]=P[\{x: a<x \leq b\}]=F(b)-F(a) \\ & P[a \leq X \leq b]=P[\{x: a \leq x \leq b\}]=F(b)-F(a)+p(a) \\ & P[a \leq X<b]=P[\{x: a \leq x<b\}]=F(b)-F(a)+p(a)-p(b) \tag{3.7} \\ & P[a<X<b]=P[\{x: a<x<b\}]=F(b)-F(a)-p(b) \end{align} To prove (3.7) , define the events , and : Then (3.7) merely expresses the facts that (since ) \begin{align} P\left[B A^{c}\right] & =P[B]-P[A] \\ P\left[B A^{c} \cup C\right] & =P[B]-P[A]+P[C] \\ P\left[B A^{c} D^{c} \cup C\right] & =P[B]-P[A]+P[C]-P[D] \tag{3.8} \\ P\left[B A^{c} D^{c}\right] & =P[B]-P[A]-P[D]. \end{align} 

The use of (3.7) in solving probability problems posed in terms of distribution functions is illustrated in example 3A.

Example 3A . Suppose that the duration in minutes of long distance telephone calls made from a certain city is found to be a random phenomenon, with a probability function specified by the distribution function , given by in which the expression is defined for any real number as the largest integer less than or equal to . What is the probability that the duration in minutes of a long distance telephone call is (i) more than six minutes, (ii) less than four minutes, (iii) equal to three minutes? What is the conditional probability that the duration in minutes of a long distance telephone call is (iv) less than nine minutes, given that it is more than five minutes, (v) more than five minutes, given that it is less than nine minutes?

 

Solution

The distribution function given by (3.9) is neither continuous nor discrete but mixed. Its graph is given in (Fig. 3D) . For the sake of brevity, we write for the duration in minutes of a telephone call and as an abbreviation in mathematical symbols of the verbal statement “the probability that a telephone call has a duration strictly greater than six minutes”. The intuitive statement is identified in our model with , the value at the set of the probability function corresponding to the distribution function given by (3.9) . Consequently, Next, the probability that the duration of a call will be less than four minutes (or, more concisely written, ) is equal to , in which is the jump in the distribution function at . A glance at the graph of , drawn in (Fig. 3D) , reveals that the graph is unbroken at . Consequently, , and

 

The probability that an observed value of the duration of a call is equal to 3 is given by in which is the jump in the graph of at . Solutions to parts (iv) and (v) of the example may be obtained similarly:

In section 2 we gave the conditions a function must satisfy in order to be a probability density function or a probability mass function. The question naturally arises as to the conditions a function must satisfy in order to be a distribution function. In advanced studies of probability theory it is shown that the properties a function must have in order to be a distribution function are the following: (i) must be non-decreasing in the sense that for any real numbers and  

(ii) the limits of , as tends to either plus or minus infinity, must exist and be given by  

(iii) at any point the limit from the right , which is defined as the limit of , as tends to through values greater than , must be equal to , so that at any point the graph of is unbroken as one approaches from the right; (iv) at any point , the limit from the left, written or , which is defined as the limit of as tends to through values less than , must be equal to ; in symbols, where we define as the probability that the observed value of the random phenomenon is equal to . Note that represents the size of the jump in the graph of at .

From these facts it follows that the graph of a typical distribution function has as its asymptotes the lines and . The graph is non-decreasing. However, it need not increase at every point but rather may be level (horizontal) over certain intervals. The graph need not be unbroken [that is, need not be continuous] at all points, but there is at most a countable infinity of points at which the graph has a break; at these points it jumps upward and possesses limits from the right and the left, satisfying (3.12) and (3.13) .

The foregoing mathematical properties of the distribution function of a numerical valued random phenomenon serve to characterize completely such functions. It may be shown that for any function possessing the first three properties listed there is a unique set function , defined on the Borel sets of the real line, satisfying axioms of section 1 and the condition that for any finite real numbers and , at which , From this fact it follows that to specify the probability function it suffices to specify the distribution function.

The fact that a distribution function is continuous does not imply that it may be represented in terms of a probability density function by a formula such as (3.3) . If this is the case, it is said to be absolutely continuous . There also exists another kind of continuous distribution function, called singular continuous , whose derivative vanishes at almost all points. This is a somewhat difficult notion to picture, and examples have been constructed only by means of fairly involved analytic operations. From a practical point of view, one may act as if singular distribution functions do not exist, since examples of these functions are rarely, if ever, encountered in practice. It may be shown that any distribution function may be represented in the form in which , and , respectively, are discrete, absolutely continuous, and singular continuous, and , and are constants between 0 and 1, inclusive, the sum of which is 1. If it is assumed that the coefficient vanishes for any distribution function encountered in practice, it follows that in order to study the properties of a distribution function it suffices to study those that are discrete or continuous.

Theoretical Exercises

Exercises