Jointly Distributed Random Variables

Two random variables, and , are said to be jointly distributed if they are defined as functions on the same probability space. It is then possible to make joint probability statements about and (that is, probability statements about the simultaneous behavior of the two random variables). In this section we introduce the notions used to describe the joint probability law of jointly distributed random variables.

The joint probability function, denoted by of two jointly distributed random variables, is defined for every Borel set of 2-tuples of real numbers by

in which denotes the sample description space on which the random variables and are defined and denotes the probability function defined on . In words, represents the probability that the 2-tuple ( ) of observed values of the random variables will lie in the set . For brevity, we usually write

instead of (5.1). However, it should be kept constantly in mind that the right-hand side of (5.2) is without mathematical content of its own; rather, it is an intuitively meaningful concise way of writing the right-hand side of (5.1) .

It is useful to think of the joint probability function of two jointly distributed random variables and as representing the distribution of a unit amount of probability mass over a 2-dimensional plane on which rectangular coordinates have been marked off, as in Fig. 7A of Chapter 4, so that to any point in the plane there corresponds a 2-tuple of real numbers representing it. For any Borel set of 2-tuples represents the amount of probability mass distributed over the set .

We are particularly interested in knowing the value for sets , which are combinatorial product sets in the plane. A set is called a combinatorial product set if it is of the form is in and is in for some Borel sets and of real numbers. If is of this form, we then write, for brevity, is in is in .

In order to know the joint probability function for all Borel sets of 2-tuples, it suffices to know it for all infinite rectangle sets , where, for any two real numbers and , we define the “infinite rectangle” set

as the set consisting of all 2-tuples whose first component is less than the specified real number and whose second component is less than the specified real number . To specify the joint probability function of and , it suffices to specify the joint distribution function , of the random variables and , defined for all real numbers and by the equation

In words, represents the probability that the simultaneous observation will have the property that and .

In terms of the probability mass distributed over the plane of Fig. 7A of Chapter 4, represents the amount of mass in the “infinite rectangle” .

The reader should verify for himself the following important formula [compare (7.3) of Chapter 4]: for any real numbers , and , such that , the probability that the simultaneous observation ( ) will be such that and may be given in terms of , by

It is important to note that from a knowledge of the joint distribution function of two jointly distributed random variables one may obtain the distribution functions and of each of the random variables and . We have the formula for any real number :

Similarly, for any real number

In terms of the probability mass distributed over the plane by the joint distribution function , the quantity is equal to the amount of mass in the half-plane that consists of all 2-tuples that are to the left of, or on, the line with equation .

The function is called the marginal distribution function of the random variable corresponding to the joint distribution function . Similarly, is called the marginal distribution function of corresponding to the joint distribution function

We next define the joint probability mass function of two random variables and , denoted by , as a function of 2 variables, with value, for any real numbers and .

It may be shown that there is only a finite or countably infinite number of 2-tuples at which . The jointly distributed random variables and are said to be jointly discrete if the sum of the joint probability mass function over the points where is positive is equal to 1. If the random variables and are jointly discrete , then they are individually discrete, with individual probability mass functions, for any real numbers and .

Two jointly distributed random variables, and , are said to be jointly continuous if they are specified by a joint probability density function.

Two jointly distributed random variables, and , are said to be specified by a joint probability density function if there is a nonnegative Borel function , called the joint probability density of and , such that for any Borel set of 2-tuples of real numbers the probability is in may be obtained by integrating over ; in symbols,

By letting in (5.10), it follows that the joint distribution function for any real numbers and may be given by

Next, for any real numbers , such that , one may verify that

The joint probability density function may be obtained from the joint distribution function by routine differentiation, since

at all 2-tuples , where the partial derivatives on the right-hand side of (5.13) are well defined.

If the random variables and are jointly continuous, then they are individually continuous, with individual probability density functions for any real numbers and given by

The reader should compare (5.14) with (3.4) .

To prove (5.14), one uses the fact that by (5.6), (5.7), and (5.11),

The foregoing notions extend at once to the case of random variables. We list here the most important notations used in discussing jointly distributed random variables . The joint probability function for any Borel set of -tuples is given by

The joint distribution function for any real numbers is given by The joint probability density function (if the derivative below exists) is given by

The joint probability mass function is given by A discrete joint probability law is specified by its probability mass function: for any Borel set of -tuples

A continuous joint probability law is specified by its probability density function: for any Borel set of -tuples

The individual (or marginal ) probability law of each of the random variables may be obtained from the joint probability law. In the continuous case, for any and any fixed number ,

An analogous formula may be written in the discrete case for .

Example 5A . Jointly discrete random variables . Consider a sample of size 2 drawn with replacement (without replacement) from an urn containing two white, one black, and two red balls. Let the random variables and be defined as follows; for or 0, depending on whether the ball drawn on the th draw is white or nonwhite. (i) Describe the joint probability law of . (ii) Describe the individual (or marginal) probability laws of and .

 

Solution

The random variables and are clearly jointly discrete. Consequently, to describe their joint probability law, it suffices to state their joint probability mass function . Similarly, to describe their individual probability laws, it suffices to describe their individual probability mass functions and . These functions are conveniently presented in the following tables:

 

Figure 2.4.1: image

Figure 2.4.1: image

 

Example 5B . Jointly continuous random variables . Suppose that at two points in a room (or on a city street or in the ocean) one measures the intensity of sound caused by general background noise. Let and be random variables representing the intensity of sound at the two points. Suppose that the joint probability law of the sound intensities, and , is continuous, with the joint probability density function given by

Find the individual probability density functions of and . Further, find and .

 

Solution

By (5.14) , the individual probability density functions are given by

 

Note that the random variables and are identically distributed. Next, the probability that each sound intensity is less than or equal to 1 is given by

The probability that the sum of the sound intensities is less than 1 is given by

Example 5C . The maximum noise intensity . Suppose that at five points in the ocean one measures the intensity of sound caused by general background noise (the so-called ambient noise). Let , and be random variables representing the intensity of sound at the various points. Suppose that their joint probability law is continuous, with joint probability density function given by

Define as the maximum intensity; in symbols, maximum . For any positive number the probability that is less than or equal to is given by

Theoretical Exercise

5.1 . Multivariate distributions with given marginal distributions . Let and be two probability density functions. An infinity of joint probability densities exist, of which and are the marginal probability density functions [that is, such that (3.4) holds]. One method of constructing is given by (3.9) ; verify this assertion. Show that another method of constructing a joint probability density function , with given marginal probability density functions and , is by defining for a given constant , such that ,

in which and are the distribution functions corresponding to and , respectively. Show that the distribution function corresponding to is given by

Equations (5.22) and (5.23) are due to E. J. Gumbel, “Distributions à plusieurs variables dont les marges sont données”, . R. Acad. Sci. Paris, Vol. 246 (1958), pp. 2717–2720.

Exercises

In exercises 5.1 to 5.3 consider a sample of size 3 drawn with replacement (without replacement) from an urn containing (i) 1 white and 2 black balls, 
(ii) 1 white, 1 black, and 1 red ball. For let or 0 depending on whether the ball drawn on the th draw is white or nonwhite.

5.1 . Describe the joint probability law of .

 

Answer

 

(i), (ii)
with
 
 
 
 otherwise0;
without,
 otherwise0.

 

5.2 . Describe the individual (marginal) probability laws of .

5.3 . Describe the individual probability laws of the random variables , and , in which . and .

 

Answer

With, if otherwise;

 

without, .

In exercises 5.4 to 5.6 consider 2 random variables, and , with joint probability law specified by the joint probability density function 
(a)
(b)

5.4 . Find (i) , (ii) , (iii) .

5.5 . Find (i) , (ii) , (iii) .

 

Answer

(a) (i) , (ii) , (iii) 0; (b) (i) , (ii) , (iii) 0.

 

5.6 . Find (i) , (ii) .

In exercises 5.7 to 5.10 consider 2 random variables, and , with the joint probability law specified by the probability mass function given for all and at which it is positive by (a) Table 5A, (b) Table 5B, in which for brevity we write for .

Figure 2.4.1: image Figure 2.4.1: image

5.7 . Show that the individual probability mass functions of and may be obtained by summing the respective columns and rows as indicated. Are and (i) jointly discrete, (ii) individually discrete?

 

Answer

Yes.

 

5.8 . Find (i) , (ii) , (iii) .

5.9 . Find (i) , (ii) , (iii) .

 

Answer

(a) (i) , (ii) , (iii) ; (b) (i) , (ii) , (iii) .

 

5.10 . Find (i) , (ii) .