The Characteristic Function of a Random Variable

It has been pointed out that the probability law of a random variable may be specified in a variety of ways. To begin with, either its probability function or its distribution function may be stated. Further, if the probability law is known to be continuous or discrete, then it may be specified by stating either its probability density function or its probability mass function . We now describe yet another function, denoted by called the characteristic function of the random variable , which has the property that a knowledge of serves to specify the probability law of the random variable . Further, we shall see that the characteristic function has properties which render it particularly useful for the study of a sum of independent random variables.

To begin our introduction of the characteristic function, let us note the following fact about the probability function and the distribution function of a random variable . Both functions can be regarded as the value of the expectation (with respect to the probability law of ) of various Borel functions . Thus, for every Borel set of real numbers

in which is a function of a real variable, called the indicator function of the set , with value at any point given by On the other hand, for every real number in which is a function of a real variable, defined by We thus see that if one knows the expectation of every bounded Borel function , with respect to the probability law of the random variable , one will know by (2.1) and (2.3) the probability function and distribution function of . Conversely, a knowledge of the probability function or of the distribution function of yields a knowledge of for every function for which the expectation exists. Consequently, stating the expectation functional of a random variable [which is a function whose argument is a function constitutes another equivalent way of specifying the probability law of a random variable.

The question arises: is there any other family of functions on the real line in addition to those of the form of (2.2) and (2.4) such that a knowledge of the expectations of these functions with respect to the probability law of a random variable would suffice to specify the probability law? We now show that the complex exponential functions provide such a family.

We define the expectation, with respect to a random variable , of a function , which takes values that are complex numbers, by

in which the symbols and , respectively, are abbreviations of the phrases “real part of” and “imaginary part of”. Note that

It may be shown that under these definitions all the usual properties of the operation of taking expectations continue to hold for complex-valued functions whose expectations exist. We define as existing if is finite. If this is the case, it then follows that

or, more explicitly,

The validity of (2.7) is proved in theoretical exercise 2.2 . In words, (2.6) states that the modulus of the expectation of a complex-valued function is less than or equal to the expectation of the modulus of the function.

The notions are now at hand to define the characteristic function of a random variable . We define as a function of a real variable , whose value is the expectation of the complex exponential function with respect to the probability law of ; in symbols,

The quantity for any real numbers and is defined by

in which is the imaginary unit, defined by or . Since , it follows that, for any random variable . Consequently, the characteristic function always exists.

The characteristic function of a random variable has all the properties of the moment-generating function of a random variable. All the moments of the random variable that exist may be obtained from a knowledge of the characteristic function by the formula

To prove (2.10), one must employ the techniques discussed in section 5 .

More generally, from a knowledge of the characteristic function of a random variable one may obtain a knowledge of its distribution function, its probability density function (if it exists), its probability mass function , and many other expectations. These facts are established in section 3 .

The importance of characteristic functions in probability theory derives from the fact that they have the following basic property. Consider any two random variables and . If the characteristic functions are approximately equal [that is, for every real number ], then their probability laws are approximately equal over intervals (that is, for any finite numbers and ) or, equivalently, their distribution functions are approximately equal [that is, for all real numbers ]. A precise formulation and proof of this assertion is given in Chapter 10.

Characteristic functions represent the ideal tool for the study of the problem of addition of independent random variables, since the sum of two independent random variables and has as its characteristic function the product of the characteristic functions of and ; in symbols, for every real number

if and are independent. It is natural to inquire whether there is some other function that enjoys properties similar to those of the characteristic function. The answer appears to be in the negative. In his paper “An essential property of the Fourier transforms of distribution functions,” Proceedings of the American Mathematical Society , Vol. 3 (1952), pp. 508510, E. Lukacs has proved the following theorem. Let be a complex valued function of two real variables and , which is a bounded Borel function of . Define for any random variable

In order that the function satisfy (2.11) and the uniqueness condition it is necessary and sufficient that have the form in which is a suitable real valued function.

Example 2A. If is , then its characteristic function is given by

To prove (2.13), we make use of the Taylor series expansion of the exponential function:

The interchange of the order of summation and integration in (2.14) may be justified by the fact that the infinite series is dominated by the integrable function .

Theoretical Exercises

Exercise