Binomial Approximation: Normal & Poisson

Some understanding of the kinds of random phenomena that obey the normal probability law can be obtained by examining the manner in which the normal density function and the normal distribution function first arose in probability theory as means of approximately evaluating probabilities associated with the binomial probability law.

The following theorem was stated by de Moivre in 1733 for the case and proved for arbitrary values of by Laplace in 1812.

The probability that a random phenomenon obeying the binomial probability law with parameters and will have an observed value lying between a and , inclusive, for any two integers and , is given approximately by

Before indicating the proof of this theorem, let us explain its meaning and usefulness by the following examples.

Example 2A. Suppose that tosses of a fair die are made. The probability that exactly of the tosses will result in a “three” is given by The probability that the number of tosses on which a “three” will occur is between 980 and 1030, inclusive, is given by the sum

It is clearly quite laborious to evaluate this sum directly. Fortunately, by (2.1), the sum in (2.2) is approximately equal to

Example 2B. In 40,000 independent tosses of a coin heads appeared 20,400 times. Find the probability that if the coin were fair one would observe in 40,000 independent tosses (i) 20,400 or more heads, (ii) between 19,600 and 20,400 heads.

Solution

Let be the number of heads in 40,000 independent tosses of a fair coin. Then obeys a binomial probability law with mean , variance , and standard deviation . According to the normal approximation to the binomial probability law, approximately obeys a normal probability law with parameters and [in making this statement we are ignoring the terms in (2.1) involving , which are known as a “continuity” correction]. Since 20,400 is four standard deviations more then the mean of the probability law of , the probability is approximately 0 that one would observe a value of more than 20,400. Similarly, the probability is that one would observe a value of between 19,600 and 20,400.

In order to have a convenient language in which to discuss the proof of (2.1), let us suppose that we are observing the number of successes in independent repeated Bernoulli trials with probability of success at each trial. Next, to each outcome let us compute the quantity

which represents the deviation of from divided by . Recall that the quantities and are equal, respectively, to the mean and standard deviation of the binomial probability law. The deviation , defined by (2.3), is a random quantity obeying a discrete probability law specified by a probability mass function , which may be given in terms of the probability mass function by

In words, (2.4) expresses the fact that for any given real number the probability that the deviation (of the number of successes from , divided by ) will be equal to is the same as the probability that the number of successes will be equal to .

The advantage in considering the probability mass function over the original probability mass function can be seen by comparing their graphs, which are given in Figs. 7.2.1 and 7.2.2 for and 100 and . The graph of the function becomes flatter and flatter and spreads out more and more widely along the -axis as the number of trials increases.

Figure 2.4.1 — **Fig. 2A** . Graphs of the binomial probability mass function for and and 100 .

The graphs of the functions , on the other hand, are very similar to each other, for different values of , and seem to approach a limit as becomes infinite. It might be thought that it is possible to find a smooth curve that would be a good approximation to , at least when the number of trials is large. We now show that this is indeed possible. More precisely, we show that if is a real number for which then, approximately, in the sense that

To prove (2.6), we first obtain the approximate expression for ; for in which . Equation (2.7) is an immediate consequence of the approximate expression for the binomial coefficient ; for any integers and

Equation (2.8), in turn, is an immediate consequence of the approximate expression for ! given by Stirling’s formula; for any integer . In (2.7) let . Then , and Then, using the expansion for some such that , valid for , we obtain that

If we ignore all terms that tend to 0 as tends to infinity in (2.11), we obtain the desired conclusion, namely (2.6) .

From (2.6) one may obtain a proof of (2.1) . However, in this book we give only a heuristic geometric proof that (2.6) implies (2.1) . For an elementary rigorous proof of (2.1) the reader should consult J. Neyman, First Course in Probability and Statistics , New York, Henry Holt, 1950, pp. 234–242. In Chapter 10 we give a rigorous proof of (2.1) by using the method of characteristic functions.

A geometric derivation of (2.1) from (2.6) is as follows. First plot in Fig. 2B as a function of ; note that for all points , except those that may be represented in the form for some integer . Next, as in Fig. 2C, plot by a series of rectangles of height , centered at all points of the form of (2.12).

From Fig. 2C we obtain (2.1). It is clear that which is equal to the sum of the areas of the rectangles in Fig.2C centered at the points of the form of (2.12), corresponding to the integers from to , inclusive. Now, the sum of the area of these rectangles is an approximating sum to the integral of the function between the limits and . We have thus obtained the approximate formula (2.1).

It should be noted that we have available two approximations to the probability mass function of the binomial probability law. From (2.5) and (2.6) it follows that

whereas from (2.1) one obtains, setting ,

In using any approximation formula, such as that given by (2.1), it is important to have available “remainder terms” for the determination of the accuracy of the approximation formula. Analytic expressions for the remainder terms involved in the use of (2.1) are to be found in J. V. Uspensky, Introduction to Mathematical Probability , McGraw-Hill, New York, 1937, p. 129, and W. Feller, “On the normal approximation to the binomial distribution”, Annals of Mathematical Statistics , Vol. 16, (1945), pp. 319–329. However, these expressions do not lead to conclusions that are easy to state. A booklet entitled Binomial, Normal, and Poisson Probabilities , by Ed. Sinclair Smith (published by the author in 1953 at Bel Air, Maryland), gives extensive advice on how to compute expeditiously binomial probabilities with 3-decimal accuracy. Smith (p. 38) states that (2.1) gives 2-decimal accuracy or better if . The accuracy of the approximation is much better for close to 0.5, in which case 2-decimal accuracy is obtained with as small as 3.

In treating problems in this book, the student will not be seriously wrong if he uses the normal approximation to the binomial probability law in cases in which .

Extensive tables of the binomial distribution function are available.

The Poisson Approximation to the Binomial Probability Law. The Poisson approximation, whose proof and usefulness was indicated in section 3 of Chapter 3, states that

The Poisson approximation applies when the binomial probability law is very far from being bell shaped; this is true, say, when .

It may happen that is very small, so that the Poisson approximation may be used; but is so large that (2.14) holds, and the normal approximation may be used. This implies that for large values of the Poisson law and the normal law approximate each other. In theoretical exercise 2.1 it is shown directly that the Poisson probability law with parameter may be approximated by the normal probability law for large values of .

Example 2C . A telephone trunking problem. Suppose you are designing the physical premises of a newly organized research laboratory. Since there will be a large number of private offices in the laboratory, there will also be a large number of individual telephones, each connecting to a central laboratory telephone switchboard. The question arises: how many outside lines will the switchboard require to establish a fairly high probability, say , that any person who desires the use of an outside telephone line (whether on the outside of the laboratory calling in or on the inside of the laboratory calling out) will find one immediately available?

Solution

We begin by regarding the problem as one involving independent Bernoulli trials. We suppose that for each telephone in the laboratory, say the th telephone, there is a probability that an outside line will be required (either as the result of an incoming call or an outgoing call). One could estimate by observing in the course of an hour how many minutes an outside line is engaged, and estimating by the ratio . In order to have repeated Bernoulli trials, we assume . We next assume independence of the events , in which is the event that the th telephone demands an outside line at the moment of time at which we are regarding the laboratory. The probability that exactly outside lines will be in demand at a given moment is, by the binomial law, given by .

Consequently, if we let denote the number of outside lines connected to the laboratory switchboard and make the assumption that they are all free at the moment at which we are regarding the laboratory, then the probability that a person desiring an outside line will find one available is the same as the probability that the number of outside lines demanded at that moment is less than or equal to , which is equal to

where the first equality sign in (2.18) holds if the Poisson approximation to the binomial applies and the second equality sign holds if the normal approximation to the binomial applies.

Define, for any and integer ,

which is essentially the distribution function of the Poisson probability law with parameter . Next, define for , such that , the symbol to denote the -percentile of the normal distribution function, defined by

One may give the following expressions for the minimum number of outside lines that should be connected to the laboratory switchboard in order to have a probability greater than a preassigned level that all demands for outside lines can be handled . Depending on whether the Poisson or the normal approximation applies, is the smallest integer such that

In writing (2.22), we are approximating by 0, since

The value of can be determined from Table I . In particular,

The solution of the inequality (2.21) can be read from tables prepared by E. C. Molina (published in a book entitled Poisson’s Exponential Binomial Limit , Van Nostrand, New York, 1942) which tabulate, to six decimal places, the function

for about 300 values of in the interval .

The value of , determined by (2.21) and (2.22), is given in Table 2A for , and .


Approximation	Poisson	Normal	Poisson	Normal	Poisson	Normal
	6	5.3	14	13.2	39	36.9
	39	38.4	106	104.3	322.8	322.8
	8	6.5	17	15.1	43	39.9
	43	42.0	113	110.4	332.4	332.4

TABLE 2A . The Values of

Determined by (2.21) and (2.22)

Theoretical Exercises

2.1. Normal approximation to the Poisson probability law. Consider a random phenomenon obeying a Poisson probability law with parameter . To an observed outcome of the random phenomenon, compute , which represents the deviation of from , divided by . The quantity is a random quantity obeying a discrete probability law specified by a probability mass function , which may be given in terms of the probability function by . In the same way that (2.6), (2.1), and (2.13) are proved show that for fixed values of , and , the following differences tend to 0 as tends to infinity:

2.2. A competition problem. Suppose that restaurants compete for the same patrons. Show that the number of seats that each restaurant should have to order to have a probability greater than that it can serve all patrons who come to it (assuming that all the patrons arrive at the same time and choose, independently of one another, each restaurant with probability ) is given by (2.22), with . Compute for and and 0.95. Express in words how the size of a restaurant (represented by ) depends on the size of its market (represented by ), the number of its competitors (represented by ), and the share of the market it desires (represented by ).

Exercises

2.1. In 10,000 independent tosses of a coin 5075 heads were observed. Find approximately the probability of observing (i) exactly 5075 heads, (ii) 5075 or more heads if the coin (a) is fair, (b) has probability 0.51 of falling heads.

Answer

(i) (a) 0.003; (b) 0.007; (ii) (a) 0.068; (b) 0.695.

2.2. Consider a room in which 730 persons are assembled. For , 730, let be the event that the th person was born on January 1. Assume that the events are independent and that each event has probability equal to . Find approximately the probability that (i) exactly 2, (ii) 2 or more persons were born on January 1. Compare the answers obtained by using the normal and Poisson approximations to the binomial law.

2.3. Plot the probability mass function of the binomial probability law with parameters and against its normal approximation. In your opinion, is the approximation close enough for practical purposes?

2.4. Consider an urn that contains 10 balls, numbered 0 to 9, each of which is equally likely to be drawn; thus choosing a ball from the urn is equivalent to choosing a number 0 to 9, and one sometimes describes this experiment by saying that a random digit has been chosen. Now let balls be chosen with replacement. Find the probability that among the numbers thus chosen the number 7 will appear between times and times, inclusive, if (i) , (ii) , (iii) . Compute the answers exactly or by means of the normal and Poisson approximations to the binomial probability law.

2.5. Find the probability that in 3600 independent repeated trials of an experiment, in which the probability of success of each trial is , the number of successes is between and , inclusive, if (i) , (ii) .

Answer

(i) 0.506; (ii) 0.532.

2.6. A certain corporation has 90 junior executives. Assume that the probability is that an executive will require the services of a secretary at the beginning of the business day. If the probability is to be 0.95 or greater that a secretary will be available, how many secretaries should be hired to constitute a pool of secretaries for the group of 90 executives?

2.7. Suppose that (i) 2, (ii) 3 restaurants compete for the same 800 patrons. Find the number of seats that each restaurant should have in order to have a probability greater than that it can serve all patrons who come to it (assuming that all patrons arrive at the same time and choose, independently of one another, each restaurant with equal probability).

Answer

(i) 423; (ii) 289.

2.8. At a certain men’s college the probability that a student selected at random on a given day will require a hospital bed is . If there are 8000 students, how many beds should the hospital have so that the probability that a student will be turned away for lack of a bed is less than 1% (in other words, find so that , where is the number of students requiring beds).

2.9. Consider an experiment in which the probability of success at each trial is . Let denote the successes in independent trials of the experiment. Show that

Consequently, if , with probability approximately equal to 0.95, the observed number of successes in independent trials will satisfy the inequalities

Determine how large should be, under the assumption that (i) , (ii) , (iii) , to have a probability of that the observed number of successes in the trials will satisfy (2.26).

Answer

Choose so that (i), (ii) ;

(iii) . One may obtain an upper bound for : (i), (ii) ; (iii) .

2.10. In his book Natural Inheritance , p. 63, F. Galton in 1889 described an apparatus known today as Galton’s quincunx . The apparatus consists of a board in which nails are arranged in rows, the nails of a given row being placed below the mid-points of the intervals between the nails in the row above. Small steel balls of equal diameter are poured into the apparatus through a funnel located opposite the central pin of the first row. As they run down the board, the balls are “influenced” by the nails in such a manner that, after passing through the last row, they take up positions deviating from the point vertically below the central pin of the first row. Let us call this point . Assume that the distance between 2 neighboring pins is taken to be 1 and that the diameter of the balls is slightly smaller than 1. Assume that in passing from one row to the next the abscissa ( -coordinate) of a ball changes by either or , each possibility having equal probability. To each opening in a row of nails, assign as its abscissa the mid-point of the interval between the 2 nails. If there is an even number of rows of nails, then the openings in the last row will have abscissas Assuming that there are 36 rows of nails, find for the probability that a ball inserted in the funnel will pass through the opening in the last row, which has abscissa .

2.11. Consider a liquid of volume , which contains bacteria. Let the liquid be vigorously shaken and part of it transferred to a test tube of volume . Suppose that (i) the probability that any given bacterium will be transferred to the test tube is equal to the ratio of the volumes and that (ii) the appearance of I particular bacterium in the test tube is independent of the appearance of the other bacteria. Consequently, the number of bacteria in the test tube is a numerical valued random phenomenon obeying a binomial probability law with parameters and . Let denote the average number of bacteria per unit volume. Let the volume of the test tube be equal to 3 cubic centimeters.
(i) Assume that the volume of the test tube is very small compared to the volume of liquid, so that is a small number. In particular, assume that and that the bacterial density bacteria per cubic centimeter. Find approximately the probability that the number of bacteria in the test tube will be greater than 1.

(ii) Assume that the volume of the test tube is comparable to the volume of the liquid. In particular, assume that cubic centimeters and . What is the probability that the number of bacteria in the test tube will be between 2400 and 2600, inclusive?

Answer

(i) 0.983; (ii) 0.979.

2.12. Suppose that among 10,000 students at a certain college 100 are red-haired.

(i) What is the probability that a sample of 100 students, selected with replacement, will contain at least one red-haired student?

(ii) How large is a random sample, drawn with replacement, if the probability of its containing a red-haired student is 0.95?

It would be more realistic to assume that the sample is drawn without replacement. Would the answers to (i) and (ii) change if this assumption were made?

Hint: State conditions under which the hypergeometric law is approximated by the Poisson law.

2.13. Let be the observed number of successes in independent repeated Bernoulli trials with probability of success at each trial. For each of the following events, find (i) its exact probability calculated by use of the binomial probability law, (ii) its approximate probability calculated by use of the normal approximation, (iii) the percentage error involved in using (ii) rather than (i).


(i)	4	0.3	(viii)	49	0.2
(ii)	9	0.7	(ix)	49	0.2
(iii)	9	0.7	(x)	49	0.2
(iv)	16	0.4	(xi)	100	0.5
(v)	16	0.2	(xii)	100	0.5
(vi)	25	0.9	(xiii)	100	0.5
(vii)	25	0.3	(xiv)	100	0.5

The Approximation of the Binomial Probability Law by the Normal and Poisson Probability Laws

Theoretical Exercises

Exercises

Quick Links

Social Media