Signal-To-Noise Ratio of a Random Variable

A question of great importance in science and engineering is the following: under what conditions can an observed value of a random variable be identified with its mean ? We have seen in section 5 that if is the arithmetic mean of a very large number of independent identically distributed random variables then for any preassigned distance an observed value of will, with high probability, be within of . In this section we discuss some conditions under which an observed value of a random variable may be identified with its mean.

If has finite mean and variance , then the condition that an observed value of is, with high probability, within a preassigned distance from its mean may be obtained from Chebyshev’s inequality: for any From (6.1) one obtains these conclusions:

If may be assumed to be approximately normally distributed, then From (6.3) one obtains these conclusions: As a measure of how close the observed value of will be to its mean , one often uses not the absolute deviation but the relative deviation assuming that .

Chebyshev’s inequality may be reformulated in terms of the relative deviation: for any From (6.6) one obtains these conclusions:

Similarly, if is approximately normally distributed,

From the foregoing inequalities we obtain this basic conclusion for a random variable with nonzero mean and finite variance.

In order that the percentage error of as an estimate of may with high probability be small, it is sufficient that the ratio be large . The quantity in (6.9) is called the measurement signal-to-noise ratio ¹of the random variable .

How large must the measurement signal-to-noise ratio of a random variable be in order that its observed value be a good estimate of its mean? By (6.7) and (6.8), various answers to this question can be obtained.

For example, if it is desired that then the measurement signal-to-noise ratio must satisfy approximately

The measurement signal-to-noise ratio of various random variables is given in Table 6A. One sees that for most of the random variables given the measurement signal-to-noise ratio is proportional to the square root of some parameter. For example, suppose the number of particles emitted by a radioactive source during a certain time interval is being counted. The number of particles emitted obeys a Poisson probability law with some parameter whose value is unknown. If the true value of is known to be very large, then the observed number of emitted particles is a good estimate of , since the measurement signal-to-noise ratio of is .

Probability Law of
Poisson, with parameter
Binomial, with parameters and
Geometric, with parameter
Uniform over the interval to
Normal, with parameters and
Exponential, with parameter
, with degrees of freedom
with , degrees of freedom	if	if

TABLE 6A. Measurement Signal-to-Noise Ratio of Random Variables Obeying Various Probability Laws

It is shown in Chapter 10 that many of the random variables in Table 6A are approximately normally distributed in cases in which their measurement signal-to-noise ratio is very large.

Example 6A. The density of an ideal gas . An ideal gas can be regarded as a collection of molecules distributed randomly in a volume . The density of the gas in a subvolume , contained in , is a random variable given by , in which is the mass of one gas molecule and is the number of molecules in the volume . Since it is assumed that each of the molecules has an independent probability of being in the subvolume , the number of molecules in obeys a binomial probability law with mean and variance , in which we have let and . The density then has mean . In speaking of the density of gas in the volume , the physicist usually has in mind the mean density. The question naturally arises: under what circumstances is the relative deviation of the true density from the mean density within a preassigned percentage error ? More specifically, what values must , , and have in order that

in which and are preassigned positive quantities. By Chebyshev’s inequality

Consequently, if the quantities , and are such that

then (6.12) holds. Because of the enormous size of (which is of the order per ), one would expect (6.14) to be satisfied for , say, as long as is not too small. In this case it makes sense to speak of the density of gas in , even though the number of molecules in is not fixed but fluctuates. However, if is very small, the fluctuations become sufficiently pronounced, and the ordinary notion of density, which identifies density with mean density, loses its meaning. The “density fluctuations” in small volumes can actually be detected experimentally inasmuch as they cause scattering of sufficiently short wavelengths.

Example 6B. The law of . The physicist Erwin Schrödinger has pointed out in the following statement ( What is Life , Cambridge University Press, 1945, p.16), “…the degree of inaccuracy to be expected in any physical law, the so-called law. The laws of physics and physical chemistry are inaccurate within a probable relative error of the order of , where is the number of molecules that cooperate to bring about that law”. From the law of Schrödinger draws the conclusion that in order for the laws of physics and chemistry to be sufficient to explain the laws governing the behavior of living organisms it is necessary that the biologically relevant processes of such an organism involve the cooperation of a very large number of atoms, for only in this case do the laws of physics become exact laws. Since one can show that there are “incredibly small groups of atoms, much too small to display exact statistical laws, which play a dominating role in the very orderly and lawful events within a living organism”, Schrödinger conjectures that it may not be possible to interpret life by the ordinary laws of physics, based on the “statistical mechanism which produces order from disorder”. We state here a mathematical formulation of the law of . If are independent random variables identically distributed as a random variable , then the sum and the sample mean have measurement signal-to-noise ratios given by

In words, the sum or average of repeated independent measurements of a random variable has a measurement signal-to-noise ratio of the order of .

Example 6C. Can the energy of an ideal gas be both constant and a distributed random variable? In example 9H of Chapter 7 it is shown that if the state of an ideal gas is a random phenomenon whose probability law is given by Gibbs’s canonical distribution then the energy of the gas is a random variable possessing a distribution with degrees of freedom, in which is the number of particles comprising the gas. Does this mean that if a gas has constant energy its state as a point in the space of all possible velocities cannot be regarded as obeying Gibbs’s canonical distribution? The answer to this question is no. From a practical point of view there is no contradiction in regarding the energy of the gas as being both a constant and a random variable with a distribution if the number of degrees of freedom is very large, for then the measurement signal-to-noise ratio of (which, from Table 6A, is equal to ) is also very large.

The terminology “signal-to-noise ratio” originated in communications theory. The mean of a random variable is regarded as a signal that one is attempting to receive (say, at a radio receiver). However, is actually received. The difference between the desired value and the received value is called noise . The less noise present, the better one is able to receive the signal accurately. As a measure of signal strength to noise strength, one takes the signal-to-noise ratio defined by (6.9). The higher the signal-to-noise ratio, the more accurate the observed value as an estimate of the desired value .

Any time a scientist makes a measurement he is attempting to obtain a signal in the presence of noise or, equivalently, to estimate the mean of a random variable. The skill of the experimental scientist lies in being able to conduct experiments that have a high measurement signal-to-noise ratio. However, there are experimental situations in which this may not be possible. For example, there is an inherent limit on how small one can make the variance of measurements taken with electronic devices. This limit arises from the noise or spontaneous current fluctuations present in such devices (see example 3D of Chapter 6). To measure weak signals in the presence of noise (that is, to measure the mean of a random variable with a small measurement signal-to-noise ratio) one should have a good knowledge of the modern theories of statistical inference.

On the one hand, the scientist and engineer should know statistics in order to interpret best the statistical significance of the data he has obtained. On the other hand, a knowledge of statistics will help the scientist or engineer to solve the basic problem confronting him in taking measurements: given a parameter , which he wishes to measure, to find random variables , whose observed values can be used to form estimates of that are best according to some criteria.

Measurement signal-to-noise ratios play a basic role in the evaluation of modern electronic apparatus. The reader interested in such questions may consult J. J. Freeman, Principles of Noise , Wiley, New York, 1958, Chapters 7 and 9.

Exercises

6.1. A random variable has an unknown mean and known variance 4. How large a random sample should one take if the probability is to be at least 0.95 that the sample mean will not differ from the true mean by (i) more than 0.1, (ii) more than of the standard deviation of , (iii) more than of the true mean of , if the true mean of is known to be greater than 10.

Answer

(i) ; (ii) ; (iii) .

6.2. Let be independent normally distributed random variables with known mean 0 and unknown common variance . Define

Since might be used as an estimate of . How large should be in order to have a measurement signal-to-noise ratio of greater than 20? If the measurement signal-to-noise ratio of is greater than 20, how good is as an estimate of ?

6.3. Consider a gas composed of molecules (with mass of the order of grams and at room temperature) whose velocities obey the MaxwellBoltzmann law (see exercise 1.15 ). Show that one may assume that all the molecules move with the same velocity, which may be taken as either the mean velocity, the root mean square velocity, or the most probable velocity.

Answer

The measurement signal-to-noise ratio of a random variable is the reciprocal of the coefficient of variation of the random variable. (For a definition of the latter, see M. G. Kendall and A. Stuart, The Advanced Theory of Statistics, Griffin, London, 1958, p. 47.) ↩︎

The Measurement Signal-To-Noise Ratio of a Random Variable

Exercises

Quick Links

Social Media