Normalizing constant

Normalizing constant

The concept of a normalizing constant arises in probability theory and a variety of other areas of mathematics.

Contents

Definition and examples

In probability theory, a normalizing constant is a constant by which an everywhere non-negative function must be multiplied so the area under its graph is 1, e.g., to make it a probability density function or a probability mass function.[1][2] For example, if we define

p(x)=e^{-x^2/2}, x\in(-\infty,\infty)

we have

\int_{-\infty}^\infty p(x)\,dx=\int_{-\infty}^\infty e^{-x^2/2}\,dx=\sqrt{2\pi\,},

if we define function  \varphi(x) as

\varphi(x)= \frac{1}{\sqrt{2\pi\,}} p(x) = \frac{1}{\sqrt{2\pi\,}} e^{-x^2/2}

so that

\int_{-\infty}^\infty \varphi(x)\,dx=\int_{-\infty}^\infty \frac{1}{\sqrt{2\pi\,}} e^{-x^2/2}\,dx=1

Function  \varphi(x) is a probability density function.[3] This is the density of the standard normal distribution. (Standard, in this case, means the expected value is 0 and the variance is 1.)

And constant  \frac{1}{\sqrt{2\pi\,}} is the normalizing constant of function p(x).

Similarly,

\sum_{n=0}^\infty \frac{\lambda^n}{n!}=e^\lambda ,

and consequently

f(n)=\frac{\lambda^n e^{-\lambda}}{n!}

is a probability mass function on the set of all nonnegative integers.[4] This is the probability mass function of the Poisson distribution with expected value λ.

Note that if the probability density function is a function of various parameters, so too will be its normalizing constant. The parametrised normalizing constant for the Boltzmann distribution plays a central role in statistical mechanics. In that context, the normalizing constant is called the partition function.

Bayes' theorem

Bayes' theorem says that the posterior probability measure is proportional to the product of the prior probability measure and the likelihood function. Proportional to implies that one must multiply or divide by a normalizing constant to assign measure 1 to the whole space, i.e., to get a probability measure. In a simple discrete case we have

P(H_0|D) = \frac{P(D|H_0)P(H_0)}{P(D)}

where P(H0) is the prior probability that the hypothesis is true; P(D|H0) is the conditional probability of the data given that the hypothesis is true, but given that the data are known it is the likelihood of the hypothesis (or its parameters) given the data; P(H0|D) is the posterior probability that the hypothesis is true given the data. P(D) should be the probability of producing the data, but on its own is difficult to calculate, so an alternative way to describe this relationship is as one of proportionality:

P(H_0|D) \propto P(D|H_0)P(H_0).

Since P(H|D) is a probability, the sum over all possible (mutually exclusive) hypotheses should be 1, leading to the conclusion that

P(H_0|D) = \frac{P(D|H_0)P(H_0)}{\displaystyle\sum_i P(D|H_i)P(H_i)} .

In this case, the reciprocal of the value

P(D)=\sum_i P(D|H_i)P(H_i) \;

is the normalizing constant.[5] It can be extended from countably many hypotheses to uncountably many by replacing the sum by an integral.

Non-probabilistic uses

The Legendre polynomials are characterized by orthogonality with respect to the uniform measure on the interval [− 1, 1] and the fact that they are normalized so that their value at 1 is 1. The constant by which one multiplies a polynomial so its value at 1 is 1 is a normalizing constant.

Orthonormal functions are normalized such that

\langle f_i , \, f_j\rangle = \, \delta_{i,j}

with respect to some inner product <fg>.

The constant 1/√2 is used to establish the hyperbolic functions cosh and sinh from the lengths of the adjacent and opposite sides of a hyperbolic triangle.

Notes

  1. ^ Continuous Distributions at University of Alabama.
  2. ^ Feller, 1968, p. 22.
  3. ^ Feller, 1968, p. 174.
  4. ^ Feller, 1968, p. 156.
  5. ^ Feller, 1968, p. 124.

References

  • Continuous Distributions at Department of Mathematical Sciences: University of Alabama in Huntsville
  • Feller, William (1968). An Introduction to Probability Theory and its Applications (volume I). John Wiley & Sons. ISBN 0-471-25708-7. 

Wikimedia Foundation. 2010.

Игры ⚽ Нужен реферат?

Look at other dictionaries:

  • Identical particles — Statistical mechanics Thermodynamics · …   Wikipedia

  • Wave function — Not to be confused with the related concept of the Wave equation Some trajectories of a harmonic oscillator (a ball attached to a spring) in classical mechanics (A B) and quantum mechanics (C H). In quantum mechanics (C H), the ball has a wave… …   Wikipedia

  • Worldwide Youth in Science and Engineering — The Worldwide Youth in Science and Engineering Academic Challenge is a high school academic competition run in Illinois and Missouri by the University of Illinois at Urbana Champaign and Missouri University of Science and Technology, respectively …   Wikipedia

  • Gordon–Newell theorem — In queueing theory, a discipline within the mathematical theory of probability, the Gordon–Newell theorem is an extension of Jackson s theorem from open queueing networks to closed queueing networks of exponential servers.[1] We cannot apply… …   Wikipedia

  • Normalization (statistics) — For other uses, see Standard score and Normalizing constant. In one usage in statistics, normalization is the process of isolating statistical error in repeated measured data. A normalization is sometimes based on a property. Quantile… …   Wikipedia

  • Bayes' theorem — In probability theory, Bayes theorem (often called Bayes law after Thomas Bayes) relates the conditional and marginal probabilities of two random events. It is often used to compute posterior probabilities given observations. For example, a… …   Wikipedia

  • Posterior probability — The posterior probability of a random event or an uncertain proposition is the conditional probability that is assigned after the relevant evidence is taken into account.The posterior probability distribution of one random variable given the… …   Wikipedia

  • Three Prisoners problem — The Three Prisoners Problem appeared in Martin Gardner s Mathematical Games column in Scientific American in 1959 [Gardner, Martin (1959a). Mathematical Games column, Scientific American , October 1959, pp. 180–182.] [Gardner, Martin (1959b).… …   Wikipedia

  • Exponential distribution — Not to be confused with the exponential families of probability distributions. Exponential Probability density function Cumulative distribution function para …   Wikipedia

  • Dirac delta function — Schematic representation of the Dirac delta function by a line surmounted by an arrow. The height of the arrow is usually used to specify the value of any multiplicative constant, which will give the area under the function. The other convention… …   Wikipedia

Share the article and excerpts

Direct link
Do a right-click on the link above
and select “Copy Link”