Normal-gamma distribution


Normal-gamma distribution
Normal-gamma
parameters: \mu\, location (real)
\lambda > 0\, (real)
\alpha \ge 1\, (real)
\beta \ge 0\, (real)
support: x \in (-\infty, \infty)\,\!, \; \tau \in (0,\infty)
pdf: f(x,\tau|\mu,\lambda,\alpha,\beta) = \frac{\beta^\alpha \sqrt{\lambda}}{\Gamma(\alpha)\sqrt{2\pi}}  \, \tau^{\alpha-\frac{1}{2}}\,e^{-\beta\tau}\,e^{ -\frac{ \lambda \tau (x- \mu)^2}{2}}
mean: [1] \operatorname{E}(X)=\mu\,\! ,\quad \operatorname{E}(\Tau)= \alpha \beta^{-1}
variance: [1] \operatorname{var}(X)= \frac{\beta}{\lambda (\alpha-1)} ,\quad 
\operatorname{var}(\Tau)=\alpha \beta^{-2}

In probability theory and statistics, the normal-gamma distribution is a bivariate four-parameter family of continuous probability distributions. It is the conjugate prior of a normal distribution with unknown mean and precision.[2]

Contents

Definition

Suppose

  x|\tau, \mu, \lambda \sim N(\mu,1 /(\lambda  \tau)) \,\!

has a normal distribution with mean μ and variance 1 / (λτ), where

\tau|\alpha, \beta \sim \mathrm{Gamma}(\alpha,\beta) \!

has a gamma distribution. Then (x,τ) has a normal-gamma distribution, denoted as

 (x,\tau) \sim \mathrm{NormalGamma}(\mu,\lambda,\alpha,\beta) \! .

Characterization

Probability density function

f(x,\tau|\mu,\lambda,\alpha,\beta) = \frac{\beta^\alpha \sqrt{\lambda}}{\Gamma(\alpha)\sqrt{2\pi}}  \, \tau^{\alpha-\frac{1}{2}}\,e^{-\beta\tau}\,e^{ -\frac{ \lambda \tau (x- \mu)^2}{2}}

Properties

Scaling

For any t > 0, tX is distributed NormalGamma(tμ,λ,α,t2β)

Marginal distributions

By construction, the marginal distribution over τ is a gamma distribution, and the conditional distribution over x given τ is a Gaussian distribution. The marginal distribution over x is a three-parameter Student's t-distribution.

Posterior distribution of the parameters

Form of the posterior for a Normal random variable with a Normal-Gamma prior:

Presume the following hierarchy for a normal random variable X with unknown mean μ and precision λ.


\begin{align}
X & \sim \mathcal{N}(\mu, \lambda^{-1}) \\
\mu | \lambda &\sim \mathcal{N}(\mu_0, {(n_0 \lambda})^{-1}) \\
\lambda &\sim \mathcal{G}\left(\frac{\nu_0}{2},\frac{2}{S_0}\right)
\end{align}

Where:

μ0 is the prior mean
S0 is the prior sum of squared errors
n0 is the prior sample size
ν0 is the prior degrees of freedom

Note the joint distribution of the parameters is Normal-Gamma. The posterior distribution of the parameters can be analytically determined by Bayes' rule working with the likelihood \mathbf{L(\lambda, \mu | X)}, and the prior π(λ,μ).


\begin{align}
\mathbf{L(\lambda, \mu | X)} & \propto \prod_{i=1}^n \lambda^{1/2} \exp[\frac{-\lambda}{2}(x_i-\mu)^2] \\ 
 &  \propto \lambda^{n/2} \exp[\frac{-\lambda}{2}\sum_{i=1}^n(x_i-\mu)^2] \\
 &  \propto \lambda^{n/2} \exp[\frac{-\lambda}{2}\sum_{i=1}^n(x_i-\bar{x} +\bar{x} -\mu)^2] \\
 &  \propto \lambda^{n/2} \exp[\frac{-\lambda}{2}\sum_{i=1}^n\left((x_i-\bar{x})^2 + (\bar{x} -\mu)^2\right)] \\
& \propto \lambda^{n/2} \exp[\frac{-\lambda}{2}\left(S + n(\bar{x} -\mu)^2\right)] 
\end{align}

where S=\sum_{i=1}^n(x_i-\bar{x})^2, the sum of squared errors.

Now consider the prior,


\mathbf{\pi}(\mu,\lambda)  \propto \lambda^{1/2}\exp[\frac{-\lambda n_0}{2}(\mu-\mu_0)^2] \lambda^{\frac{\nu_0}{2}-1}\exp[\frac{-\lambda S_0}{2}]

The posterior distribution of the parameters is proportional to the prior times the likelihood.


\begin{align}
\mathbf{P(\lambda, \mu | X}) &\propto \lambda^{n/2} \exp[\frac{-\lambda}{2}\left(S + n(\bar{x} -\mu)^2\right)] 
 \lambda^{1/2}\exp[\frac{-\lambda n_0}{2}(\mu-\mu_0)^2] \lambda^{\frac{\nu_0}{2}-1}\exp[\frac{-\lambda S_0}{2}] \\ 
 &\propto \lambda^{\frac{\nu_0+n}{2}-1} \exp[\frac{-\lambda}{2}(S + S_0) ] 
 \lambda^{1/2}\exp[\frac{-\lambda}{2}\left(n_0(\mu-\mu_0)^2 + n(\bar{x} -\mu)^2\right)]  \\ 
\end{align}

Notice the right half begins to look like the kernel of a normal pdf and the left like a gamma. After a bit of juggling and completing the square the result will appear.


\begin{align}

\mathbf{P(\lambda, \mu | X} )& \propto \lambda^{\frac{\nu_0+n}{2}-1} \exp[\frac{-\lambda}{2}(S + S_0) ] 
 \lambda^{1/2}\exp[\frac{- \lambda}{2} \left(n_0 (\mu^2 - 2 \mu \mu_0 + \mu_0^2 ) + n(\bar{x}^2-2 \mu \bar{x} + \mu^2)\right)]  \\ 
  & \propto \lambda^{\frac{\nu_0+n}{2}-1} \exp[\frac{-\lambda}{2}(S + S_0 + n_0 \mu_0^2 + n \bar{x}^2) ] 
 \lambda^{1/2}\exp[\frac{-\lambda}{2} (n+n_0) \left(\frac{n_0 \mu^2 + n \mu^2 }{n + n_0} - 2 \mu \frac{n\bar{x} +n_0\mu_0}{n+n_0} \right)]   \\
  & \propto \lambda^{\frac{\nu_0+n}{2}-1} \exp[\frac{-\lambda}{2}(S + S_0 + n_0 \mu_0^2 + n \bar{x}^2) ] 
 \lambda^{1/2}\exp[\frac{-\lambda}{2} (n+n_0) \left(\mu^2 - 2 \mu \frac{n\bar{x} +n_0\mu_0}{n+n_0} + \left (\frac{n\bar{x} +n_0\mu_0}{n+n_0}\right )^2 - \left (\frac{n\bar{x} +n_0\mu_0}{n+n_0}\right )^2\right)]   \\
  & \propto \lambda^{\frac{\nu_0+n}{2}-1} \exp[\frac{-\lambda}{2}\left(S + S_0 + n_0 \mu_0^2 + n \bar{x}^2 - \frac{\left (n\bar{x} +n_0\mu_0 \right )^2}{n+n_0}\right) ] 
 \lambda^{1/2}\exp[\frac{-\lambda}{2} (n+n_0) \left ( \mu - \frac{n\bar{x} +n_0\mu_0}{n+n_0}\right )^2]   \\
  & \propto \lambda^{\frac{\nu_0+n}{2}-1} \exp[\frac{-\lambda}{2}\left(S + S_0 + \frac{nn_0 (\bar{x}-\mu_0)^2}{n+n_0}\right) ] 
 \lambda^{1/2}\exp[\frac{-\lambda}{2} (n+n_0) \left ( \mu - \frac{n\bar{x} +n_0\mu_0}{n+n_0}\right )^2]   .\\

\end{align}

This is a normal gamma pdf with parameters 
\mathcal{NG} \left(\frac{n\bar{x} +n_0\mu_0}{n+n_0}, n+n_0, \frac{\nu_0+n}{2}, 2\left(S + S_0 + \frac{nn_0 (\bar{x}-\mu_0)^2}{n+n_0}\right)^{-1} \right) .

The reference prior is[citation needed] the limiting case as

n_0, S_0, \mu_0 \rightharpoonup 0

and \nu_0 \rightharpoonup -1

Generating normal-gamma random variates

Generation of random variates is straightforward:

  1. Sample τ from a gamma distribution with parameters α and β
  2. Sample x from a normal distribution with mean μ and variance 1 / (λτ)

Related distributions

Notes

  1. ^ a b Bernardo & Smith (1993, p.434)
  2. ^ Bernardo & Smith (1993, pages 136, 268, 434)

References

  • Bernardo, J.M.; Smith, A.F.M. (1993) Bayesian Theory, Wiley. ISBN 0-471-49464-X
  • Dearden et al. Bayesian Q-learning, Proceedings of the Fifteenth National Conference on Artificial Intelligence (AAAI-98), July 26–30, 1998, Madison, Wisconsin, USA.

Wikimedia Foundation. 2010.

Look at other dictionaries:

  • Normal-scaled inverse gamma distribution — Normal scaled inverse gamma parameters: location (real) (real) (real) (real) support …   Wikipedia

  • Normal-exponential-gamma distribution — Normal Exponential Gamma parameters: μ ∈ R mean (location) shape scale support: pdf …   Wikipedia

  • Gamma distribution — eta m = V {3m} e^{ xi m}. # If eta m > xi m^{delta 1} e^{ xi m}, then increment m and go to step 2. # Assume xi = xi m to be the realization of Gamma (delta, 1)Now, to summarize,: heta left( xi sum {i=1} ^{ [k] } {ln U i} ight) sim Gamma (k,… …   Wikipedia

  • Variance-gamma distribution — Probability distribution name =variance gamma distribution type =density pdf cdf parameters =mu location (real) alpha (real) eta asymmetry parameter (real) lambda > 0 gamma = sqrt{alpha^2 eta^2} > 0 support =x in ( infty; +infty)! pdf… …   Wikipedia

  • Inverse-gamma distribution — Probability distribution name =Inverse gamma type =density pdf cdf parameters =alpha>0 shape (real) eta>0 scale (real) support =xin(0;infty)! pdf =frac{eta^alpha}{Gamma(alpha)} x^{ alpha 1} exp left(frac{ eta}{x} ight) cdf… …   Wikipedia

  • Normal-inverse Gaussian distribution — Normal inverse Gaussian (NIG) parameters: μ location (real) α tail heavyness (real) β asymmetry parameter (real) δ scale parameter (real) support …   Wikipedia

  • Normal distribution — This article is about the univariate normal distribution. For normally distributed vectors, see Multivariate normal distribution. Probability density function The red line is the standard normal distribution Cumulative distribution function …   Wikipedia

  • Gamma function — For the gamma function of ordinals, see Veblen function. The gamma function along part of the real axis In mathematics, the gamma function (represented by the capital Greek letter Γ) is an extension of the factorial function, with its… …   Wikipedia

  • Gamma ray burst progenitors — are the types of celestial objects that can emit gamma ray bursts (GRBs). GRBs show an extraordinary degree of diversity. They can last anywhere from a fraction of a second to many minutes. Bursts could have a single profile or oscillate wildly… …   Wikipedia

  • Gamma Pricing Model — An equation for determining the fair market value of a European style option when the price movement on the underlying asset does not resemble a normal distribution. The gamma pricing model is intended to price options where the underlying asset… …   Investment dictionary