Inverse-Wishart distribution


Inverse-Wishart distribution

In statistics, the inverse Wishart distribution, also called the inverted Wishart distribution, is a probability density function defined on matrices. In Bayesian statistics it is used as the conjugate prior for the covariance matrix of a
multivariate normal distribution.

We say {mathbf B} follows an inverse Wishart distribution, denoted as mathbf{B}sim W^{-1}({mathbfPsi},m), if its probability density function is written as follows:

:frac{left|{mathbfPsi} ight|^{m/2}left|B ight|^{-(m+p+1)/2}e^{-mathrm{trace}({mathbfPsi}{mathbf B}^{-1})/2{2^{mp/2}Gamma_p(m/2)},

where {mathbf B} is a p imes p matrix. The matrix {mathbfPsi} is assumed to be positive definite.

Theorems

Distribution of the inverse of a Wishart-distributed matrix

If {mathbf A}sim W({mathbfSigma},m) and {mathbfSigma} is p*p, then {mathbf B}={mathbf A}^{-1} has an inverse Wishart distribution {mathbf B}sim W^{-1}({mathbfSigma}^{-1},m) with probability density function::p(mathbf{B}|mathbf{Psi},m) = frac{left|{mathbfPsi} ight|^{m/2}left|mathbf{B} ight|^{-(m+p+1)/2}expleft({-mathrm{tr}({mathbfPsi}{mathbf B}^{-1})/2} ight)}{2^{mp/2}Gamma_p(m/2)}where mathbf{Psi} = mathbf{Sigma}^{-1} and Gamma_p(cdot) is the multivariate gamma function. [Cite book
author = Kanti V. Mardia, J. T. Kent and J. M. Bibby
title = Multivariate Analysis
publisher = Academic Press
year = 1979
isbn = 0-12-471250-9
]

Marginal and conditional distributions from an inverse Wishart-distributed matrix

Suppose {mathbf A}sim W^{-1}({mathbfPsi},m) has an inverse Wishart distribution. Partition the matrices {mathbf A} and {mathbfPsi} conformably with each other : {mathbf{A = egin{bmatrix} mathbf{A}_{11} & mathbf{A}_{12} \ mathbf{A}_{21} & mathbf{A}_{22} end{bmatrix}, ; {mathbf{Psi = egin{bmatrix} mathbf{Psi}_{11} & mathbf{Psi}_{12} \ mathbf{Psi}_{21} & mathbf{Psi}_{22} end{bmatrix} where {mathbf A_{ij and {mathbf Psi_{ij are p_{i} imes p_{j} matrices, then we have

i) {mathbf A_{11} } is independent of {mathbf A}_{11}^{-1}{mathbf A}_{12} and {mathbf A}_{22cdot 1} , where {mathbf A_{22cdot 1 = {mathbf A}_{22} - {mathbf A}_{21}{mathbf A}_{11}^{-1}{mathbf A}_{12} is the Schur complement of {mathbf A_{11} } in {mathbf A} ;

ii) {mathbf A_{11} } sim W^{-1}({mathbf Psi_{11} }, m-p_{2}) ;

iii) {mathbf A}_{11}^{-1} {mathbf A}_{12}| {mathbf A}_{22cdot 1} sim MN_{p_{1} imes p_{2( {mathbf Psi}_{11}^{-1} {mathbf Psi}_{12}, {mathbf A}_{22cdot 1} otimes {mathbf Psi}_{11}^{-1}) , where MN_{p imes q}(cdot,cdot) is a matrix normal distribution;

iv) {mathbf A}_{22cdot 1} sim W^{-1}({mathbf Psi}_{22cdot 1}, m)

Conjugate distribution

Suppose we wish to make inference about a covariance matrix {mathbf{Sigma whose prior {p(mathbf{Sigma})} has a W^{-1}({mathbfPsi},m) distribution. If the observations mathbf{X}= [mathbf{x}_1,ldots,mathbf{x}_n] are independent p-variate gaussian variables drawn from a N(mathbf{0},{mathbf Sigma}) distribution, then the conditional distribution {p(mathbf{Sigma}|mathbf{X})} has a W^{-1}({mathbf A}+{mathbfPsi},n+m) distribution, where {mathbf{A=mathbf{X}mathbf{X}^T is n times the sample covariance matrix.

Because the prior and posterior distributions are the same family, we say the inverse Wishart distribution is conjugate to the multivariate Gaussian.

Moments

The following is based on Press, S. J. (1982) "Applied Multivariate Analysis", 2nd ed. (Dover Publications, New York), after reparameterizing the degree of freedom to be consistent with the p.d.f. definition above.

The mean::E(mathbf B) = frac{mathbfPsi}{m-p-1}.

The variance of each element of mathbf{B}::mbox{var}(b_{ij}) = frac{(m-p+1)psi_{ij}^2 + (m-p-1)psi_{ii}psi_{jj{(m-p)(m-p-1)^2(m-p-3)}The variance of the diagonal uses the same formula as above with i=j, which simplifies to::mbox{var}(b_{ii}) = frac{2psi_{ii}^2}{(m-p-1)^2(m-p-3)}.

Related distributions

A univariate specialization of the inverse-Wishart distribution is the inverse-gamma distribution. With p=1 (i.e. univariate) and alpha = m/2, eta = mathbf{Psi}/2 and x=mathbf{B} the probability density function of the inverse-Wishart distribution becomes

: p(x|alpha, eta) = frac{eta^alpha, x^{-alpha-1} exp(-eta/x)}{Gamma_1(alpha)}.

i.e., the inverse-gamma distribution, where Gamma_1(cdot) is the ordinary Gamma function.

A generalization is the normal-inverse-Wishart distribution.

ee also

*Wishart distribution
*Matrix normal distribution

References


Wikimedia Foundation. 2010.

Look at other dictionaries:

  • Inverse-gamma distribution — Probability distribution name =Inverse gamma type =density pdf cdf parameters =alpha>0 shape (real) eta>0 scale (real) support =xin(0;infty)! pdf =frac{eta^alpha}{Gamma(alpha)} x^{ alpha 1} exp left(frac{ eta}{x} ight) cdf… …   Wikipedia

  • Wishart distribution — Probability distribution name =Wishart type =density pdf cdf parameters = n > 0! deg. of freedom (real) mathbf{V} > 0, scale matrix ( pos. def) support =mathbf{W}! is positive definite pdf =frac{left|mathbf{W} ight|^frac{n p 1}{2… …   Wikipedia

  • Normal-scaled inverse gamma distribution — Normal scaled inverse gamma parameters: location (real) (real) (real) (real) support …   Wikipedia

  • Normal-inverse Gaussian distribution — Normal inverse Gaussian (NIG) parameters: μ location (real) α tail heavyness (real) β asymmetry parameter (real) δ scale parameter (real) support …   Wikipedia

  • Matrix normal distribution — parameters: mean row covariance column covariance. Parameters are matrices (all of them). support: is a matrix …   Wikipedia

  • Probability distribution — This article is about probability distribution. For generalized functions in mathematical analysis, see Distribution (mathematics). For other uses, see Distribution (disambiguation). In probability theory, a probability mass, probability density …   Wikipedia

  • Multivariate normal distribution — MVN redirects here. For the airport with that IATA code, see Mount Vernon Airport. Probability density function Many samples from a multivariate (bivariate) Gaussian distribution centered at (1,3) with a standard deviation of 3 in roughly the… …   Wikipedia

  • Chi-squared distribution — This article is about the mathematics of the chi squared distribution. For its uses in statistics, see chi squared test. For the music group, see Chi2 (band). Probability density function Cumulative distribution function …   Wikipedia

  • Uniform distribution (continuous) — Uniform Probability density function Using maximum convention Cumulative distribution function …   Wikipedia

  • Cauchy distribution — Not to be confused with Lorenz curve. Cauchy–Lorentz Probability density function The purple curve is the standard Cauchy distribution Cumulative distribution function …   Wikipedia