Karhunen-Loève theorem

Karhunen-Loève theorem

In the theory of stochastic processes, the Karhunen-Loève theorem (named after Kari Karhunen and Michel Loève) is a representation of a stochastic process as an infinite linear combination of orthogonal functions, analogous to a Fourier series representation of a function on a bounded interval. In contrast to a Fourier series where the coefficients are real numbers and the expansion basis consists of sinusoidal functions (that is, sine and cosine functions), the coefficients in the Karhunen-Loève theorem are random variables and the expansion basis depends on the process. In fact, the orthogonal basis functions used in this representation are determined by the covariance function of the process. If we regard a stochastic process as a random "function" F, that is, one in which the random value is a function on an interval ["a", "b"] , then this theorem can be considered as a random orthonormal expansion of F.

In the case of a "centered" stochastic process {X"t"}"t" ∈ ["a", "b"] (where "centered" means that the expectations E(X"t") are defined and equal to 0 for all values of the parameter "t" in ["a", "b"] ) satisfying a technical continuity condition, admits a decomposition : mathbf{X}_t = sum_{k=1}^infty mathbf{Z}_k e_k(t). where Z"k" are pairwise uncorrelated random variables and the functions "e""k" are continuous real-valued functions on ["a", "b"] which are pairwise orthogonal in "L"2 ["a", "b"] . The general case of a process which is not centered can be represented by expanding the expectation function (which is a non-random function) in the basis "e""k" .

Moreover, if the process is Gaussian, then the random variables Z"k" are Gaussian and stochastically independent. This result generalizes the "Karhunen-Loève transform". An important example of a centered real stochastic process on [0,1] is the Wiener process and the Karhunen-Loève theorem can be used to provide a canonical orthogonal representation for it. In this case the expansion consists of sinusoidal functions.

The above expansion into uncorrelated random variables is also known as the "Karhunen-Loève expansion" or "Karhunen-Loève decomposition". The empirical version (i.e., with the coefficients computed from a sample) is known as Principal component analysis, "Proper orthogonal decomposition (POD)", or the " Hotelling transform".

Formulation

We will formulate the result in terms of complex-valued stochastic processes. The results apply to real-valued processes without modification by recognizing that the complex conjugate of a real number is the number itself.

If X and Y are random variables, the inner product is defined by

: langle mathbf{X}|mathbf{Y} angle = operatorname{E}(mathbf{X^*}mathbf{Y})

where * represents complex conjugation.

Second order statistics

The inner product is defined if both X and Y have finite second moments, or equivalently, if they are both square integrable. Note that the inner product is related to covariance and correlation. In particular, for random variables of mean zero, covariance and inner product coincide. The autocovariance function "K"XX is

: K_mathrm{XX}(t,s) = operatorname{Cov} [ X(t),X(s) ] = langle mathbf{X}_t | mathbf{X}_s angle

:::::= mathrm{E} { [ X(t)-mu_X(t) ] ^* [ X(s)-mu_X(s) ] } ,

:::::= mathrm{E} { X^*(t) X(s) } - mu^*_X(t) mu_X(s) ,

:::::= R_mathrm{XX}(t,s) - mu^*_X(t) mu_X(s) . ,

If {X"t"}"t" is a centered process, then

:mu_X(t) = 0 ,

for all "t". Thus, the autocovariance "K"XX is identical to the autocorrelation "R"XX:

: K_mathrm{XX}(t,s) = R_mathrm{XX}(t,s) . ,

Note that if {X"t"}"t" is centered and "t"1, ≤ "t"2, ..., ≤ "t""N" are points in ["a", "b"] , then

: sum_{k,ell} operatorname{Cov}_{mathbf{X(t_k,t_ell) = operatorname{Var}left(sum_{k=1}^N mathbf{X}_k ight) geq 0.

Statement of the theorem

Theorem. Consider a centered stochastic process {X"t"}"t" indexed by "t" in the interval ["a", "b"] with covariance function CovX. Suppose the covariance function CovX("t","s") is jointly continuous in "t", "s". Then CovX can be regarded as a positive definite kernel and so by Mercer's theorem, the corresponding integral operator "T" on L2 ["a","b"] (relative to Lebesgue measure on ["a","b"] ) has an orthonormal basis of eigenvectors. Let {"e""i"}"i" be the eigenvectors of "T" corresponding tonon-zero eigenvalues and: mathbf{Z}_i = int_a^b mathbf{X}_t e_i(t) dt. Then Z"i" are centered orthogonal random variables and: mathbf{X}_t = sum_{i=1}^infty e_i(t) mathbf{Z}_i where the convergence is in the mean and is uniform in "t". Moreover: operatorname{Var}(mathbf{Z}_i) = operatorname{E}(mathbf{Z}_i^2) = lambda_i. where λ"i" is the eigenvalue corresponding to the eigenvector "e""i".

Cauchy sums

In the statement of the theorem, the integral defining Z"i", can be defined as the limit in the mean of Cauchy sums of random variables:: sum_{k=0}^{ell-1} mathbf{X}_{xi_k} e_i(xi_k) (t_{k+1} - t_k), where: a = t_0 leq xi_0 leq t_1 leq cdots leq xi_{ell-1} leq t_n = b

Special case: Gaussian distribution

Since the limit in the mean of jointly Gaussian random variables is jointly Gaussian, and jointly Gaussian random (centered) variables are independent if and only if they are orthogonal, we can also conclude:

Theorem. The variables Z"i" have a joint Gaussian distribution and are stochastically independent if the original process {X"t"}"t" is Gaussian.

In the gaussian case, since the variables Z"i" are independent, we can say more:

: lim_{N ightarrow infty} sum_{i=1}^N e_i(t) mathbf{Z}_i(omega) = mathbf{X}_t(omega) almost surely.

Note that by generalizations of Mercer's theorem we can replace the interval ["a", "b"] with other compact spaces "C" and Lebesgue measure on ["a", "b"] with a Borel measure whose support is "C".

The Wiener process

There are numerous equivalent characterizations of the Wiener process which is a mathematical formalization of Brownian motion. Here we regard it as the centered standard Gaussian process "B"("t") with covariance function: mathrm{K}_mathrm{BB}(t,s) = operatorname{Cov}(B(t),B(s)) = min (s,t).

The eigenvectors of the covariance kernel are easily determined. These are: e_k(t) = sqrt{2} sin left(k - frac{1}{2} ight) pi t and the corresponding eigenvalues are: lambda_k = frac{4}{(2 k -1)^2 pi^2}.

This gives the following representation of the Wiener process:

Theorem. There is a sequence {W"i"}"i" of independent Gaussian random variables with mean zero and variance 1 such that: mathbf{B}_t = sqrt{2} sum_{k=1}^infty mathbf{W}_k frac{sin left(k - frac{1}{2} ight) pi t}{ left(k - frac{1}{2} ight) pi}. Convergence is uniform in "t" and in the L2 norm, that is: operatorname{E}left(mathbf{B}_t - sqrt{2} sum_{k=1}^n mathbf{W}_k frac{sin left(k - frac{1}{2} ight) pi t}{ left(k - frac{1}{2} ight) pi} ight)^2 ightarrow 0 uniformly in "t".

References

* I. Guikhman, A. Skorokhod, "Introduction a la Théorie des Processus Aléatoires" Éditions MIR, 1977
* B. Simon, "Functional Integration and Quantum Physics", Academic Press, 1979
* K. Karhunen, Kari, "Über lineare Methoden in der Wahrscheinlichkeitsrechnung", Ann. Acad. Sci. Fennicae. Ser. A. I. Math.-Phys., 1947, No. 37, 1--79
* M. Loève, "Probability theory." Vol. II, 4th ed., Graduate Texts in Mathematics, Vol. 46, Springer-Verlag, 1978, ISBN 0-387-90262-7

ee also

*Principal component analysis
*Proper orthogonal decomposition
*Polynomial chaos


Wikimedia Foundation. 2010.

Игры ⚽ Нужна курсовая?

Look at other dictionaries:

  • Michel Loève — Born January 22, 1907(1907 01 22) Jaffa, Palestine, Ottoman Syria Died February 17, 1979(1 …   Wikipedia

  • Michel Loève — Naissance 22 janvier 1907 Jaffa Décès 17 février 1979 Berkeley Nationalité …   Wikipédia en Français

  • Kari Karhunen — Naissance 1915 Décès 1992 Nationalité  Finlande Activité principale Mathématicien, Sta …   Wikipédia en Français

  • Michel Loève — (* 22. Januar 1907 in Jaffa; † 17. Februar 1979 in Berkeley) war ein US amerikanischer Mathematiker, der sich mit Wahrscheinlichkeitstheorie beschäftigte. Loève ging auf französisch sprachige Schulen in Ägypten und studierte in Paris (Vordiplom… …   Deutsch Wikipedia

  • Mercer's theorem — In mathematics, specifically functional analysis, Mercer s theorem is a representation of a symmetric positive definite function on a square as a sum of a convergent sequence of product functions. This theorem, presented in (Mercer 1909), is one… …   Wikipedia

  • Analyse en composantes principales — Pour les articles homonymes, voir ACP. L Analyse en Composantes Principales (ACP) est une méthode de la famille de l analyse des données et plus généralement de la statistique multivariée, qui consiste à transformer des variables liées entre… …   Wikipédia en Français

  • Théorème de Mercer — En mathématiques et plus précisément en analyse fonctionnelle, le théorème de Mercer est une représentation d une fonction symétrique de type positif par le carré d une série convergente de produits de fonctions. Ce théorème est l un des… …   Wikipédia en Français

  • List of mathematics articles (K) — NOTOC K K approximation of k hitting set K ary tree K core K edge connected graph K equivalence K factor error K finite K function K homology K means algorithm K medoids K minimum spanning tree K Poincaré algebra K Poincaré group K set (geometry) …   Wikipedia

  • List of statistics topics — Please add any Wikipedia articles related to statistics that are not already on this list.The Related changes link in the margin of this page (below search) leads to a list of the most recent changes to the articles listed below. To see the most… …   Wikipedia

  • Brownian motion — This article is about the physical phenomenon; for the stochastic process, see Wiener process. For the sports team, see Brownian Motion (Ultimate). For the mobility model, see Random walk. Brownian motion (named after the botanist Robert Brown)… …   Wikipedia

Share the article and excerpts

Direct link
Do a right-click on the link above
and select “Copy Link”