- Moore-Penrose pseudoinverse
mathematics, and in particular linear algebra, the pseudoinverse of an matrix is a generalization of the inverse matrix.cite book | last=Ben-Israel | first = Adi | coauthors= Thomas N.E. Greville| title=Generalized Inverses | isbn=0-387-00293-6 | publisher=Springer-Verlag | year=2003] More precisely, this article talks about the Moore-Penrose pseudoinverse, which was independently described by E. H. Moorecite journal | last=Moore | first=E. H. | authorlink=E. H. Moore | title=On the reciprocal of the general algebraic matrix | journal= Bulletin of the American Mathematical Society| volume=26 | pages=394–395 | year=1920] in 1920 and Roger Penrosecite journal | last=Penrose | first=Roger | authorlink=Roger Penrose | title=A generalized inverse for matrices | journal= Proceedings of the Cambridge Philosophical Society| volume=51 | pages=406–413 | year=1955] in 1955. Earlier, Fredholm had introduced the concept of a pseudoinverse of integral operators in 1903. The term generalized inverseis sometimes used as a synonym for pseudoinverse.
A common use of the pseudoinverse is to compute a 'best fit' (least squares) solution to a
system of linear equations(see below under Applications). The pseudoinverse is defined and unique for all matrices whose entries are real or complex numbers. The pseudoinverse can be computed using the singular value decomposition.
The pseudoinverse of an "m"-by-"n" matrix (whose entries can be real or complex numbers) is defined as the unique "n"-by-"m" matrix satisfying all of the following four criteria:cite book | last=Golub | first=Gene H. | authorlink=Gene H. Golub | coauthors=
Charles F. Van Loan| title=Matrix computations | edition=3rd edition | publisher=Johns Hopkins | location=Baltimore | year=1996 | isbn=0-8018-5414-8 | pages = 257–258]
# ( need not be the general identity matrix, but it maps all column vectors of to themselves);
# ( is a weak inverse for the multiplicative
# ( is Hermitian); and
# ( is also Hermitian).
Here is the
conjugate transposeof a matrix "M". For matrices whose elements are real numbers instead of complex numbers, .
Proofs for some of these relations may be found on the proofs page.
* The pseudoinverse exists and is unique: for any matrix , there is precisely one matrix that satisfies the four properties of the definition.
* If the matrix is invertible, the pseudoinverse and the inverse coincide: .Citation | last1=Stoer | first1=Josef | last2=Bulirsch | first2=Roland | title=Introduction to Numerical Analysis | publisher=
Springer-Verlag| location=Berlin, New York | edition=3rd | isbn=978-0-387-95452-3 | year=2002.] rp|243
* Pseudoinversion is reversible. It is its own inverse: .rp|245
* is the
orthogonal projectoron the range of , and is the orthogonal projector on the range of .
* The pseudoinverse of a
zero matrixis its transpose.
* The pseudoinverse can be computed via a limiting process:::(see
Tikhonov regularization). These limits exist even if and do not exist.rp|263
* Pseudoinversion commutes with transposition, conjugation, and taking the conjugate transpose:rp|245 ::
* The pseudoinverse of a scalar multiple of is the reciprocal multiple of ::: for .
* If the pseudoinverse of is already known, it may be used to compute :::.
* Likewise, if is already known:::.
If "A" and "B" are such that the product is defined and
* "A" has orthonormal columns (, unitarity is a special case), or
* "B" has orthonormal rows (), or
* "A" is of full column rank, and "B" is of full row rank,then:.
The third case does not cover the first two;an orthonormal (including non-square) matrix must be of full rank, but otherwise there is no assumption made on the matrix it multiplies.
These relations have in common that the right hand side equals the left hand side multiplied with some expression. They can be used to cancel certain subexpressions or expand expressions. ::
Orthonormal columns or rows
If "A" has orthonormal columns ()or orthonormal rows (),then .
If the columns of are linearly independent, then is invertible. In this case, an explicit formula is::.It follows that is a left inverse of "A": .
If the rows of are linearly independent, then is invertible. In this case, an explicit formula is::.It follows that is a right inverse of "A": .
If both columns and rows are linearly independent (that is, for square regular matrices), the pseudoinverse is just the inverse::.
Scalars and vectors
It is also possible to define a pseudoinverse for scalars and vectors. This amounts to treating these as matrices. The pseudoinverse of a scalar is zero if is zero and the reciprocal of otherwise::
The pseudoinverse of the null vector is the transposed null vector. The pseudoinverse of a non-null vector is the conjugate transposed vector divided by its squared magnitude::
For a proof, simply check that these definitions meet the defining criteria for the pseudoinverse.
Circulant matrixthe singular value decomposition is given by the Fourier transform,that is the singular values are the Fourier coefficients.Let be the Fourier matrix, then::cite journal | last=Stallings | first=W. T. | authorlink=W. T. Stallings | title=The Pseudoinverse of an r-Circulant Matrix | journal= Proceedings of the American Mathematical Society| volume=34 | pages=385–388 | year=1972 | doi=10.2307/2038377]
Optimized approaches exist for calculating the pseudoinverse of block structured matrices.
Finding the pseudoinverse of a matrix
Using regular inverses
Let "k" be the rank of a matrix "A". Then "A" can be decomposed as , where "B" is a -matrix and "C" is a matrix. Then :.
If "A" has full row rank, so that "k" = "m", then "B" can be chosen to be the
identity matrixand the formula reduces to . Similarly, if "A" has full column rank (that is, "k" = "n"), we have
The QR method
However, computing the product or or its inverse explicitly is unnecessary and only causes additional rounding errors and computational cost. Consider first the case when "A" is full column rank. Then all that is needed is the
Cholesky decomposition:,where "R" is an upper triangular matrix. Multiplication by the inverse is then done easily by solving a system with multiple right-hand-side,:by back substitution. To compute the Cholesky decomposition without forming explicitly, use the QR decompositionof "A",where "Q" is a unitary matrix, , and "R" is upper triangular. Then :,so "R" is the Cholesky factor of . The case of full row rank is obtained by swapping and .
The general case and the SVD method
A computationally simpler and more accurate way to get the pseudoinverse is by using the
singular value decomposition. [http://www.uwlax.edu/faculty/will/svd/systems/index.html Linear Systems & Pseudo-Inverse] ] If is the singular value decomposition of "A", then . For a diagonal matrixsuch as , we get the pseudoinverse by taking the reciprocal of each non-zero element on the diagonal, and leaving the zeros in place. In numerical computation, only elements larger than some small tolerance are taken to be nonzero, and the others are replaced by zeros. For example, in the Matlabfunction pinv, the tolerance is taken to be "t" = ε•max("m","n")•max(Σ), where ε is the machine epsilon.
The cost of this method is dominated by the cost of computing the SVD, which is several times higher than matrix-matrix multiplication, if a state-of-the art implementation (such as that of
LAPACK) is used.
Updating the pseudoinverse
For the cases where "A" has full row or column rank, and the inverse of the correlation matrix ( for "A" with full row rank or for full column rank) is already known, the pseudoinverse for matrices related to can be computed by applying the
Sherman–Morrison–Woodbury formulato update the inverse of the correlation matrix, which may need less work. In particular, if the related matrix differs from the original one by only a changed, added or deleted row or column, incremental algorithmscite paper |author= Tino Gramß |title= Worterkennung mit einem künstlichen neuronalen Netzwerk |version= |publisher= Georg-August-Universität zu Göttingen |date = 1992 | url = | format = printed dissertation| accessdate = ] exist that exploit the relationship.
Similarly, it is possible to update the Cholesky factor when a row or column is added, without creating the inverse of the correlation matrix explicitly. However, updating the pseudoinverse in the general rank-deficient case is much more complicated. [Meyer, Carl D., Jr. Generalized inverses and ranks of block matrices. SIAM J. Appl. Math. 25 (1973), 597—602] [Meyer, Carl D., Jr. Generalized inversion of modified matrices. SIAM J. Appl. Math. 24 (1973), 315—323]
High quality implementations of SVD, QR, and back substitution are available in standard libraries, such as
LAPACK. Writing one's own implementation of SVD is a major programming project that requires a significant numerical expertise. In special circumstances, such as parallel computingor embedded computing, however, alternative implementations by QR or even the use of an explicit inverse might be preferable, and custom implementations may be unavoidable.
The pseudoinverse provides a least squares solution to a
system of linear equations.cite journal | last=Penrose | first=Roger | title=On best approximate solution of linear matrix equations | journal= Proceedings of the Cambridge Philosophical Society| volume=52 | pages=17–19 | year=1956]
Given a system of linear equations
in general we cannot always expect to find a vector which will solve the system; even if there exists such a solution vector, then it may not be unique. We can however always ask for a vector that brings "as close as possible" to , i.e. a vector that minimizes the
If there are several such vectors , we could ask for the one among them with the smallest Euclidean norm. Thus formulated, the problem has a unique solution, given by the pseudoinverse::
* [http://planetmath.org/encyclopedia/Pseudoinverse.html Pseudoinverse on PlanetMath]
Wikimedia Foundation. 2010.
См. также в других словарях:
Moore–Penrose pseudoinverse — In mathematics, and in particular linear algebra, a pseudoinverse A+ of a matrix A is a generalization of the inverse matrix. The most widely known type of matrix pseudoinverse is the Moore–Penrose pseudoinverse, which was independently… … Wikipedia
Moore-Penrose pseudoinverse/Proofs — Existence and Uniqueness =Let A be an m by n matrix over K , where K is either the field of real numbers or the field of complex numbers. Then there is a unique n by m matrix A^+ over K such that: #A A^+A = A #A^+A A^+ = A^+ #(AA^+)^* = AA^+… … Wikipedia
Moore-Penrose-Inverse — Die Pseudoinverse einer Matrix ist ein Begriff aus dem mathematischen Teilgebiet lineare Algebra. Sie ist eine Verallgemeinerung der inversen Matrix auf singuläre und nichtquadratische Matrizen, weshalb sie häufig auch als verallgemeinerte… … Deutsch Wikipedia
Pseudoinverse — Die Pseudoinverse einer Matrix ist ein Begriff aus dem mathematischen Teilgebiet lineare Algebra. Sie ist eine Verallgemeinerung der inversen Matrix auf singuläre und nichtquadratische Matrizen, weshalb sie häufig auch als verallgemeinerte… … Deutsch Wikipedia
Pseudoinverse — Pseudo inverse En algèbre linéaire, la notion de pseudo inverse (ou inverse généralisé) généralise celle d’inverse d’une application linéaire ou d’une matrice aux cas non inversibles en lui supprimant certaines des propriétés demandées aux… … Wikipédia en Français
Roger Penrose — Infobox Scientist box width = 300px name = Sir Roger Penrose image size = 200px caption = Roger Penrose at Brookhaven Lab, February 6, 2007 birth date = birth date and age|1931|08|08 birth place = Colchester, Essex, England death date = death… … Wikipedia
E. H. Moore — Eliakim Hastings Moore (* 28. Januar 1862 in Marietta, Ohio; † 30. Dezember 1932 in Chicago, Illinois) war ein US amerikanischer Mathematiker. Moore studierte ab 1879 an der Yale University Mathematik und Astronomie und promovierte 1885 mit der… … Deutsch Wikipedia
Eliakim Hastings Moore — (* 28. Januar 1862 in Marietta, Ohio; † 30. Dezember 1932 in Chicago, Illinois) war ein US amerikanischer Mathematiker. Moore studierte ab 1879 an der Yale University Mathematik und Astronomie und promovierte … Deutsch Wikipedia
Invertible matrix — In linear algebra an n by n (square) matrix A is called invertible (some authors use nonsingular or nondegenerate) if there exists an n by n matrix B such that where In denotes the n by n identity matrix and the multiplication used is ordinary… … Wikipedia
Inverse element — In abstract algebra, the idea of an inverse element generalises the concept of a negation, in relation to addition, and a reciprocal, in relation to multiplication. The intuition is of an element that can undo the effect of combination with… … Wikipedia