Moore–Penrose Pseudoinverse: Inversion Under Rank Deficiency
Moore–Penrose Pseudoinverse: Inversion Under Rank Deficiency
The Moore–Penrose pseudoinverse extends ordinary matrix inversion to rectangular, singular, and rank-deficient matrices. Its relevance is not merely computational: it characterizes what can be recovered when a linear system does not uniquely identify its parameters.
For \(A \in \mathbb{R}^{m \times n}\), the pseudoinverse is denoted by \(A^{+}\) and is uniquely defined by
\[ AA^{+}A=A, \qquad A^{+}AA^{+}=A^{+}, \]
\[ (AA^{+})^\top=AA^{+}, \qquad (A^{+}A)^\top=A^{+}A. \]
Least-Squares Interpretation
Consider the linear system
\[ Ax=b. \]
The pseudoinverse solution is
\[ \boxed{x^{*}=A^{+}b}. \]
It minimizes
\[ \|Ax-b\|_2^2. \]
If several least-squares solutions exist, \(A^{+}b\) selects the one with minimum Euclidean norm. Thus, a numerical solution may be well defined even when the underlying parameters are not uniquely identified.
SVD Representation
Let
\[ A=U\Sigma V^\top \]
denote the singular value decomposition, with
\[ \Sigma= \operatorname{diag} (\sigma_1,\ldots,\sigma_r,0,\ldots,0). \]
Then
\[ \boxed{ A^{+}=V\Sigma^{+}U^\top } \]
where
\[ \Sigma^{+} = \operatorname{diag} \left( \frac{1}{\sigma_1}, \ldots, \frac{1}{\sigma_r}, 0,\ldots,0 \right). \]
The pseudoinverse therefore inverts only directions associated with nonzero singular values:
\[ \boxed{ \text{invert the identified subspace, not the null space}. } \]
Numerical Illustration
Consider
\[ A= \begin{pmatrix} 1&1\\ 2&2 \end{pmatrix}, \qquad b= \begin{pmatrix} 3\\ 6 \end{pmatrix}. \]
Since
\[ \operatorname{rank}(A)=1, \]
the system identifies only
\[ x_1+x_2=3. \]
Hence, infinitely many solutions exist. The pseudoinverse is
\[ A^{+} = \begin{pmatrix} 0.1&0.2\\ 0.1&0.2 \end{pmatrix}, \]
yielding
\[ A^{+}b = \begin{pmatrix} 1.5\\ 1.5 \end{pmatrix}. \]
The pseudoinverse does not make \(x_1\) and \(x_2\) separately identifiable. It selects the minimum-norm solution among all vectors satisfying \(x_1+x_2=3\).
\[ \boxed{ \text{computability} \neq \text{identification}. } \]
Connection with OLS
For the linear model
\[ y=X\beta+\varepsilon, \]
the general least-squares estimator can be written as
\[ \boxed{ \hat{\beta}=X^{+}y. } \]
When \(X\) has full column rank,
\[ X^{+}=(X^\top X)^{-1}X^\top, \]
recovering the standard OLS estimator. Under rank deficiency, the coefficient vector is not unique, although fitted values remain well defined:
\[ \hat y=XX^{+}y. \]
The matrix \(XX^{+}\) is the orthogonal projection onto the column space of \(X\).
Identification Interpretation
If
\[ Ax_1=Ax_2, \]
then
\[ A(x_1-x_2)=0, \]
implying
\[ x_1-x_2\in\operatorname{Null}(A). \]
Differences along the null space are therefore observationally indistinguishable:
\[ \boxed{ \operatorname{Null}(A) \leftrightarrow \text{unidentified parameter directions}. } \]
This interpretation is particularly relevant in collinear regressions, redundant moment systems, factor models, mixture representations, and other low-rank econometric structures.
Final Perspective
The Moore–Penrose pseudoinverse is best understood as an operator that isolates the estimable component of a linear system.
\[ \boxed{ A^{-1} \text{ recovers the complete parameter vector under full identification;} } \]
\[ \boxed{ A^{+} \text{ recovers the component supported by the identified subspace.} } \]
This connection between rank, singular values, and identification makes the pseudoinverse a fundamental tool in modern econometrics.
