Tuesday, March 30, 2010

A set of pictures that I use for explaining Singular Value Decomposition (SVD)






According to finite-dimensional spectral theorem, a symmetric matrix has orthonormal eigenvectors, thus Q*QT=I and QT=Q-1. Let us take Q1 as eigenvectors of a symmetric matrix formed by A*AT; and Q2 as eigenvectors of a symmetric matrix formed by AT*A. According to the (not too involved) proof, SVD decomposes matrix A in a following way: A = Q1*S*Q2T, with eigenvalues being stored in S. As eigenvalues are stored in descending order, the matrix is now effectively multilayered (see pictures above), with a possibility of “peeling” off the more important vectors at first and discarding the less influential information from the latter part of decomposition.

No comments: