A set of pictures that I use for explaining Singular Value Decomposition (SVD)

According to finite-dimensional spectral theorem, a symmetric matrix has orthonormal eigenvectors, thus Q*Q^T=I and Q^T=Q^-1. Let us take Q₁ as eigenvectors of a symmetric matrix formed by A*A^T; and Q₂ as eigenvectors of a symmetric matrix formed by A^T*A. According to the (not too involved) proof, SVD decomposes matrix A in a following way: A = Q₁*S*Q₂^T, with eigenvalues being stored in S. As eigenvalues are stored in descending order, the matrix is now effectively multilayered (see pictures above), with a possibility of “peeling” off the more important vectors at first and discarding the less influential information from the latter part of decomposition.