The SVD
A couple of months back, Prof. Gilbert Strang had come to SFU as part of the distinguished speakers series (as I had written earlier.) After the talk, I got a chance to chat with the guru. I owe a lot of my understanding of linear algebra to his books and his kick-ass animations (check out these eigen-analysis demos,) and so I asked him about the single most important topic in linear algebra. Without hesitation, he immediately responded: “the SVD!”
The Singular Value Decomposition is the swiss-knife of linear algebra. Every matrix Y can be factored into three matrices: U, S, and V as
U and V are orthogonal matrices and S is a diagonal matrix. Some uses of this factorization of the matrix Y: calculating 2-norms, Frobenius norms, ranks, null spaces and ranges, (pseudo)inverses and determinants, and by extension solving systems of equations (exact, over- and under- determined), eigenvalues and eigenvectors, and approximations.
Almost every problem in engineering becomes an optimization problem (as in reducing the error under some norm) and the method of least squares makes extensive use of the SVD.
The stage has been set for the next few posts of mine.
Possibly related:
- No related posts