Calculate the Singular Value Decomposition

Introduction

Singular Value Decomposition (SVD) is a widely used technique to decompose a matrix into several component matrices, exposing many of the useful and interesting properties of the original matrix.

Computation

The aim of this article is to find the singular value decompostiion of the given matrix. Take the below matrix for example.

Let’s point out here, it’s actually very easy to find SVD for every matrices. So, what does the SVD look like? What do we want to end up with? The answer is we want a decomposition , where U and V are going to be orthogonal matrices, that is, their columns are orthonormal sets. Sigmas is going to be a diagnal matrix with non-negative entries.

In order to compute these matrices, we need two equations.

• (because and is an orthogonal matrix)

These are the two equations we need to use to find V, , and U.

Let’s start with the first one,

Now, what you notice about this equation is this is just a diagnalisation of , so we need to find the eigenvalues, and those wil be the entries of , and the eigenvectors will be the columns of V matrix.

Next, we look at the determinant of , which will be

So the eigenvalues are 20 and 80.

In order to calculate the corresponding eigenvectors, we first take

Next, we need to find the null space of this matrix. Therefore, we get the eigenvector for is . We want it to be a unit vector, remember the columns of V should be unit vectors because they’re orthonormal. Therefore, . Similarly, we get for .

Add these vectors and make them the columns of V matrix, then we can get

Good, now we can get these two of the three parts of SVD. The last thing we need to find is U matrix. For that, we need to use the second equation . So,

So now, here’s our U matrix.

Finally we have all three matrices U, V, and .

Conclusion

This is a good illustration of how to find SVD by hand. Please stay tuned as this blog will be updated regularly!