SVD takes any m x n matrix A, and then returns three matrices in return—U, Σ, and V. Here, U is an m x m unitary matrix, Σ is an m x n diagonal matrix, and V is an n x n unitary matrix. By unitary, we mean that a matrix's columns form an orthonormal basis; by diagonal, we mean that all values in the matrix are zero, except for possibly the values along its diagonal.

The significance of the SVD is that this decomposes A into these matrices so that we have A = UΣV^T ; moreover, the values along the diagonal of Σ will all be positive or zero, and are known as the singular values. We will see some applications of this soon, but you should keep in mind that the computational complexity of SVD is of the order O(mn²)—for large matrices, it is definitely a good idea to use a GPU, since this algorithm is parallelizable.

We'll now look at how we can compute the SVD of a matrix. Let's make the appropriate import statements:

import pycuda.autoinit
from pycuda import gpuarray
import numpy as np
from skcuda import linalg

We will now generate a relatively large random matrix and transfer it to the GPU:

a = np.random.rand(1000,5000).astype(np.float32)
a_gpu = gpuarray.to_gpu(a)

We can now execute the SVD. This will have three outputs corresponding to the matrices that we just described. The first parameter will be the matrix array we just copied to the GPU. Then we need to specify that we want to use cuSolver as our backend for this operation:

U_d, s_d, V_d = linalg.svd(a_gpu,  lib='cusolver')

Now let's copy these arrays from the GPU to the host:

U = U_d.get()
s = s_d.get()
V = V_d.get()

s is actually stored as a one-dimensional array; we will have to create a zero matrix of size 1000 x 5000 and copy these values along the diagonal. We can do this with the NumPy diag function, coupled with some array slicing:

S = np.zeros((1000,5000))
S[:1000,:1000] = np.diag(s)

We can now matrix-multiply these values on the host with the NumPy dot function to verify that they match up to our original array:

print 'Can we reconstruct a from its SVD decomposition? : %s' % np.allclose(a, np.dot(U, np.dot(S, V)), atol=1e-5)

Since we are using only float32s and our matrix is relatively large, a bit of numerical error was introduced; we had to set the "tolerance" level (atol) a little higher than usual here, but it's still small enough to verify that the two arrays are sufficiently close.