Chapter 5. Linear Algebra in NumPy

NumPy is designed for numeric computations; underneath the hood it is still the powerful ndarray object, but at the same time NumPy provides different types of objects to solve mathematical problems. In this chapter, we will cover the matrix object and polynomial object to help you solve problems using a non-ndarray way. Again, NumPy provides a lot of standard mathematical algorithms and supports multi-dimensional data. While a matrix can't perform three-dimensional data, using the ndarray objects with the NumPy functions of linear algebra and polynomials is more preferable (the extensive SciPy library is another good choice for linear algebra, but NumPy is our focus in this book). Let's use NumPy to do some math now!

The topics that will be covered in this chapter are:

  • Matrix and vector operations
  • Decompositions
  • Mathematics of polynomials
  • Regression and curve fitting

The matrix class

For linear algebra, using matrices might be more straightforward. The matrix object in NumPy inherits all the attributes and methods from ndarray, but it's strictly two-dimensional, while ndarray can be multi-dimensional. The well-known advantage of using NumPy matrices is that they provide matrix multiplication as the * notation; for example, if x and y are matrices, x * y is their matrix product. However, starting from Python 3.5/NumPy 1.10, native matrix multiplication is supported with the new operator "

However, starting from Python 3.5/NumPy 1.10, native matrix multiplication is supported with the new operator "@". So that is one more good reason to use ndarray ( https://docs.python.org/3/whatsnew/3.5.html#whatsnew-pep-465 ).

However, matrix objects still provide convenient conversion such as inverse and conjugate transpose while an ndarraydoes not. Let's start by creating NumPy matrices:

In [1]: import numpy as np 
In [2]: ndArray = np.arange(9).reshape(3,3) 
In [3]: x = np.matrix(ndArray) 
In [4]: y = np.mat(np.identity(3)) 
In [5]: x 
Out[5]: 
matrix([[0, 1, 2], 
        [3, 4, 5], 
        [6, 7, 8]]) 
In [6]: y 
Out[6]: 
matrix([[1., 0., 0.], 
        [0., 1., 0.], 
        [0., 0., 1.]]) 

There are a couple of ways to create or convert to a NumPy matrix object, and the more preferred way is to use numpy.mat() or numpy.matrix(). Both methods create matrices, but numpy.matrix() creates a copy while numpy.mat() changes the view only; it's equivalent to numpy.matrix(data, copy = False). In the previous example, we create two matrices, both of which are from the ndarray object (the np.identity(3) returns a 3 x 3 identity array). Of course you can use a string or list to create a matrix, for example: np.matrix('0 1 2; 3 4 5; 6 7 8'), and np.matrix([[0,1,2],[3,4,5],[6,7,8]]) will create the same matrix as x. In the following example, we are going to do some basic matrix operations:

In [7]: x + y 
Out[7]: 
matrix([[ 1.,  1.,  2.], 
        [ 3.,  5.,  5.], 
        [ 6.,  7.,  9.]]) 
In [8]: x * x 
Out[8]: 
matrix([[ 15,  18,  21], 
        [ 42,  54,  66], 
        [ 69,  90, 111]]) 
In [9]: np.dot(ndArray, ndArray) 
Out[9]: 
array([[ 15,  18,  21], 
       [ 42,  54,  66], 
       [ 69,  90, 111]]) 
In [10]: x**3 
Out[10]: 
matrix([[ 180,  234,  288], 
        [ 558,  720,  882], 
        [ 936, 1206, 1476]])
In [11]: z = np.matrix(np.random.random_integers(1, 50, 9).reshape(3,3)) 
In [12]: z 
Out[12]: 
matrix([[32, 21, 28], 
        [ 2, 24, 22], 
        [32, 20, 22]]) 
In [13]: z.I 
Out[13]: 
matrix( [[-0.0237 -0.0264  0.0566] 
         [-0.178   0.0518  0.1748] 
         [ 0.1963 -0.0086 -0.1958]]) 
 
In [14]: z.H 
Out[14]: 
matrix([[32  2 32] 
        [21 24 20] 
        [28 22 22]]) 

You can see from the previous example that, when we use the * notation, it applies the matrix multiplication as you use numpy.dot() for ndarray (we will talk about this in the next section). Also, the ** power notation is done in a matrix way. We also created a matrix z from random functions to show when the matrix is invertible (not singular). You can obtain the inverse matrix using numpy.matrix.I. We can also do a conjugate (Hermitian) transpose using numpy.matrix.H.

Now we know how to create a matrix object and do some basic operations, it's time for some practice. Let's try to solve a simple linear equation. Suppose we have a linear equation as A x = b and we want to know the value of x. A possible solution will be as follows:

A-1A x = A-1 b 
I x = A-1 b 
x = A-1 b 

We obtain x by multiplying the inverse of A and b, so let's do this with numpy.matrix:

In [15]: A = np.mat('3 1 4; 1 5 9; 2 6 5') 
In [16]: b = np.mat([[1],[2],[3]]) 
In [17]: x = A.I * b 
In [18]: x 
Out[18]: 
matrix([[ 0.2667], 
        [ 0.4667], 
        [-0.0667]]) 
In [21]: np.allclose(A * x, b) 
Out[21]: True 

We obtained x, and we used numpy.allclose() to compare the LHS and the RHS within a tolerance. The default absolute tolerance is 1e-8. The result returns True, meaning that LHS and RHS are equal within the tolerance, which verifies our solution. Though numpy.matrix() takes an ordinary matrix form, in most cases ndarray would be good enough for you to do linear algebra. Now we will simply compare the performance between ndarray and matrix:

In [20]: x = np.arange(25000000).reshape(5000,5000) 
 
In [21]: y = np.mat(x) 
 
In [22]: %timeit x.T 
10000000 loops, best of 3: 176 ns per loop 
 
In [23]: %timeit y.T 
1000000 loops, best of 3: 1.36 µs per loop 

This example shows a huge performance difference between ndarray and matrix when doing a transpose. Both x and y have 5,000 by 5,000 elements, but x is a two-dimensional ndarray, while y converted it to the same shape matrix. The NumPy matrix will always do operations in the matrix way, even if the computation has been optimized by NumPy.

While ndarray here by default reverses the dimensions instead of permuting the axes (the matrix always permutes the axes), that's a huge performance improvement trick done in ndarray. Therefore, ndarray is preferred when doing linear algebra especially for large sets of data considering its performance. Use matrix only when necessary. Before we go on to the next section, let's go through two more matrix object properties that convert a matrix to a basic ndarray:

In [24]: A.A 
Out[24]: 
array([[3, 1, 4], 
       [1, 5, 9], 
       [2, 6, 5]]) 
In [25]: A.A1 
Out[25]: array([3, 1, 4, 1, 5, 9, 2, 6, 5]) 

The previous examples use the matrix A we created in the linear equation practice. numpy.matrix.A returns the basic ndarray and numpy.matrix.A1 returns a one-dimensional ndarray.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.149.232.152