© The Author(s), under exclusive license to APress Media, LLC, part of Springer Nature 2022
A. DanialPython for MATLAB Developmenthttps://doi.org/10.1007/978-1-4842-7223-7_11

11. NumPy and SciPy

Albert Danial1  
(1)
Redondo Beach, CA, USA
 

NumPy is the foundational module for numeric computation in Python. Its primary data structure, the NumPy n-dimensional array, can be considered the Python equivalent of a MATLAB matrix. SciPy is a collection of modules built upon NumPy to solve a wide range of computational problems in clustering, differential equations, image processing, interpolation, linear algebra, linear programming, optimization, sparse matrices, statistics, and symbolic mathematics, among others. NumPy and SciPy together span the capabilities of the core MATLAB product plus functions from many toolboxes.

In this chapter, I’ll show how to create, manipulate, subset, aggregate, and serialize NumPy arrays, then survey the broad capabilities found in NumPy and SciPy. Comparisons to MATLAB equivalents are shown throughout.

11.1 NumPy Arrays

A NumPy array, or, more formally, a numpy.ndarray, meaning “n-dimensional array,” is functionally equivalent to a MATLAB matrix. Both are containers for a single (usually numeric) data type, both allow vectorized operations on their values, and both are accompanied by a large library of high performance mathematical, scientific, and statistical functions.

The NumPy module must be explicitly imported before NumPy arrays and functions can be used in a program or interactive computing session. This is most commonly done with
import numpy as np

The prefix np. will therefore represent the NumPy module throughout this book.

11.1.1 Formatting NumPy Array Values

We’ll see many numbers in the following sections, so it’s helpful to have control over their appearance. MATLAB’s format command has a collection of useful styles. NumPy offers that and more; it gives control not only of the numeric format but also the line width and summary options if the array exceeds a given size.

MATLAB:
>> a = reshape(1:6, 3,2)/pi/14326
   2.2219e-05   8.8876e-05
   4.4438e-05   1.1110e-04
   6.6657e-05   1.3331e-04
Python:
In : a = np.arange(1,7).reshape(2,3).T/np.pi/14326
In : a
Out:
array([[2.22190344e-05, 8.88761374e-05],
       [4.44380687e-05, 1.11095172e-04],
       [6.66571031e-05, 1.33314206e-04]])
MATLAB:
>> format long e
>> a
   2.221903435598148e-05     8.887613742392592e-05
   4.443806871196296e-05     1.110951717799074e-04
   6.665710306794443e-05     1.333142061358889e-04
Python:
In : np.set_printoptions(precision=20)
In : a
Out:
array([[2.2219034355981481e-05, 8.8876137423925925e-05],
       [4.4438068711962962e-05, 1.1109517177990740e-04],
       [6.6657103067944433e-05, 1.3331420613588887e-04]])

Occasionally , in MATLAB you’ll accidentally type the name of a massive array or enter an expression that produces volumes of numbers without a trailing semicolon. The result is many screens full of numbers scrolling by. NumPy is more forgiving in that it will simply print leading and trailing terms:

MATLAB:
>> a = reshape(1:1000000, 1000, 1000)
     ... entire matrix printed ...
Python:
In : a = np.arange(1000000).reshape(1000,1000)
In : a
Out:
array([[     0,      1,      2, ...,    997,    998,    999],
       [  1000,   1001,   1002, ...,   1997,   1998,   1999],
       [  2000,   2001,   2002, ...,   2997,   2998,   2999],
       ...,
       [997000, 997001, 997002, ..., 997997, 997998, 997999],
       [998000, 998001, 998002, ..., 998997, 998998, 998999],
       [999000, 999001, 999002, ..., 999997, 999998, 999999]])
In : np.set_printoptions(linewidth=40,edgeitems=2)
In : a
Out:
array([[     0,      1, ...,    998,
           999],
       [  1000,   1001, ...,   1998,
          1999],
       ...,
       [998000, 998001, ..., 998998,
        998999],
       [999000, 999001, ..., 999998,
        999999]])

If you’re fortunate to have a large monitor, setting linewidth=200 (or something similar) can be a big help when poring over data.

11.1.2 Differences Between NumPy Arrays and MATLAB Matrices

The next five sections explain the primary differences between NumPy arrays and MATLAB matrices.

11.1.2.1 MATLAB Matrices Made of Numeric Literals Default to Double Precision

In contrast, NumPy array types are inferred from the inputs. Unless one of the terms has a decimal point or exponent, NumPy will create an array of 64-bit integers:

MATLAB:

Python:

>> a = [1 2 3];

>> class(a)

    'double'

>> a = [1 2. 3];

>> class(a)

    'double'

In : a = np.array([1, 2, 3])

In : a.dtype

Out: dtype('int64')

In : a = np.array([1, 2., 3])

In : a.dtype

Out: dtype('float64')

11.1.2.2 MATLAB Matrices Have a Minimum of Two Dimensions

NumPy arrays, in contrast, may have just one dimension.

MATLAB:

Python:

>> a = [1 2 3]

>> size(a)

   1   3

>> ndims(a)

   2

In : a = np.array([1, 2, 3])

In : a.shape

Out: (3,)

In : a.ndim

Out: 1

In the preceding example, a in MATLAB is a 1 x 3 matrix, while Python’s a just has three terms. MATLAB can therefore distinguish between row vectors and column vectors which is useful for some linear algebra expressions.

One-dimensional NumPy arrays have no concept of vertical or horizontal arrangement; thus, their row or column affinity becomes a matter of context. A second dimension can be added explicitly to convey the same row or column sense as a MATLAB vector; this process is explained in Section 11.1.12.

11.1.2.3 MATLAB Numeric Scalars Are 1 x 1 Matrices

NumPy scalars are distinct from NumPy arrays; scalars have just one value.

MATLAB:

Python:

>> a = 2.7183;

>> size(a)

   1   1

>> ndims(a)

   2

In : a = 2.7183

In : a.shape

AttributeError Traceback

----> 1 a.shape

AttributeError: 'float' object

has no attribute 'shape'

11.1.2.4 NumPy Arrays Are Row Major

By default a NumPy array uses row major storage internally, while MATLAB arrays are stored column major. In other words, if the numbers 1 through 12 were stored as consecutive values in memory, they would look like this as 3 × 4 matrix:

MATLAB:

Python:

>> a = 1:12

a =

 1 2 3 4 5 6 7 8 9 10 11 12

>> reshape(a, [3,4])

    1    4    7   10

    2    5    8   11

    3    6    9   12

In : a = np.arange(1, 13)

In : a

Out: array([ 1, 2, 3,  4,  5,  6,

             7, 8, 9, 10, 11, 12])

In : a.reshape(3,4)

array([[ 1,  2,  3,  4],

       [ 5,  6,  7,  8],

       [ 9, 10, 11, 12]])

NumPy is capable of creating column-major arrays using the order='F' (Fortran-order) optional argument to the numpy.array() function.

Python:
In : a = np.arange(1, 13)
In : a
Out: array([ 1, 2, 3,  4,  5,  6,
             7, 8, 9, 10, 11, 12])
In : a.reshape(3,4, order='F')
array([[ 1,  4,  7, 10],
       [ 2,  5,  8, 11],
       [ 3,  6,  9, 12]])

11.1.2.5 NumPy Arrays Must Be Explicitly Resized to Add Terms

MATLAB matrices can be expanded simply by referencing new indices, but NumPy array sizes are fixed once the array is created. One can create new NumPy arrays using the vertical or horizontal stack functions np.vstack() or np.hstack() to add rows or columns from existing arrays. Alternatively, one can call the np.resize() function to add the desired number of terms in any dimension. MATLAB, by comparison, allows rows or columns to be appended by merely referencing a new index on the left-hand side matrix. This example shows how one would append a row to an existing 2 × 2 matrix using the .resize() method :

MATLAB:

Python:

>> a = [1 2; 3 4]

a =

    1  2

    3  4

>> a(3,:) = [5 6]

a =

    1  2

    3  4

    5  6

In : a = np.array([[1,2], [3,4]])

In : a

Out: array([[1, 2]

            [3, 4]])

In : a.resize( (3,2) )

In : a[2,] = [5, 6]

In : a

Out: array([[1, 2],

            [3, 4],

            [5, 6]])

While MATLAB’s matrix growing capability is convenient, it comes with a performance penalty. It is much faster to preallocate a MATLAB matrix, for example, by first creating a matrix of zeros, then populating existing rows or columns with values rather than starting with an empty matrix and growing it in a loop. Preallocation is fine when the final size of a matrix is known up front, but this is not always the case. Reading values from a file may require either two passes through the file, once to learn what the final size will be and a second time to populate matrix entries, or reverting to growing the matrix.

11.1.3 NumPy Data Types

Table 11-1 lists the possible types for np.ndarrays along with their MATLAB counterparts.
Table 11-1

NumPy data types

MATLAB Type

NumPy Type

Notes

logical

np.bool

8-bit Boolean

uint8

np.ubyte

8-bit unsigned integer

uint8

np.uint8

8-bit unsigned integer

int8

np.int8

8-bit signed integer

uint16

np.uint16

16-bit unsigned integer

int16

np.int16

16-bit signed integer

uint32

np.uint32

32-bit unsigned integer

int32

np.int32

32-bit signed integer

uint64

np.uint64

64-bit unsigned integer

int64

np.int, np.int64

64-bit signed integer

uint64

np.uint

64-bit unsigned integer

not available

np.float16

16-bit (short) float

single

np.float32

32-bit float

double

np.float64

64-bit float

double

np.float

64-bit float

not available

np.float128

128-bit (quad precision) float1

complex

np.complex

pair of 64-bit floats

single2

np.complex64

Pair of 32-bit floats

double3

np.complex128

Pair of 64-bit floats

not available

np.complex256

Pair of 128-bit floats4

char

np.str

Fixed-length strings

datetime

datetime.datetime

Datetime object

11.1.4 Typecasting Scalars and Arrays

MATLAB’s typecasting mechanism is straightforward: if you have a scalar or array in one data type (values are double precision by default), you just need to prefix the variable with the type name.

While NumPy’s typecasting notation for scalars matches MATLAB’s, NumPy uses different notation to typecast arrays. One must write X.astype(np.uint8) to cast array X to np.uint8:

MATLAB:

Python:

>> p = 4.32;

>> uint8(p)

 4

>> p = [4.32 1.23];

>> uint8(p)

   4  1

In : p = 4.32

In : np.uint8(p)

Out: 4

In : p = np.array([4.32, 1.23])

In : p.astype(np.uint8)

Out: array([4, 1], dtype=uint8)

Both MATLAB and NumPy support versions of “like” casting, meaning creating a new array using the type of an existing array:

MATLAB:

Python:

>> p = [4.32 1.23];

>> q = uint8([0 0]);

>> cast(p,'like',q)

  1×2 uint8 row vector

   4   1

In : p = np.array([4.32, 1.23])

In : q = np.zeros((1,2),dtype=np.uint8)

In : p.astype(q.dtype)

Out: array([4, 1], dtype=uint8)

np.zeros() and np.zeros_like() are described in Section 11.1.6.

11.1.5 Hex, Binary, and Decimal Representations

The content of some numeric data, specifically integers, may have properties that become apparent when represented in different bases. Bit mask operations in particular are more easily understood when their values are shown in binary form.

11.1.5.1 Decimal to Binary

MATLAB:

Python:

>> dec = 1234;

>> dec2bin(dec)

   10011010010

In : dec = 1234

In : np.binary_repr(dec)

Out: '10011010010'

11.1.5.2 Binary to Decimal

MATLAB:

Python:

>> bin2dec('10011010010')

  1234

In : np.int('10011010010', 2)

Out: 1234

11.1.5.3 Decimal to Hexadecimal

MATLAB’s dec2hex() function returns a hexadecimal string representing the provided integer. NumPy lacks a dedicated function for this conversion; instead, one can use %x and %X format strings. These work the same way in MATLAB and Python: %x produces lowercase hexadecimal, while %X produces uppercase. Additionally, a leading zero before the field width will zero-pad the resulting string. This provides a bit more control over the output than dec2hex().

MATLAB:

Python:

>> dec2hex(8765)

 223D

>> sprintf('%x', 8765)

 223d

>> sprintf('%X', 8765)

 223D

>> sprintf('%06X', 8765)

 00223D

In : '%x' % 8765

Out: '223d'

In : '%X' % 8765

Out: '223D'

In : '%06X' % 8765

Out: '00223D'

11.1.5.4 Hexadecimal to Decimal

MATLAB:

Python:

>> hex2dec('223D')

  8765

In : np.int('223D', 16)

Out: 8765

11.1.6 Creating Arrays

NumPy arrays can be created many ways. The canonical constructor is the np.array() function which takes a list and optional settings for the data type (default np.float64) and storage scheme (default row major). Here, we see how basic matrices are created in both languages:

MATLAB:

Python:

>> a = [9 8 7; 6 5 4]

a =

   9   8   7

   6   5   4

In : a = np.array([[9, 8, 7],[6, 5, 4]])

In : a

array([[9, 8, 7],

       [6, 5, 4]])

11.1.6.1 Zeros

Array allocation or initialization is often done by creating a zero-filled array with the desired shape and type. NumPy has two functions, np.zeros() and np.zeros_like(), for this purpose. The np.zeros_like() function is interesting because rather than taking the array size as an argument, it takes an existing matrix, then returns a zero-filled array having the same size and type of the given array. Another array creation option is np.empty() which creates an array without initializing its contents.

MATLAB’s zeros() function supports a 'like' option, but it doesn’t return the correct result in 2020b:

MATLAB 2020b:

Python:

>> a = zeros(2,3)

   0   0   0

   0   0   0

>> b = [2; 4]

     2

     4

>> c = zeros('like',b)

     0

In : a = np.zeros((2,3))

In : a

Out: array([[0, 0, 0],

            [0, 0, 0]])

In : b = np.array([[2], [4]])

In : b

Out: array([[2],

            [4]])

In : c = np.zeros_like(b)

In : c

Out: array([[0],

            [0]])

11.1.6.2 Ones

An array of ones is another useful initializer. Its result is sometimes scaled to produce arrays filled with the same value, although a special function, np.full(), exists just for that purpose; its use is illustrated in Section 11.1.6.3. As with np.zeros_like(), np.ones() has a np.ones_like() companion:

MATLAB:

Python:

>> a = ones(3,1)

   1

   1

   1

In : a = np.ones((3,1))

In : a

Out: array([[1],

            [1],

            [1]])

11.1.6.3 NaNs

MATLAB has a nan() function which creates a matrix of NaN—not a number—values. NumPy has a more generic matrix initializer, np.full(), that initializes an array with the given value:

MATLAB:

Python:

>> nan(2,3)

   NaN   NaN   NaN

   NaN   NaN   NaN

In : np.full((2,3), np.NaN)

array([[nan, nan, nan],

       [nan, nan, nan]])

11.1.6.4 Range

A range of consecutive numbers can be created two ways in Python. The first is with the standard range() function and the second is with NumPy’s np.arange() function which is more useful for numeric work because the return value is a NumPy array. Both have the same calling arguments, but neither, perhaps confusingly to MATLAB programmers, include the stop value in the output. This can be especially vexing when stepping with non-unit values:

MATLAB:

Python:

>> b = -8:-4

b =

  -8  -7  -6  -5  -4

>> b = -8:1.5:-4

b =

   -8.00 -6.50 -5.00

In : b = np.arange(-8,-3)

In : b

Out: array([-8, -7, -6, -5, -4])

In : b = np.arange(-8,-4,1.5)

In : b

Out: array([-8. , -6.5, -5. ])

11.1.6.5 Identity

Identity matrices are created with eye() and np.eye(), respectively:

MATLAB:

Python:

>> eye(3)

   1   0   0

   0   1   0

   0   0   1

In : a = np.eye(3)

array([[1., 0., 0.],

       [0., 1., 0.],

       [0., 0., 1.]])

11.1.6.6 Diagonal Matrices

Diagonal matrices can be created by passing diag() and np.diag() a 1D vector. Additionally, these functions return the diagonal of a matrix when given a 2D array:

MATLAB:

Python:

>> diag([9 8 7])

   9   0   0

   0   8   0

   0   0   7

>> diag(eye(2))

   1

   1

In : np.diag([9,8,7])

array([[9, 0, 0],

       [0, 8, 0],

       [0, 0, 7]])

In : np.diag(np.eye(2))

Out: array([1., 1.])

11.1.6.7 Upper and Lower Triangular Matrices

These are formed in MATLAB with triu() and tril() and in NumPy with np.triu() and np.tril(). All four functions support an optional offset from the diagonal.

We saw at the beginning of Section 11.1 that MATLAB matrices are column major, while NumPy arrays, by default, are row major. The MATLAB definition of A in the following example is therefore transposed to yield the same pattern of numbers as NumPy's A. We’ll use A to explore upper and lower triangle extractions:

MATLAB:

Python:

>> A = reshape(5:16, 4,3)'

A =

    5    6    7    8

    9   10   11   12

   13   14   15   16

>> triu(A)

    5    6    7    8

    0   10   11   12

    0    0   15   16

>> triu(A,1)

    0    6    7    8

    0    0   11   12

    0    0    0   16

>> tril(A)

    5    0    0    0

    9   10    0    0

   13   14   15    0

>> tril(A,-1)

    0    0    0    0

    9    0    0    0

   13   14    0    0

In : A = np.arange(5,17).reshape(3,4)

In : A

array([[ 5,  6,  7,  8],

       [ 9, 10, 11, 12],

       [13, 14, 15, 16]])

In : np.triu(A)

array([[ 5,  6,  7,  8],

       [ 0, 10, 11, 12],

       [ 0,  0, 15, 16]])

In : np.triu(A,1)

array([[ 0,  6,  7,  8],

       [ 0,  0, 11, 12],

       [ 0,  0,  0, 16]])

In : np.tril(A)

array([[ 5,  0,  0,  0],

       [ 9, 10,  0,  0],

       [13, 14, 15,  0]])

In : np.tril(A,-1)

array([[ 0,  0,  0,  0],

       [ 9,  0,  0,  0],

       [13, 14,  0,  0]])

11.1.6.8 Random

Pseudorandom arrays can be created using a variety of distribution functions, as can be seen by doing tab completion on np.random in an ipython session:
In : np.random.<Tab>
absolute_import        mtrand                  RandomState
beta()                 multinomial()           ranf()
binomial()             multivariate_normal()   rayleigh()
bytes()                negative_binomial()     sample()
chisquare()            noncentral_chisquare()  seed()
choice()               noncentral_f()          set_state()
dirichlet()            normal()                shuffle()
division               np                      standard_cauchy()
exponential()          operator                standard_exponential()
f()                    pareto()                standard_gamma()
gamma()                permutation()           standard_normal()
geometric()            poisson()               standard_t()
get_state()            power()                 test
gumbel()               print_function          triangular()
hypergeometric()       rand()                  uniform()
laplace()              randint()               vonmises()
Lock()                 randn()                 wald()
logistic()             random()                warnings
lognormal()            random_integers()       weibull()
logseries()            random_sample()         zipf()
np.random.rand() produces a uniform random distribution, like MATLAB’s rand() function:

MATLAB:

Python:

>> rand(2)

   0.095692   0.232535

   0.813753   0.892878

>> rand(3,2)

   0.456106   0.908961

   0.274357   0.755147

   0.113177   0.900217

In : np.random.rand(2,2)

array([[0.59219917, 0.62549076],

       [0.21467385, 0.70801235]])

In : np.random.rand(3,2)

array([[0.664537, 0.219990],

       [0.974484, 0.251934],

       [0.827411, 0.063566]])

11.1.6.9 Equally Spaced Distribution (linspace)

MATLAB:

Python:

>> linspace(21,23,5)

  21.0 21.5 22.0 22.5 23.0

In : np.linspace(21,23,5)

Out: array([21., 21.5, 22., 22.5, 23. ])

11.1.6.10 Logarithmically Spaced Distribution (logspace)

MATLAB:

Python:

>> logspace(0.1, 1, 3)

    1.2589    3.5481   10.0000

In : np.logspace(0.1, 1, 3)

Out: array([ 1.25892541, 3.54813389, 10. ])

11.1.6.11 Horizontal and Vertical Stacking

MATLAB has a convenience edge on NumPy when it comes to creating new matrices by stacking terms of existing matrices; MATLAB matrix names merely need to be placed side by side within brackets. NumPy, in contrast, requires calls to special functions np.hstack() and np.vstack(). The new arrays d and e are created by horizontally and vertically stacking a, b, and c:

MATLAB:

Python:

>> a = [1; 2]

     1

     2

>> b = [0; -3]

     0

    -3

>> c = [-1; 0]

    -1

     0

>> d = [a b c]

     1     0    -1

     2    -3     0

>> e = [a;b;c]

     1

     2

     0

    -3

    -1

     0

In : a = np.array([[1],[2]])

In : a

array([[1],

       [2]])

In : b = np.array([[0],[-3]])

In : b

array([[ 0],

       [-3]])

In : c = np.array([[-1],[0]])

In : c

array([[-1],

       [ 0]])

In : d = np.hstack([a,b,c])

In : d

array([[ 1,  0, -1],

       [ 2, -3,  0]])

In : e = np.vstack([a,b,c])

In : e

array([[ 1],

       [ 2],

       [ 0],

       [-3],

       [-1],

       [ 0],

11.1.6.12 Meshgrid

meshgrid() in MATLAB and its NumPy namesake np.meshgrid() are useful for creating coordinate pairs on a regular mesh. These, in turn, are often used as substrates over which 2D data is mapped for visualization or interpolation.

MATLAB:

Python:

>> [y,x] = meshgrid([1 2 3],

                  [3.5 3.6])

y =

   1   2   3

   1   2   3

x =

   3.5000   3.5000   3.5000

   3.6000   3.6000   3.6000

In : y,x = np.meshgrid([1,2,3],

                     [3.5,3.6])

In : y

array([[1, 2, 3],

       [1, 2, 3]])

In : x

array([[3.5, 3.5, 3.5],

       [3.6, 3.6, 3.6]])

11.1.6.13 Inflating Matrices

The scipy.ndimage module has a function zoom() which can be used to inflate a matrix by interpolation. (Interpolation is covered in Section 11.4.) The same result can be achieved in MATLAB, of course, but with more operations. The following example increases the number of rows by 3× and the number of columns by 2×:

MATLAB:

Python:

>> a = [4 -4; 4 0]

     4    -4

     4     0

>> nY = 2*3; nX = 2*2;

>> Xs = linspace(0,1,nX);

>> Ys = linspace(0,1,nY);

>> [Xq,Yq] = meshgrid(Xs, Ys);

>> interp2(X,Y,a,Xq,Yq)

  4.0000  1.3333 -1.3333 -4.0000

  4.0000  1.6000 -0.8000 -3.2000

  4.0000  1.8667 -0.2667 -2.4000

  4.0000  2.1333  0.2667 -1.6000

  4.0000  2.4000  0.8000 -0.8000

  4.0000  2.6667  1.3333       0

In : from scipy.ndimage import zoom

In : a = np.array([[4.0, -4], [4, 0]])

In : a

array([[ 4., -4.],

       [ 4.,  0.]])

In : zoom(a, [3, 2], order=1)

array([[ 4. , 1.3333, -1.3333, -4.  ],

       [ 4. , 1.6   , -0.8   , -3.2 ],

       [ 4. , 1.8667, -0.2667, -2.4 ],

       [ 4. , 2.1333,  0.2667, -1.6 ],

       [ 4. , 2.4   ,  0.8   , -0.8 ],

       [ 4. , 2.6667,  1.3333,  0.  ]])

11.1.6.14 Test Matrices

Algorithm development typically begins with test data, or in its absence, random data with prescribed properties. The array creation methods of this section can be combined in interesting ways to yield data having properties useful for exploring algorithm robustness, error handling, or exercising various logic branches. Here, we explore some of those methods.

Note

The MATLAB and Python matrices in this section are generated from random number generators and therefore will not match each other.

Integers between -5 and +5

MATLAB:

Python:

>> floor(10*rand(4,4) - 5)

  -5  -3   4  -4

  -4   4   1   4

  -3   0  -1   3

   3  -3   0  -4

In : np.random.randint(-5,6,size=(4,4))

array([[-3,  5,  4,  5],

       [ 0, -1,  1,  3],

       [ 0,  3,  1, -2],

       [-1, -1,  5, -5]])

4 × 3 array of integers in a linear sequence, column major

MATLAB:

Python:

>> reshape(1:12, 4,3)

    1    5    9

    2    6   10

    3    7   11

    4    8   12

In : np.arange(1,13).reshape(3,4).T

array([[ 1,  5,  9],

       [ 2,  6, 10],

       [ 3,  7, 11],

       [ 4,  8, 12]])

4 × 3 array of integers in a linear sequence, row major

MATLAB:

Python:

>> reshape(1:12, 3,4)'

    1    2    3

    4    5    6

    7    8    9

   10   11   12

In : np.arange(1,13).reshape(4,3)

array([[ 1,  2,  3],

       [ 4,  5,  6],

       [ 7,  8,  9],

       [10, 11, 12]])

4 × 3 array of floats, half are NaN

MATLAB:

Python:

>> a = 100*rand(4,3);

>> a(a < 50) = nan

a =

   69.082      NaN      NaN

      NaN   73.604      NaN

      NaN   63.711      NaN

   51.612   76.674   63.956

In : a = 100*np.random.rand(4,3)

In : a[a < 50] = np.NaN

In : a

array([[60.551, 99.574, 78.620],

       [   nan, 66.137, 64.343],

       [52.505,    nan,    nan],

       [96.109, 56.539,    nan]])

Note

MATLAB and Python generate different random matrices. The two matrices earlier are not intended to match.

11.1.7 Complex Scalars and Arrays

MATLAB and NumPy take slightly different approaches to creating complex scalars and arrays. MATLAB treats both i and j as $$ sqrt{-1} $$, but NumPy only recognizes j. Both also support a complex() function that takes real and imaginary arguments:

MATLAB:

Python:

>> 1+2i

   1.0000 + 2.0000i

>> 3 - 4j

   3.0000 - 4.0000i

>> complex(5,6)

   5.0000 + 6.0000i

In : 1+2i

    1+2i

SyntaxError: invalid syntax

In : 3 - 4j

Out: (3-4j)

In : complex(5,6)

Out: (5+6j)

Curiously, neither the default Python complex() nor the NumPy version np.complex() accept array arguments. Instead, to create a complex numeric array from real and imaginary arrays in NumPy, one must multiply the array containing the imaginary part by the complex unit value 1j:

MATLAB:

Python:

>> N = ones(2,3);

>> complex(N, -5*N)

 1.00-5.00i 1.00-5.00i 1.00-5.00i

 1.00-5.00i 1.00-5.00i 1.00-5.00i

In : N = np.ones((2,3))

In : N - 5*N*(1j)

array([[1.-5.j, 1.-5.j, 1.-5.j],

       [1.-5.j, 1.-5.j, 1.-5.j]])

Once we have a complex variable, we can extract its real and imaginary components similarly:

MATLAB:

Python:

>> z = complex([2.2;3.3],...

               [-4.4;5.5])

   2.2000 - 4.4000i

   3.3000 + 5.5000i

>> real(z)

    2.2000

    3.3000

>> imag(z)

   -4.4000

    5.5000

In : z=np.array([[2.2],[3.3]]) + (1j)*

...:   np.array([[-4.4],[5.5]])

In : z

array([[2.2-4.4j],

       [3.3+5.5j]])

In : z.real

array([[2.2],

       [3.3]])

In : z.imag

array([[-4.4],

       [ 5.5]])

11.1.8 Linear Indexing

MATLAB matrices allow linear indexing whereby a single scalar index can be used to access terms of a multidimensional array. The scalar index is the offset from the beginning of the array to the desired term, striding along columns (or, more generally, incrementing the index of leftmost dimension most rapidly).

The same is possible with NumPy arrays using either their .take() or .item() methods. Keep in mind, though, that (1) NumPy arrays are row major so a different offset is needed to arrive at the same term, and (2) the argument to .take() is zero index based.

In this example, we extract the 12th item from the same array—but get different results because MATLAB counts down columns, while Python counts across rows.

MATLAB:

Python:

>> a = [11 14 17 20 23;

        12 15 18 21 24;

        13 16 19 22 25];

>> a(12)

  22

In : a = np.array(

     [[11, 14, 17, 20, 23],

      [12, 15, 18, 21, 24],

      [13, 16, 19, 22, 25]])

In : a.take(11)

Out: 16

Both languages have convenience functions that return row and column indices corresponding to a linear index given the shape of the original array:

MATLAB:

Python:

>> [r,c]=ind2sub([3,5],12)

r =  3

c =  4

>> sub2ind([3,5],3,4)

  12

In : np.unravel_index(11,(3,5))

Out: (2, 1)

In : np.ravel_multi_index((2,1),(3,5))

Out: 11

sub2ind() and np.ravel_multi_index() give the linear offset into a multidimensional array given the indices of the desired term.

The .take() method also accepts lists of indices and an optional argument specifying the dimension to extract terms from. This turns out to be useful when extracting rows or columns of NumPy arrays within MATLAB because MATLAB syntax precludes indexing NumPy arrays with brackets.

Here, we show how the first row and third column of a can be extracted with .take() and the axis keyword argument:

Python:
In : a
Out:
array([[11, 14, 17, 20, 23],
       [12, 15, 18, 21, 24],
       [13, 16, 19, 22, 25]])
In : a.take(0,axis=0)
Out: array([11, 14, 17, 20, 23])
In : a.take(2,axis=1)
Out: array([17, 18, 19])
A practical application of this technique appears in Section 12.​7, where a 1D NumPy array of RGBA values is pulled from a 2D NumPy colormap array within MATLAB; the line is
color = colors.take(col_ind + int64(4)*color_row);

11.1.9 Reading/Writing Arrays to/from Text Files

Sections 7.​1 and 7.​6 covered text and binary I/O of generic data. Here, we’ll examine I/O for the special case of numeric arrays to and from text files.

11.1.9.1 Reading an Array from a Text File

Numeric values can be read from a text file several ways in both languages, but the easiest is with the MATLAB load() function and NumPy’s np.loadtxt(). This is demonstrated with the text file a.txt containing this 3 × 2 matrix:
 -3.2  4.3
 1.e-5 2.9
 6.6  -9.2
Both MATLAB and Python load functions allow the file to be annotated by comments—any text following a pound sign—and both skip blank lines.

MATLAB:

Python:

>> a = load('a.txt')

a =

  -3.2000e+00   4.3000e+00

   1.0000e-05   2.9000e+00

   6.6000e+00  -9.2000e+00

In : a = np.loadtxt('a.txt')

In : a

array([[-3.2e+00,  4.3e+00],

       [ 1.0e-05,  2.9e+00],

       [ 6.6e+00, -9.2e+00]])

(See Section 7.​1.​5 for reading data from comma-separated value files.)

11.1.9.2 Writing an Array to a Text File

Writing numberic data to a text file can also be done multiple ways, but again the easy way is with single function calls—save() in MATLAB and np.savetxt() in Python. We’ll use the same a matrix as loaded from the a.txt file in the previous section:

MATLAB:

Python:

>> save("aM.txt","a","-ascii")

In : np.savetxt('aP.txt', a)

It produces these files:

aM.txt from MATLAB:
  -3.20000000e+00 4.30000000e+00
  1.00000000e-05 2.90000000e+00
  6.60000000e+00 -9.20000000e+00
aP.txt from Python:
-3.200000000000000178e+00 4.299999999999999822e+00
1.000000000000000082e-05 2.899999999999999911e+00
6.599999999999999645e+00 -9.199999999999999289e+00
np.savetxt()’s format can be configured with the optional fmt keyword argument:
In : np.savetxt('aP.txt', a, fmt='% 10.3f')
In : cat aP.txt
    -3.200      4.300
     0.000      2.900
     6.600     -9.200

11.1.10 Reading/Writing Arrays to/from Binary Files

Performing input and output with arrays to and from binary files is much faster, and nearly always more space efficient, than I/O with text files. Binary I/O can be done either with just the raw bytes of numeric data in an array or the raw bytes plus additional metadata storing the array dimensions and data types. Here, we’ll cover the four combinations of reading and writing with and without metadata and describe how to manage little/big endian byte conversions.

11.1.10.1 Reading a Raw Array from a Binary File

This can be done by opening a file with read binary mode in MATLAB or with NumPy’s np.fromfile() function. This example shows how to read a binary file containing 64,800 single-precision floating-point numbers (representing, e.g., sea surface temperature spaced at 1° interval of latitude and longitude), in native byte order.

MATLAB:

Python:

>> fh = fopen('f64800.bin');

>> sst = fread(fh, [360 ...

        180], 'single');

>> fclose(fh);

In : sst = np.fromfile('f64800.bin',

            dtype=np.float32)

In : sst = sst.reshape(360,180)

Given the same binary file, Python and MATLAB will load the same 1D array of terms. However, if the terms are loaded into a 2D or higher-dimensional array, the row versus column-major ordering difference between the two languages means terms will be ordered differently. In the preceding example, MATLAB’s sst would have to be loaded like this to have sst(i+1,j+1) in MATLAB equal sst[i,j] in Python:

MATLAB:
>> fh = fopen('f64800.bin');
>> sst = fread(fh, 360*180, 'single');
>> fclose(fh);
>> sst = reshape(sst,180,360)';

11.1.10.2 Writing a Raw Array to a Binary File

Numeric arrays are easily written to binary files. Here, we’ll write the sample sea surface temperature file used in the previous example:

MATLAB:

Python:

>> sst = single(rand(360,180))

>> fh = fopen('f64800.bin','w');

>> fwrite(fh,sst,'single');

>> fclose(fh);

In : sst = np.random.rand(360,

        180).astype(np.float32)

In : sst.tofile('f64800.bin')

11.1.10.3 Writing Arrays to a Binary File with Metadata

This creates files that are more convenient to use later because the data type and dimensions are also stored. The disadvantage is such file formats are then harder for other applications to deal with. MATLAB’s save() without arguments writes a binary file with one or more arrays (or other data structures). The saved file, in .mat format, includes MATLAB-specific headers and thus has more than a pure numeric representation of the array’s bytes.

NumPy has a pair of analogous functions, np.save() and np.savez(), which save one or multiple arrays to a file in “.npy” or “.npz” formats. Like .mat files, .npy/.npz files contain additional metadata on the arrays’ shapes and types. Sadly, the names of the arrays themselves are not captured by default; to add this information to an .npz file, one must call np.savez() with the unusual notation of “array=name”—in our case, X=X and Y=Y. If we omit that, the function stores our arrays with the names “arr_0”, “arr_1”, and so on.

MATLAB:

Python:

>> X=reshape([-4:4], 3,3)'

X =

  -4  -3  -2

  -1   0   1

   2   3   4

>> Y = [pi e]

Y =

   3.1416   2.7183

>> save 'XY.mat'  X Y

In : X = np.arange(-4,5).reshape(3,3)

In : X

array([[-4, -3, -2],

       [-1,  0,  1],

       [ 2,  3,  4]])

In : Y = np.array([np.pi, np.e])

In : Y

Out: array([3.14159265, 2.71828183])

In : np.savez('XY.npz', X=X, Y=Y)

Saving NumPy arrays with np.save() or np.savez() is more efficient and more portable across Python versions than using the pickle module (Section 7.​13).

In Section 11.1.10.4, we’ll see that XY.mat and XY.npz can be loaded with MATLAB’s load() and NumPy’s np.loadz() functions, respectively.

Before loading the arrays back in, we’ll write another pair of binary files, this time saving only bytes of the numeric data. This is useful when writing data for subsequent ingest by code written in other languages—C, C++, Java, Fortran, and so on—that do not understand .mat or .npz formats. A pure binary representation of an array also makes it possible for MATLAB to read arrays written by Python to binary files (although the converse is unnecessary because Python, through the SciPy module, can read .mat files, ref. Section 7.​14).

The native binary write in MATLAB requires the write technique shown in Section 7.​6. NumPy arrays, though, have a method, .tofile(), just for this purpose. In both cases, we expect the resulting binary file for array X to have 9 terms x 8 bytes per term = 72 bytes.

MATLAB:

Python:

>> fh = fopen('Xm.bin','wb');

>> fwrite(fh, X, 'double');

>> fclose(fh);

>> ls -l X?.bin

-rw-rw-r-- 1 al al 72 Nov 15 15:27 Xm.bin

-rw-rw-r-- 1 al al 72 Nov 15 15:28 Xp.bin

In : X.tofile('Xp.bin')

While both files are 72 bytes, as we’ll see in the next section, their contents are quite different (spoilers: MATLAB’s array is written column-wise, and Python’s is row-wise; MATLAB’s array is of type double, while Python’s is 64-bit integers).

11.1.10.4 Reading Arrays from a Binary File

The complementary functions to save and np.save()/np.savez() are, as one might expect, load() and np.load() (which reads both /.npy and /.npz files). Both languages read their respective files with single function calls, but MATLAB conveniently returns them directly into the existing namespace, while Python returns the results in a dictionary which is keyed by the saved name. Had we omitted calling np.savez() with the ‘X=X, Y=Y’ arguments, the data dictionary would have to be keyed by the default names, that is, data['arr_0'] for X and data['arr_1'] for Y.

MATLAB:

Python:

>> load XY  % .mat is implied

>> X

X =

  -4  -3  -2

  -1   0   1

   2   3   4

>>Y

Y =

   3.1416   2.7183

In : data = np.load('XY.npz')

In : data['X']

array([[-4, -3, -2],

       [-1,  0,  1],

       [ 2,  3,  4]])

In : data['Y']

Out: array([3.14159265, 2.71828183])

Next, we’ll look at the raw binary files Xm.bin and Xp.bin we wrote in the previous section. As these contain only bytes of numeric data, they can be read by any programming language capable of reading a binary file and mapping the data to a numeric array. However, the lack of metadata means we need a priori knowledge of the number of dimensions, the size along each dimension, and the data type.

As done in Section 7.​6, we’ll open the files with the generic read function using binary mode and then use the struct module to recast the bytes into our numeric data types. We’ll also need to know that

  • The Python-written file Xp.bin contains 64-bit (8-byte) signed integers (the default data type from np.arange() when start, end, and increments are integers).

  • The MATLAB-written file Xm.bin contains 64-bit (8-byte) floats (the MATLAB default for all numbers).

  • The struct format characters for 64-bit signed integers and 64-bit floats are “q” and “d”, respectively (Table 7-1).

  • Both files contain 2D arrays of size 3 × 3.

We’ll read each file in Python and MATLAB:

Python:
In : import struct
#    Python-written file
In : with open('Xp.bin', 'rb') as fh:
...:     raw = fh.read() # load entire contents into raw
...:     n_terms  = len(raw) // 8 # integer division
...:     Ints = struct.unpack(f'{n_terms}q', raw)
In : Ints
Out: (-4, -3, -2, -1, 0, 1, 2, 3, 4)
In : X = np.array(Ints).reshape(3,3)
In : X
array([[-4, -3, -2],
       [-1,  0,  1],
       [ 2,  3,  4]])
#    MATLAB-written file
In : with open('Xm.bin', 'rb') as fh:
...:     raw = fh.read()
...:     n_terms  = len(raw) // 8
...:     Flts = struct.unpack(f'{n_terms}d', raw)
In : X = np.array(Flts).reshape(3,3)
array([[-4., -1.,  2.],
       [-3.,  0.,  3.],
       [-2.,  1.,  4.]])

There’s our first surprise—Python loaded a transpose of the MATLAB data. This happens because MATLAB’s arrays are stored internally in column-major order, while NumPy arrays, by default, are stored row-major.

The MATLAB code to read the files is shown as follows. The second surprise appears if we forget the Python values are integers rather than doubles:

MATLAB:
>> fh = fopen('Xm.bin', 'rb');
>> X = fread(fh, 'double');
>> fclose(fh);
>> reshape(X, 3,3)
    -4    -3    -2
    -1     0     1
     2     3     4
>> fh = fopen('Xp.bin', 'rb');
>> X = fread(fh, 'double');
>> fclose(fh);
>> reshape(X, 3,3)
  9.9e-323 *
       NaN       NaN    0.1000
       NaN         0    0.1500
       NaN    0.0500    0.2000
>> fh = fopen('Xp.bin', 'rb');
>> X = fread(fh, 'int64');
>> fclose(fh);
>> reshape(X, 3,3)
    -4    -1     2
    -3     0     3
    -2     1     4

11.1.10.5 Endian Conversions

Little-to-big and big-to-little endian conversions are sometimes necessary when transferring numeric data between computers with different architectures. NumPy allows easy determination and manipulation of word endian format. MATLAB allows one to change an array’s endian order but not determine what its current order is.

NumPy arrays have an attribute, .dtype, which stores the data type such as np.float16 or np.complex256. The data type itself is a NumPy object with the interesting property that endianness can be determined from the .str representation of the data type—if it starts with >, it is big-endian; with < it is little-endian:

Python:
In : a = np.array([1234, 5678], dtype=np.int32)
In : a.dtype
Out: dtype('int32')
In : a.dtype.str
Out: '<i4'

Endian conversion on I/O can be done directly in NumPy simply by providing np.tofile() or np.fromfile() the explicit dtype to read or write. Revisiting the sea surface temperature example of Section 11.1.10.2, this time explicitly writing and reading a big-endian file looks like this:

Python:
In : sst = np.random.rand(360, 180).astype(np.float32)
In : sst.tofile('f64800.bin', dtype='>f4')
In : sst_copy = np.fromfile('f64800.bin', dtype='>f4')
MATLAB variables do not have metadata indicating their endianness. However, one can change the endianness with swapbytes(). NumPy’s .byteswap() method does the same thing:

MATLAB:

Python:

>> a = int32([1234 5678]);

>> b = swapbytes(a)

  1×2 int32 row vector

   -771489792    773193728

>> dec2hex(a)

  2×4 char array

    '04D2'

    '162E'

>> dec2hex(b)

  2×8 char array

    'D2040000'

    '2E160000'

In : a = np.array([1234, 5678],

              dtype=np.int32)

In : b = a.byteswap()

In : b

Out: array([-771489792,  773193728],

           dtype=int32)

In : hex(a[0]),hex(a[1])

Out: ('0x4d2', '0x162e')

In : hex(b[0]), hex(b[1])

Out: ('-0x2dfc0000', '0x2e160000')

Python’s hex() function treats signed numbers differently than MATLAB’s dec2hex(), but we can at least tell from the integer representations that the same operation is being performed.

11.1.11 Primitive Array Operations

11.1.11.1 Addition, Subtraction

Array addition and subtraction work identically in MATLAB and NumPy: if the array dimensions match, the terms are added or subtracted term-wise. If the array dimensions differ, an attempt is made to broadcast or perform an outer-loop add or subtract by expanding the array sizes until they match (broadcasting is explained in more detail in Section 11.1.13):

MATLAB:

Python:

>> a = [2, -3]

a =

   2  -3

>> b = [7; 8; 9]

b =

   7

   8

   9

>> a+b

    9    4

   10    5

   11    6

In : a = np.array([[2,-3]])

In : a

Out: array([[ 2, -3]])

In : b = np.array([[7],[8],[9]])

In : b

array([[7],

       [8],

       [9]])

In : a+b

array([[ 9,  4],

       [10,  5],

       [11,  6]])

11.1.11.2 Elementwise Operations

NumPy's array operations default to working elementwise. In other words, multiplication, division, and exponentiation with *, /, and ** yield elementwise results. The MATLAB equivalents have a . preceding elementwise operations (.*, ./, and .ˆ).

MATLAB:

Python:

>> a = [2  7;  3 8;  4 9];

>> b = [5 -6; -1 2; -3 4];

>> a.^2

    4   49

    9   64

   16   81

>> a.*b

   10  -42

   -3   16

  -12   36

In : a = np.array([[2,7],[3,8],[4,9]])

In : a**2

array([[ 4, 49],

       [ 9, 64],

       [16, 81]])

In : a*b

array([[ 10, -42],

       [ -3,  16],

       [-12,  36]])

11.1.11.3 Bitwise Operations

Operations that work on individual bits typically make sense only with unsigned integers—np.uintn and uintn in NumPy and MATLAB, for n = 8, 16, 32, or 64. Table 11-2 shows the MATLAB and Python bit operations.
Table 11-2

Bit operations

Operation

MATLAB

Python

Left shift bits of x y times

bitshift(x,y)

x << y

Right shift bits of x y times

bitshift(x,-y)

x >> y

x AND y

bitand(x,y)

x & y

x OR y

bitor(x,y)

x | y

x XOR y

bitxor(x,y)

x ˆ y

Flip bits of x

bitcmp(x)

∼ x

11.1.11.4 Transpose and Hermitian Operations

Array transposition differs slightly between MATLAB and NumPy. In MATLAB, the apostrophe after a matrix returns the transpose if the matrix is real and the complex conjugate transpose, or Hermitian, if the matrix is complex. In Python, the .T method returns the (purely structural) transpose for both real and complex arrays. To obtain the Hermitian, one must first invoke the .conj() method to obtain the complex conjugate, then .T:

MATLAB:

Python:

>> a = [1 2; 3 4]

>> a = [1 2 ; 3 4]

a =

   1   2

   3   4

>> a'

   1   3

   2   4

>> a = a + j*eye(2)

a =

   1 + 1i   2 + 0i

   3 + 0i   4 + 1i

>> a'

   1 - 1i   3 - 0i

   2 - 0i   4 - 1i

In : a = np.array([[1,2],[3,4]])

In : a

array([[1, 2],

       [3, 4]])

In : a.T

array([[1, 3],

       [2, 4]])

In : a = a + 1j*np.eye(2)

In : a

array([[1.+1.j, 2.+0.j],

       [3.+0.j, 4.+1.j]])

In : a.T

array([[1.+1.j, 3.+0.j],

       [2.+0.j, 4.+1.j]])

In : a.conj().T

array([[1.-1.j, 3.-0.j],

       [2.-0.j, 4.-1.j]])

11.1.11.5 Array Multiplication

Array multiplication is done with the * operator in MATLAB and the less intuitive @ operator in Python. Alternatively, one can use np.dot() to perform the matrix multiplication:

MATLAB:

Python:

>> a = [2 7; 3 8; 4 9]

a =

   2   7

   3   8

   4   9

>> b = [ 1 5]

b =

   1   5

>> a*b

   37

   43

   49

In : a = np.array([[2,7],[3,8],[4,9]])

In : a

array([[2, 7],

       [3, 8],

       [4, 9]])

In : b = np.array([1,5])

In : b

Out: array([1, 5])

In : a@b

Out: array([37, 43, 49])

In : np.dot(a,b)

Out: array([37, 43, 49])

MATLAB views all numeric values, whether they are scalars, one-dimensional vectors, or multidimensional matrices, as having at least two dimensions; MATLAB considers the preceding vector as having three rows and one column. In NumPy, one-dimensional arrays have just one dimension; concepts of “row” and “column” do not apply.

Warning

The * operator in Python performs an elementwise array product, like .* in MATLAB. This can be a source of bugs when translating MATLAB expressions to Python.

11.1.12 Adding Dimensions

MATLAB’s “all variables have at least two dimensions” property is useful when the distinction between a row and column vector is important. As a simple example, consider the vector products vTv and vvT. The first produces a scalar, while the second gives a square matrix with v’s dimension. MATLAB can express these products as v'*v and v*v'.

In contrast, a one-dimensional NumPy array has no knowledge of row or column orientation; v.T@v and [email protected] (or v.T.dot(v) and v.dot(v.T)) both return vT v. We need an extra dimension to compute the outer product.

A one-dimensional NumPy array can get a second (or higher) dimension using one of three ways:

  • By explicitly reshaping it

In : v = np.array([1, 2, 3])
In : v.reshape(1,3)       # row
Out: array([[1, 2, 3]])
In : v.reshape(3,1)       # column
array([[1],
       [2],
       [3]])

  • By passing it to np.atleast 2d()5

In : v = np.array([1, 2, 3])
In : np.atleast_2d(v)     # row
Out: array([[1, 2, 3]])
In : np.atleast_2d(v).T  # column
array([[1],
       [2],
       [3]])

  • By indexing it with np.newaxis or None as an additional subscript

In : v = np.array([1, 2, 3])
In : v[np.newaxis,:]     # row
Out: array([[1, 2, 3]])
In : v[:,None]           # column
array([[1],
       [2],
       [3]])

The last method using np.newaxis as a new index appears frequently when broadcasting arrays, a topic covered in the next section. As its name implies, np.newaxis brings into existence a new dimension with size 1 at the designated location. Indexing the one-dimensional array v in the preceding example as v[np.newaxis, :, np.newaxis] yields a 3D array with dimensions 1 × 3 × 1 array.

11.1.13 Array Broadcasting

Broadcasting in both MATLAB and Python refers to implicitly extending array dimensions by copying existing rows or columns (or submatrices) to satisfy rules for numeric operations. Specifically, an array can be broadcast to the needed shape if its trailing dimensions are one or match the trailing dimensions of the other arrays in the computation; the nonmatching dimensions are simply copied to inflate the array to the correct size. np.newaxis often plays a useful role as the last dimension since its size is 1 and therefore pairs with the last dimension of other arrays.

Broadcasting is best explained with examples. In the following, the vectors (11, 12) and (4, 5, 6) are added by broadcasting two different ways, first by inflating them to 2 x 3 matrices:
$$ left[egin{array}{l}11kern0.5em 11kern0.5em 11\ {}12kern0.5em egin{array}{cc}12&amp; 12end{array}end{array} ight]+left[egin{array}{l}egin{array}{ccc}4&amp; 5&amp; 6end{array}\ {}egin{array}{cc}egin{array}{cc}4&amp; 5end{array}&amp; 6end{array}end{array} ight] $$
then by going to 3 × 2 matrices:
$$ left[egin{array}{l}egin{array}{cc}11&amp; 12\ {}11&amp; 12end{array}\ {}11kern0.5em 12end{array} ight]+left[egin{array}{l}egin{array}{cc}4&amp; 4\ {}5&amp; 5end{array}\ {}egin{array}{cc}6&amp; 6end{array}end{array} ight] $$
sum the unequally-sized arrays (11, 12) and (4, 5, 6) directly, then the successful. The following code shows the errors MATLAB and NumPy give when attempting to sum when the arrays are altered to satisfy broadcasting rules:

MATLAB:

Python:

>> a = [11 12];

>> b = [4 5 6];

>> a + b

error: operator +: noncon-

formant arguments (op1 is

1x2, op2 is 1x3)

>> a' + b

   15   16   17

   16   17   18

>> a + b'

   15   16

   16   17

   17   18

In : a = np.array([11,12])

In : b = np.array([4, 5, 6])

In : a + b

ValueError: operands could

not be broadcast together

with shapes (2,) (3,)

In : a[:,np.newaxis] + b

array([[15, 16, 17],

       [16, 17, 18]])

In : a + b[:,np.newaxis]

array([[15, 16],

       [16, 17],

       [17, 18]])

11.1.13.1 Broadcasting Requires More Memory

Broadcasting—in MATLAB as in Python—is generally computationally efficient as it eliminates the need to write explicit loops. However, as more dimensions are copied, broadcasting can become a substantial memory and CPU sink. Say you have 1000 frames captured by a 4k camera where each frame has 4096 x 2160 pixels of uint32 values, and you need to multiply each frame’s pixels by a filter matrix. If you were to use broadcasting, a temporary array containing 1000 copies of the filter would be made first before the multiplication takes place.

Writing a loop to iterate over each frame may be slower but could save you from an out-of-memory error.

11.1.13.2 Broadcasting Example 1: Normalize Vectors

In this example, we have five 3D vectors stored in a 5 x 3 array. We want to divide each vector by its magnitude to make them all unit vectors. The computation could be done by writing a loop to iterate over each vector, but broadcasting lets us skip the loop and implement a more efficient solution. We’ll start by computing the magnitude of the vectors:

MATLAB:

Python:

>> V = [-7.72 -2.84 -7.55; ...

         0.79  4.42  6.10; ...

        -9.99  4.24  9.29; ...

         5.84  7.65  0.20; ...

        -6.85  3.73  7.56];

>> mag = vecnorm(V');

In : V = np.array(

        [[-7.72, -2.84, -7.55],

         [ 0.79,  4.42,  6.10],

         [-9.99,  4.24,  9.29],

         [ 5.84,  7.65,  0.20],

         [-6.85,  3.73,  7.56]])

In : mag = np.linalg.norm(V,axis=1)

MATLAB conveniently figures out the correct broadcast expansion, but NumPy must be told which dimension to add; simply doing V/mag raises an error saying “operands could not be broadcast together with shapes (5,3) (5,).”

MATLAB:

Python:

>> V = V./mag';

>> V

   -0.6914   -0.2544   -0.6762

    0.1043    0.5835    0.8054

   -0.6993    0.2968    0.6503

    0.6067    0.7947    0.0208

   -0.6306    0.3434    0.6960

In : V = V/mag[:,np.newaxis]

In : V

array([[-0.6914, -0.2544, -0.6762],

       [ 0.1043,  0.5835,  0.8053],

       [-0.6993,  0.2968,  0.6504],

       [ 0.6067,  0.7947,  0.0208],

       [-0.6306,  0.3434,  0.6960]])

11.1.13.3 Broadcasting Example 2: The Distance Matrix

Section 11.12.2 shows how the Traveling Salesman Problem can be solved with the stochastic optimization technique of simulated annealing. The TSP cost function is the sum of distances between adjacent cities in the route. The cities’ locations remain constant, so we can speed up the inner loops of the TSP solvers by precomputing all city-to-city distances up front.

Let’s say our city locations can be described on a 2D grid using Cartesian coordinates (x, y). Table 11-3 shows locations of four cities that will be used in the following examples.
Table 11-3

City locations on a Cartesian grid

 

city0

city1

city2

city3

x

56.2

27.7

96.1

51.7

y

9.7

71.0

51.9

65.1

We want to compute the 4 × 4 matrix of distances for all city pairs i and j: $$ {D}_{ij}=sqrt{{left({x}_i-{x}_j ight)}^2+{left({y}_i-{y}_j ight)}^2} $$. The matrix Dij is symmetric and will have zeros on the diagonal. We’ll begin by computing the 4 × 4 matrix of x coordinate differences, xi − xj, with broadcasting:

MATLAB:

Python:

>> x = [56.2 27.7 96.1 51.7]

x =

 56.200 27.700 96.100 51.700

>> x'

   56.200

   27.700

   96.100

   51.700

>> x' - x

   0.00  28.50 -39.90   4.50

 -28.50   0.00 -68.40 -24.00

  39.90  68.40   0.00  44.40

  -4.50  24.00 -44.40   0.00

In : x = np.array([56.2,27.7,96.1,51.7])

Out: x

array([56.2, 27.7, 96.1, 51.7])

In : x[:,np.newaxis]

array([[56.2],

       [27.7],

       [96.1],

       [51.7]])

In : x[:,np.newaxis] - x

array([[  0. ,  28.5, -39.9,   4.5],

       [-28.5,   0. , -68.4, -24. ],

       [ 39.9,  68.4,   0. ,  44.4],

       [ -4.5,  24. , -44.4,   0. ]])

Squaring the terms to get (xi − xj)2 is straightforward:

MATLAB:

Python:

>> (x' - x).^2

    0.00  812.25 1592.01   20.25

  812.25    0.00 4678.56  576.00

 1592.01 4678.56    0.00 1971.36

   20.25  576.00 1971.36    0.00

In : (x[:,np.newaxis] - x)**2

array([[   0.  ,  812.25, 1592.01,   20.25],

       [ 812.25,    0.  , 4678.56,  576.  ],

       [1592.01, 4678.56,    0.  , 1971.36],

       [  20.25,  576.  , 1971.36,    0.  ]])

Applying broadcasting to the y differences as well lets us compute the distance matrix with a single line of code:

MATLAB:

Python:

>> y = [9.7 71.0 51.9 65.1];

>> D = sqrt((x'-x).^2 + (y'-y).^2)

D =

  0.0000 67.6013 58.0762 55.5824

 67.6013  0.0000 71.0166 24.7145

 58.0762 71.0166  0.0000 46.3206

 55.5824 24.7145 46.3206  0.0000

In : y = np.array([9.7,71.0,51.9,65.1])

In : D = np.sqrt((x[:,np.newaxis]-x)**2 +

...:             (y[:,np.newaxis]-y)**2)

[[ 0.    , 67.6013, 58.0762, 55.5824],

 [67.6013,  0.    , 71.0166, 24.7145],

 [58.0762, 71.0166,  0.    , 46.3206],

 [55.5824, 24.7145, 46.3206,  0.    ]])

11.1.14 Index Masks

MATLAB and Python allow one to identify and operate on array terms according to the value of an index mask. In addition to working as filters, index masks help one write vectorizable code and therefore are powerful tools for achieving high performance.

The following example shows mathematical operations are only performed at indices where the mask is non-zero or logical true (MATLAB) or True (Python).

MATLAB:

Python:

>> a = reshape(0:8, 3,3)'

a =

     0     1     2

     3     4     5

     6     7     8

>> mask=false(3,3);

>> mask(2:end,2:end) = true;

>> mask

  3×3 logical array

   0   0   0

   0   1   1

   0   1   1

>> a(mask)'

   4   7   5   8

>> a(mask) = -a(mask)

     0     1     2

     3    -4    -5

     6    -7    -8

In : a = np.arange(9).reshape(3,3)

In : a

array([[0, 1, 2],

       [3, 4, 5],

       [6, 7, 8]])

In : mask = np.zeros((3,3), dtype=np.bool)

In : mask[1:,1:] = True

In : mask

array([[False, False, False],

       [False,  True,  True],

       [False,  True,  True]])

In : a[mask]

Out: array([4, 5, 7, 8])

In : a[mask] *= -1

In : a

array([[ 0,  1,  2],

       [ 3, -4, -5],

       [ 6, -7, -8]])

11.1.14.1 Creating Index Masks

Index masks can be created by applying Boolean expressions to arrays. The primary distinction between MATLAB and Python in this regard is how AND and OR operators are implemented. MATLAB uses & and | while NumPy uses * and +:

MATLAB:

Python:

>> a = -2:2

a =

  -2  -1   0   1   2

>> -1 < a

   0   0   1   1   1

>> a < 2

   1   1   1   1   0

>> (-1 < a) & (a < 2)

   0   0   1   1   0

>> abs(a) > 1

   1   0   0   0   1

>> (-1 < a) | (abs(a) > 1)

   1   0   1   1   1

In : a = np.arange(-2,3)

In : a

Out: array([-2, -1,  0,  1,  2])

In : -1 < a

Out: array([False, False,  True,  True,  True])

In : a < 2

Out: array([ True,  True,  True,  True, False])

In : (-1 < a) * (a < 2)

Out: array([False, False,  True,  True, False])

In : np.abs(a) > 1

Out: array([ True, False, False, False,  True])

In : (-1 < a) + (np.abs(a) > 1)

Out: array([ True, False,  True,  True,  True])

11.1.14.2 Using Index Masks

Once we have an index mask, we can perform operations on terms of similarly sized arrays wherever the mask has a value of True (Python) or 1 (MATLAB). As an example, consider an array of longitudes with values ranging from –180 to +180 degrees that needs to be passed to a function expecting longitudes between 0 to 360 degrees. Longitudes are cyclic in that a longitude of λ is identical to λ+ 360 degrees, or λ + 2π radians. To satisfy the function’s desired input range we merely need to degrees that needs to be passed to a function expecting longitudes between 0 and 360 add 360 degrees to any negative longitude, a task well suited to index arrays:

MATLAB:

Python:

>> Lon = [40 -40 60 -120];

>> mask = Lon < 0;

>> Lon(mask)=Lon(mask)+360

Lon =

   40   320    60   240

In : Lon = np.array([40, -40, 60, -120])

In : mask = Lon < 0

In : Lon[mask] += 360

In : Lon

Out: array([40, 320, 60, 240])

The mask variable itself need not be explicitly saved; the Boolean expression defining the mask can be used as the array subscript directly:

MATLAB:

Python:

>> Lon = [40 -40 60 -120];

>> Lon(Lon<0)=Lon(Lon<0)+360

Lon =

   40   320    60   240

In : Lon = np.array([40, -40, 60, -120])

In : Lon[Lon < 0] += 360

In : Lon

Out: array([40, 320, 60, 240])

11.1.15 Extracting and Updating Submatrices

Submatrices of an array can be accessed and updated easily within MATLAB and Python using either row and column slices or index matrices. Here, we make a 6 × 4 matrix, extract the lower-right 2 × 3 terms, then subtract 20 from the lower-right 3 × 4 terms:

MATLAB:

Python:

>> a = reshape([1:24], 4,6)'

a =

    1    2    3    4

    5    6    7    8

    9   10   11   12

   13   14   15   16

   17   18   19   20

   21   22   23   24

>> a(4:5,2:4)

   14   15   16

   18   19   20

>> a(4:5,2:4)=a(4:5,2:4)-20

a =

    1    2    3    4

    5    6    7    8

    9   10   11   12

   13   -6   -5   -4

   17   -2   -1    0

   21   22   23   24

In : a = np.arange(1,25).reshape(6,4)

In : a

array([[ 1,  2,  3,  4],

       [ 5,  6,  7,  8],

       [ 9, 10, 11, 12],

       [13, 14, 15, 16],

       [17, 18, 19, 20],

       [21, 22, 23, 24]])

In : a[3:5,1:4]

array([[14, 15, 16],

       [18, 19, 20]])

In : a[3:5,1:4] -= 20

In : a

array([[ 1,  2,  3,  4],

       [ 5,  6,  7,  8],

       [ 9, 10, 11, 12],

       [13, -6, -5, -4],

       [17, -2, -1,  0],

       [21, 22, 23, 24]])

MATLAB can just as easily use arbitrary index arrays instead of index slices to define our submatrix. NumPy makes this harder than it should be, though, because it permits only one dimension to be indexed by an array at a time. Say we wish to extract the 3 × 2 submatrix made from the third, fifth, and sixth rows and the second and fourth columns:

MATLAB conveniently lets us define our submatrix with a(I,J), where I and J are arrays of our rows and columns. NumPy (as of v1.20.2) disappoints here though as a[I,J] raises an IndexError. To achieve the same extraction as MATLAB, the special accessor function np.ix_() is needed:

MATLAB:

Python:

>> a = reshape([1:24], 4,6)'

a =

    1    2    3    4

    5    6    7    8

    9   10   11   12

   13   14   15   16

   17   18   19   20

   21   22   23   24

>> I = [3 5 6];

>> J = [2  4];

>> a(I,J)

   10   12

   18   20

   22   24

In : a = np.arange(1,25).reshape(6,4)

In : a

array([[ 1,  2,  3,  4],

       [ 5,  6,  7,  8],

       [ 9, 10, 11, 12],

       [13, 14, 15, 16],

       [17, 18, 19, 20],

       [21, 22, 23, 24]])

In : I = [2, 4, 5]

In : J = [1, 3]

In : a[np.ix_(I,J)]

array([[10, 12],

       [18, 20],

       [22, 24]])

Alternatively , we can use index chaining (Section 3.​4.​5) with a pair of single-dimension index arrays:

Python:
In : a[I,J]
Traceback (most recent call last):
    a[I,J]
IndexError: shape mismatch: indexing arrays could not be broadcasttogether with shapes (3,) (2,)
In : a[I,:]
Out:
array([[ 9, 10, 11, 12],
       [17, 18, 19, 20],
       [21, 22, 23, 24]])
In : a[:,J]
Out:
array([[ 2,  4],
       [ 6,  8],
       [10, 12],
       [14, 16],
       [18, 20],
       [22, 24]])
In : a[I][:,J]
Out:
array([[10, 12],
       [18, 20],
       [22, 24]])

The notation a[I][:,J] is unintuitive though and is not encouraged.

11.1.16 Finding Terms of Interest

Both NumPy and MATLAB have functions useful for locating terms within an array that satisfy desired properties. The following sections will use the following 2D arrays to demonstrate searching for terms in an array with and without NaNs. (NaNs are used in fields such as the atmospheric sciences to represent unknown values, e.g., a satellite image of terrain may use NaNs to represent ground pixels hidden by clouds.)

In each example, we seek both the values satisfying the properties and indices of those values.

MATLAB:

Python:

>> X = [0.22, -.47; ...

        0.97, -.31; ...

        0.32, 0.05];

>> Z = [0.22, -.47; ...

        0.97, -.31; ...

        nan , 0.05];

In : X = np.array(

        [[0.22, -.47,],

         [0.97, -.31,],

         [0.32, 0.05,]])

In : Z = np.array(

        [[0.22  , -.47,],

         [0.97  , -.31,],

         [np.NaN, 0.05,]])

11.1.16.1 Find the Smallest Term

MATLAB's and NumPy's min() and max() functions work similarly except when handling NaNs—in NumPy one must explicitly call a NaN-aware function if one anticipates working with arrays containing NaNs. This example uses X and Z from the previous section:

MATLAB:

Python:

>> min(X,[],'all')

 -0.47000

>> min(Z,[],'all')

 -0.47000

In : np.min(X)

Out: -0.47

In : np.min(Z)

Out: nan

In : np.nanmin(Z)

Out: -0.47

Python is also a bit awkward in that the index of the minimum value returned by np.argmin() is a linear index, which, unlike MATLAB, requires the use of the .take() method for arrays having two or more dimensions. Alternatively, the linear index and the array’s dimensions can be passed to np.unravel_index() to give the row and column (and higher dimensions as needed) indices corresponding to the linear index.

MATLAB:

Python:

>> [Rval, Rind] = min(X)

Rval =

   0.22000  -0.47000

Rind =

   1   1

>> [Cval, Cind] = min(Rval)

Cval = -0.47000

Cind =  2

>> R = Rind(Cind); C = Cind;

>> R, C

R =  1

C =  2

>> X(R,C)

 -0.47000

In : lin_i = np.argmin(X)

In : lin_i

Out: 1

In : X.take(lin_i)

Out: -0.47

In : R, C = np.unravel_index(lin_i, X.shape)

In : R, C

Out: (0, 1)

In : X[R,C]

Out: -0.47

The identical process works in MATLAB for the NaN-containing Z array. In Python, however, one must call np.nanargmin(Z) to arrive at –0.47. Calling np.argmin(Z) leads to the linear index of 4, corresponding to the NaN term, instead of 1.

One aspect of finding minimum values that MATLAB and Python share is that if the matrix has multiple copies of the minimum value, both languages return the location of the first occurrence.

11.1.16.2 Find the Largest Term

Unsurprisingly , the process for finding indices of the largest term is identical to finding the smallest term except all function calls have max switched for min:

MATLAB:

Python:

>> [Rval, Rind] = max(X);

>> [Cval, Cind] = max(Rval);

>> R = Rind(Cind); C = Cind;

>> R, C

R =  2

C =  1

>> X(R,C)

ns =  0.97000

In : lin_i = np.argmax(X)

In : X.take(lin_i)

Out: 0.97

In : R, C = np.unravel_index(lin_i, X.shape)

In : R, C

Out: (1, 0)

In : X[R,C]

Out: 0.97

The call to argmax() in Python can also be written as a method call on the X array; lin_i = X.argmax() is equivalent to lin_i = np.argmax(X).

11.1.16.3 Find Term Nearest a Given Value

The location of the term closest to a given value can be found by getting the location of the smallest delta between all terms of the array and the desired value. Here, we find the indices of the term closest to –0.5:

MATLAB:

Python:

>> delta = abs(X - (-0.5));

>> [Rval, Rind] = min(delta);

>> [Cval, Cind] = min(Rval);

>> R = Rind(Cind); C = Cind;

>> R, C

R =  1

C =  2

>> X(R,C)

 -0.47000

In : delta = np.abs(X - (-0.5))

In : lin_i = delta.argmin()

In : R, C = np.unravel_index(lin_i, X.shape)

In : R, C

Out: (0, 1)

In : X[R, C]

Out: -0.47

Once again, in Python the equivalent search over terms of the NaN-containing Z array requires nanargmin() instead of argmin(). The MATLAB code shown earlier works with both X and Z.

11.1.16.4 Find Term Nearest a Given Value in a Sorted 1D Array

A special case of finding the term closest to a given value is when the array to be searched is one-dimensional and already sorted. The sorted data permits bisection searches to locate the desired terms rapidly. A typical use case is working with data accumulated chronologically; subsequent analyses often need to know what data was recorded nearest to a given time T.

MATLAB has an undocumented (and not guaranteed to exist in future versions) bisection search capability oddly invoked as builtin('_ismemberhelper',A,S).6 However, it only works for exact, not nearest, matches. In this case, we’re no better off than the O(N ) min() method of Section 11.1.16.3.

Python’s solution is the .searchsorted() method available to all NumPy arrays. It does an O(log2(N )) binary search for the locations where the sought elements should be inserted to maintain the sort order.

MATLAB:

Python:

X = 1000:1700;

for t = [1234.4, 1245.6]

  delta = abs(X - t);

  [val, ind] = min(delta);

  fprintf('%.2f ', X(ind))

end

1234.00

1246.00

In : A = np.arange(1000.0, 1701.0)

In : t = [1234.5, 1245.6]

In : ind = A.searchsorted(t)

In : ind

Out: array([235, 246])

In : A[ind]

Out: array([1235., 1246.])

11.1.16.5 Find Indices of Terms Satisfying a Condition

The locations of terms satisfying a condition can be found with NumPy’s argwhere() function which returns an array of indices for those matching terms. MATLAB’s find() function does the same thing, but it returns linear indexes into the array instead of index sets (e.g., (i,j) pairs for 2D arrays) as Python does.

This example shows how to get indices of terms whose magnitude is less than 0.3:

MATLAB:

Python:

>> ind = find(abs(X) < .3)

   1

   6

In : ind = np.argwhere(np.abs(X)<.3)

In : ind

array([[0, 0],

       [2, 1]])

The returned value from MATLAB’s find() can directly subscript the array, but the same is not true from Python’s np.argwhere(). There, one must call the tuple() function on each index pair first:

MATLAB:

Python:

>> X(ind)

   0.220000

   0.050000

In : X[ind]

array([[[ 0.22, -0.47],   # not

        [ 0.22, -0.47]],  # what

       [[ 0.32,  0.05],   # we

        [ 0.97, -0.31]]]) # wanted!

In : X[tuple(ind[0])]

Out: 0.22

In : X[tuple(ind[1])]

Out: 0.05

Section 11.1.14.1 shows more complex array conditions that include numeric AND and OR operators.

11.1.17 Object-Oriented Programming and Computational Performance

Section 10.​2 in the previous chapter mentioned object-oriented programming can have a negative impact on computational performance. To demonstrate this, we’ll create simulations of balls bouncing within a rectangular enclosure. One sim will use conventional procedure–based coding with all data stored in arrays, while the other uses object-oriented techniques. The simulation has two computationally intensive parts: determining collisions between the balls and updating velocity vectors for colliding ball pairs. Here, we’ll just implement the collision detection portion of the simulation.

If we store the ball’s coordinates and radii in conventional arrays, we can take full advantage of vectorization and easily compute collisions between a thousand balls at a rate of 60 Hz with both MATLAB and Python. Both languages take full advantage of broadcasting to compute the collision matrix, an N x N symmetric Boolean matrix which has 1 (MATLAB) or True (Python) at indices (i, j) and (i, j) if balls i and j are in contact for a simulation with N balls.

MATLAB:
% file: code/OO/collision_matrix.m
function coll = collision_matrix(x,y,R)
  dx = x - x';
  dy = y - y';
  sep = sqrt( dx.^2 + dy.^2 );
  sum_r = R + R';
  coll = sep < sum_r;
end
% file: code/OO/run_sim.m
function res = run_sim(N, n_iter)
  fprintf(‘n balls=%d ’, N)
  for i = 1:n_iter
    box_x = 10.;
    box_y = 6.;
    r_min = 0.1;
    r_max = 0.3;
    x = r_max + (box_x-2*r_max)*rand(N,1);
    y = r_max + (box_y-2*r_max)*rand(N,1);
    R = 0.1 + 0.8*rand(N,1);
    tic
    coll = collision_matrix(x, y, R);
    n_coll = (sum(sum(coll)) - N)/2;
    fprintf(‘n coll=%d Hz = %.2f ’, n_coll, 1/toc)
  end
end
Python :
#!/usr/bin/env python3
# file: code/OO/collision_matrix.py
import numpy as np
from numpy.random import rand
import time
def collision_matrix(x, y, R):
    """
    Use broadcasting to return a square matrix of Boolean
    values with True at position [i,j] and [j,i] if
    circles i and j intersect.
    """
    dx = x[:,np.newaxis] - x[np.newaxis,:]
    dy = y[:,np.newaxis] - y[np.newaxis,:]
    sep = np.sqrt( dx**2 + dy**2 )
    sum_r = R[:,np.newaxis] + R[np.newaxis,:]
    return sep < sum_r
def run_sim(N, n_iter):
  box_x, box_y = 10., 6.
  r_min, r_max = 0.1, 0.3
  print(f'n balls={N}')
  for i in range(n_iter):
    x = r_max + (box_x-2*r_max)*rand(N)
    y = r_max + (box_y-2*r_max)*rand(N)
    R = 0.1 + 0.8*rand(N)
    T_s = time.time()
    coll = collision_matrix(x, y, R)
    n_coll = (np.sum(coll) - N)//2
    print(f'n coll={n_coll} Hz = '
          f'{1/(time.time()-T_s):.2f}')
MATLAB and Python perform equally well:

MATLAB:

Python:

>> run_sim(1000, 10)

n balls=1000

n coll=29965 Hz = 59.29

n coll=28679 Hz = 47.92

n coll=29406 Hz = 45.44

n coll=30536 Hz = 62.94

n coll=28888 Hz = 62.21

n coll=29597 Hz = 62.99

n coll=29443 Hz = 62.64

n coll=29441 Hz = 63.83

n coll=30324 Hz = 65.24

n coll=29286 Hz = 63.38

In : run_sim(1000, 10)

n balls=1000

n coll=28836 Hz = 32.22

n coll=28728 Hz = 43.19

n coll=29613 Hz = 45.33

n coll=28559 Hz = 65.21

n coll=30694 Hz = 56.81

n coll=29023 Hz = 64.17

n coll=29755 Hz = 58.66

n coll=28404 Hz = 64.25

n coll=29196 Hz = 60.42

n coll=29824 Hz = 64.44

We’ll repeat the simulation, this time defining each ball as a Ball object (a variation of Circle that includes a collision detection function) and adding a class method to detect if the ball is in contact with another given ball.

MATLAB:
% file: code/OO/Ball.m
classdef Ball
  properties
      x {mustBeNumeric}
      y {mustBeNumeric}
      r {mustBeNumeric}
  end
  methods
   function obj = Ball(x,y,r)
     obj.x = x;
     obj.y = y;
     obj.r = r;
   end
   function coll = collides_with(obj, other)
        dx = obj.x - other.x;
        dy = obj.y - other.y;
        sep = sqrt( dx^2 + dy^2 );
        sum_r = obj.r + other.r;
        coll = sep < sum_r;
   end
 end
end
% file: code/OO/run_oo_sim.m
function res = run_oo_sim(N, n_iter)
  fprintf('n balls=%d ', N)
  box_x = 10.;
  box_y =  6.;
  r_min = 0.1;
  r_max = 0.3;
  for i = 1:n_iter
    tic
    balls = {};
    for j = 1:N
      x = r_max + (box_x-2*r_max)*rand();
      y = r_max + (box_y-2*r_max)*rand();
      R = 0.1 + 0.8*rand();
      balls{j} = Ball(x,y,R);
    end
    coll = eye(N);
    for j = 1:N
      for k = j+1:N
        coll(j,k) = balls{j}.collides_with(balls{k});
        coll(k,j) = coll(j,k);
      end
    end
    n_coll = (sum(sum(coll)) - N)/2;
    fprintf('n coll=%d Hz = %.2f ', n_coll, 1/toc)
  end
end
Python:
#!/usr/bin/env python3
# file: code/OO/Ball.py
import numpy as np
from numpy.random import rand
import time
class Ball:
  def __init__(self, x,y,r):
    self.x = x
    self.y = y
    self.r = r
  def collides_with(self, other):
    dx = self.x - other.x
    dy = self.y - other.y
    sep = np.sqrt( dx**2 + dy**2 )
    sum_r = self.r + other.r
    return sep < sum_r
def run_oo_sim(N, n_iter):
  box_x, box_y = 10., 6.
  r_min, r_max = 0.1, 0.3
  print(f'n balls={N}')
  for i in range(n_iter):
    balls = []
    for j in range(N):
      x = r_max + (box_x-2*r_max)*rand()
      y = r_max + (box_y-2*r_max)*rand()
      R = 0.1 + 0.8*rand()
      balls.append( Ball(x,y,R) )
    coll = np.eye(N, dtype=np.bool)
    T_s = time.time()
    for j in range(N):
      for k in range(j+1,N):
        coll[j,k] = balls[j].collides_with(balls[k])
        coll[k,j] = coll[j,k]
    n_coll = (np.sum(coll) - N)//2
    print(f'n coll={n_coll} Hz = '
          f'{1/(time.time()-T_s):.2f}')
def main():
  run_oo_sim(1000, 10)
if __name__ == "__main__": main()
In addition to taking more code, the object-oriented solution is much slower than the vectorized solution; updates happen at a frequency of less than 1 Hz, compared to about 60 Hz in the vectorized solution.

MATLAB:

Python:

>> run_oo_sim(1000, 10)

n balls=1000

n coll=29788 Hz = 0.77

n coll=29959 Hz = 0.84

n coll=29236 Hz = 0.85

n coll=29422 Hz = 0.85

n coll=30776 Hz = 0.85

n coll=29626 Hz = 0.86

n coll=30527 Hz = 0.83

n coll=28534 Hz = 0.85

n coll=29844 Hz = 0.85

n coll=30211 Hz = 0.85

In : run_oo_sim(1000, 10)

n balls=1000

n coll=28442 Hz = 0.96

n coll=28054 Hz = 0.97

n coll=29002 Hz = 0.96

n coll=29128 Hz = 0.97

n coll=28722 Hz = 0.91

n coll=30064 Hz = 0.93

n coll=29300 Hz = 0.94

n coll=28034 Hz = 0.93

n coll=28609 Hz = 0.93

n coll=30782 Hz = 0.94

11.2 Linear Algebra

MATLAB and Python, via NumPy and SciPy, offer similar capabilities—and similar computational performance—for linear algebra. There are three primary differences:

  • MATLAB’s operators for linear algebra notation are more concise than NumPy’s or SciPy’s.

  • MATLAB has a uniform collection of linear algebra functions, while Python’s are spread across NumPy and SciPy, which have overlapping capabilities in identically named linear algebra submodules, numpy.linalg and scipy.linalg. In general, SciPy’s linear algebra offerings are more extensive than NumPy’s.

  • MATLAB’s functions apply to both dense and sparse matrices. In both NumPy and SciPy, one must use differently named functions to operate on sparse matrices. Python’s sparse matrix capability is discussed in Section 11.3.

11.2.1 Linear Equations

11.2.1.1 Ax = b

Linear equations with a square matrix A can be solved like this in the two languages:

MATLAB:

Python:

>> A = [1,2,-3; ...

        4,-5,6; ...

       -7,8,9];

>> b = [-1,0; 0,1; 1,0];

>> x = A

x =

  -0.375000   0.175000

  -0.250000   0.050000

   0.041667   0.091667

In : import numpy as np

In : A = np.array([[1,2,-3],

                   [4,-5,6],

                   [-7,8,9]])

In : b = np.array([[-1,0], [ 0,1],

                   [ 1,0]])

In : x = np.linalg.solve(A,b)

In : x

array([[-0.375     ,  0.175     ],

       [-0.25      ,  0.05      ],

       [ 0.04166667,  0.09166667]])

To obtain just the LU factors of a full-rank matrix for subsequent use with forward- and back-substitution, one can use SciPy’s version of the linalg module:

MATLAB:

Python:

>> A = [1,2,-3; ...

        4,-5,6; ...

       -7,8,9];

>> [L, U, P] = lu(A)

L =

  1.00000  0.00000  0.00000

 -0.14286  1.00000  0.00000

 -0.57143 -0.13636  1.00000

U =

 -7.00000  8.00000  9.00000

  0.00000  3.14286 -1.71429

  0.00000  0.00000 10.90909

P =

Permutation Matrix

   0   0   1

   1   0   0

   0   1   0

In : import scipy.linalg as spla

In : A = np.array([[1,2,-3],

                   [4,-5,6],

                   [-7,8,9]])

In : LU, P = spla.lu_factor(A)

In : LU, P

(array([[-7.        ,  8.        ,  9.        ],

        [-0.14285714,  3.14285714, -1.71428571],

        [-0.57142857, -0.13636364, 10.90909091]]),

 array([2, 2, 2], dtype=int32))

Other options include Cholesky (scipy.linalg.cholesky, scipy.linalg.cholesky_banded()) and LDL (scipy.linalg.ldl()) factorizations of symmetric matrices. Forward- and back-substitution are then done with lu_solve() (or cho_solve() for a Cholesky factor):

MATLAB:

Python:

>> b = [1; 3; 5];

>> x = U(L(P*b))

  9.750000000000000e-01

  8.500000000000000e-01

  5.583333333333333e-01

In : b = np.array([1,3,5])

In : x = spla.lu_solve( (LU, P), b )

In : x

Out: array([0.975 , 0.85 , 0.55833333])

11.2.2 Singular Value Decomposition

NumPy and SciPy have identically named SVD functions, numpy.linalg.svd() and scipy.linalg.svd(). The SciPy version offers slightly more functionality by optionally checking for Inf and NaN values (use check_finite=True) and offering a choice of the LAPACK routine to use (lapack_driver='gesdd' is the default, while lapack_driver='gesvd' selects the same routine used by MATLAB).

MATLAB:

Python:

>> A = [9 8 7; 5 4 3; ...

       -1 2 -1; -5 5 0];

>> [U,S,V] = svd(A,0)

U =

 -8.9152e-01  4.7790e-02  2.0140e-01

 -4.5163e-01 -1.9355e-02 -3.8250e-01

  2.1957e-03  2.9513e-01 -8.6635e-01

  3.4816e-02  9.5407e-01  2.5014e-01

S =

  1.5615e+01      0      0

      0  7.3818e+00      0

      0      0  1.2919e+00

V =

 -6.6973e-01 -6.4105e-01 -3.7485e-01

 -5.6100e-01  7.6749e-01 -3.1021e-01

 -4.8656e-01 -2.5282e-03  8.7365e-01

In : A = np.array([[9,8,7],[5,4,3],

...:      [-1,2,-1],[-5,5,0.]])

In : U, S, V = np.linalg.svd(A,

...:         full_matrices=False)

In : U

array([[-0.89152435,  0.04778986,  0.20140207],

       [-0.45162746, -0.01935495, -0.38250147],

       [ 0.00219575,  0.29512661, -0.86634739],

       [ 0.0348158 ,  0.95406593,  0.25014403]])

In : S

array([15.61537691,  7.38180298,  1.29189342])

In : V

array([[-0.66973287, -0.56100049, -0.48655557],

       [-0.64105237,  0.76749298, -0.00252817],

       [-0.37484629, -0.3102144 ,  0.87364597]])

MATLAB and Python results for U and S match, but V is transposed. Here, we compute the solution’s error by subtracting the reconstructed A from the original:

MATLAB:

Python:

>> A - U*S*V'

   7.1054e-15   1.7764e-15   4.4409e-15

   2.6645e-15   1.7764e-15   1.3323e-15

   2.2204e-16   1.5543e-15  -1.1102e-16

   8.8818e-16   6.2172e-15   1.1657e-15

In : A - [email protected](S)@V

array(

[[ 7.10542736e-15, 1.77635684e-15, 4.44089210e-15],

 [ 2.66453526e-15, 1.77635684e-15, 1.33226763e-15],

 [ 2.22044605e-16, 1.55431223e-15,-1.11022302e-16],

 [ 8.88178420e-16, 6.21724894e-15, 1.12317600e-15]])

11.2.3 Eigenvalue Problems

MATLAB’s ability to overload functions is appealing for solving eigenvalue problems since only two functions, eig() and eigs(), handle everything—eigenvalues only, eigenvalues and eigenvectors, simple form, general form, symmetric or asymmetric inputs, full or partial solution. In contrast, NumPy and SciPy together have nearly a dozen functions (several of which are identically named) to cover these cases. Table 11-4 shows how eigensolving capability is fragmented in Python compared to MATLAB. The “asym.” column refers to a function’s ability to work with asymmetric matrices. A function without a mark in this column only works with real symmetric or complex Hermitian matrices. The specialized Python functions eigvals banded() and eigh tridiagonal() are not included.
Table 11-4

Capabilities of MATLAB, NumPy, and SciPy eigensolvers

 

dense

sparse

Ax = λx

Ax = λBx

asym.

full

partial

MATLAB

       

eig

•

•

•

•

•

•

 

eigs

•

•

•

•

•

 

•

numpy.linalg

       

eig

•

 

•

 

•

•

 

eigh

•

 

•

  

•

 

eigvals

•

 

•

 

•

•

 

eigvalsh

•

 

•

  

•

 

scipy.linalg

•

   

•

•

 

eig

•

 

•

•

•

•

 

eigh

•

 

•

•

 

•

 

eigvals

•

 

•

•

•

•

 

eigvalsh

•

 

•

•

 

•

 

scipy.sparse.linalg

       

eig

 

•

•

•

•

 

•

eigsh

 

•

•

•

  

•

MATLAB:

Python:

>> n = 3;

>> ones = ones(n);

>> off  = 2*tril(ones,-2);

>> A = (n+2)*eye(n) - ones + off

A =

     4    -1    -1

    -1     4    -1

     1    -1     4

>> [v,d] = eig(A)

v =

   -0.5774    0.0000    0.7071

   -0.5774    0.7071    0.7071

    0.5774   -0.7071   -0.0000

d =

    4.0000         0         0

         0    5.0000         0

         0         0    3.0000

>> A*v - v*d

   1.0e-14 *

   -0.0444    0.0020         0

   -0.2220    0.2665    0.1776

    0.2220   -0.2665   -0.1228

In : import numpy as np

In : n = 3

In : ones = np.ones((n,n))

In : off  = 2*np.tril(ones,-2)

In : A = (n+2)*np.eye(n) - ones + off

In : A

Out:

array([[ 4., -1., -1.],

       [-1.,  4., -1.],

       [ 1., -1.,  4.]])

In : [d,v] = np.linalg.eig(A)

In : v

Out: array(

  [[ 2.22045e-15,  5.77350e-01,  7.07107e-01],

   [ 7.07107e-01,  5.77350e-01,  7.07107e-01],

   [-7.07107e-01, -5.77350e-01,    4.44089e-16]])

In : d

Out: array([5., 4., 3.])

In : A@v - [email protected](d)

Out: array(

  [[-5.55111e-16, -4.44089e-16, 4.44089e-16],

   [ 4.88498e-15,  3.10862e-15, 2.22046e-15],

   [-1.77636e-15,  -8.88178e-16, -4.44089e-16]])

Aside from Python’s capability fragmentation, solutions, computational performance, and accuracy are comparable those in MATLAB.

11.3 Sparse Matrices

Sparse matrices are central to many areas of numerical analysis including computational fluid dynamics, finite element analysis, network theoretic computations, and neural network computations for machine learning and artificial intelligence. SciPy’s sparse module provides sparse matrix capability to Python. Under the hood, sparse uses the C and Fortran libraries SuperLU and ARPACK to solve systems of equations and eigenproblems.

SciPy’s sparse matrix capability is good but not as convenient as MATLAB’s. MATLAB has two advantages: its sparse matrices are more fully integrated into the language, and there is a single sparse storage scheme which is both efficient and flexible. SciPy provides many sparse matrix operations, but its sparse matrices may be used in only a subset of NumPy functions. Also, before creating a sparse matrix in SciPy, one must decide which of the seven (!) storage formats to use. Each storage format has different indexing capabilities and computational performance. Conversion between formats is easy however.

11.3.1 Sparse Matrix Creation with COO, CSC, CSR

A sparse matrix can be created from a dense matrix by passing the dense matrix to any of the sparse creation functions. This example uses the compressed row storage function csr_matrix() :

MATLAB:

Python:

>> x = [1 2 0; 0 3 0; 0 4 5]

x =

     1     2     0

     0     3     0

     0     4     5

>> y = sparse(x)

y =

   (1,1)        1

   (1,2)        2

   (2,2)        3

   (3,2)        4

   (3,3)        5

import scipy.sparse as sp

In : x = np.array([[1, 2, 0],

...:     [0, 3, 0],[0, 4, 5]])

In : x

array([[1, 2, 0],

       [0, 3, 0],

       [0, 4, 5]])

In : y = sp.csr_matrix(x)

In : print(y)

Out

  (0, 0)        1

  (0, 1)        2

  (1, 1)        3

  (2, 1)        4

  (2, 2)        5

Conversion from a dense matrix is generally only useful for testing sparse algorithms because only relatively small sparse matrices can be created this way. More typically, one builds up a sparse matrix from scratch. The most straightforward way to do that is with SciPy’s “coordinate” format, also known as “COO.” Its creation method is efficient and closely matches MATLAB’s—one provides the constructor function linear arrays of row, column, and values. For example, the code to define this matrix
$$ left(egin{array}{l}0kern0.5em egin{array}{cc}1&amp; 0end{array}\ {}egin{array}{ccc}8&amp; 0&amp; 2end{array}\ {}egin{array}{ccc}0&amp; 4&amp; -3end{array}\ {}egin{array}{cc}0&amp; egin{array}{cc}0&amp; 0end{array}end{array}end{array} ight) $$
looks like this:

MATLAB:

Python:

>> I = [1 2 2 3 3];

>> J = [2 1 3 2 3];

>> V = [1 8 2 4 -3];

>> A = sparse(I,J,V,4,3)

A =

Compressed Column Sparse

(rows = 3, cols = 3,

 nnz = 5 [56%])

  (2, 1) ->  8

  (1, 2) ->  1

  (3, 2) ->  4

  (2, 3) ->  2

  (3, 3) -> -3

import scipy.sparse as sp

In : I = [0, 1, 1, 2, 2]

In : J = [1, 2, 2, 1, 2]

In : V = [1, 8, 2, 4, -3]

In : A = sp.coo_matrix((V,(I,J)),(4,3))

In : A

<4x3 sparse matrix of type

 '<class 'numpy.int64'>'

 with 5 stored elements in COOrdinate

 format>

In : print(A)

  (0, 1)        1

  (1, 2)        8

  (1, 2)        2

  (2, 1)        4

  (2, 2)       -3

MATLAB’s output is sorted by column, while Python’s is sorted by row; the matrices actually contain the same values at the same indices. This can be seen by viewing the dense representation of each matrix:

MATLAB:

Python:

>>  full(A)

   0   1   0

   8   0   2

   0   4  -3

   0   0   0

In : A.todense()

matrix([[ 0,  1,  0],

        [ 8,  0,  2],

        [ 0,  4, -3],

        [ 0,  0,  0]])

11.3.1.1 Repeated COO Indices

MATLAB’s sparse() function sums terms where indices are repeated. In other words, if the index entry (i, j) appears three times in the input index arrays with corresponding values V1, V2, and V3 in the value array, MATLAB’s matrix value at (i, j) will be V1 + V2 + V3. The same behavior is true for all SciPy matrix creation functions although the summation may be implicit—the three duplicate terms are stored separately, but all mathematical operations on the sparse matrix yield the same result as if the terms were summed. To see this in action, we’ll create the sparse matrix
$$ left(egin{array}{cc}1&amp; 3\ {}0&amp; 4end{array} ight) $$
by repeating the last row and column’s indices twice, once with value 10 and a second time with value –6. This example omits the trailing dimension argument because the largest row and column indices in the input arrays I and J are enough to define the dimension:

MATLAB:

Python:

.

>> I = [1 1 2 2 ];

>> J = [1 2 2 2 ];

>> V = [1 3 10 -6 ];

>> A = sparse(I,J,V)

A =

Compressed Column Sparse

 (rows = 2, cols = 2,

  nnz = 3 [75%])

  (1, 1) ->  1

  (1, 2) ->  3

  (2, 2) ->  4

In : import scipy.sparse as sp

In : I = [0, 0, 1, 1]

In : J = [0, 1, 1, 1]

In : V = [1, 3, 10, -6]

In : A = sp.coo_matrix((V,(I,J)))

In : A

<2x2 sparse matrix of type

 '<class 'numpy.int64'>'

 with 4 stored elements in

 COOrdinate format>

In : print(A)

  (0, 0)        1

  (0, 1)        3

  (1, 1)       10

  (1, 1)       -6

Although Python stores the repeated index values separately, computation with the matrix shows it uses the summed value of 4 for the last term:

MATLAB:

Python:

>> A * [1; 1]

   4

   4

In : A.dot([1,1])

Out: array([4, 4], dtype=int64)

The Python repeated index terms can be explicitly summed with the sparse matrix object’s .sum_duplicates() method :

Python:
In : print(A)
  (0, 0)        1
  (0, 1)        3
  (1, 1)        10
  (1, 1)        -6
In : A.sum_duplicates()
In : print(A)
  (0, 0)        1
  (0, 1)        3
  (1, 1)        4

11.3.1.2 Extracting Indices and Values

Indices and values of a COO sparse matrix are stored as object attributes .row, .col, and .data. These correspond to the output of MATLAB’s find() function on a sparse matrix. Using A from Section 11.3.1.1:

MATLAB:

Python:

>> [I J V] = find(A);

>> I'

   1 1 2

>> J'

   1 2 2

> V'

   1 3 4

In : A.sum_duplicates()

In : A.row

Out: array([0, 0, 1], dtype=int32)

In : A.col

Out: array([0, 1, 1], dtype=int32)

In : A.data

Out: array([1, 3, 4])

Without the call to .sum_duplicates(), the index and value attributes would have shown the repeated terms.

Only COO matrices have .row and .col attributes. However, conversion from other formats to COO via matrices’ .tocoo() methods is inexpensive. See Section 11.3.5.2 for a complete list of components available in the other formats.

11.3.1.3 Compressed Row, Column Storage

The Compressed Sparse Row (CSR) and Compressed Sparse Column (CSC) storage schemes allow more rapid matrix operations such as factoring and multiplication. Although their creation functions csr_matrix() and csc_matrix() can take the same arguments as coo_matrix(), it is usually faster to create matrices with coo_matrix() and then convert them to CSR or CSC formats using the matrices’ .tocsr() or .tocsc() methods. Duplicate indices are summed during these conversions.

While csr_matrix() and csc_matrix() can accept explicit row and column index arrays, one can directly supply the tighter index and index pointer arrays native to the CSR and CSC internal representations. For csr_matrix(), this means an array of column indices (J in the preceding example) and an array of pointers that contain counts of terms in each row. The array of pointers, indptr, are defined such that indices for row i are stored in J[indptr[i]:indptr[i + 1]] and their corresponding values are stored in data[indptr[i]:indptr[i + 1]]. These inputs repeat the earlier example:

Python:
In : J = np.array([1,0,2,1,2])  # column indices
In : V = np.array([1,8,2,4,-3]) # matrix data values
In : indptr = np.array([0, 1, 3, 5, 5])
In : A = sp.csr_matrix((V, J, indptr), shape=(4, 3))
In : A.todense()
The indptr array bears additional clarification—how did we arrive at [0, 1, 3, 5, 5]? One way to think of it is the difference indptr[i + 1] - indptr[i] is the count of terms in row i, and each term in indptr includes the sum of terms before it. Table 11-5 shows the contents of the index array J needed to capture indices of the sparse matrix at (0, 1), (1, 0), (1,2), (2, 1), and (2, 2).
Table 11-5

indptr values for each row

Row

  

Column Indices

0

J[ indptr[0]:indptr[1] ]

J[0:1]

[1]

1

J[ indptr[1]:indptr[2] ]

J[1:3]

[0,2]

2

J[ indptr[2]:indptr[3] ]

J[3:5]

[1,2]

3

J[ indptr[3]:indptr[4] ]

J[5:5]

[]

To convert an array of row indices, I = [1,0,2,1,2] in our case, into index pointers, we can take advantage of a data container, Counter, from the collections module. A Counter takes an iterable such as a list or array and returns a dictionary whose keys are unique entries in the iterable and whose values are the number of times those entries appeared. If the dictionary is keyed by a nonexistent key, rather than raising a KeyError, it simply returns zero .

Python:
In : I
Out: array([0, 1, 1, 2, 2])
In : from collections import Counter
In : n_terms_this_row = Counter(I)
In : indptr = [0]
In : for i in range(nRows):
...:     indptr.append( indptr[-1] + n_terms_this_row[i])
In : indptr
Out: [0, 1, 3, 5, 5]

Index pointer inputs to csc_matrix() work the same way as for csc_matrix(), except one supplies explicit row indices and pointers to column indices instead of the other way around.

11.3.2 Sparse Matrix Creation with LIL, DOK

The three storage formats of the previous section, coordinate and compressed row and column, require explicit entries for each row index, column index, and value to be given up front. Rather than compute locations and values and then create the sparse matrix from these, sparse matrices can be built up incrementally. In addition, many engineering applications, notably finite element analysis, produce sparse matrices from many smaller submatrices. The resulting sparse matrix has small blocks of non-zero terms in adjacent rows and columns.

SciPy has three storage formats, list of lists (LIL) , dictionary of keys (DOK) , and block compressed row (BSR), that enable efficient incremental assembly, access, and manipulation of such structured sparse matrices.

Here, we’ll recreate the matrix
$$ left(egin{array}{l}egin{array}{ccc}0&amp; 1&amp; 0\ {}8&amp; 0&amp; 2\ {}0&amp; 4&amp; -3end{array}\ {}0kern0.5em 0kern0.5em 0end{array} ight) $$
using scalar inserts for the 1 in the top row and the 8 in the left column, then a block insert to place the submatrix
$$ left(egin{array}{cc}0&amp; 2\ {}4&amp; -3end{array} ight) $$

into the second and third rows and columns.

This example uses the LIL constructor:

MATLAB:

Python:

.

>> A = sparse(4,3)

>> A(1,2) = 1;

>> A(2,1) = 8;

>> A(2:3,2:3) = [0 2;4 -3];

>> A

A =

Compressed Column Sparse

   (rows = 4, cols = 3,

    nnz = 5 [42%])

  (2, 1) ->  8

  (1, 2) ->  1

  (3, 2) ->  4

  (2, 3) ->  2

  (3, 3) -> -3

import scipy.sparse as sp

A = sp.lil_matrix( (4,3) )

A[0,1] = 1

A[1,0] = 8

A[1:3,1:3] = np.array([[0, 2],

                       [4,-3]])

In : print(A)

  (0, 1)        1.0

  (1, 0)        8.0

  (1, 2)        2.0

  (2, 1)        4.0

  (2, 2)       -3.0

The dictionary of keys (DOK) constructor is functionally identical to the LIL constructor. The only difference in the following code is calling dok_matrix() instead of lil_matrix():

MATLAB:

Python:

.

>> A = sparse(4,3)

>> A(1,2) = 1;

>> A(2,1) = 8;

>> A(2:3,2:3) = [0 2;4 -3];

>> A

A =

Compressed Column Sparse

   (rows = 4, cols = 3,

    nnz = 5 [42%])

  (2, 1) ->  8

  (1, 2) ->  1

  (3, 2) ->  4

  (2, 3) ->  2

  (3, 3) -> -3

import scipy.sparse as sp

A = sp.dok_matrix( (4,3) )

A[0,1] = 1

A[1,0] = 8

A[1:3,1:3] = np.array([[0, 2],

                       [4,-3]])

In : print(A)

  (0, 1)        1.0

  (1, 0)        8.0

  (1, 2)        2.0

  (2, 1)        4.0

  (2, 2)        -3.0

In addition to direct insertion of terms, MATLAB and SciPy LIL and DOK sparse matrices allow summation of blocks with updates to the sparse structure where necessary:

MATLAB:

Python:

>> A(2:3,2:3) = ...

   A(2:3,2:3) + [0 2;4 -3];

A = sp.dok_matrix( (4,3) )

A[1:3,1:3] += np.array([[0, 2],

                        [4,-3]])

11.3.3 Sparse Matrix Creation with BSR, DIA

The two remaining sparse formats, block compressed row (BSR) and diagonal (DIA), are more specialized than the previous five and are therefore seen less frequently.

As its name suggests , the block compressed row format stores a collection of dense submatrices. It has drawbacks though: submatrices must be equally sized, and direct indexing is not supported. The BSR constructor function bsr_matrix() takes as input a 3D array of data blocks, and 1D arrays containing the block column indices, and pointers into the column indices for each row. This example creates an 8 x 6 sparse matrix from four 2 x 2 matrices—the sparse matrix then has four block rows and three block columns:

Python:
In : V = np.array([[[1,1],[1,1]],
...:               [[2,2],[2,2]],
...:               [[3,3],[3,3]],
...:               [[4,4],[4,4]]])
In : J = np.array([1, 2, 0, 2])  # column indices
In : indptr = np.array([0, 1, 2, 2, 4])
In : A = sp.bsr_matrix((V, J, indptr), shape=(8, 6))
In : A.todense()
matrix([[0, 0, 1, 1, 0, 0],
        [0, 0, 1, 1, 0, 0],
        [0, 0, 0, 0, 2, 2],
        [0, 0, 0, 0, 2, 2],
        [0, 0, 0, 0, 0, 0],
        [0, 0, 0, 0, 0, 0],
        [3, 3, 0, 0, 4, 4],
        [3, 3, 0, 0, 4, 4]])

The indptr array is computed from block row indices (which would be I = [0,1,3,3] in this example) the same way indptr are created for csr_matrix() (refer to Section 11.3.1).

The diagonal storage format, DIA, is well suited for computational fluid dynamics problems that employ iterative solutions on finite difference grids. The dia_matrix() constructor resembles MATLAB’s spdiags() function :

MATLAB:
>> data = [-2 -4 -6 -8 -10 -12 -14;
           10 11 12 13  14  15  16;
           -1 -3 -5 -7  -9 -11 -13]' ;
>> offsets = [2 0 -1];
>> A = spdiags(data, offsets, 7,7);
>> full(A)
   10    0   -6    0    0    0    0
   -1   11    0   -8    0    0    0
    0   -3   12    0  -10    0    0
    0    0   -5   13    0  -12    0
    0    0    0   -7   14    0  -14
    0    0    0    0   -9   15    0
    0    0    0    0    0  -11   16
Python:
In : data = np.array([[-2,-4,-6,-8,-10,-12,-14],
...:                  [10,11,12,13, 14, 15, 16],
...:                  [-1,-3,-5,-7, -9,-11,-13]])
In : offsets = np.array([2,0,-1])
In : A = sp.dia_matrix((data, offsets), shape=(7,7))
In : A.todense()
matrix([[ 10,   0,  -6,   0,   0,   0,   0],
        [ -1,  11,   0,  -8,   0,   0,   0],
        [  0,  -3,  12,   0, -10,   0,   0],
        [  0,   0,  -5,  13,   0, -12,   0],
        [  0,   0,   0,  -7,  14,   0, -14],
        [  0,   0,   0,   0,  -9,  15,   0],
        [  0,   0,   0,   0,   0, -11,  16]])

Note that in both MATLAB and Python, terms of data are dropped to accommodate the size reduction as one moves away from the diagonal. In our case, –2 and –4 do not appear above the diagonal, and –13 does not appear below the diagonal.

As with BSR, DIA sparse matrices cannot be subscripted.

11.3.4 Test Matrices

A few examples of dense test matrices were presented in Section 11.1.6.14; here, we do the same for sparse matrices. As with the dense random matrices, the MATLAB sparse random matrices will not match the Python sparse matrices.

11.3.4.1 Random Sparse 4 × 3 Arrays with Density of 40%

MATLAB:

Python:

>> A = sprand(4,3,.4);

>> full(A)

   0.000  0.000  0.000

   0.617  0.000  0.349

   0.000  0.000  0.000

   0.881  0.165  0.335

In : A = sp.rand(4, 3, density=0.4,

                 format='dok')

In : A.todense()

matrix([[0.688, 0.401, 0.   ],

        [0.   , 0.   , 0.   ],

        [0.337, 0.   , 0.   ],

        [0.   , 0.023, 0.   ]])

11.3.4.2 Random Sparse via I,J,V

Here, we’ll also use the spy() function in MATLAB and Python’s matplotlib to view the sparse matrix topology:

MATLAB:

Python:

nR = 50;

nC = 50;

nNZ = 10*nC;

I = ceil(nR*rand(nNZ,1));

J = ceil(nC*rand(nNZ,1));

V = ceil(nC*rand(nNZ,1));

A = sparse(I,J,V);

spy(A)

import scipy.sparse as sp

import numpy as np

import matplotlib.pyplot as plt

nR = 50

nC = 50

nNZ = 10*nC# number of non-zeros

I = np.random.randint(0, nR, size=(nNZ,))

J = np.random.randint(0, nC, size=(nNZ,))

V = np.random.randint(0, nC, size=(nNZ,))

A = sp.coo_matrix((V,(I,J)),shape=(nR,nC))

plt.spy(A)

plt.show()

11.3.4.3 Arrow Matrix

MATLAB:

Python:

>> A = speye(4);

>> A(1,:) = 1;

>> A(:,1) = 1;

>> full(A)

   1   1   1   1

   1   1   0   0

   1   0   1   0

   1   0   0   1

In : A = sp.identity(4).todok()

In : A[0,:] = 1

In : A[:,0] = 1

In : A.todense()

matrix([[1., 1., 1., 1.],

        [1., 1., 0., 0.],

        [1., 0., 1., 0.],

        [1., 0., 0., 1.]])

11.3.5 Sparse Matrix I/O

MATLAB’s save and load commands make it easy to write sparse matrices to .mat files and to read them back. Unlike their NumPy counterparts, SciPy sparse matrices lack built-in methods for file input and output. Ironically, SciPy’s savemat() and loadmat() functions support reading and writing sparse matrices to and from MATLAB .mat files.

11.3.5.1 Write to MATLAB .mat Files

MATLAB:

Python:

>> A = sprand(4,3,.4);

>> save A  % to A.mat

>> clear A

>> whos A

 Name Size Bytes Class  Attributes

 A    4x3    112 double sparse

>> clear A

>> load A  % from A.mat

>> whos A

 Name Size Bytes Class  Attributes

 A    4x3    112 double sparse

In : A = sp.rand(4, 3, 0.4)

In : A.todense()

matrix([[0.   , 0.   , 0.264],

        [0.   , 0.   , 0.   ],

        [0.334, 0.743, 0.   ],

        [0.059, 0.   , 0.   ]])

In : import scipy.io as io

In : io.savemat('A.mat',{'A':A})

In : A = None

In : data = io.loadmat('A.mat')

In : data['A'].todense()

matrix([[0.   , 0.   , 0.264],

        [0.   , 0.   , 0.   ],

        [0.334, 0.743, 0.   ],

        [0.059, 0.   , 0.   ]])

This method is also useful for passing sparse matrices between MATLAB and Python.

11.3.5.2 Write Component Parts As NumPy Arrays

The component parts of a sparse matrix—the data values and row and column indices—are NumPy arrays, so we can use NumPy’s savez() and loadz() functions to read and write them efficiently. The catch here is that the item names vary by storage format, so one must know the format of the sparse matrix (stored in its .format attribute) before accessing its components:

  • COO: A.data, A.row, A.col

  • CSR: A.data, A.indices, A.indptr

  • CSC: A.data, A.indices, A.indptr

  • LIL: A.data, A.rows (contains both row and column indices)

  • BSR: A.data, A.indices, A.indptr

  • DIA: A.data, A.offsets

DOK is unique in that its components are stored in a dictionary rather than NumPy arrays. Its entries can be retrieved with A.keys() and A.values(). DOK matrices can either be converted to other formats to save as NumPy arrays or written to pickle files as shown in the next section.

After loading the components, the matrix has to be regenerated by calling the appropriate constructor. The process looks like this for a COO matrix:

Python:
In : A = sp.rand(4, 3, 0.4, format='coo')
In : A.todense()
matrix([[0.44250527, 0.        , 0.        ],
        [0.61852124, 0.        , 0.49900225],
        [0.        , 0.        , 0.10741277],
        [0.        , 0.        , 0.        ]])
In : np.savez('A_coo.npz', V=A.data, I=A.row, J=A.col, dims=A.shape)
In : !ls -l A_coo.npz
-rw-rw-r-- 1 al al 788 May 29 13:36 A_coo.npz
In : A = None
In : fh = np.load('A_coo.npz')
In : V  = fh['V']
In : I  = fh['I']
In : J  = fh['J']
In : dm = fh['dims']
In : fh.close()
In : A = sp.coo_matrix((V, (I, J)), shape=dm)
In : A.todense()
matrix([[0.44250527, 0.        , 0.        ],
        [0.61852124, 0.        , 0.49900225],
        [0.        , 0.        , 0.10741277],
        [0.        , 0.        , 0.        ]])

11.3.5.3 Write a Python Pickle Object

SciPy sparse matrices are conventional Python variables and can therefore be written to pickle files (ref. Section 7.​13):

Python:
In : import pickle
In : import scipy.sparse as sp
In : A = sp.rand(4, 3, 0.4, format='csr')
In : A.todense()
Out: matrix([[0.   , 0.   , 0.899],
             [0.   , 0.054, 0.   ],
             [0.   , 0.   , 0.   ],
             [0.   , 0.625, 0.603]])
In : with open('A.pkl', 'wb') as f:
In :     pickle.dump(A, f, 4) # protocol v 4
In : !ls -l A.pkl
In : -rw-rw-r-- 1 al al 457 May 29 14:15 A.pkl
In : A = None
In : with open('A.pkl', 'rb') as f:
In :     A = pickle.load(f)
In : A.todense()
Out: matrix([[0.   , 0.   , 0.899],
             [0.   , 0.054, 0.   ],
             [0.   , 0.   , 0.   ],
             [0.   , 0.625, 0.603]])

11.3.6 Linear Algebra

SciPy’s sparse matrix linear algebra functions are in the scipy.sparse.linalg module which must be imported independently from scipy.sparse itself.

As a representative matrix, we’ll use a finite element–based symmetric sparse matrix, fidap037,7 from the National Institute of Standards and Technology’s Matrix Market.8 It can be loaded into Python with scipy.io’s mmread() function and into MATLAB with mmread.m.9 SciPy’s mmread() returns a sparse matrix in coordinate form (COO). If the matrix is to be factored, it will need to be converted to a form such as CSC.

The following code loads the file into the sparse variable K and defines a matrix b with an equal number of rows and two columns, the first of which has all +1 values, while the other column has –1.

MATLAB:

Python:

>> K = mmread('fidap037.mtx');

>> [nR,nC] = size(K);

>> b = ones(nR,2);

>> b(:,2) = -1;

from scipy.io import mmread

import scipy.sparse as sp

import scipy.sparse.linalg as spla

K = mmread('fidap037.mtx') # COO format

K = K.tocsc() # for factorization later

nR,nC = K.shape

b = np.ones((nR,2))

b[:,1] = -1

Matrix summary

The following commands print the dimensions and number of non-zero terms and show the sparsity pattern. Other matrix summary capabilities vary between the two systems. MATLAB has a function to estimate the condition number but not the norm, while SciPy supports the reverse. The Matrix Market values for these are 2.26e+02 and 1.5e+03.

MATLAB:

Python:

>> size(K)

     3565    3565

>> nnz(K)

    67591

>> spy(K)

>> condest(K)

  226.3745

In : K.shape

Out: (3565, 3565)

In : K.nnz

Out: 67591

In : import matplotlib.pyplot as plt

In : plt.spy(K,markersize=1)

In : plt.show()

In : spla.norm(K)

Out: 1472.080010077052

Figure 11-1

MATLAB

Figure 11-2

Python

Matrix-vector product

Unlike NumPy arrays which use @ for matrix multiplication, SciPy sparse matrices can use *. The .dot() method also works for sparse matrices. Only a portion of the 3565 rows of the product are shown:

MATLAB:

Python:

>> Kb = K*b;

>> Kb(191:195,:)

    1.0000   -1.0000

   13.4848  -13.4848

   52.3418  -52.3418

    1.0000   -1.0000

   27.2083  -27.2083

In : Kb = K*b

In : Kb[190:195,:]

Out:

array([[  1.        ,  -1.        ],

       [ 13.48478231, -13.48478231],

       [ 52.3417648 , -52.3417648 ],

       [  1.        ,  -1.        ],

       [ 27.20825099, -27.20825099]])

Solution to linear equations

MATLAB’s backslash operator works with both dense and sparse matrices. SciPy uses spsolve() to solve for x in Kx = b and splu() to factor K to lower and upper triangular matrices, stored together as LU, with subsequent solution with LU.solve(b):

MATLAB:

Python:

>> x = K;

>> x(501:505,:)

    1.0000   -1.0000

    0.0712   -0.0712

    0.0453   -0.0453

    1.0000   -1.0000

    0.0421   -0.0421

>> [L,U,P,Q] = lu(K);

>> x = Q*(U(L(P*b)));

>> x(501:505,:)

    1.0000   -1.0000

    0.0712   -0.0712

    0.0453   -0.0453

    1.0000   -1.0000

    0.0421   -0.0421

In : x = spla.spsolve(K,b)

In : x[500:505,:]

Out:

array([[ 1.        , -1.        ],

       [ 0.07122914, -0.07122914],

       [ 0.04526046, -0.04526046],

       [ 1.        , -1.        ],

       [ 0.04214178, -0.04214178]])

In : LU = spla.splu(K)

In : x = LU.solve(b)

In : x[500:505,:]

Out:

array([[ 1.        , -1.        ],

       [ 0.07122914, -0.07122914],

       [ 0.04526046, -0.04526046],

       [ 1.        , -1.        ],

       [ 0.04214178, -0.04214178]])

LU factorization performance is comparable between MATLAB and SciPy. Additionally, both have a collection of similarly named iterative solvers including conjugate gradient and minimum residual methods.

Solution to the standard eigenvalue problem

As indicated in Table 11-4, MATLAB’s eigs() function handles all forms of sparse eigensolutions with options controlled by various arguments. SciPy has two functions, eigs() and eigsh(), to handle asymmetric and symmetric inputs. Both support a shift-and-invert argument, sigma, to search for solutions near a desired eigenvalue region, and both support extracting a given number of eigenpairs.

The following example solves Kx = λx for the ten eigenvalues and eigenvectors of our symmetric matrix K nearest a shift of σ = -1. This problem is especially challenging because it has 1651 repeated eigenvalues of 1.0, found by computing the Sturm number (explained later). Shifting near –1 does not guarantee the returned eigenvalues are complete in either MATLAB or Python. Although MATLAB returned seven eigenvalues of 1.0 compared to Python’s six, ideally both should have returned ten values of 1.0.

MATLAB:

Python:

>> [vec,val] = eigs(K,10,-1);

> format long e

>> diag(val)

     9.999999999999947e-01

     9.999999999999947e-01

     9.999999999999956e-01

     9.999999999999969e-01

     9.999999999999978e-01

     1.000000000000000e+00

     1.000000000000000e+00

     2.319122761249388e+00

     2.390762542515986e+00

     2.500701541757246e+00

In : val,vec = spla.eigsh(K,k=10,sigma=-1)

In : val

Out:

array([1.        ,

       1.        ,

       1.        ,

       1.        ,

       1.        ,

       1.        ,

       2.31912276,

       2.39076254,

       2.50070154,

       2.63542116])

Sturm sequence number

Neither MATLAB nor Python sparse eigensolvers can tell if the requested number of solutions is comprehensive, that is, if n eigensolutions were requested near a shift of σ, are the returned n eigenvalues the closest to σ, or were some omitted? The Sturm sequence number for the general eigenvalue problem Kx = λMx, removes guesswork from this question. This number is the count of negative values on the diagonal of the LU factor of K - σM, and it equals the number of eigenvalues below σ.

The following code computes the Sturm sequence number for our K at σ1 = 0.99 and σ2 = 1.01. Since we’re solving the standard eigenvalue problem, Kx = λx, M in this case is simply the identity matrix:

MATLAB:

Python:

>> sigma = 0.99;

>> [L,U] = lu(K-sigma*eye(nR));

>> dU = diag(U);

>> sum(dU < 0)

           0

>> sigma = 1.01;

>> [L,U] = lu(K - sigma*eye(nR));

>> dU = diag(U);

>> sum(dU < 0)

        1651

In : sigma = 0.99

In : LU = spla.splu(K - sigma*sp.eye(nR))

In : dU = LU.U.diagonal()

In : np.sum(dU < 0)

Out:    0

In : sigma = 1.01

In : LU = spla.splu(K - sigma*sp.eye(nR))

In : dU = LU.U.diagonal()

In : np.sum(dU < 0)

Out: 1651

Therefore, we know Kx = λx has 1651 eigenvalues between 0.99 and 1.01.

Solution to the general eigenvalue problem

The general eigenvalue problem, Kx = λMx, arises in structural vibration and buckling problems, among others. MATLAB’s and SciPy’s sparse eigensolvers both accept a second matrix to solve such problems. We’ll manufacture a diagonal M matrix with values linearly spaced from 1 to 3565 to accompany our K.

Although SciPy’s eigenvalues are returned in sorted order, neither it nor MATLAB’s eigs() guarantees this order. MATLAB’s output is explicitly sorted here to simplify the comparison.

MATLAB:

Python:

>> M = spdiags([1:nR]', 0, nR, nR); sigma = 1.8;

>> [vec,val] = eigs(K,M,10,sigma);

>> sort(diag(val))

     1.097093867635186e+00

     1.129980804603547e+00

     1.362226236223728e+00

     1.593598298009953e+00

     1.743881417538397e+00

     1.837195925535993e+00

     2.042699252311738e+00

     2.171083950484060e+00

     2.236999096156634e+00

     2.411152544918247e+00

M = sp.diags(np.arange(1,nR+1))

sigma = 1.8

val,vec = spla.eigsh(K,M=M,

             k=10,sigma=sigma)

array([1.09709389,

       1.12998084,

       1.36222621,

       1.59359813,

       1.74388143,

       1.83719593,

       2.04269926,

       2.1710838 ,

       2.23699914,

       2.4111525 ])

11.3.7 Summary of Sparse Formats and Capabilities; Recommendations

Table 11-6 summarizes SciPy’s sparse matrix formats and capabilities.
Table 11-6

Comparison of SciPy sparse storage schemes

 

Creation

Subscriptable

Change Sparse Structure

Matrix-Vector Product

Linear Algebra

MATLAB

fast

yes

yes

fast

yes

COO

fast

no

no

medium

no

CSC/CSR

Indices

fast

yes

yes

fast

yes

CSC/CSR

Subscripting

very slow

yes

yes

fast

yes

LIL

slow

yes

fast

fast

yes

DOK

very slow

yes

yes

fast

yes

BSR

medium

no

no

fast

yes

DIA

very fast

no

no

fast

yes

Performance measurements suggest two of the seven formats, DOK and BSR, could be dropped from SciPy with no impact on capability. LIL does everything DOK can do, only faster, and BSR, despite being tailored for block matrices, is both slow and inflexible.

The most effective way to work with sparse matrices in Python is to create them as COO, CSC, or CSR matrices using index arrays. COO matrices can be created marginally more quickly than CSC or CSR, but COO matrices cannot be subscripted or used for linear algebra operations beyond matrix multiplication. Converting COO matrices to CSC or CSR can be done quickly though.

Subscripted growth, that is, creation with A[i,j] += v, should be avoided if performance is a concern. Conversion between formats is cheap, usually less than 1% of the effort it took to create the matrix, so if many indexed updates are necessary, it may pay off to convert a matrix from COO/CSR/CSC to LIL (which can do subscripted updates the fastest), make the subscripted update, then convert back to CSC or CSR.

One wonders if these format switches could be abstracted away to give a single, seamless sparse matrix experience—like MATLAB’s.

11.4 Interpolation

Numerical data must often be resampled at intervals different than originally recorded. This is especially true of observations taken from in situ measurements, test equipment, and so on.

While both NumPy and SciPy have interpolation functions, SciPy’s offer more capability.

11.4.1 One-Dimensional Interpolation

The following figure shows five points on the interval 0 to 1. We’ll demonstrate linear and spline interpolation methods to estimate values between known points. Extrapolation before the first and beyond the last points is best done with a curve fitting method such as linear regression (Section 11.5.1).

11.4.1.1 One-Dimensional Linear Interpolation

One dimensional linear interpolation can be done with NumPy’s np.interp(). It takes three input arguments: an array of x values at which we want the corresponding y values, then the set of known x and y:
In : x_known = np.array([0.194, 0.429, 0.512, 0.821, 0.917])
In : y_known = np.array([0.347, 0.543, 0.661, 0.811, 0.793])
In : x_desired = np.linspace(0, 1, num=21)
In : x_desired
array([0.  , 0.05, 0.1 , 0.15, 0.2 ,
       0.25, 0.3 , 0.35, 0.4 , 0.45,
       0.5 , 0.55, 0.6 , 0.65, 0.7 ,
       0.75, 0.8 , 0.85, 0.9 , 0.95, 1.])
In : y_desired = np.interp(x_desired, x_known, y_known)

The interpolated points (x_desired, y_desired) are shown as blue plus signs:

The extrapolated values are merely copies of the first and last known points. This can be changed to other values (such as np.NaN) with the optional keyword arguments left= and right= to np.interp().

11.4.1.2 One-Dimensional Spline Interpolation

Spline interpolation in one dimension requires the interp1d() function from the SciPy scipy.interpolate module . Instead of taking three arguments like np.interp(), interp1d() takes only the known x and y values as inputs, along with optional keywords kind which describes the interpolation method to use (options are listed in Table 11-7), bound_error, and fill_value. The bound_error and fill_value optional arguments are important because, unlike np.interp(), interp1d() will not extrapolate values before the first point or beyond the last point. By default, bound_error will be True which means an attempt to extrapolate will cause the code to raise a ValueError exception. We can avoid the exception by setting bound_error to False, but in this case we need to supply the value to use when x_desired is outside the range of x_known. That’s what fill_value is for. A commonly used value for the out-of-bounds result is np.NaN, which can be used to inform downstream routines that the interpolator failed at the corresponding x value. Naturally, those downstream routines must be prepared to deal with np.NaN values. Conveniently, points with np.NaN values do not appear in plots.
Table 11-7

Options for kind in scipy.interpolate.interp1d()

kind

Description

'linear'

Linear; same as np.interp()

'nearest'

Nearest neighbor

'zero'

Zeroth-order spline

'slinear'

First-order spline

'quadratic'

Second-order spline

'cubic'

Third-order spline

'previous'

Value of the previous point

'next'

Value of the next point

interp1d()’s return value is a new function which takes the array of desired x values as its sole argument. The return value from this function is the y array for the given x’s. The code below uses the same x_known, y_known, and x_desired as in Section 11.4.1.1.

In : from scipy.interpolate import interp1d
In : new_fn = interp1d(x_known, y_known,
                kind='cubic', bounds_error=False,
                fill_value=np.NaN)
In : y_desired = new_fn(x_desired)

which yields

11.4.2 Two-Dimensional Interpolation

Two-dimensional interpolation is a common task in the fields of geographic information systems (GIS), mapping, and remote sensing. As with 1D interpolation, 2D interpolation takes values at known locations and returns estimated values at neighboring locations, typically a regularly spaced grid. Values on a regular grid are well suited for visualization as heat maps, contour plots, or map overlays.

As a simple example, let’s look at the function f(x, y) = x+ 2(cos 7x+ sin 5y)+ 3y sin 6x on the square bounded by −0.6 < x < 1 and −1 < y < 0.4. A contour plot of f(x, y) looks like this:

To test our 2D interpolation methods, we’ll subsample f(x, y) at 114 points taken from a region of a spiral and defined by these x and y values (Boolean index masks such as in_range are explained in Section 11.1.14):
K = 0.005
t = np.linspace(0,800,num=1100)
Spiral = np.array([K*t*np.cos(t)+1,
                   K*t*np.sin(t)-1])
in_range = (-0.6 < Spiral[0,:]) * (Spiral[0,:] < 1.0) *
           (-1.0 < Spiral[1,:]) * (Spiral[1,:] < 0.4)
x = Spiral[0,in_range]
y = Spiral[1,in_range]

Both MATLAB and Python employ a function named griddata() to perform 2D interpolation of irregularly spaced data onto regular grids; Python’s exists in SciPy’s scipy.interpolate module. griddata()’s input arguments are the coordinates of the points at which we know data values, the data values themselves, then a pair of two-dimensional arrays holding the X and Y of coordinates at which we want the function to interpolate values. The X and Y arrays typically contain coordinates on a regularly spaced grid as returned by mgrid[] or np.meshgrid() (explained later); however, any arrangement of points may be employed. Points whose indices differ by one are assumed to be topologically adjacent for the interpolation computations.

We want results spaced over a regular grid, so we first have to create the coordinate values for that grid. This can be done two ways in Python. The first method uses NumPy’s meshgrid() function :

MATLAB:

Python:

>> nX = 3; nY = 4;

>> format short

>> [X, Y] = meshgrid(...

         linspace(-.6,  1,nX),...

         linspace(-1 ,-.4,nY))

X =

   -0.6000    0.2000    1.0000

   -0.6000    0.2000    1.0000

   -0.6000    0.2000    1.0000

   -0.6000    0.2000    1.0000

Y =

   -1.0000   -1.0000   -1.0000

   -0.8000   -0.8000   -0.8000

   -0.6000   -0.6000   -0.6000

   -0.4000   -0.4000   -0.4000

In : nX, nY = 3, 4

In : X, Y = np.meshgrid(

...:     np.linspace(-.6,  1,nX),

...:     np.linspace(-1 ,-.4,nY))

In : X

array([[-0.6,  0.2,  1. ],

       [-0.6,  0.2,  1. ],

       [-0.6,  0.2,  1. ],

       [-0.6,  0.2,  1. ]])

In : Y

array([[-1. , -1. , -1. ],

       [-0.8, -0.8, -0.8],

       [-0.6, -0.6, -0.6],

       [-0.4, -0.4, -0.4]])

The second method uses NumPy’s mgrid object (not a function!) indexed by ranges with imaginary step sizes. Ordinary list and NumPy index ranges permit only integer values, but mgrid index ranges may have floating-point start, end, and increment values. Another quirk is that the x and y positions are reversed compared to np.meshgrid() :

MATLAB:

Python:

matlab code

>> nX = 3; nY = 4;

>> format short

>> [X, Y] = meshgrid(...

      linspace(-.6,  1,nX),...

      linspace(-1 ,-.4,nY))

X =

   -0.6000    0.2000    1.0000

   -0.6000    0.2000    1.0000

   -0.6000    0.2000    1.0000

   -0.6000    0.2000    1.0000

Y =

   -1.0000   -1.0000   -1.0000

   -0.8000   -0.8000   -0.8000

   -0.6000   -0.6000   -0.6000

   -0.4000   -0.4000   -0.4000

In : nX, nY = 3, 4

In : Y, X = np.mgrid[-1.0:-0.4:nY*1j,

...:                 -0.6: 1.0:nX*1j]

In : X

array([[-0.6,  0.2,  1. ],

       [-0.6,  0.2,  1. ],

       [-0.6,  0.2,  1. ],

       [-0.6,  0.2,  1. ]])

In : Y

array([[-1. , -1. , -1. ],

       [-0.8, -0.8, -0.8],

       [-0.6, -0.6, -0.6],

       [-0.4, -0.4, -0.4]])

We now have all inputs needed for the 2D interpolation: coordinates of 114 locations where we know the function values and coordinates of a regular grid where we want interpolated values. By default, griddata() performs a linear interpolation but can also do cubic and nearest-neighbor interpolation. We’ll evaluate all three on a 200 × 200 grid:

MATLAB:

Python:

% code/griddata/spiral_interp.m

K = 0.005;

% subsample f(x,y) at 114 points

t = linspace(0,800,1100);

X = [K*t.*cos(t)+1; K*t.*sin(t)-1];

in_range = (-0.6 < X(1,:)) & ...

            (X(1,:) < 1.0) & ...

           (-1.0 < X(2,:)) & ...

            (X(2,:) < 0.4);

x_known = X(1,in_range);

y_known = X(2,in_range);

F_known = f(x_known, y_known);

% make a 200 x 200 regular grid

nX = 200; nY = 200;

[X, Y] = meshgrid(...

    linspace(-.6,  1,nX),...

    linspace(-1 ,-.4,nY));

% interpolate  to the regular grid

F_near   = griddata(...

        x_known, y_known, ...

        F_known, X,Y, 'nearest');

F_linear = griddata(...

        x_known, y_known,...

        F_known, X,Y, 'linear');

F_cubic  = griddata(...

        x_known, y_known,...

        F_known, X,Y, 'cubic');

function [S] = f(x,y)

   S = x + 2*(cos(7*x) + ...

       sin(5*y)) + 3*y.*sin(6*x);

end

# code/griddata/spiral_interp.py

import numpy as np

from scipy.interpolate import griddata

def f(x,y):

   return x + 2*(np.cos(7*x)+

      np.sin(5*y)) + 3*y*np.sin(6*x)

K = 0.005

# subsample f(x,y) at 114 points

t = np.linspace(0,800,num=1100)

X = np.array([K*t*np.cos(t)+1,

              K*t*np.sin(t)-1])

in_range = (-0.6 < X[0,:]) *

            (X[0,:] < 1.0) *

           (-1.0 < X[1,:]) *

            (X[1,:] < 0.4)

x_known = X[0,in_range]

y_known = X[1,in_range]

F_known = f(x_known, y_known)

# make a 200 x 200 regular grid

nX, nY = 200, 200

Y, X = np.mgrid[-1.0:-0.4:nY*1j,

                -0.6: 1.0:nX*1j]

# interpolate to the regular grid

F_near   = griddata(

     (x_known, y_known),

     F_known, (X,Y), method="nearest")

F_linear = griddata(

     (x_known, y_known),

     F_known, (X,Y),method='linear')

F_cubic  = griddata(

     (x_known, y_known),

     F_known, (X,Y),method='cubic')

Which interpolation method is best? As with many problems, the answer is “it depends.” The cubic method most faithfully captures the gradients of the original function, especially where there is a concentration of known data points. However, both the cubic and linear methods fail to capture the local peak at the top edge.

This failure is caused by griddata()’s linear and cubic methods’ use of a Delaunay triangulation over the known data points. The triangulation fills a convex hull around the known points with elements, and any desired point inside the convex hull will receive an interpolated value from its bounding element. Points along the top and left edges are joined by thin slivers of elements that unhelpfully associate remotely spaced points.

11.4.3 Two-Dimensional Interpolation on a Grid

A simplified, but still common, form of two-dimensional interpolation takes evenly spaced values on a coarse grid and interpolates the values to a finer grid. In image processing, this interpolation is referred to as image zoom since the finer grid has many more data points (or pixels) than the original grid.

Both interp2 in MATLAB and griddata in Python, discussed in the previous section, can be used for this task. Both of those, however, require additional arguments that define the interpolation point positions. A simpler method is available in Python’s scipy.ndimage module. Its zoom() function needs only two arguments: the input array and the amount to grow (or reduce it by). By default, zoom() performs a cubic spline interpolation, but the spline order can be set between 0 and 5. The following example interpolates values of a 3 x 5 matrix to fill a 10 x 12 matrix.

The second argument describes the amount by which each dimension should grow. We want 10 rows instead of 3 and 12 columns instead of 5, so the scaling argument is [10/3, 12/5]:

Python:
from scipy.ndimage import zoom
In : a = np.array([[0, 0, 0, 5, 6],
                   [0, 0, 0, 5, 6],
                   [4, 4, 5, 6, 8]])
In : b = zoom(a, [10/3, 12/5])
In : b.shape
Out: (10, 12)
In : b
array([[ 0,  0,  0,  0,  0,  0,  1,  3,  5,  6,  6,  6],
       [ 0,  0,  0,  0, -1, -1,  1,  3,  5,  6,  6,  6],
       [ 0,  0,  0,  0, -1, -1,  0,  2,  5,  6,  6,  6],
       [ 0,  0,  0, -1, -1, -1,  0,  2,  4,  6,  6,  6],
       [ 0,  0,  0,  0, -1, -1,  0,  2,  5,  6,  6,  6],
       [ 0,  0,  0,  0,  0,  0,  1,  3,  5,  6,  6,  6],
       [ 1,  2,  2,  1,  1,  2,  2,  4,  5,  6,  7,  7],
       [ 3,  3,  3,  3,  3,  3,  4,  4,  5,  6,  7,  7],
       [ 4,  4,  4,  4,  4,  4,  5,  5,  6,  7,  7,  8],
       [ 4,  4,  4,  4,  4,  5,  5,  5,  6,  7,  8,  8]])

If the scaling argument were a single value, both dimensions would be increased by the same factor.

The negative values are an artifact of the cubic spline that fits the step function across the 0/5 boundary. A bilinear interpolation can be done by setting (order=1):

Python:
from scipy.ndimage import zoom
In : a = np.array([[0, 0, 0, 5, 6],
                   [0, 0, 0, 5, 6],
                   [4, 4, 5, 6, 8]])
In : b = zoom(a, [10/3, 12/5], order=1)
In : b
array([[0, 0, 0, 0, 0, 0, 1, 3, 5, 5, 6, 6],
       [0, 0, 0, 0, 0, 0, 1, 3, 5, 5, 6, 6],
       [0, 0, 0, 0, 0, 0, 1, 3, 5, 5, 6, 6],
       [0, 0, 0, 0, 0, 0, 1, 3, 5, 5, 6, 6],
       [0, 0, 0, 0, 0, 0, 1, 3, 5, 5, 6, 6],
       [0, 0, 0, 0, 0, 1, 1, 3, 5, 5, 6, 6],
       [1, 1, 1, 1, 1, 2, 2, 4, 5, 6, 6, 7],
       [2, 2, 2, 2, 2, 3, 3, 4, 5, 6, 7, 7],
       [3, 3, 3, 3, 3, 4, 4, 5, 6, 6, 7, 8],
       [4, 4, 4, 4, 4, 5, 5, 6, 6, 7, 7, 8]])

11.5 Curve Fitting

11.5.1 Linear Regression

Linear regression finds the slope and vertical intercept of a line which best fits a collection of points. The “best fit” is defined as the line which minimizes the sum of the square of errors—the distances the points are away from the line. Motivations for solving linear regression problems generally fall into two categories: the need for just the slope and intercept of the best-fit line and the need for a comprehensive solution that returns error residuals, skew, kurtosis, and numerous other characteristic values.

In this section, we’ll concern ourselves with the simple version of the problem. The comprehensive solution, including weighted least squares, appears in Section 11.7.

Both MATLAB and Python can solve the linear regression problem with their implementations of polyfit(), a general polynomial fit solver, by setting the degree of the polynomial to one. While the functions return the same solution, the Python function returns the vertical intercept and slope, while the MATLAB function returns them in the opposite order:

MATLAB:

Python:

>> Pts = [7.312 15.878;

          7.657 16.308;

          7.934 16.690;

          7.962 16.902;

          8.614 17.013;

          8.623 17.766];

>> polyfit(Pts(:,1),Pts(:,2),1)

   1.1346   7.6638

from numpy.polynomial.polynomial

    import polyfit

In : Pts = np.array([

            [ 7.312, 15.878],

            [ 7.657, 16.308],

            [ 7.934, 16.690],

            [ 7.962, 16.902],

            [ 8.614, 17.013],

            [ 8.623, 17.766]])

In : b, m = polyfit(Pts[:,0],Pts[:,1],1)

In : m, b

Out: (1.134557, 7.663752)

11.5.2 Fitting Higher-Order Polynomials

The polyfit() function shown in the previous section can of course be used to find coefficients of best-fit polynomials with a degree greater than one. Here, we compute the coefficients of a cubic which best fits another collection of points. Note that in this case the points are not sorted in ascending value of X coordinate—point order is irrelevant.

The solution has four values, a3, a2, a1, and a0, which are the coefficients to the cubic
$$ {a}_3{x}^3+{a}_2{x}^2+{a}_1x+{a}_0 $$
that best fits the points. Aside from returning the coefficients in opposite order (MATLAB returns a0, a1, a2, and a3, while Python returns a3, a2, a1, and a0), the values match.

MATLAB:

Python:

Pts = [-0.938  16.875;

        0.326  21.290;

        1.787  22.317;

        2.968  28.767;

        4.038  10.210;

        5.358 -53.774];

>> polyfit(Pts(:,1),Pts(:,2),3)

 -1.356 4.306 2.967 15.960

from numpy.polynomial.polynomial

    import polyfit

In : Pts = np.array([

       [ -0.938,  16.875],

       [  0.326,  21.290],

       [  1.787,  22.317],

       [  2.968,  28.767],

       [  4.038,  10.210],

       [  5.358, -53.774]])

In : polyfit(Pts[:,0],Pts[:,1],3)

Out: array([15.960, 2.967,

            4.306, -1.356])

11.5.3 Fitting to Models

Few real-world datasets fit linear or polynomial models, at least not for wide ranges. More common distributions, at least among physical phenomena, are Gaussian, sinusoidal, and exponential or logarithmic decay or growth—or combinations of these.

The scipy.optimize module has a number of functions that can be used to fit models to data. Here, we’ll examine two of them, curve_fit() and differential_evolution(). MATLAB has a powerful Curve Fitting Toolbox, but for simpler curves the generic optimization function fminsearch() is often effective.

scipy.optimize.curve_fit() aims to find parameters of a continuously smooth model (e.g., the equation of a curve) that best fit a dataset. It works well when the model is monotonic, for example, the logarithmic growth seen in Figure 11-3.
Figure 11-3

Logarithmic growth data

The scipy.optimize.curve_fit() function and MATLAB’s fminsearch() fare poorly on more varying data such as sinusoidally decaying data from an underdamped system such as the point distribution shown in Figure 11-4.
Figure 11-4

Sinusoidally decaying data

For such data we’ll need a hardier method. One such option is the differential evolution algorithm [3], implemented in scipy.optimize.differential_evolution(). It is less sensitive to local minima in the objective function and often finds excellent solutions over wide input domains. A MATLAB implementation of this algorithm is available on the FileExchange.10

11.5.3.1 Case 1: Logarithmic Growth

The points of Figure 11-3 were created with the function

f (t) = A ln Bt + C

using A = 5, B = 14, and C = -13.7 and then adding noise in the form of random values:

MATLAB:

Python:

% code/optim/log_test_data.py

A = 5; B = 14; C = -13.7;

Fn = @(t,A,B,C)(A*log(B*t) + C);

rand('seed',234567);

n_pts = 1000;

t = 2 + linspace(.2,30,n_pts)+...

        .3*rand(1,n_pts);

y_true = Fn(t, A, B, C);

y_meas = y_true + 0.2*A*(0.5 -...

        rand(1,n_pts));

t_meas = t  + 0.1*A*(0.5 -...

        rand(1,n_pts));

#!/usr/bin/env python3

# code/optim/log_test_data.py

import numpy as np

A, B, C = 5, 14, -13.7

def Fn(t, A, B, C):

    return A*np.log(B*t) + C

np.random.seed(234567)

n_pts = 1000

t = 2 + np.linspace(.2,30,n_pts)+

        .3*np.random.rand(n_pts)

y_true = Fn(t, A, B, C)

y_meas = y_true + 0.2*A*(0.5 -

        np.random.rand(n_pts))

t_meas = t  + 0.1*A*(0.5 -

        np.random.rand(n_pts))

Our task is to determine values for A, B, and C using only the noisy data points and the model equation, f(t) = A ln Bt + C. Before we begin we'll need a way to assess how well a given set of  A, B, and C values fit our model. A reasonable choice is the R2 coefficient of determination.

11.5.3.2 R2 to Score the Solution

A plot overlaying the noisy data on a curve of the model equation with the solution parameters will give us a subjective idea on how good the solution is. While the plot is useful, a numeric estimate of the “goodness of fit” is also valuable. On the Python side, we’ll use the r2_score() function from the sklearn.metrics (machine learning) module. It returns the R2 coefficient of determination between known and estimated values; 1.0 represents excellent agreement, and 0.0 (or worse, a negative score) means terrible agreement. MATLAB's Statistics and Machine Learning Toolbox has functions to compute R2. Implementations can also be found on the FileExchange.11

11.5.3.3 Fitting Logarithmic Growth with curve_fit()

The curve_fit() function has only three required arguments: the model function (Fn()), the independent variable (t_meas), and the dependent variable (y_meas). It returns two variables, popt, the optimal values of the parameters, and pcov, the estimated covariance of the optimal values.

Formulating the MATLAB solution takes a bit more work. In addition to the model function, we also have to define an error function in the form of a sum of squares of differences function which itself is invoked from another anonymous function. Adding to the difficulty is the need to estimate initial values for A, B, and C.

MATLAB:

Python:

% code/optim/log_test_data_full.py

A = 5; B = 14; C = -13.7;

Fn = @(t,A,B,C)(A*log(B*t) + C);

rand('seed',234567);

n_pts = 1000;

t = 2 + linspace(.2,30,n_pts)+...

        .3*rand(1,n_pts);

y_true = Fn(t, A, B, C);

y_meas = y_true + 0.2*A*(0.5 -...

        rand(1,n_pts));

t_meas = t  + 0.1*A*(0.5 -...

        rand(1,n_pts));

opt_Fn = @(x)sum_sq(x,t_meas,y_meas);

popt = fminsearch(opt_Fn,[1 1 1])

function ss = sum_sq(x,t,ydata)

  A = x(1); B = x(2); C = x(3);

  ss = sum((ydata - A*log(B*t) + C).^2);

end

In : run log_test_data.py

In : from scipy.optimize import curve_fit

In : from sklearn.metrics import r2_score

In : popt, pcov = curve_fit(Fn,t_meas,y_meas)

In : popt

Out: array([  5.00663437,

             12.05087493,

            -12.99449958])

In : r2_score(y_meas, y_curve_fit)

Out: 0.9920846142243493

MATLAB’s results aren’t great. They are also sensitive to the initial guess:

MATLAB:
>> log_test_data_full
>> popt = fminsearch(opt_Fn,[1 1 1])
popt =
    4.9880    0.5889   -2.1631
>> popt = fminsearch(opt_Fn,[1 10 -10])
popt =
    4.9881   11.9819   12.8655

The Python R2 score of 0.99 is excellent. Our estimates of A = 5.007, B = 12.05, and C = –12.99 are reasonably close to the true values of 5, 14, and –13.7; more impressively, the resulting curve matches the true curve so closely as to be indistinguishable from it: 

11.5.3.4 Case 2: Sinusoidal Decay

More complex models require increasingly more fiddling with curve_fit()’s options such as parameter bounds, initial values, uncertainty estimates, and Jacobian matrix. Instead we’ll switch to the more robust (but more computationally expensive) differential_evolution() function.

We’ll apply this model of an underdamped system:

f(t) = Ae−Bt sin Ct + D  (11.1)

to estimate values of A, B, C, and D that best fit the points shown in Figure 11-4. The points in that figure are made with the following code. It begins with known values for A…D and then adds noise in the form of random values to the model function.

Python:
# code/optim/sin_decay_test_data.py
import numpy as np
A, B, C, D = 1, .2, 1.4, 10
def Fn(t, A, B, C, D):
return A*np.exp(-B*t)*
         np.sin(C*t) + D
np.random.seed(234567)
n_pts = 1000
t = np.linspace(1,20,n_pts) + .3*np.random.rand(n_pts)
y_true = Fn(t, A, B, C, D)
y_meas = y_true + .11*(0.5 - np.random.rand(n_pts))

11.5.3.5 Fitting Sinusoidal Decay with curve_fit()

Sending our noisy data and model function to curve_fit() leads to an unsatisfactory result.

Python:
In : from scipy.optimize import curve_fit
In : from sklearn.metrics import r2_score
run sin_decay_test_data.py # set t, y_meas, y_true
In : popt, pcov = curve_fit(Fn,t,y_meas)
In : y_curve_fit = Fn(t, *popt)
In : popt
Out: [99.999 3.843 5.886 9.995]
In : r2_score(y_meas, y_curve_fit)
Out: 0.033069669065311835

The R2 score of 0.03 means our solution is worthless. The green curve of Figure 11-5 in the next section shows exactly how bad it is.

11.5.3.6 Fitting Sinusoidal Decay with differential_evolution()

scipy.optimize.differential_evolution(), unlike curve_fit(), requires upper and lower bounds on the parameters. We’ll give these wide ranges to illustrate the method’s surprising robustness:
$$ {displaystyle egin{array}{l}0.01&lt;=A&lt;=100\ {}-10&lt;=B&lt;=10\ {}0.01&lt;=C&lt;=10\ {}-100&lt;=D&lt;=100end{array}} $$
Python:
In : from scipy.optimize import differential_evolution
In : from sklearn.metrics import r2_score
In : def func(parameters, *data):
...:     A, B, C, D = parameters
...:     Fn, t, Y = data
...:     return np.linalg.norm(Fn(t,A,B,C,D)-Y)
In : bounds=([0.01, 100],
...:         [ -10,  10],
...:         [0.01,  10],
...:         [-100, 100])
run sin_decay_test_data.py # set t, y_meas, y_true
In : args = (Fn, t, y_meas)
In : DE_result = differential_evolution(func, bounds, args=args)
In : DE_result
     fun: 1.0028485230670718
     jac: array([ 1.15463195e-06, -2.70894418e-06,  9.99200722e-07, -3.79696274e-06])
 message: 'Optimization terminated successfully.'
    nfev: 3430
     nit: 55
 success: True
       x: array([1.00574221, 0.1992207 , 1.40038483, 9.99905623])
In : y_DE_fit = Fn(t, *DE_result.x)
In : r2_score(y_true, y_DE_fit)
Out: 0.9998993756587
This time the R2 score is just about 1.0, indicating an excellent fit. Figure 11-5 shows the curve_fit() and differential_evolution() solutions to this problem.
Figure 11-5

Sinusoidal decay modeled with curve fit and differential evolution

Rather than attempting to solve this more challenging problem with MATLAB’s fminsearch(), the recipe in Section 11.6 shows how to solve the curve fit in MATLAB using SciPy’s differential_evolution(). Of course, if the FileExchange implementation of differential_evolution() is available to you, it will be a more convenient alternative than using Python. It goes without saying that the Curve Fitting Toolbox is better still.

11.6 Recipe 11-1: Curve Fitting with differential_evolution()

The Curve Fitting Toolbox is the package of choice for determining equations of best-fit curves to data. In its absence, differential_evolution() from SciPy’s optimize module is a more powerful alternative than fminsearch from the core MATLAB product. Here, we demonstrate using the Python implementation of differential_evolution() with MATLAB to find the coefficients A, B, C, and D of the decaying sinusoid function, as was done using only Python in Section 11.5.3.6.

This recipe reveals a limitation of MATLAB’s py module : MATLAB functions passed as arguments to Python functions cannot be called from Python. This prevents us from calling optimizers such as differential_evolution() directly from MATLAB since their first argument is the function to be minimized or maximized.

There are at least two possible solutions: (1) write the cost function in Python and perform the optimization entirely in a Python bridge module or (2) call the MATLAB cost function from Python through the matlab.engine module (Section 6.​10). The second method is much more powerful since it should be able to work with any MATLAB function. In practice, though, Python calls to matlab.engine objects frequently fail with the MATLAB error “failed to connect to matlab.engine: Unable to connect to MATLAB session.” For this reason, only the first method is demonstrated.

When evaluating the cost function in Python, the MATLAB portion merely passes data points to a Python bridge module which contains both the cost function and the call to the optimizer. A function in the bridge module then returns the best-fit coefficients to MATLAB.

Python:
# code/optim/bridge_de.py
import numpy as np
from scipy.optimize import differential_evolution
def Fn(t, A, B, C, D):
    return A*np.exp(-B*t)*np.sin(C*t) + D
def func(parameters, *data):
    A, B, C, D = parameters
    Fn, t, Y = data
    return np.linalg.norm(Fn(t,A,B,C,D)-Y)
def compute_ABCD(bounds, t, y_meas):
    args = (Fn, t, y_meas)
    DE_result = differential_evolution(func, bounds, args=args)
    soln = { 'success' : DE_result.success,
             'message' : DE_result.message,
             'x' : DE_result.x,
           }
    if DE_result.success:
        y_DE_fit = Fn(t, *DE_result.x)
    else:
        y_DE_fit = []
    soln['y_fit'] = y_DE_fit
    return soln

The following MATLAB driver sends parameter bounds and test data to the Python compute_ABCD() function which returns a dictionary containing the solution—if one is found:

MATLAB:
% code/optim/fit_decaying_sine.m
Im = @py.importlib.import_module;
optim = Im('bridge_de');
np = Im('numpy');
sk = Im('sklearn.metrics');
A = 1; B = .2; C = 1.4; D = 10;
rng(234567);
n_pts = 1000;
t = linspace(1,20,n_pts) + .3*rand(1,n_pts);
y_true = Fn(t, A, B, C, D);
y_meas = y_true + .11*(0.5 - rand(1,n_pts));
bounds = py.tuple({[0.01, 100], ...
                   [ -10,  10], ...
                   [0.01,  10], ...
                   [-100, 100]});
solution = optim.compute_ABCD(bounds, np.array(t), np.array(y_meas));
if solution.get('success')
   R2 = sk.r2_score(np.array(y_true), np.array(solution.get('y_fit')));
   ABCD = py2mat(solution.get('x'));
   fprintf('A=%.6f, B=%.6f, C=%.6f, D=%.6f ', ABCD)
   fprintf('score = %f ', R2)
else
   fprintf('Failed: %s ', solution.get('message'))
end
function [x] = Fn(t, A, B, C, D)
    x = A*exp(-B*t).*sin(C*t) + D;
end

The solution from MATLAB matches the Python solution of Section 11.5.3.6:

MATLAB:
>> fit_decaying_sine
A=1.005742, B=0.199221, C=1.400385, D=9.999056
score = 0.999899

11.7 Regression

11.7.1 Ordinary Least Squares

The Python module statsmodels has its own statistics-flavored linear regression, or least squares, solvers that return richer results than NumPy’s polyfit() function described in Section 11.5.1. The primary functions are OLS(), for ordinary least squares, and WLS(), for weighted least squares.

The following examples generate points (x, y) that roughly approximate the line y = mx + b for a given slope, m, and vertical intercept, b. We’ll call OLS() and WLS() to see how well we can recover m and b.

MATLAB:

Python:

% create random X,Y

m = 3.77;

b = -5.5;

nPts = 20;

X = -5 + 12*rand(nPts,1);

X = sort(X);

noise = 8*(rand(nPts,1)-.5);

Y = m*X + b + noise;

% recover m and b

mb = polyfit(X,Y,1);

m_ls = mb(1);

b_ls = mb(2);

fprintf("ML : %f %f ", ...

        m_ls,b_ls)

Y_pred = m_ls*X + b_ls;

plot(X,Y,'g.')

hold on

plot(X,Y_pred,'r-')

grid

---------- Output:

ML : 3.558505 -4.843925

import numpy as np

import statsmodels.api as stats

import matplotlib.pyplot as plt

# create random X,Y

m = 3.77 # true slope

b = -5.5 # true intercept

nPts = 20

X = -5 + 12*np.random.rand(nPts)

X.sort()

noise = 8*(np.random.rand(nPts)-.5)

Y = m*X + b + noise

% recover m and b

Xb = stats.add_constant(X,0)

ols = stats.OLS(Y, Xb).fit()

m_ls, b_ls = ols.params

print(f'OLS {m_ls} {b_ls}')

Y_pred = ols.predict(Xb)

plt.plot(X,Y,'g.')

plt.plot(X,Y_pred,'r-')

plt.grid(True)

plt.show()

---------- Output:

OLS 3.86476 -6.23757

The MATLAB and Python implementations will see different point pairs, so we do not expect their respective solutions for m and b to match for small sample sizes. If we were to use 1000 points instead of 20, both Python and MATLAB will return values within 1% of the true values.

The ols object from the preceding Python example has numerous methods and properties. The following text is the output of print(ols.summary()):
                            OLS Regression Results
===========================================================================
Dep. Variable:                   y   R-squared:                       0.973
Model:                         OLS   Adj. R-squared:                  0.972
Method:              Least Squares   F-statistic:                     653.9
Date:             Fri, 26 Jun 2020   Prob (F-statistic):           1.33e-15
Time:                     15:05:59   Log-Likelihood:                -42.682
No. Observations:               20   AIC:                             89.36
Df Residuals:                   18   BIC:                             91.36
Df Model:                        1
Covariance Type:         nonrobust
===========================================================================
                 coef    std err         t     P>|t|     [0.025     0.975]
---------------------------------------------------------------------------
x1             3.8648      0.151    25.571     0.000      3.547     4.182
const         -6.2376      0.511   -12.214     0.000     -7.310    -5.165
===========================================================================
Omnibus:                        0.521   Durbin-Watson:                2.171
Prob(Omnibus):                  0.771   Jarque-Bera (JB):             0.578
Skew:                           0.055   Prob(JB):                     0.749
Kurtosis:                       2.174   Cond. No.                     3.62
===========================================================================

11.7.2 Weighted Least Squares

Data points may have disproportionate importance—imagine a measurement campaign performed with different models of test equipment. Values observed with more accurate equipment should be accorded greater significance when aggregate properties of all points are computed.

MATLAB can perform weighted least squares fits with the fittype() and fit() functions from the Curve Fitting Toolbox or the nlinfit() function from the Statistics and Machine Learning Toolbox. Python’s statsmodule solves such problems with WLS(), which takes the same inputs as OLS() shown in the previous section plus additional argument for weights.

The following example repeats the ordinary least squares example with weights assigned to the points linearly from left to right. Point size represents weight in the following plot:

Python:
#!/usr/bin/env python3
# code/statistics/wls.py
import numpy as np
import statsmodels.api as stats
import matplotlib.pyplot as plt
m = 3.77 # true slope
b = -5.5 # true intercept
nPts = 20
X = -5 + 12*np.random.rand(nPts)
X.sort()
W = X - np.min(X) + 1
noise = 8*(np.random.rand(nPts)-.5)
Y = m*X + b + noise
Xb = stats.add_constant(X,0)
wls = stats.WLS(Y, Xb, weight=W).fit()
m_ls, b_ls = wls.params
print(f'WLS {m_ls} {b_ls}')
Y_pred = wls.predict(Xb)
plt.scatter(X,Y, marker='o',s=5*W)
plt.plot(X,Y_pred,'r-')
plt.grid(True)
plt.title('Weighted Least Squares')
plt.show()
which produces
WLS 3.774306 -4.79019

Solve the weighted least squares problem in MATLAB with the recipe in Section 11.8.

11.7.3 Confidence and Prediction Intervals

The coefCI() function from MATLAB’s Statistics and Machine Learning Toolbox returns confidence intervals for coefficients of linear regression models . Plotting these takes a bit of extra effort though. In the absence of this toolbox, one can make confidence interval plots quite easily with the regplot() function from the Seaborn Python module.

This code repeats a portion of the ordinary least squares problem, increases the noise, and adds more points:

Python:
#!/usr/bin/env python3
# code/statistics/conf_interval.py
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
m = 3.77 # true slope
b = -5.5 # true intercept
nPts = 80
X = -5 + 12*np.random.rand(nPts)
X.sort()
noise = 30*(np.random.rand(nPts) - 0.5)
Y = m*X + b + noise
sns.regplot(x=X, y=Y, marker='.')
plt.xlabel('X')
plt.ylabel('Y')
plt.grid()
plt.title('Confidence interval with seaborn.regplot()')
plt.savefig('conf_interval.png', bbox_inches='tight',
            pad_inches=0.1, transparent=True)
plt.show()

Prediction intervals are arguably more useful than confidence intervals. They can be computed with the predint() function in MATLAB’s Curve Fitting Toolbox or with the predband() function [1] in Python, shown as follows:

Python:
#!/usr/bin/env python3
# code/statistics/predict_interval.py
import numpy as np
import matplotlib.pyplot as plt
import scipy.stats
import statsmodels.api as stats
import seaborn as sns
def predband(x, xd, yd, p, func, conf=0.95):
    """
    https://apmonitor.com/che263/index.php/Main/PythonRegressionStatistics
    x = requested points
    xd = x data
    yd = y data
    p = additional arguments to func, after xd
    func = function name
    """
    alpha = 1.0 - conf  # significance
    N = xd.size         # data sample size
    var_n = len(p)      # number of parameters
    # Quantile of Student's t distribution for p=(1-alpha/2)
    q = scipy.stats.t.ppf(1.0 - alpha / 2.0, N - var_n)
    # Stdev of an individual measurement
    se = np.sqrt(1. / (N - var_n) *
                 np.sum((yd - func(xd, *p)) ** 2))
    # Auxiliary definitions
    sx = (x - xd.mean()) ** 2
    sxd = np.sum((xd - xd.mean()) ** 2)
    # Predicted values (best-fit model)
    yp = func(x, *p)
    # Prediction band
    dy = q * se * np.sqrt(1.0+ (1.0/N) + (sx/sxd))
    # Upper & lower prediction bands.
    lpb, upb = yp - dy, yp + dy
    return lpb, upb
def model_function(x, m, b):
    return m*x + b
m = 3.77 # true slope
b = -5.5 # true intercept
nPts = 80
X_even = np.linspace(-5, 7)
X = -5 + 12*np.random.rand(nPts)
X.sort()
noise = 30*(np.random.rand(nPts) - 0.5)
Y = m*X + b + noise
Xb = stats.add_constant(X,0)
ols = stats.OLS(Y, Xb).fit()
m_ls, b_ls = ols.params # best fit slope, y-intercept
lpb, upb = predband(X_even, X, Y, [m_ls, b_ls], model_function, conf=0.95)
sns.regplot(x=X, y=Y, marker='.')
plt.plot(X_even, lpb, 'k--',label='95% Prediction Band')
plt.plot(X_even, upb, 'k--')
plt.xlabel('X')
plt.ylabel('Y')
plt.grid()
plt.title('Prediction and confidence intervals')
plt.legend(loc='best')
plt.savefig('pred_interval.png', bbox_inches='tight',
            pad_inches=0.1, transparent=True)
plt.show()

11.8 Recipe 11-2: Weighted Least Squares in MATLAB

A direct invocation of Python’s weighted least squares function from MATLAB leads to a disappointing result. Here is such an attempt using the example problem of the previous section:

MATLAB:
m = 3.77; % true slope
b = -5.5; % true intercept
nPts = 20;
X = -5 + 12*rand(nPts,1);
X = sort(X);
W = X - min(X) + 1;
noise = 8*(rand(nPts,1)-.5);
Y  = py.numpy.array(m*X + b + noise);
Xb = py.statsmodels.api.add_constant(X,0);
wls = py.statsmodels.api.WLS(Y, Xb, pyargs('weight', W)).fit();
[m_ls, b_ls] = wls.params
The program fails on the last line with this error:
Unrecognized method, property, or field 'params' for class
'py.statsmodels.regression.linear_model.RegressionResultsWrapper'.

The reason for the failure is the wls object returned by py.statsmodels.api.WLS() lacks a .params attribute—as well as many other attributes and methods—so the attempt to access the nonexistent .params throws an error.

We’ll need a small bridge module which contains a function that takes primitive MATLAB inputs, in this case the X, Y, and W arrays, calls statsmodels.api.WLS(), then returns desired portions of the solution in a form MATLAB understands. In other words, the call to WLS() happens entirely in Python without a traversal of the MATLAB/Python interface. The entire module defines only one function in ten lines:

Python:
# file bridge_WLS.py
import os
import sys
import statsmodels.api as stats
def slope_intercept(X, Y, W):
    if sys.platform == 'win32':
        os.environ['KMP_DUPLICATE_LIB_OK'] = 'TRUE'
    Xb = stats.add_constant(X,0)
    wls = stats.WLS(Y, Xb, weight=W).fit()
    m_ls, b_ls = wls.params
    Y_pred = wls.predict(Xb)
    if sys.platform == 'win32':
        del os.environ['KMP_DUPLICATE_LIB_OK']
    return { 'm' : m_ls,
             'b' : b_ls,
             'Y_predict' : Y_pred }

The returned dictionary is recast by py2mat() as a struct containing the slope (.m), intercept (.b), and array of predicted y values (.Y_predict). This MATLAB code calls Python’s weighted least squares correctly:

MATLAB:
% code/matlab_py/weighted_LS.m
np  = py.importlib.import_module('numpy');
WLS = py.importlib.import_module('bridge_WLS');
m = 3.77; % true slope
b = -5.5; % true intercept
nPts = 20;
X = -5 + 12*rand(nPts,1);
X = sort(X);
W = X - min(X) + 1;
noise = 8*(rand(nPts,1)-.5);
Y  = np.array(m*X + b + noise);
X  = np.array(X);
W  = np.array(W);
mbY = WLS.slope_intercept(X,Y,W);
mbY = py2mat(mbY);
fprintf('slope= % .3f  intercept = % .3f ', mbY.m, mbY.b)
scatter(X,Y,5*W,'b','filled')
hold on
plot(X,mbY.Y_predict,'r-')
grid on
saveas(gcf,'matlab_WLS.png')
Note

The Python and MATLAB codes for this example create points randomly, so their locations on the two plots will differ.

11.9 Recipe 11-3: Confidence and Prediction Intervals in MATLAB

The following MATLAB programs replicate the confidence and prediction interval plots shown in Section 11.7.3. The only catch is Seaborn wants NumPy arrays, so X and Y are explicitly converted from MATLAB-native to Python-native variables with the mat2py() utility.

Confidence interval:

MATLAB:
% code/statistics/conf_interval.m
Im = @py.importlib.import_module;
plt   = Im('matplotlib.pyplot');
mpl   = Im('matplotlib');
sns   = Im('seaborn')
if ispc
    mpl.use('WXAgg')
else
    mpl.use('TkAgg')
end
m = 3.77; % true slope
b = -5.5; % true intercept
nPts = 80;
X = -5 + 12*rand(nPts,1);
X = sort(X);
noise = 30*rand(nPts,1) - 0.5;
Y = m*X + b + noise;
% seaborn wants Python-native arrays
X = mat2py(X);
Y = mat2py(Y);
sns.regplot(pyargs('x',X,'y',Y,'marker','.'))
plt.xlabel('X')
plt.ylabel('Y')
plt.grid()
plt.title('Confidence interval with seaborn.regplot()')
plt.show()

Prediction interval:

The following program imports both model_function() and predband() from the pure Python version of the prediction interval example:

MATLAB:
% code/statistics/predict_interval.m
Im = @py.importlib.import_module;
plt   = Im('matplotlib.pyplot');
mpl   = Im('matplotlib');
sns   = Im('seaborn');
np    = Im('numpy');
%scipy_stats = Im('scipy.stats')
stats = Im('statsmodels.api');
JH    = Im('predict_interval');
if ispc
    mpl.use('WXAgg');
else
    mpl.use('TkAgg');
end
m = 3.77; % true slope
b = -5.5; % true intercept
nPts = 80;
X_even = linspace(-5, 7);
X = -5 + 12*rand(nPts,1);
X = sort(X);
noise = 30*(rand(nPts,1) - 0.5);
Y = m*X + b + noise;
X = mat2py(X);
Y = mat2py(Y);
Xb = stats.add_constant(X,0);
ols = stats.OLS(Y, Xb).fit();
[m_ls, b_ls] = ols.params % best fit slope, y-intercept
[lpb, upb] = JH.predband(X_even, X, Y, [m_ls, b_ls], ...
                         JH.model_function, pyargs('conf',0.95));
sns.regplot(pyargs('x',X,'y',Y,'marker','.'))
plt.plot(X_even, lpb, 'k--',pyargs('label','95% Prediction Band'))
plt.plot(X_even, upb, 'k--')
plt.xlabel('X')
plt.ylabel('Y')
plt.grid()
plt.title('Prediction and confidence intervals')
plt.legend(pyargs('loc=','best'))
plt.show()

11.10 Finding Roots

11.10.1 Univariate

The root_scalar() function from SciPy’s optimization module is the analog to MATLAB’s fzero() function. root_scalar() offers eight algorithms for finding roots of a scalar equation: bisect, brentq, brenth, ridder, toms748, newton, secant, halley. The first five require limits on the independent variable, while the newton, secant, and halley methods work on unbounded domains. Additional information such as a starting guess or a slope function may be provided to accelerate convergence. If no method is specified, the solver chooses the best option for the given inputs (whether or not bounds were specified, slope function provided, and so on).

The following example finds x where f(x) = 0 on the interval 0 ≤ x ≤ 1 where
$$ f(x)=2{x}^3-7{x}^2+12x-4 $$
without specifying a method. fzero() does not have an option to define boundaries; instead, we have to supply an initial guess to the solution. root_scalar() returns an object with the solution—if one was found—in its .root attribute:

MATLAB:

Python:

>> F = @(x)(2*x.^3-7*x.^2+12*x-4);

>> fzero(F, 0)

     4.265215061673047e-01

In : import numpy as np

In : import scipy.optimize as so

In : def F(x):

In :   return 2*x**3-7*x**2+12*x-4

In : x  = np.linspace(-1,2, num=100)

In : f_zero = so.root_scalar(F,

...:           bracket=[0, 1])

In : f_zero

Out[14]:

      converged: True

           flag: 'converged'

 function_calls: 9

     iterations: 8

           root: 0.42652150616730566

In : f_zero.root

Out: 0.42652150616730566

The following invocation shows how a particular method is specified to solve the problem without bounds on x. The secant method does not require a slope function (newton and halley do), but needs two starting guesses, x0 and x1. Again, fzero() doesn’t have comparable options, so instead we’ll show its robustness by starting with a guess that is far from the true solution.

MATLAB:

Python:

>> F = @(x)(2*x.^3-7*x.^2+12*x-4);

>> fzero(F, 100)

     4.265215061673049e-01

In : f_zero = so.root_scalar(F, x0=100,

...:            x1=200, method="secant")

Out[25]:

      converged: True

           flag: 'converged'

 function_calls: 25

     iterations: 24

           root: 0.4265215061673047

11.10.2 Multivariate

SciPy’s more general multivariate root finder is scipy.optimize.root(). As with scipy.optimize.root_scalar(), one can select among several solution methods and optionally supply Jacobian functions. Its use is demonstrated by finding a point of intersection between a plane, a sphere, and an ellipsoid:
$$ {displaystyle egin{array}{l}x+y+z=1\ {}{x}^2+{y}^2+{z}^2=1\ {}{x}^2+4{y}^2+{z}^2=2end{array}} $$

MATLAB can solve this with fsolve() from the Optimization Toolkit. Alternatively, MATLAB can use scipy.optimize.root() to find the solution; this is demonstrated with the recipe in Section 11.11.

Python:
In : import scipy.optimize as so
In : def F(x):
...:   return x[0]+x[1]+ x[2]-1 - (x[0]**2+  x[1]**2+x[2]**2-1),
...:       x[0]+x[1]+x[2]-1 - (x[0]**2+4*x[1]**2+x[2]**2-2),
...:       (x[0]**2+x[1]**2+x[2]**2-1) - (x[0]**2+4*x[1]**2+x[2]**2-2)
In : so.root(F, [0,0,0])
    fjac: array([[-0.21557015,  0.57423188,  0.78980203],
                 [-0.78752535, -0.5804519 ,  0.20707345],
                 [ 0.57735027, -0.57735027,  0.57735027]])
     fun: array(
       [-1.08435e-12,  1.44386e-11,  1.55230e-11])
 message: 'The solution converged.'
    nfev: 37
     qtf: array(
       [ 7.84383e-09, -6.23725e-09, -4.21542e-22])
       r: array( [-2.19228e+00, -1.97436e+00,  8.33239e-01,
                  4.49321e-01, -2.26699e+00, -1.70549e-13])
  status: 1
 success: True
       x: array(
       [ 0.16314374,  0.57735027, -0.29406851])

Selecting a starting point far from the solution can make the method fail:

Python:
In : so.root(F, [0,10,10])
    fjac: array([[-7.06972124e-01, -7.07241361e-01, -2.69237135e-04],
                 [ 4.08481434e-01, -4.08015101e-01, -8.16496537e-01],
                 [-5.77350270e-01,  5.77350270e-01, -5.77350268e-01]])
     fun: array([-180., -479., -299.])
 message: 'The iteration is not making good progress, as measured by
           the   improvement from the last ten iterations.'
    nfev: 17
     qtf: array([ 4.66104096e+02,  3.66045040e+02, -5.80555266e-07])
       r: array([ 3.62141354e+01, -1.88545072e+04,  1.36198852e+03,
                  7.18371924e+01,  1.06466345e-01,  1.09677462e-07])
  status: 5
 success: False
       x: array([ 0., 10., 10.])

11.11 Recipe 11-4: Solving Simultaneous Nonlinear Equations

The plane/sphere/ellipsoid intersection problem of Section 11.10.2 can be solved in MATLAB with SciPy’s multivariate root finder. As with the curve fitting recipe (Section 11.6), we will need to implement the function to be solved in Python because Python cannot evaluate MATLAB functions passed as arguments.

Here’s the bridge module. The function that wraps the call to the solver, F_roots(), uses (0,0,0) as a default starting point if one isn’t given:

Python:
# code/optim/bridge_nonlinear_roots.py
import scipy.optimize as so
def F(x):
  return x[0]+x[1]+ x[2]-1 - (x[0]**2+  x[1]**2+x[2]**2-1),
         x[0]+x[1]+x[2]-1 - (x[0]**2+4*x[1]**2+x[2]**2-2),  
        (x[0]**2+x[1]**2+x[2]**2-1) - (x[0]**2+4*x[1]**2+x[2]**2-2)
def F_roots(guess=[0,0,0]):
return so.root(F, guess)

Calling it from MATLAB is simple:

MATLAB:
>> soln = py.bridge_nonlinear_roots.F_roots();
>> soln.get('success')
  logical
     1
>> soln.get('message')
  Python str with no properties.
    The solution converged.
>> py2mat(soln.get('x'))
     1.631437374762493e-01  5.773502691851446e-01  -2.940685072817079e-01

Once more, this time supplying a bad starting location:

MATLAB :
>> soln = py.bridge_nonlinear_roots.F_roots(pyargs('guess',{0,10,10}));
>> soln.get('success')
  logical
     0
>> soln.get('message')
  Python str with no properties.
    The iteration is not making good progress, as measured by the
      improvement from the last ten iterations.

11.12 Optimization

SciPy’s scipy.optimize module has many functions to solve optimization problems. We already saw one of its most powerful global optimizers, scipy.optimize.differential_evolution(), in action Section 11.5.3. Here, we’ll examine a wider scope of problems including local uni- and multivariate optimization with linear and nonlinear constraints, linear programming, and combinatorial optimization for problems such as route planning.

11.12.1 Linear Programming

Linear programming (LP) problems are ubiquitous in industrial engineering, operations research, logistics, and scheduling. MATLAB’s Optimization Toolbox and SciPy have a function named linprog() that can solve real-valued LP problems. More difficult mixed-integer LP problems require modules not found in SciPy; for these we’ll turn to PuLP, which can be added to Anaconda installations via conda install.

As a simple representative LP problem, say you’ve been tasked to assemble a computer server room to support your organization’s various engineering and research departments. Some departmental needs are modest, while others require many powerful servers. The departments agree to a charge-back cost structure according to the resources they use; your goal is to figure out equitable parameters of the cost model. The server properties and negotiated charge-back rates are shown in Table 11-8.
Table 11-8

Computer categories and properties

 

Cores

Power [kW]

Volume [ft3]

Charge Back [$/hr]

Small

 4

 50

10

0.10

Medium

 8

105

20

0.60

Large

12

192

40

0.96

Huge

48

391

40

2.83

Your decision boils down to how many of each of Small, Medium, Large, and Huge servers to purchase such that

  • The total power consumption is less than 30,000 kW.

  • The total volume needed to house the servers is less than 80,000 ft3.

  • The aggregate count of compute cores exceeds 1000.

  • The number of cores in Large servers exceeds the count of cores in Huge servers.

The objective function is to maximize the charge-back

maximize  0.1S + 0.6M + 0.96L + 2.83H

(the vision is that the more departments contribute, the greater the opportunity to grow the facility in the future) subject to
$$ {displaystyle egin{array}{r}50S+105M+192L+391Hle 30000\ {}10S+20M+40L+40Hle 80000\ {}4S+8M+12L+48Hge 1000\ {}12Lge 48Hend{array}} $$

As the values for S … H can only take on integer values, this is an integer-valued LP problem. Nonetheless, we’ll see how close the real-valued solutions of SciPy’s linprog() function match fully integer-based solutions from PuLP.

11.12.1.1 SciPy linprog()

Calling arguments to SciPy’s linprog() are nearly identical to those for the Optimization Toolbox’s version. Inputs are coefficients of the function to minimize, matrices of coefficients that describe the inequality relationships, equality relationships (if such exist), and upper and lower bounds. Inequality relationships must be expressed as less-than inequalities with independent variables on the left and constant values on the right. Greater-than relationships can be remapped to less-than by changing signs on both sides.

Our problem then becomes minimize -0.1S - 0.6M - 0.96L - 2.83H with constraints:
$$ {displaystyle egin{array}{l}50S+105M+192L+391Hle 30000\ {}kern0.85em 10S+20M+40L+40Hle 80000\ {}kern0.97em -4S-8M-12L-48Hle -1000\ {}kern4.089998em -12L+48Hle 0 end{array}} $$

The left-hand side coefficients define the matrix A, the right-hand side terms define b, and the coefficients of the cost function define f. A final constraint is that we seek only positive solutions so the first value in each of the four (lower, upper) bound tuples in variable UL_bounds is zero.

The Python solution looks like this:

Python:
from scipy.optimize import linprog
f = [ -0.1, -0.6, -0.96, -2.83])
A = [[ 50, 105, 192, 391],
     [ 10,  20,  40,  40],
     [ -4,  -8, -12, -48],
     [  0,   0, -12,  48]]
b  = [ 30000, 80000, -1000, 0]
UL_bounds = [ (0, None), (0, None),
              (0, None), (0, None) ]
sol = linprog(f,A,b,bounds=UL_bounds)

The function returns an object, sol in our example, with these attributes:

Python:
In : print(sol)
     con: array([], dtype=float64)
     fun: -172.64883520249523
 message: 'Optimization terminated successfully.'
     nit: 13
   slack: array([2.90929165e-08, 7.48231234e+04,
                 1.48490078e+03, 1.47610990e-09])
  status: 0
 success: True
       x: array([1.15833172e-10, 1.37570134e-08,
                 1.03537532e+02, 2.58843831e+01])
In : print(sol.status)
Out: 0
In : print(sol.x)
    array([1.15833172e-10, 1.37570134e-08,
           1.03537532e+02, 2.58843831e+01])

A solution status of 0 indicates a successful convergence, and the solution to our independent variables is in attribute .x. The closest integer values to our solution leaves us with S = 0, M = 0, L = 104, H = 26, which corresponds to an aggregate charge value of $172.42 per hour.

If you don’t have the Optimization Toolbox, you can still solve the problem in MATLAB by calling SciPy’s linprog():

MATLAB:
f = [ -0.1; -0.6; -0.96; -2.83];
A = [ 50 105 192 391; ...
      10  20  40  40; ...
      -4  -8 -12 -48; ...
       0   0 -12  48 ];
b = [ 30000; 80000; -1000; 0];
UL_bounds = { py.tuple({0, py.None}), py.tuple({0, py.None}), ...
              py.tuple({0, py.None}), py.tuple({0, py.None}) };
opt = py.importlib.import_module('scipy.optimize');
sol = opt.linprog(f,A,b,pyargs('bounds',py.list(UL_bounds)))

Running the preceding code gives the correct solution, but, frustratingly, the solution components are not accessible in MATLAB:

Python:
sol =
  Python OptimizeResult with no properties.
         con: array([], dtype=float64)
         fun: -172.64883520249523
     message: 'Optimization terminated successfully.'
         nit: 13
       slack: array([2.90929165e-08, 7.48231234e+04, 1.48490078e+03, 1.47599621e-09])
      status: 0
     success: True
           x: array([1.15833172e-10, 1.37570134e-08, 1.03537532e+02, 2.58843831e+01])
>> y = sol.x
Unrecognized method, property, or field 'x' for class
  'py.scipy.optimize.optimize.OptimizeResult'.
Once again, we’ll need to write a small Python bridge module with a function that extracts and returns the components we want.

MATLAB:

Python:

LP = py.importlib.import_module(...

    'bridge_linprog');

py.importlib.reload(LP);

f = [ -0.1; -0.6; -0.96; -2.83];

A = [ 50 105 192 391; ...

      10  20  40  40; ...

      -4  -8 -12 -48; ...

       0   0 -12  48 ];

b = [ 30000; 80000; -1000; 0];

UL_bounds = { py.tuple({0, py.None}), ...

              py.tuple({0, py.None}), ...

              py.tuple({0, py.None}), ...

              py.tuple({0, py.None}) };

py_sol = LP.solver(f,A,b,pyargs(...

    'bounds',py.list(UL_bounds)));

m_sol = py2mat(py_sol);

# file bridge_linprog.py

from scipy.optimize import linprog

def solver(f,A,b,bounds=None):

   sol = linprog(f,A,b,bounds=bounds)

   return { 'fun' : sol.fun,

            'message' : sol.message,

            'status' : sol.status,

            'success' : sol.success,

            'x' : sol.x}

Now the five components we captured in the bridge module can be used in MATLAB:

MATLAB:
>> m_sol
  struct with fields:
        fun: -172.6488
    message: "Optimization terminated successfully."
     status: 0
    success: 1
          x: [1.1583e-10 1.3757e-08 103.5375 25.8844]

11.12.1.2 PuLP

The Python PuLP module has two appealing aspects: one can define variables as being integer or real, and constraints can be expressed in a more natural form than other modules:

Python:
import pulp
model = pulp.LpProblem("Server room", pulp.LpMaximize)
S = pulp.LpVariable('S', lowBound=0, cat='Integer')
M = pulp.LpVariable('M', lowBound=0, cat='Integer')
L = pulp.LpVariable('L', lowBound=0, cat='Integer')
H = pulp.LpVariable('H', lowBound=0, cat='Integer')
# constants
C_s, C_m, C_l, C_h = .10,  .60,  .96, 2.83  # cost
T_s, T_m, T_l, T_h = 50, 105, 192, 391      # thermal
O_s, O_m, O_l, O_h =  4,   8,  12, 48       # cores
V_s, V_m, V_l, V_h = 10,  20,  40, 40       # volume
# the function to maximize
model += C_s*S +  C_m*M +  C_l*L +  C_h*H, "Charge"
# subject to constraints
model += T_s*S +  T_m*M +  T_l*L +  T_h*H  <= 30000 # thermal
model += O_s*S +  O_m*M +  O_l*L +  O_h*H  >=  1000 # cores
model += O_l*L >= O_h*H # cores
model += V_s*S +  V_m*M +  V_l*L +  V_h*H  <= 80000 # volume
# compute the solution
model.solve()
print(pulp.LpStatus[model.status])
The result is Python:
   0 S,   30 M,   93 L,   23 H   charge = 172.37
Optimal

11.12.2 Simulated Annealing

Simulated annealing is a stochastic global optimization method based on the concept of crystal formation in metals as temperature drops. Briefly, the algorithm makes a random guess at a solution, then either uses a perturbation of that solution for the next iteration or makes a completely new guess. The decision as to whether to perturb the solution or make a new guess depends on the solution’s cost and how far along the “annealing schedule” the algorithm has progressed. Early on, a guess that is worse than its predecessor will likely be accepted, but as iterations progress and the “temperature” drops, improvements to the solution are more likely to survive to the next round.

The Python module simanneal, available through conda-forge, provides a convenient framework to solve optimization problems with simulated annealing. simanneal requires the user to provide the following:

  • A class which inherits from simanneal’s Annealer class and provides a constructor (the __init__() function), a move() function which perturbs an existing solution, and an energy() function which evaluates the solution’s cost function

  • An initial guess to the solution

  • An annealing schedule

To illustrate how these three items might be implemented, we’ll solve the Traveling Salesman Problem (TSP) using a simplification of the TSP solution given in simanneal’s documentation.12

The Annealer-derived class is defined with lines 12–22. Our problem’s cost function is defined with the energy() method on line 20. The .state attribute is the collection of variables we wish to optimize—in our case, it is a list of city indices that defines the route. For us, the cost of a solution is the sum of distances between adjacent cities in the route. Here, we access the .distance_matrix attribute which is a look-up table of distances between cities; it is an N x N matrix where the term at index (i, j) is the distance between cities i and j. The distance matrix is populated in the class’s constructor at line 13. Aside from adding .distance_matrix as an attribute, the constructor is entirely boilerplate. Finally, the move() method at line 16 defines how to apply a random perturbation to the current state, in other words, how to make a random change to our route.

The compute_distances() function at line 24 computes the distance matrix using the broadcasting technique mentioned in Section 11.1.13.

Lines 31–34 define 80 cities arranged on an 8 x 10 grid. The simplicity of this layout makes it easy to identify optimal solutions.

The annealing schedule is defined on line 42. The argument tsp.auto(minutes=0.5) tells the module to estimate schedule parameters so that a solution is returned in roughly 30 seconds of elapsed time. Alternatively, you can control the parameters yourself with

Python:
schedule = {
    'tmax' : 20000.0, # starting temperature
    'tmin' :     2.0, # ending temperature
    'steps' : 50000 , # n iterations
    'updates' : 100 , # n updates to stdout
}
tsp.set_schedule(schedule)

In practice, the automatic estimator provides an ideal balance between convenience, solution fitness, and knowledge of how long you’ll need to wait for an answer. If your solution isn’t sufficiently optimal, simply increase the amount of time you’ll let the annealer to run (via the minutes parameter to tsp.auto()).

Python:
 1   import numpy as np
 2   import random
 3   from simanneal import Annealer
 4   import matplotlib.pyplot as plt
 5
 6   def plot_route(route, coordinates, fname):
 7       circuit = [_ for _ in route] + [ route[0] ]
 8       plt.plot(coordinates[circuit,0], coordinates[circuit,1])
 9       plt.scatter(coordinates[:,0], coordinates[:,1], color='r')
10       plt.show()
11
12   class TravellingSalesmanProblem(Annealer):
13       def __init__(self, state, distance_matrix):
14           self.distance_matrix = distance_matrix # store distances in the object
15           super(TravellingSalesmanProblem, self).__init__(state)
16       def move(self): # swap two cities in the route; return that distance
17           a, b = random.sample(range(len(self.state)), k=2)
18           self.state[a], self.state[b] = self.state[b], self.state[a]
19           return self.energy()
20       def energy(self): # for this problem, energy = route distance
21           S = self.state
22           return sum([self.distance_matrix[S[i-1],S[i]] for i in range(len(S))])
23
24   def compute_distances( city_coord ): # uses broadcasting
25       X, Y = city_coord[:,0], city_coord[:,1]
26       dX = X[:,np.newaxis] - X[np.newaxis,:]
27       dY = Y[:,np.newaxis] - Y[np.newaxis,:]
28       return np.sqrt( dX**2 + dY**2 )
29
30   def main():
31       nR, nC = 8, 10
32       J, I = np.meshgrid(range(nR),range(nC))
33       city_coord = np.vstack( [I.ravel(), J.ravel()] ).T
34       distance_matrix = compute_distances( city_coord )
35
36       # initial state, a randomly-ordered itinerary
37       starting_sequence = list(range(nR*nC))
38       random.shuffle(starting_sequence) # modifies starting_sequence in-place
39       plot_route(starting_sequence, city_coord, 'tsp_start')
40
41       tsp = TravellingSalesmanProblem(starting_sequence, distance_matrix)
42       tsp.set_schedule(tsp.auto(minutes=0.5))
43       tsp.copy_strategy = "slice" # state is a list; slice method is fastest
44       state, e = tsp.anneal()     # start the optimization
45
46       print(f'Route distance = {e:.2f}, route = ',state)
47       plot_route(state, city_coord, 'tsp_solved')
48   if __name__ == '__main__': main()
Figure 11-6

Initial random guess to the TSP

Figure 11-7

A solution to the TSP found with simanneal with tsp.auto(minutes=0.25)

Figure 11-8

A solution to the TSP found with simanneal with tsp.auto(minutes=2.5)

11.13 Differential Equations

SciPy’s collection of differential equation solvers is comparable to MATLAB’s family of ode*() routines. The calling method is quite different though. MATLAB’s routines return the full solution (if one can be found), while SciPy’s return an iterator that must be called in a loop.

As a sample problem, we’ll solve the Duffing oscillator equation:13
$$ frac{d^2u}{dt^2}+cfrac{du}{dt}+ ku+{k}_3{u}^3=Asin left(omega t ight) $$
Since both MATLAB and Python solvers only work with first-order differential equations, this second-order equation must be linearized to a pair of first-order equations:
$$ {displaystyle egin{array}{l}frac{du_1}{dt}={u}_2\ {}frac{du_2}{dt}=-{ku}_1-{cu}_2-{k}_3{u}_1^3+Asin left(omega t ight)end{array}} $$
The following MATLAB14 and Python functions Duff() return a two-item array of values corresponding to the two right-hand sides of these equations:

MATLAB:

Python:

% file: code/ode/duffing.m

Time = [0 40];

U_0   = [0; 0];

c = 0.1;

k = 0.7;

k3 = 1;

A = 44.5;

omega = 1.1;

options = odeset('RelTol',1e-7);

[t,u] = ode45(@(t,u)Duff(...

    t,u,k,c,k3,A,omega), ...

    Time,U_0,options);

figure

plot(u(:,1),u(:,2));

xlabel('Position');

ylabel('Velocity');

title('Duffing oscillator')

function u = Duff(t,u,k,c,k3,A,omega)

  u = [u(2); -k*u(1)-c*u(2)- ...

           k3*u(1)^3+A*sin(omega*t)];

end

# file: code/ode/duffing.py

import numpy as np

from scipy.integrate import RK45

import matplotlib.pyplot as plt

def Duff(t,u,k=0.7,c=0.1,k3=1,

         A=44.5,omega=1.1):

return [u[1], -k*u[0]-c*u[1] -

        k3*u[0]**3+A*np.sin(omega*t)]

T0, Tn = 0, 40

U_0 = [0, 0]

t, u = [], []

sol = RK45(Duff, T0, U_0, Tn,

           rtol=1e-7)

while sol.t < Tn:

  t.append(sol.t)

  u.append([sol.y[0], sol.y[1]])

  sol.step()

u = np.array(u)

plt.plot(u[:,0], u[:,1])

plt.xlabel('Position');

plt.ylabel('Velocity');

plt.title('Duffing oscillator');

plt.show()

The single line call to ode45() on the left corresponds to a call to RK45() followed by a while loop that successively calls the solver object’s .step() method in Python. Both solvers are configured to return solutions with a relative accuracy of 10−7, but MATLAB needs 5093 iterations, while Python only does 1002. A phase space plot of the solution shows good agreement:

11.14 Symbolic Mathematics

The Python module SymPy is a Computer Algebra System (CAS) comparable to MATLAB’s Symbolic Toolkit. SymPy is an enormously capable CAS covering, among other topics, calculus, differential geometry, combinatorics, number theory, cryptography, series, and ordinary and partial differential equations. In addition to computing results symbolically, it can also express the solutions in LaTeX or as computer code in Python, MATLAB, and many other languages as shown in Table 11-9.
Table 11-9

SymPy code generators

Language

sympy.printing Function

C

ccode()

C++

cxxcode()

Fortran

fcode()

GLSL

glsl_code()

JavaScript

jscode()

Julia

julia_code()

Maple

maple_code()

Mathematica

mathematica_code()

MATLAB

octave_code()

Python

pycode()

R

rcode()

Rust

rust_code()

The examples shown in this section barely scratch the surface of SymPy’s power.

11.14.1 Defining Symbolic Variables

Variables are identified in SymPy by importing them from the sympy.abc module. For example, the expression from sympy.abc import x, y, phi identifies x, y, and phi as variables to be treated symbolically.

11.14.2 Derivatives

Derivatives are obtained with the sympy.diff() function. This example shows how to compute the derivative of
$$ fleft(phi ight)=frac{phi }{sin^2left(phi ight)}+cos left(phi ight) $$
Python:
In : from sympy import cos, sin, diff
In : from sympy.abc import phi
In : from sympy.printing.latex import print_latex, octave_code
In : f = phi/sin(phi)**2 + cos(phi)
In : d = diff(f)
In : d.doit()
In : d.doit()
Out: -2*phi*cos(phi)/sin(phi)**3 - sin(phi) + sin(phi)**(-2)
In : print_latex(d)
- frac{2 phi cos{left(phi ight)}}{sin^{3}{left(phi ight)}} -
     sin{left(phi ight)} + frac{1}{sin^{2}{left(phi ight)}}
The LaTeX output for the derivative renders like this:
$$ -frac{2phi cos left(phi ight)}{sin^3left(phi ight)}-sin left(phi ight)+frac{1}{sin^2left(phi ight)} $$

and MATLAB code for the input and output functions can be produced like so:

Python:
In : print(octave_code(f))
phi./sin(phi).^2 + cos(phi)
In : print(octave_code(d))
-2*phi.*cos(phi)./sin(phi).^3 - sin(phi) + sin(phi).^(-2)

11.14.3 Integrals

Integration is done with sympy.integrate(). For example, the integral
$$ int frac{x}{1+x+3{x}^2} dx $$

is found with

Python:
In : from sympy import integrate
In : from sympy.abc import x
In : from sympy.printing.latex import print_latex
In : s = integrate(x/(1 + x + 3*x**2))
In : s.doit()
Out: log(x**2 + x/3 + 1/3)/6 - sqrt(11)*atan(6*sqrt(11)*x/11 + sqrt(11)/11)/33
In : print_latex(s)
frac{log{left(x^{2} + frac{x}{3} + frac{1}{3} ight)}}{6} - frac{sqrt{11}
    operatorname{atan}{left(frac{6 sqrt{11} x}{11} + frac{sqrt{11}}{11}
     ight)}}{33}
which looks like this:
$$ frac{log left({x}^2+frac{x}{3}+frac{1}{3} ight)}{6}-frac{sqrt{11};mathrm{atan};left(frac{6sqrt{11};x}{11}+frac{sqrt{11}}{11} ight)}{33} $$

11.14.4 Solving Equations

The equations
$$ {displaystyle egin{array}{l}{left(x-1 ight)}^2+{left(y+3 ight)}^2={R}^2\ {}kern3.959998em y=kern0.5em mx+bend{array}} $$

describe a circle and a line. The following code solves for the points of intersection (x, y) in terms of R, m, and b:

Python:
In : from sympy import Eq
In : from sympy.solvers import solve
In : from sympy.abc import x,y,R,m,b
In : circle = (x-1)**2 + (y+3)**2 - R**2
In : line = m*x + b
In : solve(Eq(circle,line), (x,y))
Out: [(m/2 - sqrt(4*R**2 + 4*b + m**2 +
       4*m - 4*y**2 - 24*y - 36)/2 + 1, y),
      (m/2 + sqrt(4*R**2 + 4*b + m**2 +
       4*m - 4*y**2 - 24*y - 36)/2 + 1, y)]

11.14.5 Linear Algebra

11.14.5.1 Perform a Symbolic LU Factorization of

The matrix

$$ left[egin{array}{ccc}1&amp; 2&amp; -3\ {}4&amp; -5&amp; 6\ {}-7&amp; 8&amp; 9end{array} ight] $$

is factored to lower and upper triangular matrices using exact arithmetic with the following code:

Python:
In : from sympy import Matrix
In : A = Matrix([[1,2,-3],
                 [4,-5,6],
                 [-7,8,9]])
In : L, U, perm =
...:    A.LUdecomposition()
In : L
Matrix([
[ 1,      0, 0],
[ 4,      1, 0],
[-7, -22/13, 1]])
In : U
Matrix([
[1,   2,     -3],
[0, -13,     18],
[0,   0, 240/13]])
In : perm
Out: []
In : L*U - A
Matrix([
[0, 0, 0],
[0, 0, 0],
[0, 0, 0]])

11.14.5.2 Symbolically Solve Ax = b

This series of three equations and three unknowns

$$ left[egin{array}{ccc}1&amp; 2&amp; -3\ {}4&amp; -5&amp; 6\ {}-7&amp; 8&amp; 9end{array} ight]x=left{egin{array}{c}1\ {}-2\ {}1end{array} ight} $$

is solved symbolically for x with Gauss-Jordan elimination using this code:

Python:
In : b = Matrix([1, -2, 1])
In : x, params = A.gauss_jordan_solve(b)
In : x
Matrix([
[ 1/20],
[ 3/10],
[-7/60]])
In : params
Out: Matrix(0, 1, [])

11.14.6 Series

SymPy has support for Fourier series, power series, series summation, and series expansion. This example shows how the closed-form summation
$$ sum limits_{k=1}^{N+1}frac{e^{ikpi}}{N^2} $$

can be found:

Python:
In : from sympy import I, Sum, exp, pi
In : from sympy.abc import k, N
In : s = Sum( exp(I*k*pi)/N**2, (k, 1, N+1))
In : s.doit()
Out: (-(-1)**(N + 2)/2 - 1/2)/N**2

11.15 Recipe 11-5: Using SymPy in MATLAB

The series summation from Section 11.14.6 can be done from MATLAB after defining symbolic variables and SymPy-specific constants for i and π. We’ll prefix all Python variables with “p” to avoid name collisions with built-in MATLAB variables even though in this case only pi would have caused a collision.

MATLAB:
>> pI = py.sympy.I;
>> pPi = py.sympy.pi;
>> pk = py.sympy.abc.k;
>> pN = py.sympy.abc.N;
>> s = py.sympy.Sum( py.sympy.exp(pI*pk*pPi)/pN^2, py.tuple({pk,1,pN+1}));
>> s.doit()
  Python Mul with properties:
                       args: [1×2 py.tuple]
               assumptions0: [1×1 py.dict]
        canonical_variables: [1×1 py.dict]
          expr_free_symbols: [1×1 py.set]
               free_symbols: [1×1 py.set]
                         func: [1×1 py.sympy.core.assumptions.ManagedProperties]
               is_algebraic: [1×1 py.NoneType]
                               :
                is_rational: [1×1 py.NoneType]
                    is_real: [1×1 py.NoneType]
          is_transcendental: [1×1 py.NoneType]
                    is_zero: [1×1 py.NoneType]
    N**(-2.0)*(-(-1)**(N + 2.0)/2 - 0.5)

The output from s.doit() is a Python variable within MATLAB; to get a MATLAB character string containing the solution, we merely need to access the Python solution’s .char attribute :

MATLAB:
>> s_soln = s.doit();
>> m_soln = s_soln.char;
>> m_soln
   'N**(-2.0)*(-(-1)**(N + 2.0)/2 - 0.5)'

11.16 Recipe 11-6: Compute Laplace Transforms

The laplace() function in MATLAB’s Symbolic Toolbox can compute Laplace transforms. The laplace_transform() function in SymPy can do these too. The following example computes the transform of
$$ f(t)={e}^{- at}cos omega t $$
MATLAB 2020b:
>> sympy = py.importlib.import_module('sympy');
>> s = sympy.symbols('s');
>> t = sympy.symbols('t');
>> a = sympy.symbols('a', pyargs('real','True', 'positive','True'));
>> omega = sympy.symbols('omega', pyargs('real','True'));
>> f = sympy.exp(-a*t)*sympy.cos(omega*t);
>> L = sympy.laplace_transform(f, t, s)
L =
  Python tuple with no properties.
    ((a + s)/(omega**2 + (a + s)**2), -a, Eq(2*Abs(arg(omega)), 0))
>> latex = py.importlib.import_module('sympy.printing.latex');
>> latex.print_latex(L{1})
frac{a + s}{omega^{2} + left(a + s ight)^{2}}
The LaTex output renders as
$$ frac{a+s}{omega^2+{left(a+s ight)}^2} $$

11.17 Unit Systems

The Python module pint allows one to assign units to numeric scalars and arrays, much like the similar capability in MATLAB’s Symbolic Toolbox. Once units are assigned to variables, attempts to mix terms with incompatible dimensions in a computation—for example, adding an area to a temperature–raise a DimensionalityError. Mixing terms with compatible dimensions but different unit systems—adding a length in meters to a length in inches—works correctly because the appropriate scale factors are applied first.

In both pint and the Symbolic Toolbox, one must first initialize the units object (u in the following examples for both languages). Subsequent variables can receive units by multiplying a numeric value or variable by one of the units object’s many attributes. Here, we show how one can convert metric values for aluminum 6061’s15 modulus of elasticity and density from the metric system to the US customary system.

Python:
In : from pint import UnitRegistry
In : u = UnitRegistry()
In : E = 68.9 * u.GPa
In : rho = 2.70 * u.g/u.cm**3
In : E_us = E.to(u.psi)
In : rho_us = rho.to(u.lb/u.inch**3)
In : E_us
Out: 9993100.129611416 <Unit('pound_force_per_square_inch')>
In : rho_us
Out: 0.09754368840022593 <Unit('pound / inch ** 3')>

Both pint and the Symbolic Toolbox support hundreds of base and derived units; complete lists can be found at https://github.com/hgrecco/pint/blob/master/pint/default_en.txt and www.mathworks.com/help/symbolic/units-list.html.

Units can be assigned to a NumPy array; operations on individual terms or slices from that array will work only with unitless or other consistent unit variables. It is not possible to create a NumPy array with mixed units, though. This precludes the use of pint in applications that employ arrays whose rows or columns have different unit types. One such example is finite element matrices that contain both linear and rotational stiffnesses.

11.17.1 Defining Units in pint

Arbitrary units can be defined easily. In the next example, currency units of “dollar” and “euro” are defined. From these, prices per unit area are defined and then applied to a collection of real estate lots with four properties per lot. Finally, the price of each collection is given in dollars and euros. Note that pint does not automatically simplify the compound units when the cost vector is computed; we have to explicitly convert to dollars or euros:

Python:
In : from pint import UnitRegistry
In : u = UnitRegistry()
In : u.define('dollar = 1 = USD')
In : u.define('euro = 1.31 * dollar = EU')
In : USD_per_ft2 = 2.1 * u.USD/u.foot**2
In : EU_per_m2 = USD_per_ft2.to(u.EU/u.meter**2)
In : EU_per_m2
Out: 17.2551 <Unit('euro / meter ** 2')>
In : a = np.arange(20).reshape(5,4) * u.acre
In : a
array([[ 0,  1,  2,  3],
       [ 4,  5,  6,  7],
       [ 8,  9, 10, 11],
       [12, 13, 14, 15],
       [16, 17, 18, 19]]) <Unit('acre')>
In : P = np.ones(4) * USD_per_ft2
In : P
Out: array([2.1, 2.1, 2.1, 2.1]) <Unit('dollar / foot ** 2')>
In : cost = a@P
In : cost
Out: array([ 12.6,  46.2,  79.8, 113.4, 147. ])
            <Unit('acre * dollar / foot ** 2')>
In : cost.to(u.USD)
array([ 548858.19543059, 2012480.04991215, 3476101.90439371,
       4939723.75887528, 6403345.61335684]) <Unit('dollar')>
In : cost.to(u.euro)
array([ 418975.72170274, 1536244.31291004, 2653512.90411734,
       3770781.49532464, 4888050.08653194]) <Unit('euro')>

The pint documentation says adding units to variables causes a performance hit when performing computations with those variables. That’s true each time pint has to check units. If the variable is a NumPy array, though, this units check is done only once for the entire array, not for each term in the array. By and large, programs whose performance is dominated by numeric array computations run as quickly with units as without.

11.18 Recipe 11-7: Using pint in MATLAB

The units example of Section 11.17 requires some modification for use in MATLAB. The primary difference is that units cannot be accessed from the registry variable using dot notation. Instead, we have to use the .Units() accessor function. The “meter” unit, for example, must be referenced in MATLAB as u.Unit('meter') instead of just u.meter as it is in Python. Also note that the a matrix is defined as the transpose of a 4 × 5 to get the same row-major values as Python.

MATLAB:
>> u = py.pint.UnitRegistry;
>> u.define('dollar = 1 = USD');
>> u.define('euro = 1.31 * dollar = EU');
>> USD_per_ft2 = 2.1 * u.Unit('USD')/u.Unit('foot')^2;
>> EU_per_m2 = USD_per_ft2.to(u.Unit('EU')/u.Unit('meter')^2);
>> a = reshape(0:19, 4,5)';
>> b = py.numpy.array(a)*u.Unit('acre');
>> P = py.numpy.ones(int64(4)) * USD_per_ft2;
>> cost = b.dot(P)
cost =
  Python Quantity with properties :
                     T: [1×1 py.pint.quantity.Quantity]
                              :
                 units: [1×1 py.pint.unit.Unit]
    [12.6 46.2 79.8 113.4 147.0] acre * dollar / foot ** 2
>> cost.to(u.Unit('USD'))
  Python Quantity with properties:
                     T: [1×1 py.pint.quantity.Quantity]
                              :
                 units: [1×1 py.pint.unit.Unit]
    [548858.195 2012480.050 3476101.904 4939723.759 6403345.613] dollar
>> cost.to(u.Unit('EU'))
  Python Quantity with properties:
                     T: [1×1 py.pint.quantity.Quantity]
                              :
                 units: [1×1 py.pint.unit.Unit]
    [418975.722 1536244.313 2653512.904 3770781.495 4888050.087] euro

The numeric value of the Python units array can be converted to a MATLAB matrix like this:

MATLAB:
>> euros = cost.to(u.Unit('EU'));
>> matlab_euros = double(euros.m)
   4.1898e+05   1.5362e+06   2.6535e+06   3.7708e+06   4.8881e+06

11.19 References

  1. [1]

    John Hedengren. Programming for Engineers: Regression Statistics with Python. 2021. URL: https://apmonitor.com/che263/index.php/Mai/PythonRegressionStatistics

     
  2. [2]

    MathWorks Support Team. “How do I perform a binary search of a presorted array?” In: (June 2017). URL: www.mathworks.com/matlabcentral/answers/92533-how-do-i-perform-a-binary-search-of-a-presorted-array

     
  3. [3]

    Storn, R.; Price, K. (1997). “Differential evolution – a simple and efficient heuristic for global optimization over continuous spaces.” Journal of Global Optimization. 11 (4): 341–359. doi:10.1023/A:1008202821328

     
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.188.57.172