The true power of NumPy lies in its fast mathematical operations. The approach used by NumPy is to avoid stepping into the Python interpreter by performing element-wise calculation using optimized C code. Broadcasting is a clever set of rules that enables fast array calculations for arrays of similar (but not equal!) shape.

Whenever you do an arithmetic operation on two arrays (like a product), if the two operands have the same shape, the operation will be applied in an element-wise fashion. For example, upon multiplying two shape (2,2) arrays, the operation will be done between pairs of corresponding elements, producing another (2, 2) array, as shown in the following code:

    A = np.array([[1, 2], [3, 4]]) 
    B = np.array([[5, 6], [7, 8]]) 
    A * B 
    # Output:
    # array([[ 5, 12],           
    #        [21, 32]]) 

If the shapes of the operands don't match, NumPy will attempt to match them using broadcasting rules. If one of the operands is a scalar (for example, a number), it will be applied to every element of the array, as the following code illustrates:

    A * 2 
    # Output: 
    # array([[2, 4], 
    #        [6, 8]]) 

If the operand is another array, NumPy will try to match the shapes starting from the last axis. For example, if we want to combine an array of shape (3, 2) with one of shape (2,), the second array will be repeated three times to generate a (3, 2) array. In other words, the array is broadcasted along a dimension to match the shape of the other operand, as shown in the following figure:

If the shapes mismatch, for example, when combining a (3, 2) array with a (2, 2) array, NumPy will throw an exception.

If one of the axis's size is 1, the array will be repeated over this axis until the shapes match. To illustrate that point, consider that we have an array of the following shape:

    5, 10, 2 

Now, consider that we want to broadcast it with an array of shape (5, 1, 2); the array will be repeated on the second axis 10 times, which is shown as follows:

    5, 10, 2 
    5,  1, 2 → repeated 
    - - - - 
    5, 10, 2 

Earlier, we saw that it is possible to freely reshape arrays to add axes of size 1. Using the numpy.newaxis constant while indexing will introduce an extra dimension. For instance, if we have a (5, 2) array and we want to combine it with one of shape (5, 10, 2), we can add an extra axis in the middle, as shown in the following code, to obtain a compatible (5, 1, 2) array:

    A = np.random.rand(5, 10, 2) 
    B = np.random.rand(5, 2) 
    A * B[:, np.newaxis, :] 

This feature can be used, for example, to operate on all possible combinations of the two arrays. One of these applications is the outer product. Consider that we have the following two arrays:

    a = [a1, a2, a3] 
    b = [b1, b2, b3] 

The outer product is a matrix containing the product of all the possible combinations (i, j) of the two array elements, as shown in the following snippet:

    a x b = a1*b1, a1*b2, a1*b3 
            a2*b1, a2*b2, a2*b3 
            a3*b1, a3*b2, a3*b3 

To calculate this using NumPy, we will repeat the [a1, a2, a3] elements in one dimension, the [b1, b2, b3] elements in another dimension, and then take their element-wise product, as shown in the following figure:

Using code, our strategy will be to transform the a array from shape (3,) to shape (3, 1), and the b array from shape (3,) to shape (1, 3). The two arrays are broadcasted in the two dimensions and get multiplied together using the following code:

    AB = a[:, np.newaxis] * b[np.newaxis, :] 

This operation is very fast and extremely effective as it avoids Python loops and is able to process a high number of elements at speeds comparable with pure C or FORTRAN code.