Linear algebra is of primary importance in machine learning and it gives us an array of tools that are especially handy for the purpose of manipulating data and extracting patterns from it. Moreover, when data must be processed in batches as in much machine learning, great runtime efficiencies are gained from using the "vectorized" form as an alternative to traditional looping constructs when implementing software solutions in optimization or data pre-processing or any number of operations in analytics.

We will consider only the domain of real numbers in what follows. Thus, a vector represents an array of n real-valued numbers. A matrix is a two-dimensional array of m rows and n columns of real-valued numbers.

Some key concepts from the foundation of linear algebra are presented here.

Vector

The vector x (lowercase, bold, by convention; equivalently, ) can be thought of as a point in n-dimensional space. Conventionally, we mean column-vector when we say vector. The transpose of a column vector is a row vector with the same number of elements, arranged in a single row.

Scalar product of vectors

Also known as the dot product, the scalar product is defined for two vectors of equal length. The result of the operation is a scalar value and is obtained by summing over the products of the corresponding elements of the vectors. Thus, given vectors x and y:

The dot product xTy is given as: