A read operation reads a value from the memory. But since we have many memory blocks in our memory matrix, which one do we need to select to read from the memory? That is determined by the weight vector. The weight vector specifies which region in the memory is more important than others. We use an attention mechanism to get this weight vector. We will explore more on how exactly we compute this weight vector in the upcoming section. The weight vector is normalized, meaning that its value ranges from zero to one, and the sum of the value equals one. The following diagram shows the weight vector of the length, N:

Let's denote this normalized weight vector with w_t, where the subscript, t, implies time and w_t(i) denotes an element in the weight vector at an index, i, and time, t:

Our memory matrix consists of N rows and M columns, as shown in the following diagram. Let's represent our memory matrix at the time t as M_t:

Now that we have the weight vector and memory matrix, we perform a linear combination of a memory matrix, M_t, and a weight vector, w_t, to get the read vector, r_t, as shown in the following figure:

This can be expressed as the following:

As you can see in the preceding figure, we have the memory matrix of the N rows and M columns, and the weight vector of size N containing weights for all of the N locations. Performing a linear combination of these two, we get a read vector of the length M.