NTM is an interesting algorithm that has the ability to store and retrieve information from memory. The idea of NTM is to augment the neural network with an external memory—that is, instead of using hidden states as a memory, it uses an external memory for storing and retrieving the information. The architecture of NTM is shown in the following figure:

The important components of the NTM are as follows:

• Controller: This is basically a feedforward neural network or recurrent neural network. It reads from and writes to the memory.
• Memory: The memory matrix or memory bank, or simply the memory, is where we will store the information. Memory is basically a two-dimensional matrix composed of memory cells. The memory matrix contains N rows and M columns. Using the controller, we access the content from the memory. So, the controller receives input from the external environment and emits responses by interacting with the memory matrix.
• Read and write heads: The read head and write head are the pointers containing addresses of the memory from which it has to read from and write to.

Okay, but how can we access information from the memory? Can we access the information from the memory by specifying the row and column index? Yes, we can. But the problem is that, if we access information by index, we can't train our NTM using gradient descent because we can't calculate gradients for an index. So, the author of NTM defines blurry operations for reading and writing using a controller. The blurry operation will interact with all of the elements in the memory to some extent. It is basically an attention mechanism that strongly focuses on a particular location in the memory that is important to read/write, while ignoring the focus on the other location. So, we use special read and write operations to determine which location on the memory to focus on. We will explore more on read and write operations in the upcoming section.