This chapter will briefly cover the Message Passing Interface (MPI), which is a specification for message exchange. The primary goal of the MPI is to establish an efficient, flexible, and portable standard for message exchange communication.

Mainly, we will show the functions of the library that include synchronous and asynchronous communication primitives, such as (send/receive) and (broadcast/all-to-all), the operations of combining the partial results of the calculation (gather/reduce), and finally, the synchronization primitives between processes (barriers).

Furthermore, the control functions of the communication network will be presented by defining the topologies.

In this chapter, we will cover the following recipes:

• Using the mpi4py Python module
• Implementing point-to-point communication
• Avoiding deadlock problems
• Collective communication using a broadcast
• Collective communication using the scatter function
• Collective communication using the gather function
• Collective communication using Alltoall
• The reduction operation
• Optimizing communication