1.4 PARALLEL COMPUTING DESIGN CONSIDERATIONS

This section discusses some of the important aspects of the design of parallel computing systems. The design of a parallel computing system requires considering many design options. The designer must choose a basic processor architecture that is capable of performing the contemplated tasks. The processor could be a simple element or it could involve a superscalar processor running a multithreaded operating system.

The processors must communicate among themselves using some form of an interconnection network. This network might prove to be a bottleneck if it cannot support simultaneous communication between arbitrary pairs of processors. Providing the links between processors is like providing physical channels in telecommunications. How data are exchanged must be specified. A bus is the simplest form of interconnection network. Data are exchanged in the form of words, and a system clock informs the processors when data are valid. Nowadays, buses are being replaced by networks-on-chips (NoC) [13]. In this architecture, data are exchanged on the chip in the form of packets and are routed among the chip modules using routers.

Data and programs must be stored in some form of memory system, and the designer will then have the option of having several memory modules shared among the processors or of dedicating a memory module to each processor. When processors need to share data, mechanisms have to be devised to allow reading and writing data in the different memory modules. The order of reading and writing will be important to ensure data integrity. When a shared data item is updated by one processor, all other processors must be somehow informed of the change so they use the appropriate data value.

Implementing the tasks or programs on a parallel computer involves several design options also. Task partitioning breaks up the original program or application into several segments to be allocated to the processors. The level of partitioning determines the workload allocated to each processor. Coarse grain partitioning allocates large segments to each processor. Fine grain partitioning allocates smaller segments to each processor. These segments could be in the form of separate software processes or threads. The programmer or the compiler might be the two entities that decide on this partitioning. The programmer or the operating system must ensure proper synchronization among the executing tasks so as to ensure program correctness and data integrity.