In the last chapter, we learned the history and philosophy of the Python programming language.
In this short chapter, we will study the concepts and history of supercomputing.
Concept of the Supercomputer
A supercomputer is a special computer with a lot of processing power. This is the simplest definition of the term supercomputer. The key feature which distinguishes supercomputers from other classes of computers is their massive processing power.
Supercomputers are used in computationally intensive applications. Mostly these are scientific applications. A few examples are the following:
Weather prediction
Climate research
Molecular modelling
Physical Simulations
Quantum mechanics
Oil and gas exploration
Brief history of Supercomputers
Control Data Corporation (CDC) is the cradle of supercomputers. Here, in 1964, Seymour Cray built CDC 6600. It was dubbed as the first supercomputer as it outperformed all other contemporary computers. Its processing speed was about 10 MHz. In 1968, Seymour Cray built CDC 7600. Its processing speed was 35 MHz. Again, it was the fastest computer and outran all other computers in terms of computing power.
This was the genesis of supercomputers. Eventually, Cray left CDC and formed his own company for designing and developing supercomputers. Cray created some of the most successful supercomputers in history, namely Cray 1, Cray X-MP, Cray 2, and Cray Y-MP. The 1990s saw the era of massively parallel supercomputers with thousands of processors connected to each other in various configurations. A notable example of this is the Intel Paragon which could have many Intel i860 processors.
The speed of a supercomputer is measured in Floating Point Operations Per Second (FLOPS) instead of MIPS (Million Instructions Per Second). Intel ASCI Red was the first TFLOPS (Tera FLOPS ) supercomputer. In 2008, IBM Roadrunner became the first supercomputer with the speed of PFLOPS (Peta FLOPS ).
The next breakthrough in the area of supercomputing will be the Exascale supercomputer with the processing speed to be measured in Exa-FLOPS.
I am not providing a list here of the top 10 supercomputers or the fastest supercomputers. This is because the list keeps on changing every year. Also, supercomputers are ranked based on various parameters, so no ranking from different sources based on diverse parameters will ever be the same.
Cluster
While designing massively parallel computing systems, generally two approaches are followed. The first one is to involve thousands of computers spread across a wide geographical area using the Internet to solve a particular problem. This works well over a wide area network like the Internet. These types of systems are called distributed systems. Another approach to this is to place thousands of processing nodes in close proximity to each other. This saves a lot of time on communication and most of the processing power is used to solve the computationally massive problem. This approach is known as clustering. All supercomputers fall in this category.
A computer cluster is defined as a group of loosely or tightly coupled computers that work together. The computers in a cluster are known as nodes. All the nodes in a cluster do exactly same type of task.
The mini-supercomputer we are going to develop is going to be a cluster of Pis. All supercomputers are clusters, but all clusters are not supercomputers. As we have learned in the definition of the supercomputer, supercomputers have massive processing power. That’s why every cluster does not qualify to be called a supercomputer. The cluster we will build here is not a supercomputer as it pales before a real-world supercomputer in terms of processing power, but it works on the same principle as a real-world supercomputer. Hence we will call it a mini-supercomputer. Ever since the introduction of massively parallel systems, the boundaries between large-scale clusters and less powerful computers have begun to fade. Few homemade clusters today are as powerful as 1980s supercomputers. Based on their configuration, commodity clusters are classified into the following two categories.
Heterogenous Cluster
When all the nodes of the cluster do not have the exactly same hardware configuration, then the cluster is called a heterogeneous cluster. When making my cluster, I used two units of Pi B+, one unit of Pi 2, and one unit of Pi 3, so my cluster is a heterogeneous cluster.
Beowulf Cluster
Unlike the heterogeneous clusters, all the nodes in a Beowulf cluster have exactly the same configuration. We can make homogeneous and Beowulf clusters out of commodity-grade hardware as well as SBCs like Raspberry Pi. Almost all clusters use some distribution of Linux as the OS for their nodes.
Depending on the availability of the models of Pi near you, you can make either a heterogeneous or a Beowulf cluster.
Parallelism and Concurrency
Let’s explore a few important terms in the area of supercomputing.
Parallelism
Parallelism means the computational tasks are carried out in parallel. This means that the tasks are carried out simultaneously (at the same time). Parallelism is usually employed in cases where the computational problems are very large. Large problems are often divided into smaller sub-problems and then solved parallelly by the computers. With the introduction of multi-core processors, execution of parallel programs is supported by the hardware itself. Another way to run parallel programs is to create parallel systems by using multiple different computers. Parallel is an antonym of serial which means in series and one after the other. Parallelism is closely related to another term, concurrency.
Let me explain parallelism in simple words. Suppose there are two jobs to be completed and two people are available to take up the jobs. Both the people are assigned one job each and they start working independently of each other. This is known as parallelism.
Concurrency
Concurrency means many computational tasks are concurrently progressing. It is not necessary for the tasks to progress at the same time. In parallelism, all the tasks are executing at the same time. In concurrency, they need not. A concurrent system is one where a computation can advance without waiting for all other computations to complete, and more than one computation can advance at the same time. The best example of concurrency is the process scheduling in the OS.
Let me explain concurrency in simple words. Suppose two jobs are to be completed and only one person is available to do all the work. The person decides to start with the first job. He does 30% of it and then switches to the second job. He completes 40% of the second job and switches back to the first job. This type of switch happens multiple times. We can say that the work on both the jobs is in progress. Although the jobs are not done simultaneously, the jobs are progressing towards completion. In the end, both jobs are finished. Concurrent is an antonym of sequential.
Parallel Programming
All clusters and supercomputers use parallelism to decompose a computationally massive task into smaller chunks and then collect the results back for the final output. The programming paradigm which supports this type of operation is called parallel programming. Message Passing Interface (MPI) is one of the most-used parallel programming standards in industry and academics. We will study how to install it on the Pi for Python 3 in the next chapter.
Conclusion
In this short chapter, we learned a few important concepts related to supercomputing and we also studied the history of supercomputers.
In the next chapter, we will study how to set up a node of a Raspberry Pi cluster.