Parallel and distributed computing are similar technologies designed to increase the amount of processing power available for a specific task. Generally, these methods are used to solve problems that require large computational capabilities.
When the problem is divided into many small pieces, individual sections of the problem can be calculated by many processors simultaneously. This allows more processing power to be exercised on the problem than can be provided by a single processor.
The main difference between parallel and distributed processing is that parallel configurations include many processors within a single system, while distributed configurations exploit the processing power of many computers simultaneously.
Let's look at the other differences:
| Parallel processing | Distributed processing |
| Parallel processing has the advantage of providing reliable processing power with a very low degree of latency. | Distributed processing is not extremely efficient on a processor-by-processor basis, as the data must travel over the network rather than through the internal connections of a single system. |
| By concentrating all the processing power in one system, speed loss due to data transfer is minimized. | Each processor will contribute much less processing power than any processor in a parallel system since data transfer creates a bottleneck that limits processing power. |
| The only real limit is the number of processors incorporated in the system. | The system is almost infinitely scalable since there is no actual upper limit to the number of processors in a distributed system. |
However, in the context of computer applications, it is customary to distinguish between local and distributed architectures:
| Local architectures | Distributed architectures |
| All the components are on the same machine. | Applications and components can reside on different nodes that are connected by a network. |
The advantages of using distributed computing consist mainly of the possibility of concurrent use of the programs, the centralization of the data, and the distribution of the processing load, which all come at the price of greater complexity, especially with communication between the various components.