Numerous load balancing algorithms exist for a variety of problems:

• Global algorithms: These require global knowledge of the computation being performed, which often adds a lot of overhead.
• Local algorithms: These rely only on information that is local to the task in question, which reduces overhead compared to global algorithms, but they are usually worse at finding optimal agglomeration and mapping.

However, the reduced overhead may reduce the execution time, even though the mapping is worse by itself. If the tasks rarely communicate other than at the start and end of the execution, then a task-scheduling algorithm is often used, which simply maps tasks to processors as they become idle. In a task-scheduling algorithm, a task pool is maintained. Tasks are placed in this pool and are taken from it by workers.

There are three common approaches in this model:

• Manager/worker: This is the basic dynamic mapping scheme in which all the workers connect to a centralized manager. The manager repeatedly sends tasks to the workers and collects the results. This strategy is probably the best for a relatively small number of processors. The basic strategy can be improved by fetching tasks in advance so that communication and computation overlap each other.
• Hierarchical manager/worker: This is the variant of a manager/worker that has a semi-distributed layout. Workers are split into groups, each with their own manager. These group managers communicate with the central manager (and possibly among themselves as well), while workers request tasks from the group managers. This spreads the load among several managers and can, as such, handle a larger number of processors if all workers request tasks from the same manager.
• Decentralize: In this scheme, everything is decentralized. Each processor maintains its own task pool and communicates with the other processors in order to request tasks. How the processors choose other processors to request tasks varies and is determined on the basis of the problem.