Multithreaded mergesort

In this example, we amend the sort function once again, so that, after the initial division into chunks, it spawns a thread per part. Each thread uses the single-threaded version of the algorithm to sort its part, and then at the end we use the multi-merge technique to calculate the final result. Translating into Python:

# ms/algo/mergesort_thread.py
from functools import reduce
from math import ceil
from concurrent.futures import ThreadPoolExecutor, as_completed
from .mergesort import sort as _sort, merge

def sort(v, workers=2):
if len(v) == 0:
return v
dim = ceil(len(v) / workers)
chunks = (v[k: k + dim] for k in range(0, len(v), dim))
with ThreadPoolExecutor(max_workers=workers) as executor:
futures = [
executor.submit(_sort, chunk) for chunk in chunks
]
return reduce(
merge,
(future.result() for future in as_completed(futures))
)

We import all the required tools, including executors, the ceiling function, and sort and merge from the single-threaded version of the algorithm. Notice how I changed the name of the single-threaded sort into _sort upon importing it.

In this version of sort, we check whether v is empty first, and if not we proceed. We calculate the dimension of each chunk using the ceil function. It's basically doing what we were doing with max in the previous snippet, but I wanted to show you another way to solve the issue.

When we have the dimension, we calculate the chunks and prepare a nice generator expression to serve them to the executor. The rest is straightforward: we define a list of future objects, each of which is the result of calling submit on the executor. Each future object runs the single-threaded _sort algorithm on the chunk it has been assigned to.

Finally as they are returned by the as_completed function, the results are merged using the same technique we saw in the earlier multipart example.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.144.202.167