Before you start rewriting your code

Don't rush into rewriting your code in a concurrent fashion just yet. There are plenty of reasons not to write concurrent code. Let's look at a few reasons here:

  • First of all, don't do it if you don't need that boostany type of concurrency adds code complexity and makes debugging exponentially harder.
  • Second, the code for many specific computation-demanding tasks is already written. For example, multiple sklearn models support multicore executionyou just need to specify the number of CPUs to use. Some solutions, such as Numba, can release the GIL for specific operations, without large code changes being made. 
  • Some important packages do not support concurrent operationsasynchronous execution in particularsuch as sqlalchemy and most database access tools in general.

All in all, make sure you really need your code to run concurrently or in parallel before investing your time and effort. As cool as it sounds, concurrent code is notoriously difficult and takes significantly more time to develop, optimize, and maintain.

If you want to get a deeper understanding of concurrency in Python (which is a very wide topic), we can recommend the following resources:

PEP 554 (currently in draft status) proposes to use sub interpreters (isolated instances, controlled by the main interpreter process) to allow better multiprocessing without GIL getting in the way. To learn more about this proposal, read the PEP: https://www.python.org/dev/peps/pep-0554/#about-subinterpreters.

Speaking of maintenance, let's talk about the other side of the same performance coin apart from code performance, there is coding performance. On many occasions, the ability to write code faster, add changes quickly, and introduce fewer bugs could be even more valuable than the speed of code itself. Thus, let's talk about best practices when it comes to coding and tools that will help you be a better developer.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.137.198.96