Cover image for Parallel Programming with Python

Parallel Programming with Python

Author Jan Palach
Release Date: 2014/06/01
ISBN: 9781783288397
Topic:
0%
57
Chapters
0-1
Hours read
0k
Total Words
    Start Reading Now    
       Add to Wishlist       
View table of contents

Book Description

Develop efficient parallel systems using the robust Python environment

In Detail

Starting with the basics of parallel programming, you will proceed to learn about how to build parallel algorithms and their implementation. You will then gain the expertise to evaluate problem domains, identify if a particular problem can be parallelized, and how to use the Threading and Multiprocessor modules in Python.

The Python Parallel (PP) module, which is another mechanism for parallel programming, is covered in depth to help you optimize the usage of PP. You will also delve into using Celery to perform distributed tasks efficiently and easily. Furthermore, you will learn about asynchronous I/O using the asyncio module. Finally, by the end of this book you will acquire an in-depth understanding about what the Python language has to offer in terms of built-in and external modules for an effective implementation of Parallel Programming.

This is a definitive guide that will teach you everything you need to know to develop and maintain high-performance parallel computing systems using the feature-rich Python.

What You Will Learn

  • Explore techniques to parallelize problems
  • Integrate the Parallel Python module to implement Python code
  • Execute parallel solutions on simple problems
  • Achieve communication between processes using Pipe and Queue
  • Use Celery Distributed Task Queue
  • Implement asynchronous I/O using the Python asyncio module
  • Create thread-safe structures

Downloading the example code for this book. You can download the example code files for all Packt books you have purchased from your account at http://www.PacktPub.com. If you purchased this book elsewhere, you can visit http://www.PacktPub.com/support and register to have the files e-mailed directly to you.

Table of Contents

  1. Parallel Programming with Python
    1. Table of Contents
    2. Parallel Programming with Python
    3. Credits
    4. About the Author
    5. Acknowledgments
    6. About the Reviewers
    7. www.PacktPub.com
      1. Support files, eBooks, discount offers, and more
        1. Why subscribe?
        2. Free access for Packt account holders
    8. Preface
      1. What this book covers
      2. What you need for this book
      3. Who this book is for
      4. Conventions
      5. Reader feedback
      6. Customer support
        1. Downloading the example code
        2. Errata
        3. Piracy
        4. Questions
    9. 1. Contextualizing Parallel, Concurrent, and Distributed Programming
      1. Why use parallel programming?
      2. Exploring common forms of parallelization
      3. Communicating in parallel programming
        1. Understanding shared state
        2. Understanding message passing
      4. Identifying parallel programming problems
        1. Deadlock
        2. Starvation
        3. Race conditions
      5. Discovering Python's parallel programming tools
        1. The Python threading module
        2. The Python multiprocessing module
        3. The parallel Python module
        4. Celery – a distributed task queue
      6. Taking care of Python GIL
      7. Summary
    10. 2. Designing Parallel Algorithms
      1. The divide and conquer technique
      2. Using data decomposition
      3. Decomposing tasks with pipeline
      4. Processing and mapping
        1. Identifying independent tasks
        2. Identifying the tasks that require data exchange
        3. Load balance
      5. Summary
    11. 3. Identifying a Parallelizable Problem
      1. Obtaining the highest Fibonacci value for multiple inputs
      2. Crawling the Web
      3. Summary
    12. 4. Using the threading and concurrent.futures Modules
      1. Defining threads
        1. Advantages and disadvantages of using threads
        2. Understanding different kinds of threads
        3. Defining the states of a thread
        4. Choosing between threading and _thread
      2. Using threading to obtain the Fibonacci series term with multiple inputs
      3. Crawling the Web using the concurrent.futures module
      4. Summary
    13. 5. Using Multiprocessing and ProcessPoolExecutor
      1. Understanding the concept of a process
        1. Understanding the process model
          1. Defining the states of a process
      2. Implementing multiprocessing communication
        1. Using multiprocessing.Pipe
        2. Understanding multiprocessing.Queue
      3. Using multiprocessing to compute Fibonacci series terms with multiple inputs
      4. Crawling the Web using ProcessPoolExecutor
      5. Summary
    14. 6. Utilizing Parallel Python
      1. Understanding interprocess communication
        1. Exploring named pipes
        2. Using named pipes with Python
          1. Writing in a named pipe
          2. Reading named pipes
      2. Discovering PP
      3. Using PP to calculate the Fibonacci series term on SMP architecture
      4. Using PP to make a distributed Web crawler
      5. Summary
    15. 7. Distributing Tasks with Celery
      1. Understanding Celery
        1. Why use Celery?
      2. Understanding Celery's architecture
        1. Working with tasks
        2. Discovering message transport (broker)
        3. Understanding workers
        4. Understanding result backends
      3. Setting up the environment
        1. Setting up the client machine
        2. Setting up the server machine
      4. Dispatching a simple task
      5. Using Celery to obtain a Fibonacci series term
      6. Defining queues by task types
      7. Using Celery to make a distributed Web crawler
      8. Summary
    16. 8. Doing Things Asynchronously
      1. Understanding blocking, nonblocking, and asynchronous operations
        1. Understanding blocking operations
        2. Understanding nonblocking operations
        3. Understanding asynchronous operations
      2. Understanding event loop
        1. Polling functions
        2. Using event loops
      3. Using asyncio
        1. Understanding coroutines and futures
          1. Using coroutine and asyncio.Future
          2. Using asyncio.Task
          3. Using an incompatible library with asyncio
      4. Summary
    17. Index
18.221.239.148