Distributed task management with Celery

Celery is a Python framework that manages distributed tasks by following the object-oriented middleware approach. Its main feature is handling many small tasks and distributing them on many computational nodes. Finally, the result of each task will then be reworked in order to compose the overall solution.

To use Celery, a message broker is required. This is an independent (from Celery) software component that has the function of middleware, which is used to send and receive messages to distributed task workers.

In fact, a message broker—also known as message middleware—deals with the exchange of messages in a communication network.: the addressing scheme of this type of middleware is no longer of the point-to-point type, but is message-oriented addressing.

The reference architecture, with which the message broker manages the exchange of messages, is based on the so-called publish/subscribe paradigm, which is depicted as follows:

Message broker architecture

Celery supports many types of brokers. However, the more complete ones are RabbitMQ and Redis.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.144.84.155