One additional functionality offered by APScheduler is the ability to distribute scheduled jobs to be executed across multiple CPU cores (or processes). In this example, you will learn how to do that with a background scheduler. Navigate to the Chapter19/example3.py file and inspect the included code, as follows:

# Chapter19/example3.py

from datetime import datetime
import time
import os

from apscheduler.schedulers.background import BackgroundScheduler

def task():
    print(f'From process {os.getpid()}: The time is {datetime.now()}')
    print(f'Starting job inside {os.getpid()}')
    time.sleep(4)
    print(f'Ending job inside {os.getpid()}')

if __name__ == '__main__':
    scheduler = BackgroundScheduler()
    scheduler.add_executor('processpool')
    scheduler.add_job(task, 'interval', seconds=3, max_instances=3)
    scheduler.start()

    try:
        while True:
            time.sleep(1)
    except KeyboardInterrupt:
        pass

scheduler.shutdown()

In this program, the job that we would like to schedule (the task() function) prints out the identifier of the process that is running it at each call (using the os.getpid() method) and is designed to last for around four seconds. In the main program, we are using the same background scheduler we used in the last example, but we are specifying that scheduled jobs should be executed in a pool of processes:

scheduler.add_executor('processpool')

Remember that the default value of the number of processes in this pool is 10, and can be changed to a different value. Next, as we add the job to the scheduler, we also have to specify that this job can be executed in more than one process instance (in this case, three instances); this allows our process pool executor to be utilized fully and efficiently:

scheduler.add_job(task, 'interval', seconds=3, max_instances=3)

The first few lines of my output, after running the program, are as follows:

> python3 example3.py
From process 1213: The time is 2018-11-01 10:18:00.559319
Starting job inside 1213
From process 1214: The time is 2018-11-01 10:18:03.563195
Starting job inside 1214
Ending job inside 1213
From process 1215: The time is 2018-11-01 10:18:06.531825
Starting job inside 1215
Ending job inside 1214
From process 1216: The time is 2018-11-01 10:18:09.531439
Starting job inside 1216
Ending job inside 1215
From process 1217: The time is 2018-11-01 10:18:12.531940
Starting job inside 1217
Ending job inside 1216
From process 1218: The time is 2018-11-01 10:18:15.533720
Starting job inside 1218
Ending job inside 1217
From process 1219: The time is 2018-11-01 10:18:18.532843
Starting job inside 1219
Ending job inside 1218
From process 1220: The time is 2018-11-01 10:18:21.533668
Starting job inside 1220
Ending job inside 1219
From process 1221: The time is 2018-11-01 10:18:24.535861
Starting job inside 1221
Ending job inside 1220
From process 1222: The time is 2018-11-01 10:18:27.531543
Starting job inside 1222
Ending job inside 1221
From process 1213: The time is 2018-11-01 10:18:30.532626
Starting job inside 1213
Ending job inside 1222
From process 1214: The time is 2018-11-01 10:18:33.534703
Starting job inside 1214
Ending job inside 1213

As you can see from the printed process identifiers, the scheduled job was being executed in different processes. You will also notice that the ID of the first process was 1213, and, as soon as our scheduler started to use the process with the ID of 1222, it then switched back to the 1213 process (notice the last few lines of the preceding output). This is because our process pool contains 10 workers, and the 1222 process was the last element of the pool.