The pickle module, from the Python standard library, offers tools to convert Python objects into byte streams, and vice versa. Even though there is a partial overlap in the API that pickle and json expose, the two are quite different. As we have seen previously in this chapter, JSON is a text format, human readable, language independent, and supports only a restricted subset of Python data types. The pickle module, on the other hand, is not human readable, translates to bytes, is Python specific, and, thanks to the wonderful Python introspection capabilities, it supports an extremely large amount of data types.

Regardless of these differences, though, which you should know when you consider whether to use one or the other, I think that the most important concern regarding pickle lies in the security threats you are exposed to when you use it. Unpickling erroneous or malicious data from an untrusted source can be very dangerous, so if you decide to adopt it in your application, you need to be extra careful.

That said, let's see it in action, by means of a simple example:

# persistence/pickler.py
import pickle
from dataclasses import dataclass

@dataclass
class Person:
    first_name: str
    last_name: str
    id: int

    def greet(self):
        print(f'Hi, I am {self.first_name} {self.last_name}'
              f' and my ID is {self.id}'
        )

people = [
    Person('Obi-Wan', 'Kenobi', 123),
    Person('Anakin', 'Skywalker', 456),
]

# save data in binary format to a file
with open('data.pickle', 'wb') as stream:
    pickle.dump(people, stream)

# load data from a file
with open('data.pickle', 'rb') as stream:
    peeps = pickle.load(stream)

for person in peeps:
    person.greet()

In the previous example, we create a Person class using the dataclass decorator, which we have seen in Chapter 6, OOP, Decorators, and Iterators. The only reason I wrote this example with a data class is to show you how effortlessly pickle deals with it, with no need for us to do anything we wouldn't do for a simpler data type.

The class has three attributes: first_name, last_name, and id. It also exposes a greet method, which simply prints a hello message with the data.

We create a list of instances, and then we save it to a file. In order to do so, we use pickle.dump, to which we feed the content to be pickled, and the stream to which we want to write. Immediately after that, we read from that same file, and by using pickle.load, we convert back into Python the whole content of that stream.

Just to make sure that the objects have been converted correctly, we call the greet method on both of them. The result is the following:

$ python pickler.py
Hi, I am Obi-Wan Kenobi and my ID is 123
Hi, I am Anakin Skywalker and my ID is 456  

The pickle module also allows you to convert to (and from) byte objects, by means of the dumps and loads functions (note the s at the end of both names). In day-to-day applications, pickle is usually used when we need to persist Python data that is not supposed to be exchanged with another application. One example I stumbled upon recently was the session management in a flask plugin, which pickles the session object before sending it to Redis. In practice, though, you are unlikely to have to deal with this library very often.

Another tool that is possibly used even less, but that proves to be very useful when you are short of resources, is shelve.