Introduction to Transactions

As we mentioned above, a transaction is a logical unit of processing in a database that includes one or more database operations, which can be read or write operations. There are situations where your application may require reads and writes to multiple documents (in one or more collections) as part of this logical unit of processing. An important aspect of a transaction is that it is never partially completed—it either succeeds or fails.

How to Use Transactions

MongoDB provides two APIs to use transactions. The first is a similar syntax to relational databases (e.g., start_transaction and commit_transaction) called the core API and the second is called the callback API, which is the recommended approach to using transactions.

The core API does not provide retry logic for the majority of errors and requires the developer to code the logic for the operations, the transaction commit function, and any retry and error logic required.

The callback API provides a single function that wraps a large degree of functionality when compared to the core API, including starting a transaction associated with a specified logical session, executing a function supplied as the callback function, and then committing the transaction (or aborting on error). This function also includes retry logic that handle commit errors. The callback API was added in MongoDB 4.2 to simplify application development with transactions as well as make it easier to add application retry logic to handle any transaction errors.

In both APIs, the developer is responsible for starting the logical session that will be used by the transaction. Both APIs require operations in a transaction to be associated with a specific logical session (i.e., pass in the session to each operation). A logical session in MongoDB tracks the time and sequencing of the operations in the context of the entire MongoDB deployment. A logical session or server session is part of the underlying framework used by client sessions to support retryable writes and causal consistency in MongoDB—both of these features were added in MongoDB version 3.6 as part of the foundation required to support transactions. A specific sequence of read and write operations that have a causal relationship reflected by their ordering is defined as a causally consistent client session in MongoDB. A client session is started by an application and used to interact with a server session.

In 2019, six senior engineers from MongoDB published a paper at the SIGMOD 2019 conference entitled “Implementation of Cluster-wide Logical Clock and Causal Consistency in MongoDB”.¹ This paper provides a deeper technical explanation of the mechanics behind logical sessions and causal consistency in MongoDB. The paper documents the efforts from a multiteam, multiyear engineering project. The work involved changing aspects of the storage layer, adding a new replication consensus protocol, modifying the sharding architecture, refactoring sharding cluster metadata, and adding a global logical clock. These changes provide the foundation required by the database before ACID-compliant transactions can be added.

The complexity and additional coding required in applications are the main reasons to recommend the callback API over the core API. These differences between the APIs are summarized in Table 8-1.

To understand the differences between these two APIs, we can compare the APIs using a simple transaction example for an ecommerce site where an order is placed and the corresponding items are removed from the available stock as they are sold. This involves two documents in different collections in a single transaction. The two operations, which will be the core of our transaction example, are:

    orders.insert_one({"sku": "abc123", "qty": 100}, session=session)
    inventory.update_one({"sku": "abc123", "qty": {"$gte": 100}},
                         {"$inc": {"qty": -100}}, session=session)

First, let’s see how the core API can be used in Python for our transaction example. The two operations of our transaction are highlighted in Step 1 of the program listing below:

# Define the uriString using the DNS Seedlist Connection Format 
# for the connection
uri = 'mongodb+srv://server.example.com/'
client = MongoClient(uriString)

my_wc_majority = WriteConcern('majority', wtimeout=1000)

# Prerequisite / Step 0: Create collections, if they don't already exist. 
# CRUD operations in transactions must be on existing collections.

client.get_database( "webshop",
                     write_concern=my_wc_majority).orders.insert_one({"sku":
                     "abc123", "qty":0})
client.get_database( "webshop",
                     write_concern=my_wc_majority).inventory.insert_one(
                     {"sku": "abc123", "qty": 1000})

# Step 1: Define the operations and their sequence within the transaction
def update_orders_and_inventory(my_session):
    orders = session.client.webshop.orders
    inventory = session.client.webshop.inventory


    with session.start_transaction(
            read_concern=ReadConcern("snapshot"),
            write_concern=WriteConcern(w="majority"),
            read_preference=ReadPreference.PRIMARY):

        orders.insert_one({"sku": "abc123", "qty": 100}, session=my_session)
        inventory.update_one({"sku": "abc123", "qty": {"$gte": 100}},
                             {"$inc": {"qty": -100}}, session=my_session)
        commit_with_retry(my_session)

# Step 2: Attempt to run and commit transaction with retry logic
def commit_with_retry(session):
    while True:
        try:
            # Commit uses write concern set at transaction start.
            session.commit_transaction()
            print("Transaction committed.")
            break
        except (ConnectionFailure, OperationFailure) as exc:
            # Can retry commit
            if exc.has_error_label("UnknownTransactionCommitResult"):
                print("UnknownTransactionCommitResult, retrying "
                      "commit operation ...")
                continue
            else:
                print("Error during commit ...")
                raise

# Step 3: Attempt with retry logic to run the transaction function txn_func
def run_transaction_with_retry(txn_func, session):
    while True:
        try:
            txn_func(session)  # performs transaction
            break
        except (ConnectionFailure, OperationFailure) as exc:
            # If transient error, retry the whole transaction
            if exc.has_error_label("TransientTransactionError"):
                print("TransientTransactionError, retrying transaction ...")
                continue
            else:
                raise

# Step 4: Start a session.
with client.start_session() as my_session:

# Step 5: Call the function 'run_transaction_with_retry' passing it the function
# to call 'update_orders_and_inventory' and the session 'my_session' to associate
# with this transaction.

    try:
        run_transaction_with_retry(update_orders_and_inventory, my_session)
    except Exception as exc:
        # Do something with error. The error handling code is not
        # implemented for you with the Core API.
        raise

Now, let’s look at how the the callback API can be used in Python for this same transaction example. The two operations of our transaction are highlighted in Step 1 of the program listing below:

# Define the uriString using the DNS Seedlist Connection Format 
# for the connection
uriString = 'mongodb+srv://server.example.com/'
client = MongoClient(uriString)

my_wc_majority = WriteConcern('majority', wtimeout=1000)

# Prerequisite / Step 0: Create collections, if they don't already exist.
# CRUD operations in transactions must be on existing collections.

client.get_database( "webshop",
                     write_concern=my_wc_majority).orders.insert_one({"sku":
                     "abc123", "qty":0})
client.get_database( "webshop",
                     write_concern=my_wc_majority).inventory.insert_one(
                     {"sku": "abc123", "qty": 1000})

# Step 1: Define the callback that specifies the sequence of operations to
# perform inside the transactions.

def callback(my_session):
    orders = my_session.client.webshop.orders
    inventory = my_session.client.webshop.inventory

    # Important:: You must pass the session variable 'my_session' to 
    # the operations.

    orders.insert_one({"sku": "abc123", "qty": 100}, session=my_session)
    inventory.update_one({"sku": "abc123", "qty": {"$gte": 100}},
                         {"$inc": {"qty": -100}}, session=my_session)

#. Step 2: Start a client session.

with client.start_session() as session:

# Step 3: Use with_transaction to start a transaction, execute the callback,
# and commit (or abort on error).

    session.with_transaction(callback,
                             read_concern=ReadConcern('local'),
                             write_concern=my_write_concern_majority,
                             read_preference=ReadPreference.PRIMARY)
}

Tuning Transaction Limits for Your Application

There are a few parameters that are important to be aware of when using transactions. They can be adjusted to ensure your application can make the optimal use of transactions.