Unit of Change and Unit of Consistency

An aggregate is a unit of change and a unit of consistency.

A unit of change will be every entity in your database model that can handle all requested changes without a need to go to other documents. In other words, changes we make are propagating within one aggregate.

The second important property of an aggregate is that they represent a unit of consistency . A consistent database will, at any point in time, contain only valid data. All information stored in a database will conform to all business rules, constraints, and validation checks. No matter what changes we apply to our aggregates, if we use our validation rules, they will remain valid when saved to the database.

Going back to our ubiquitous Order example, if we add one more line to the Lines collection, we also need to update the Total property by recalculating the new total value of our Order. If it contains some more properties, like VAT or shipping costs, they will need to be updated. The system we are building can have some more advanced rules – e.g., we might offer a volume discount for shipping overseas. In that case, adding just one more order line will trigger a set of updates, rule checks, validation checks, and some more updates resulting from various calculations. After applying all updates, we will persist Order in the database in an atomic manner; our Order aggregate contains a set of units that changed together. This changeset was validated to conform to all business and other validation rules. We are now sure that our database holds Order that contains only valid data.

Distributed Systems

However, a database is just one component of the software system we are building. Taking a broader look, we will see that modern applications are complex. They consist of many components, coming in various forms – from utility classes, over in-process services to physically separated services and executing across a network of interconnected machines. Services are communicating synchronously or asynchronously.

These are three isolated components existing on separated machines, communicating over the network. A system composed in this way is considered to be a distributed system.

Sending confirmation email, creating a new Order, and pushing it into the ERP system are one transactional operation from the perspective of your business logic. However, in a distributed system like this one is, the standard notion of transaction you are used to does not apply anymore. For example, what happens if the ERP system is not available when you try to send it new Order? Shall you abandon it? In that scenario, when will we send a confirmation email?

Distributed systems are challenging because they are complex and because many assumptions you usually make do not hold anymore. Operations you were executing as synchronous ones before might be asynchronous in the distributed system. Each time you communicate over the network, you must provide a fallback for the scenario of communication failure. Messages sent between distributed components will sometimes be queued and delivered dozens of seconds, maybe even minutes after you initially send them.

As a result, in most cases, in distributed systems, you will not be able to count on the transactional nature of your applications. Distributed systems have a notion of eventual consistency. Your business transaction will eventually complete, but there are no guarantees how long it will take for changes you ordered to be delivered and accepted by the components of the system. At the end of this propagation process , your distributed system will become consistent, but this fact is something that you must be aware of all the time.

Aggregates in the Distributed System

Revisiting the notion of the aggregate in the context of distributed systems reveals a significant impact on the design of aggregate boundaries. Namely, the scope of aggregate boundaries will determine units of data that will be part of it. Coming back to the fact that an aggregate is a unit of change and a unit of consistency, we can conclude that modifying and saving aggregate into the database will represent one transaction.

What is a transaction? The transaction represents an individual indivisible operation that either succeeds or fails as a complete unit. Partial completeness of the transaction is not possible, and when transaction processing is finished, you will have precise and reliable feedback about the outcome. This property is usually called atomicity, precisely because of the indivisible nature of the transaction.

What would happen if just one of these two operations is executed? You would either charge for a product that would not be included in the Order or have the product in the Order without increasing the Total. In other words, you would either create financial damage to yourself or your customer.

Hence, it is clear that we need to treat both of these operations as one atomic unit. As a result, both of them will succeed (new order line is created and Total is correctly updated), or both fail (order total remains the same since no new order lines were added). Transaction failure is nothing dramatic per se. You can handle it in several standard ways, including retrying it, before showing an error message. However, what is essential is that in both cases of successful or unsuccessful outcomes, your data will remain in a consistent state.

Aggregates as Transaction Boundaries

We can conclude that aggregates are supporting transactions. We will load a Document that represents aggregate, update it, and save it back to the database in a transactional manner on a database level, which results in a guarantee of consistency of our Document.

You can ask yourself – why can we not update two documents in one transaction? This is possible indeed. As we already mentioned, modeling is a subtle intellectual activity. There are no complex rules, and everything you read so far is a recommendation based on best practices learned through the practical application of these principles. So, updating two aggregate documents in one transaction is possible. However, recommended approach would be to group things that change together transactionally into one aggregate and then propagate changes through the rest of the system in an eventually consistent manner. Also, on many occasions, you will be forced by the nature of your distributed system to go with eventual consistency.

One typical example would be a separation of your application into microservices. Every microservice will usually have its database. Transactions that are spanning two or more aggregates hosted within different databases are unviable and hard. This approach will create more problems than benefits for you as you develop, maintain, and evolve such application.

Hence, it would help if you looked at the boundaries of your aggregate as transactional boundaries. For most of the changes in your application, you will have to load, change, and save just one Document – a document that represents your aggregate.

Everything else that happens in your system will not be transactional but eventually consistent. You should take this into account and implement error handling mechanisms. Also, you should carefully approach every single business scenario in your application and determine when synchronous communication is a must and when you can use asynchronous communication.

Embedding

With both age and shipping address, we applied a technique called embedding . Their separate existence was not justified in both cases, and we inserted these small documents inside bigger documents representing entities.

Referencing

These two models are simplified, but they are showing basic info. Looking closely at the Resume model, we can see property Employee: “employees/1” directly linking Resume to an Employee.

This is a reference. References in RavenDB are nothing more than properties containing the string ID of some other document.

What we now have is Employee holding a reference to a Resume: Resume: "resumes/1". This reference can be read as owns – Employee owns a Resume.

Employee and Resume are now pointing at each other via properties containing the other Document’s ID. In an application displaying a screen with Employee details, we can easily offer the Resume screen because Employee holds a reference to his CV. Similarly, the screen showing Resume can offer a link leading to an accompanied Employee since we can easily access reference from the Employee property.

In this case, we embedded not a Value Object without identity but an Entity with a clear identity. This is possible due to a clear ownership situation – Employee has exactly one Resume, and every Resume belongs to a specific Employee.

This fourth approach to modeling one-to-one relationships is entirely legitimate. It is up to you to examine all possible directions, think about processes and workflows in your application, and determine acceptable modeling solutions.

Also, it would be best if you thought about the future. What will be possible ways your application will evolve? What might be requests coming from your users? What will happen as the amount of data grows or if your documents are growing?

With this in mind, let’s observe the modeling variant of Employee presented in Listing 2-5. For simplicity purposes, we intentionally made both Employee and Resume artificially oversimplified. However, if you recall that real-life Resumes can be pretty big, sometimes three or four pages, you will realize that you will embed a substantially extensive Document. Not only that, in the scenario where Resume contains a list of projects Employee participated in, you will see Resume growing over months and years. If you need to load, manipulate, and save Employee to the database, that means that the size of the Employee document will grow over time.

Furthermore, suppose you decide to apply same technique and embed some more documents in the same manner. In that case, you will inevitably see your Employee document becoming very large and very cumbersome to manipulate.

Finally, what will happen if requirements change and company management decided to introduce more than one Resume for every Employee – scenario where the Company applies for contracts and, as a part of a bid campaign, generates custom-tailored Resume of all project members. This will break your one-to-one modeling, which will transform into one-to-many (one Employee can have many Resumes). How much effort will it take to modify the database model and your code to accommodate this change request? Will a straightforward sentence like “we want to enable one Employee to have more than one Resume in our database” result in an inappropriately large amount of work?

All of this is showing that modeling is intricate. You will never be in a situation where you complete the model and declare it as “best possible.” Hence, you should never strive for perfection and “one right way.” Also, it would help if you never tried to anticipate the future and then build all variants you can foresee as a part of your domain model. What you should do instead is to make the simplest possible model that is covering present needs but also does not obstruct change requests that will be a normal and expected part of your application life cycle.

Referential Integrity in RavenDB

In this example, we created a reference to a nonexistent Resume. There was no mechanism on the database level to prevent us from introducing data corruption. Hence, it is natural to ask – relational databases were protecting us from data corruption at the database level, is this behavior exposed by RavenDB a step back? Why would I want to work with a database that is not safeguarding me from making mistakes that would corrupt my data?

Additionally, with JSON format , you will not be able to specify a type of specific property. Hence, if you have a property “age” of type integer, RavenDB will not prevent you from storing “John” in it. With a relational database, such an attempt will result in an error. So, how to explain this? Why would you go with a database that would make your application less reliable and error-prone? With a database that would let you introduce corruption in your data?

Yes, looking at the database as an isolated part of your system, that is true indeed. Relational databases like SQL Server or PostgreSQL will provide you with a way to strongly type columns and to check referential integrity, preventing you from establishing a reference to a nonexistent entity. However, observing databases in the broader context of data persistence gives a significantly different angle.

Unlike the 1980s and 1990s, today’s modeling methodologies (such as domain-driven design) are not centered around databases as a starting point in the modeling process. Instead, they recommend modeling the business domain in the programming language and then persisting domain entities into the database. In the past, most planning meetings were starting in front of the whiteboard, where developers would draw relational database tables, fields, and lots of Foreign Keys establishing relationships between them. This process looked very similar to UML diagrams construction, except that the database was “center of the world” and whole systems planning started with modeling a database. Following persistence model, entities in the programming language were modeled, along with CRUD operations to manage these entities.

From the perspective of database “centrism ,” offloading validation to the database is something you are naturally striving for. However, as domain modeling techniques advanced and developed, we reached the present day where modeling is done at the conceptual level of Aggregates, Entities, and Value Objects. We perform analysis of the processes, divide enterprise projects into subdomains, and conduct modeling at the scope of a subdomain. This activity is performed to look purely at the business and process aspect of our subdomain, keeping in mind that artifacts of this activity will have to be eventually persisted. Nevertheless, persistence is a minor consideration, not a major one.

Every subdomain usually has its entities. And not only that – every subdomain has its own set of business rules, and entities should be validated according to these business constraints. As a general rule of thumb, a carefully crafted business domain should prevent the instantiation of invalid objects. One of the good rules is a principle that “impossible states should be irrepresentable.” In other words, you can declare someone’s age as an integer and offload that check to the database, but would it be correct from the perspective of the business domain to allow the creation and storing of an Employee with age -50 or zero? Data integrity mechanisms exposed by the relational database can cover just one minor part of the overall business validity of your data. A complete set of validation and data integrity rules is a vast one. It usually contains various static and dynamic constraints, often encompassing multiple different entities in various states. For example, a minor shopper should not be able to add a beer to a shopping cart.

Examining the primary role of a database , we can conclude that when you are storing entities into it, the database should provide a reliable mechanism to persist in-memory objects in a verbatim and sound way. Hence, you can look at the database as a way to mirror your object graph from memory into persistent media (usually a hard drive).

So, looking through narrow lenses, it is true that relational database is indeed capable of performing type-level and reference integrity checks, throwing errors at attempts to store invalid data. However, layers of business rules and checking mechanisms of high-level rules should be present in every reliable application. With a good software development approach, you will never let your objects exist in an invalid state.

Hence, the database should be placed at the proper position during application development – not in the center anymore, but as a persistence mechanism. Its role should be to store validated objects reliably and provide fast and reliable means to index and query such data.