This chapter covers

• The hazards of too-shallow modeling
• What deep modeling feels like
• Security flaws in the form of broken business integrity
• Deep modeling to mitigate risk

This is a real story about how negative numbers can cause severe economic loss. It’s based on a case we worked on with a client, but to be able to share the details, we’ve obfuscated the context. Most importantly, we’ve changed what the business sold. We can assure you that it wasn’t books. Interestingly enough, there are other examples that actually did involve books. Amazon had a similar bug around the year 2000.¹  But for those cases, we don’t know the under-the-hood details.

This is also a story about how a serious security problem persisted in production for a long time without being detected and without anything being broken—at least, not in the technical sense. Nevertheless, it still caused money to bleed from the enterprise. Although the company could have uncovered who benefited unfairly, for practical reasons it wasn’t possible for it to recoup its losses.

Finally, this story is about how an international retail business accidently gave its customers do-it-yourself discount vouchers in its online store. We’ll show how its loss was the result of a shallow design with incomplete or missing modeling, something we often encounter.²  And we’ll discuss how an explicit and conscious modeling effort makes a difference.

This online store is nothing unusual; it’s the typical kind of business where the customer puts books in a basket, checks out, and pays with a credit card, and the books ship (figure 2.1). The store has been in production for a while, with ongoing development since its initial release. The retail business brings in a security team to do some auditing and testing. Specifically, the team audits how the system is set up in production as well as in the codebase. They also do tests where they try to manipulate the system from the outside to find security flaws. The team has a pretty open mandate to follow up on anything strange they find.

The infrastructure seems solid as the security team pokes around. They probe the firewalls. They scan for open operating system ports. They throw malicious packages at the web server. Still, everything works fine. This isn’t a big surprise. Nowadays, security problems are seldom the result of broken infrastructure. We’ve learned that things that shouldn’t be exposed to the public should be cut off from the public.

At the same time, other members of the team investigate the online store application from a technical perspective. They search for ways to circumvent the login. They see whether they can kidnap an open customer session. They try to poison the cookies. Same thing here—no success. These things would also be OK if the web server had been properly configured and, in this example, someone had obviously made the effort to read the documentation to implement the configuration.

A breakthrough comes when one of the team members, Joe Tester, gets curious about the Quantity field (where you specify how many books you want) on the order form. He passes in a JavaScript snippet to see if it executes, but nothing happens. Then he attempts to provoke a SQL injection—still nothing.³  Finally, he gets curious and enters -1 as the quantity for a copy of Hamlet priced at $39. Phrased in another way, Joe Tester tries to buy a negative Hamlet —an anti-Hamlet.

Joe’s surprised that he receives no error message. The store accepts the order, and it goes all the way through the order flow. He checks out with a credit card and gets an email confirmation that the order is accepted. “Strange,” he thinks, makes a comment in his notebook, and continues working. The next afternoon, there’s a knock on the security team’s door. A lady hesitantly enters.

“I’m from accounting,” she presents herself, “and I wonder if any of you know anything about a person named Joe Tester.” To clarify, she adds, “Because I asked around, and someone said you might know.”

The lady from accounting continues: “I was running the accounts receivable ledger, and the system issued a credit invoice to him for $39. But when we were going to mail the book to him, we noticed a strange thing about his customer address; it’s the same as our address here at headquarters. That’s why I got suspicious and started asking around.”

The system tried to pay real money to Joe Tester—not good.

2.1 An online bookstore with business integrity issues

Let’s step back from the case for a moment to see what happened. It’s definitely strange that the online store accepted an order of -1 copy of Hamlet. But now that this has happened, let’s think about the logical consequences.

If someone buys a book that costs $39, then the value of the order is $39, and it makes sense that the customer pays $39 to the store. In this case, the customer, Joe Tester, didn’t buy a copy of Hamlet, he bought a negative copy, so the value of the order is -$39 (figure 2.2). He should pay -$39 to the store, or the store should pay him $39. But a store isn’t meant to pay out money in this fashion. Perhaps Joe Tester should give the store a copy of Hamlet.

From a security perspective, this is a security breach. One aspect of security talks about the integrity of data, which roughly means that data hasn’t changed or isn’t generated in an unauthorized manner. Most often, you think about integrity in a technical way—providing checksums and cryptographic signatures to ensure data only changes according to the rules. In this case, the rules aren’t technical rules but business rules. It’s not sound business for a store to send customers money for anti-books. What we have is a breach of business integrity.

Suspicions raised, the security team starts investigating what’s really going on. It turns out that the online store system calculates the “to pay” value for the order to be -$39, which is logical, although weird. The amount of -$39 passes through several online store systems, one after another (figure 2.3).

An interesting aspect of this story is that the security problem can’t be understood without understanding each of these systems and how they react and interact. We’ll start with two of them: the billing system and the accounts receivable ledger.

The purpose of the billing system is to collect payments from customers. If a customer sets the payment preference to a credit card and checks out an order worth $347, then that sum is charged to the customer’s credit card. Customers can have other payment preferences too; for example, invoice, accumulated invoicing, or gift cards. Some customers have different payment methods for different amounts. Large amounts might be paid directly, whereas small amounts are accumulated into an end-of-month invoice.

When Joe Tester takes his order of -$39 to checkout, that amount is sent to the billing system. But the credit card module of the billing system doesn’t know how to handle negative amounts. From the perspective of the billing system, a negative amount means there’s no payment to collect. The payment task drops through without action.

2.1.1 The inner workings of the accounts receivable ledger

Now let’s turn to the accounts receivable ledger. Part of the accounting system, the ledger keeps track of customers who owe the company money, which is the case when they’ve bought books but the company has yet to receive payment. In short, the accounts receivable ledger handles the balance of each customer. For example, if someone checks out an order of $347 and selects pay by invoice, then $347 is added onto that customer’s balance. Later, when the company receives the payment, the balance in the ledger is cleared. Sometimes people pay too much, perhaps $350 in this case. The balance then drops to a negative, -$3, meaning that the company owes money to the customer. For a bank, it’s often normal that a company owes money to the customer, even in the long run. But for an online bookstore, such a situation is only acceptable as a temporary condition. If it arises, the store should try to clear its debt as soon as possible.⁴ 

The normal procedure for the online book company is to pay out money owed to customers by sending a credit invoice. Those checks run as a batch job, which was what the lady from accounting referred to when she said, “I was running the accounts receivable ledger….” When Joe Tester makes his anti-purchase, the online store system sends a payment amount of -$39 to the accounts receivable ledger, which immediately puts him at an advantage to the company. The next night, the job runs and finds this outstanding debt, so it creates a credit invoice to be sent to him to clear the ledger, effectively sending our tester a payment (figure 2.4). This is the invoice that a perceptive lady in accounting catches as suspicious and starts asking around about.

As these tests are done in production, it’s obvious that the production system has had this flaw for a while. The only reason this particular case was caught was that the strange address raised suspicion. But there may have been similar cases earlier.

A financial investigation is started to see how big the problem is. Operations and the security team join forces to do a cross-check of all credit invoices that have been issued. Sieving away credit invoices to suppliers and partners leaves the customer-facing credit invoices. Most of those are valid credits issued for damaged goods or other legitimate reasons. Only a small portion is left, so the problem turns out not to be particularly big—or so it seems at the time. Still, it’s strange that sending out money for nothing has gone undetected. The technical investigation continues to unveil the entire scope of what happens when someone orders an anti-book. And two more important systems are involved: inventory and shipping.

2.1.2 How the inventory system tracks books in the store

The inventory system for the store keeps track of how many books of each kind the store has in stock and can sell from the warehouse. A good starting point for that is how many books of each kind are on the shelves in the warehouse, but, unfortunately, it’s not quite that straightforward. For example, say that there are 17 copies of Hamlet in the warehouse. A customer has bought two of them, but those two haven’t yet been picked from the shelf and shipped to the customer. These two copies shouldn’t be counted, because they aren’t sellable, and they don’t belong to the store any more. The inventory of Hamlet should be 15 copies, not 17.

Another situation might be that the shelf for Pride and Prejudice is empty. The retailer has bought another 100 copies from the publisher, but those copies are still on a truck that hasn’t yet arrived at the warehouse. As the copies are in the possession of the store, they are sellable and should be included in the inventory. The inventory of Pride and Prejudice should be 100 copies, not 0. There’s lots of other strange situations that might occur as well. The inventory system is a complicated piece of logic in and of itself.

If the online store sells three copies of Hamlet, it sends a message to the inventory system, decreasing the inventory level of Hamlet from 15 to 12. But what happens if instead Joe Tester buys -1 copy of that book? The inventory level of Hamlet was 15 and should now be reduced by -1, resulting in an inventory level of 16 (figure 2.5). Selling one anti-Hamlet increases the inventory level by one!

2.1.3 Shipping anti-books

The shipping system ensures the books are shipped to the customers. When an order arrives from the online store, the shipping system iterates through the order lines and compiles pick lists for the warehouse workers to pack the boxes. This is a complicated system that tries to minimize the work for the warehouse workers by letting workers pick books for several orders simultaneously, for example. The system handles that optimization well.

What the shipping system doesn’t handle well is a negative number of books. Such an order line causes a runtime exception that’s logged to the system log, together with a multitude of other messages. Unfortunately, no one ever looks at that log. In effect, the order line is discarded.

2.1.4 Systems living the same lie

Here dawns an interesting realization. From a financial perspective, the IT systems are consistent with each other. The billing system and shipping system are of less importance, as they don’t change their state. The more interesting systems are the inventory system and the accounts receivable ledger.

• The inventory system falsely believes the inventory of Hamlet is 16 when it should be 15.
• The accounts receivable ledger falsely believes the retailer owes money to someone else.

From a financial perspective, this balances out: instead of having 15 books and owing nothing, the retailer has 16 books and has a debt of the value of one book. Both systems live a lie, but they live the same lie. The illusion is consistent.

Because the systems are consistent, the regular reports won’t show any discrepancies. As a matter of fact, there are reports that run every night. More comprehensive reports are also run each quarter as part of the financial reporting. And none of these have reported any discrepancies because there are no discrepancies between the IT systems. The discrepancy that exists is the inconsistency between the inventory system (16 books) and what’s actually in the warehouse (15 books). But that won’t be noticed until the end-of-year inventory, when the inventory of the warehouse is counted manually and fed into the bookkeeping system. Then, and only then, will the missing book be noticed.

At the end-of-year inventory, there’s nothing strange in finding a discrepancy. Books are physical objects, and things happen at a warehouse. A delivery from a supplier might arrive that should contain 134 books but only contains 133. Not all boxes are counted, and sometimes there’s a mistake in counting. A book might be dropped, damaged, and discarded. This should be noted, but sometimes people are in a hurry and forget to do so. And sometimes, there’s theft. All wrapped together, this is reported by finance as a loss-on-warehouse, and a certain level is expected.

As a matter of fact, the end-of-year inventory a few months earlier reported a higher level of loss-on-warehouse than usual. Management’s analysis was that there was a motivational problem with the people at the warehouse: either they had gotten less careful and damaged more books or they had started stealing books. As a result, management sent them on a day retreat with a motivational coach to get their values more aligned with the company’s ethics. The folks at the warehouse were confused and not happy.

2.1.5 A do-it-yourself discount voucher

On further study of the discrepancies, things turn out to be worse than they initially seemed. The inventory difference in the warehouse is much larger than the total number of credit invoices. But there’s something more going on.

Realizing that the usual reports can’t be trusted, the team starts a deeper investigation. It turns out that getting a credit invoice wasn’t the most usual way the flaw had been exploited. A much more popular version was to give yourself a discount by ending your shopping trip in the e-store with some negative books to reduce the total amount of your order before paying (figure 2.6). The rumor about this strange feature had obviously spread, because quite a lot of customers used it.

The investigation shows that the company has lost a significant amount of money through this loophole. Now it’s time to decide what to do.

The technical flaw will be addressed, and we’ll return to that soon. But what about all the money customers owe the business because they’ve given themselves discounts? In the end, it’s the board of directors that gets the call to decide what to do. After careful consideration, they decide to let things be. Chasing down customers, many of them returning and frequent customers, would generate more ill will and would hurt the company more than simply accepting the loss, patching the hole, and moving forward.

This continuous breach of business integrity had been going on for months without anything being technically broken. And it most probably would have continued unnoticed had it not been for the curious lady in accounting and a security tester with an interest in how the business worked—the domain of book sales. We’re convinced that, throughout the world today, there are many similar flaws in existence, being continuously exploited without triggering any alarms.

2.2 Shallow modeling

A company leaking money this way obviously has a security problem. How could this occur? And, more importantly, how could it have been avoided? Our observation is that this kind of situation is often the result of modeling that stops short at the first model that seems to fit, without digging deeper or questioning and without planning or consideration. Let’s refer to this ad hoc style as shallow modeling (in contrast to deep modeling).

Let’s start by asking the question, “How could things go wrong this way?” Looking at it after the fact, it seems obvious that a quantity can’t be an integer without restrictions. But why did someone design it that way? As we mentioned in chapter 1, design consists of all the active decisions you make when developing software. In this case, the design included the (active) choice to make a quantity an unrestricted integer. It might not have been a decision that was well thought through, but it was a decision nevertheless.

Let’s look at the rest of the concepts in the design—and there are lots of concepts in the domain of online sales. Some of these are more important and some less. When designing, someone chooses some concepts to be the most crucial: order, order line, book, and quantity, for example. We often see designs where the main focus seems to be on answering the question, “How can I represent this?” With this mindset, design is about finding a way to code it. When you find a way to code it, you’re done. And, in this case, the shortest distance between business and code is achieved if you can represent things using the language primitives: integers, floats, booleans, and strings (figure 2.7).⁵ 

Different things are represented in different ways in the domain, some explicit and some implicit. For example, the model contains order as a concept, and an order has a monetary value. An order also consists of order lines, each with a book and a quantity. A book has a title, an ISBN, and a price. Order, order line, and book are explicit concepts in the domain. Quantity, title, ISBN, and price are implicit concepts, represented by integers, strings, and so forth. Rephrased, a quantity is an integer without restrictions. Looking at it this way, it seems strange that order, order line, and book are all well elaborated, but quantity is left as an integer without consideration. How did it become this way?

2.2.1 How shallow models emerge

We think many of these mistakes boil down to the modeling being incomplete or even missing. Imagine a conversation in the early stages of the project between a sales person, Sal Esperson, and a developer, Deve Loper. The discussion might go something along these lines:

“And then you can add books to the order,” says Sal.

“So, how do we describe a book?” questions Deve.

“We show a title and a price,” answers Sal.

“What can the price be? Is it always a whole number?” asks Deve.

“Well no, a book can be priced $19.50, tax excluded,” clarifies Sal.

Deve thinks, “So a book has a title and a price as attributes. The title is a string. The price is not an int, it’s a float.”

And Deve asks, “Is that all there is to a book?”

“Nah,” answers Sal, “it’s also important that it has an ISBN, so we can keep hardbacks and paperbacks separate.”

“OK. And then we add books to the order,” says Deve. “For example, Moby Dick, Pride and Prejudice, Hamlet, Moby Dick again, 1984, and Moby Dick again?”

“Well, almost. We would say three Moby Dick books, as we don’t care about what order you buy them in.”

“OK,” Deve thinks. “It isn’t a float, it’s an integer.”

Later on, this discussion is turned into code. Deve creates a class Book that gets the attributes title, isbn, and price. The Order class gets a new method, addOrderLine(Book book, int quantity). Because order, order line, and book get explicit representations, those each become classes. The type system enforces that these are used at the proper places in the code. If you try to pass anything else where a Book is expected, you’ll get a compilation error.

On the other hand, quantity is implicitly represented by the primitive type int. But using quantity isn’t enforced by the type system and compiler. You can accidentally pass in some other integer, such as the temperature outside, without getting a compiler error. The only hint that a quantity is expected is the parameter name quantity, as shown in the following:

class Book {
    String title;
    String isbn;
    double price;
    ...
}

class Order {
    void addOrderLine(Book book, int quantity) {
    ...
    }
}

In the conversation, note that Deve asks no further questions about what a title is or about the ISBN, jumping to the conclusion that they are simply text, and he represents them using Strings in the code. But, most probably, a title can’t be any string, and an ISBN certainly can’t.

Deve isn’t incompetent when it comes to modeling. He does ask an interesting question about the nature of price (“Is it always a whole number?”) but leaves it there. Also, he completely misses the hint that prices might be more complicated when Sal answers, “…tax excluded.” The drive for Deve seems to be “Can I represent this in code?” and not “Do I understand how this works?”

We’ve seen that shallow modeling like this leads to having interesting business concepts represented as primitives: int, float/double, string, boolean, and so forth. Our experience is that these kinds of implicit representations are common. We often see systems where almost everything is represented by strings, integers, and floats. Unfortunately, this has several drawbacks.

2.2.2 The dangers of implicit concepts

You’ve seen that something as simple as leaving quantity as a primitive integer can cause severe security problems. This type doesn’t capture the crucial restrictions. In the same way, having the title and ISBN as unrestricted strings provides too much leeway. Strange things can happen when a system assumes some data is formatted as a proper ISBN when it’s not.

Credit card numbers and Social Security numbers (SSNs) are two other examples that we often see as implicit concepts represented by strings. Obviously, this risks having data that’s not a valid credit card number or SSN, and it might not even have the right format. But worse is the risk that you might not treat it properly.

Both credit card numbers and SSNs have strong restrictions on how they can be revealed, put into logs, and so on. If they are represented as strings, there’s a risk that they’ll be accidentally put into logs or shown. Later, we’ll see how representing these things as domain classes can avoid such mistakes; for example, by using read-once objects (we cover this in chapter 5).

Back to our implicit concepts represented by language primitives, such representations also create very funky code. Ponder the following method signature:

void addCust(String name, String phone, String fax, int creditStatus,
   int vipLevel, String contact, String contactPhone, boolean partner)

This code has only eight parameters, but if you accidentally swap two of them, then the customer might get a credit status or a VIP level they shouldn’t have. As both the creditStatus and vipLevel are ints, the compiler won’t catch that you’ve sent them in the wrong order. These kinds of mistakes can lead to subtle and hard-to-find bugs, sometimes with security connotations. And eight parameters isn’t a long list. We’ve seen parameter lists with tens of parameters, all strings. Constructors especially seem to be at risk for this specific problem.

Shallow modeling and the resulting implicit concepts lead to a high risk of buggy and insecure code. The alternative is a more conscious approach with deep modeling and explicit concepts. Now let’s turn to what this story would look like if the modeling had been a conscious effort to implement a deep model.

2.3 Deep modeling

To understand deep modeling, you must first acknowledge that any model you come up with is a choice. In any domain, there are uncountably many different models possible. When you design, you choose what set of concepts to build your design around, and what meaning you load into those words. Using the terminology from Domain-Driven Design, this particular choice makes up the domain model, the chosen distillation of the domain. In our work with security and design, we’ve gotten lots of inspiration from Domain-Driven Design and its focus on understanding and modeling the domain in a strict way.⁶ 

Making a conscious effort when modeling means that you actively search for ways to understand the domain. The drive isn’t “How do I code this concept?” but rather “How can I understand this concept?” This leads to much deeper dialogues when modeling and often to an iterative process of discussions and coding. The result is that you unveil more concepts that need to be represented explicitly to capture the full understanding of your model.

2.3.1 How deep models emerge

Let’s go back to the discussion between Deve Loper and Sal Esperson to see how it might evolve with a mindset of deep modeling:

“And then you can add books to the order,” says Sal.

“So, how do we describe a book?” questions Deve.

“We show a title and a price,” answers Sal.

“What can the price be? Is it always a whole number?” asks Deve.

“Well no, a book can be priced $19.50, tax excluded,” clarifies Sal.

Deve thinks, “So, a book has title and price as attributes. And price seems to be a complicated issue in itself because you mentioned tax. I’ll need to dive into those later,” and asks, “Is that all there is to a book?”

“Nah,” answers Sal, “it’s also important that it has an ISBN so we keep hardbacks and paperbacks separate.”

“OK. And then we add books to the order,” says Deve. “For example, Moby Dick, Pride and Prejudice, Hamlet, Moby Dick again, 1984, and Moby Dick again?”

“Well, almost. We would say you have a quantity of three Moby Dick books, as we don’t care about what order you buy them in,” says Sal.

“Can you buy half a Moby Dick?” asks Deve.

“Of course not, silly.”

“You used the word quantity,” Deve says. “I want to understand that better. What happens if you have a quantity of three Moby Dick books and then they’re removed? Do you then have a quantity of zero Moby Dick books?”

“Eehhh, not really. I mean, a quantity of zero isn’t really a quantity at all. We’d say no quantity,” Sal clarifies.

“This quantity seems to have some rules around it,” Deve says. “So, how big can a quantity be? Two billion books?”

“Haha. Well, certainly not. Seriously, I think we’re limited by the through-store logistics flow, and it can’t handle orders bigger than a total quantity of 240.”

“The through-store flow?” asks Deve.

“Yep, that’s what they call it. It’s how the orders from the online store are handled at the warehouse; it’s about box sizes, packing stations, and stuff. Orders bigger than that must go to the warehouse bulk flow. But we can’t use that from the online store,” Sal explains.

“What is the total quantity of an order? Can you give me an example?”

“That’s simply adding the quantity of all books. If you have three Hamlets, four Pride and Prejudices, and one Moby Dick, then you have a total quantity of eight,” Sal says.

Deve makes a note that a single quantity can’t be larger than 240, and the same goes for the total quantity of an order. Later on, this knowledge is captured as code:

class Book {    ①  
    BookTitle title;

    ISBN isbn;
    Money price;
    ...
}

class Quantity {    ②  

  ...
  Quantity(int quantityOfBooks) {
      isTrue(0 < quantityOfBooks, "Quantity must be positive");
      isTrue(quantityOfBooks <= 240,
        "Quantity must fit in through-store flow, which is limited to 240");
      ...
  }
}

class Order {
    void addOrderLine(Book book,
                      Quantity quantity) {    ③  

        ...
    }

    Quantity totalQuantity() {
        ...
    }
}

In a later chapter, we’ll dig deeper into code like this. Chapter 4 covers contracts such as isTrue(0 < quantityOfBooks.... The class Quantity is an example of a domain primitive, to which chapter 5 is dedicated. Creating entities such as the Order class is the subject of chapters 6 and 7.

2.3.2 Make the implicit explicit

With deep modeling, you can find lots more concepts that are interesting—too interesting to be left implicit. Our standard advice is to make implicit concepts explicit. When you find an implicit concept like “quantity” in your story, take a few minutes to discuss it a little bit more deeply. If it seems interesting enough, make it into an explicit concept instead—spell it out as a part of the design. Later on in the code, quantity will show up as a class of its own and will uphold its own constraints. Using the concept of quantity also makes the rest of the code more expressive.

A common objection is that making all these concepts explicit creates a lot of classes. We’d like to point out that the code in those classes is necessary in any case: all the interesting business rules have to be caught in code, or else you’re creating a worse system. Having explicit concepts as classes makes a difference in how your code is organized. Extracting interesting concepts into classes of their own makes them easier to find than if the same code is spread out into service methods in large service classes.⁷ 

Shallow modeling is a missed opportunity for learning. As you’ve seen, it’s also a potential source of security vulnerabilities. To grab the opportunity would be to ask, “What do you mean by quantity? Can there be variants? Are there restrictions?” Most probably, you’d learn that a quantity of books can never be a negative value. Perhaps you’d even learn that it can’t be zero because “We only use the word quantity with a number if there are books; otherwise, we say there’s no quantity.”

Discussing the lower bound might lead to a discussion about an upper bound. Is it sensible to be able to order 2,147,483,647 books? Asking about such an order might lead the domain expert to start explaining how logistics work, how books are loaded on pallets, and so on. Such a discussion will again give you a deeper understanding and, yet again, reduce the risk of business integrity problems.

A design like this is much more expressive, much more robust, and much less prone to contain security vulnerabilities. We’ll spend the following chapters elaborating on how to achieve this and what design guidelines we’ve found most effective to avoid security flaws. We’ll start by looking at some of the concepts of Domain-Driven Design we’ve found most useful.

Summary

• Incomplete, missing, or shallow modeling leads to a design with security flaws.
• A security flaw in the form of broken business integrity can live in production for a long time, bleeding money from your enterprise.
• Conscious, explicit design results in a much more robust solution.