Decomposition Gotchas

The law of large numbers says that random errors tend to cancel each other out. Beware, however, of errors that are not random. If the errors are an integral part about how you’re doing the work, they are more likely to reinforce each other than to cancel out. If you’re adding up a large number of small estimates and the errors all tend in the same direction, the resulting error will be large. This can catch you by surprise if you don’t consider the systematic errors that may exist in your estimation process.

Here are some common causes of accumulating errors. Watch out for them. As the proverbial “they” say, “forewarned is forearmed.”

Making Consistent Size Errors

You may consistently imagine the pieces as larger or smaller than they are. When you estimate a lot of small pieces, you don’t expect to get each estimate “right.” It’s perfectly fine that you’ve estimated two tasks at 3 hours each, but one takes 2 hours and the other 4. The errors cancel out and your overall estimate is useful to you.

If you have a consistent bias in your estimation, though, the errors add up and your overall estimate is likely further off than if you’d estimated the large chunk as a whole. In software development, the empirical evidence is that most developers are optimists. They underestimate much more often than they overestimate. That’s rather a shame, because underestimations are less likely to be challenged, and cause us more disappointment down the road, than overestimations.

As described in ​Calibrating to Unknown Context​, once we get started, we can easily adjust our expectations by calibrating them with actual results. This might be too late for some people, who might have built a rigid plan on the early uncalibrated estimates. It’s good enough for those who are flexible enough to accommodate reality.

Most of the time, you’d like your errors to cancel out, so your result is still accurate even if individual estimates are wrong. Sometimes, however, you’re trying to avoid some calamity. If you estimate such that all the errors are intentionally in the direction of that calamity, you should be safe should the result still be in the safe zone. This reduces risk at the cost of reduced accuracy. (See ​Danger Bearings​.)

Overlooking the Space between the Pieces

One of the typical ways to consistently underestimate the time required for a piece of work is to focus on the core of the work and neglect the periphery. Software developers think about the programming part. They may not consider the work to get their local environment ready to do the programming, setting aside any partial work and updating with the appropriate source code and libraries. They may neglect the time it takes to read the code to understand the details of the needed change. They may forget the time it takes to verify that the change is correct, and that it hasn’t broken existing functionality or introduced any new problems.

Years ago, I had a project lead who asked me to estimate how long it would take to fix a reported bug in some code written by a colleague who’d since moved to a different project. I looked at the bug report and made a list of the changes I would have to make. Then I estimated how long it would take to edit, compile, and verify each of those changes. I handed this annotated list to the project lead. He took one look at it and said, “You can’t do any task in 10 minutes. It will take you longer than that to checkout the code, find the place to change, and check it in again. Never estimate a programming task at under 30 minutes.” That’s when I realized that I was only estimating part of the required work, the programming part on which I was focused, not the necessary parts that enabled the programming. I was also leaving out the context switching from one item to another.

Overlooking Some of the Small Pieces

When you build the list of pieces of work, you try to include every task that’s required to accomplish our goals. That’s a hard job, of course. When you’re doing the work, as you go to make one change you realize “Oh, that’s connected to this; I need to change it, too.” The work itself prompts us to notice related tasks. When you’re just listing them, those related tasks may be left off the list. This almost certainly leads to underestimation.

I once worked for a company where those estimating the work for bids neglected to consider the work to migrate the data from the customer’s old system to the new. This happened for two or three bids in a row before any of the projects got far enough along to realize this omission. Since the price for the work had been heavily dependent on the estimates, this was a major blow to the profitability of those projects.

Of course, you might also add tasks that aren’t really necessary, and then overestimate. “We need a method to handle negative input values; oh, I see it already handles those.” This happens much less frequently, in my experience.

Decomposing into an Unmanageable Number of Pieces

The whole may be so big that it takes a huge number of small pieces. When you break your work down into a large number of pieces, the decomposition itself is a lot of work. It’s also subject to a lot of error, as it’s depending on pretty detailed understanding of future work.

How Many Stories Can It Be?

I’ve asked around and found it not too uncommon for people to report starting a project with a backlog containing a thousand or more stories. That’s a lot of stories! How do you fit the concept of the project in your head when you’ve broken it down into that many discrete pieces? This is an all-too-typical example of trying to get the requirements complete before starting the project. Sometimes that drive for completeness comes from a vain attempt to calculate a high-precision estimate of how much effort it will take. Striving for certainty, people spend an inordinate effort to capture all the details at the start.

Take, for example, the frequently recommended practice of breaking a planned release into User Stories for estimation. A User Story is a small slice of functionality that adds “one thing” to the system being developed. It has observable completion criteria, so you can tell when it’s done. It is a useful tool for getting things done on a development project. Getting one baby step done before going on to the next acts like a ratchet, moving us forward and preventing us from slipping backward. In order to act like a ratchet, the stories need to be pretty small—something we can accomplish in a short period of time before we go to accomplish the next small thing. As a rule of thumb, teams should size User Stories such that they take a day or two of calendar time to accomplish. It doesn’t matter how many team members might be working on the story to get it done in that time. And it’s alright if the stories are smaller. When I’m working by myself, I generally prefer them smaller.

If a three-month (13 weeks of 5 days) release is broken into User Stories that take two team-days each, then that’s about 32 stories. That’s a lot of stories, and it’s even more if multiple stories are in progress simultaneously, or if some of the stories are smaller. If half the team gets involved in each story, and half of the stories are only one day, then our story count balloons to 96. Imagine the team churning through a list of 96 stories at the start of the project so that they can know what fits into three months, or that they can know how long it will take to do what they want. Sounds like a lot of effort, doesn’t it? (And this is for a small project.)

As they expend that effort, they learn a lot of details about those stories. We’ll want to record what they learn so they don’t have to relearn it later. That will take more effort, especially to record it in a way that won’t be misinterpreted later.

How Much Work Could It Be?
	How hard is it to create a backlog of a couple thousand "story-sized" and bigger items? In one case, I’m told it took a few dozen people three months to list them all. They were deriving them from a legacy system that was to be retired, as well as asking subject matter experts in the company. Unfortunately, nobody seemed to know why some of the legacy functionality was in the system.

As they learn the details, some of these details build on details they’ve learned or decided before. But since they’ve just learned them in a short time, they often forget some. It takes time for learning to “soak in.” The prior thoughts are likely to be stronger, asserting themselves in later decisions. So far, in the hypothetical scenario, they’ve only talked about the system. They haven’t actually built anything with which they can interact. That’s a clue that some of the details they’ve “learned” are certainly wrong, but they don’t know which ones, yet. As they build the system and interact with it, they’ll learn more, and they’ll learn more deeply. Do they abandon the early pre-start learning, or do they try to maintain it, fixing it where it’s incorrect and incorporating new things they’ve learned? Either represents extra work.

What Happens to These Stories?

While the attempt to be complete may seem heroic, it turns out to be misplaced effort. In some cases, the stories start to be quietly deleted as early as two or three months after building the list. Why quietly? It seems embarrassing to throw away the result of so much work. In fact, I suspect that some people realize there are a lot of extraneous entries in the backlog far earlier, but may be unwilling to speak up. The Sunk Cost Fallacy makes us value what has been costly, just because we paid that cost. Adding insult to injury, in all the cases I’ve heard of people developing a "complete backlog" at the beginning of a project, there was always new information uncovered later. That complete backlog continued to grow over time, in spite of having unneeded items included.

I frequently hear or read about people suggesting the use of User Stories for relatively long-range planning. Sometimes they mean something as short as a release in a few months. Sometimes they’re talking about multiple releases over a year or two. In all of these cases, they’re talking about breaking down the work to be done into small slices, so that they can better measure it’s perceived size for predicting the future. This is a great deal of planning up-front, when people know the least about the work they need to do.

All in all, creating a long product backlog of User Stories is very reminiscent of creating a detailed Work Breakdown Structure at the start of the project. It starts to separate the work into distinct phases that are executed in a linear fashion. The details of User Stories are typically oriented more in functional terms than construction terms, but it’s still a difficult and error-prone way of defining the work. It does, of course, let us come up with numerical estimates.

Making the Whole Look Larger

What is the length of the coastline of Great Britain? It depends on the length of your measuring stick. Large measuring sticks skip over the nuances. The smaller the unit of measurement—that is, the closer you look—the larger the length of the coastline. This is known as the Coastline Paradox. Before you know it, you’re counting the grains of sand on the perimeter.

Estimation is Fractal
	"It turns out that estimation is fractal. The more fine-grained you break down the requirements, the more ’edges’ you will discover. This means that the more detailed your estimate, the more the total will tend towards infinity, simply due to rounding errors and the fear factors that we multiply into fine grained estimates." -- Dan North[9]

Measuring the size of software development follows a similar pattern. The deeper you look at it, the more you discover and the larger the work appears to us. The deeper you look into requirements, the more you discover and the more details you want to specify, making the work demonstrably larger. More features always seems better than fewer when looking at things in isolation where there’s no counterbalancing tendency to keep it in check.

Losing Focus

When you decompose your big picture into lots of tiny details, you lose sight of the original intent. It becomes hard to remember which aspects are essential and which are elaboration. It’s even harder to communicate the focus to someone else. They see an undifferentiated sea of details, not a clear picture of the important goals.

When you change your focus from “delivering what the customer wants” to “delivering these items on your planned backlog,” you make it easier for the project to go astray. The end customer doesn’t want a collection of stories. They want a system that works coherently to accomplish the work they need.

Imagining Pieces Not Appropriate for the Implementation

I remember a manager handing me a work breakdown list to estimate so he could create a Gantt chart for the executives. As one of the programmers, I was supposed to estimate how long it would take me to do each of the tasks on the list. Unfortunately, I couldn’t even recognize what some of the tasks were. Those that I recognized clearly split the system between a user interface and a database without considering the complexity of anything in between. Where was the interaction between different actions and business rules? It was clear to me that the listed development tasks would not result in the system we wanted to build.

Requirements-Based Decomposition

Think back to the example User Story in ​User Stories​, where a user can act on behalf of another user. One naive approach would treat this as a new module of code, mostly duplicating the code for acting on your own behalf. Keeping these modules in sync will be nearly impossible. If you notice this duplication, you might instead modify every place where a user action is implemented, modifying it to accommodate acting on your own behalf or acting on another’s behalf. This will touch LOTS of places in the code. The need for "shotgun surgery" will alert you that there is a concept that needs encapsulation. In fact, the old concept of "user" needs splitting into two concepts, "actor" and "beneficiary." It’s unreasonably optimistic to expect this level of analysis before starting implementation, so estimates based on premature decomposition will likely expect such a naive decomposition based on the stated requirements. If this decomposition becomes the implementation plan, then the implementation will be naive and hard to build and maintain.

Even if you’re not starting with a bogus list of programming tasks, the initial design is likely to be the most prosaic and naive implementation. Insight comes from working with the system.

When the details are about the way you’re going to build the system, that is, components of the system, you’re locking in the design of the system from the very start. If you find that some other arrangement of components works better, that invalidates the entire estimate and plan.

A similar problem happens even if you’re the one breaking things down into functional slices. Those slices, when there are a lot of them, become very detailed. They are no longer describing the outcomes that the user wants, but the intended way of delivering those outcomes. When you discover that some of those outcomes are unimportant but others, not previously envisioned, are important, then it’s very difficult to change the plan. Which of these stories can you pull out to affect only these outcomes?

You may discover a simpler design that’s less work to implement. You may notice that things that don’t work as smoothly as you’d like, and redesign to improve that. Or you may live with the existing design, and suffer delays from its inadequacies. Sometimes you get in the middle of the implementation and realize that your design won’t quite accomplish the goals. Do you change the goals, or redesign?

There are a lot of scenarios where the system you’d like to write doesn’t follow the design of the system you’ve assumed in your estimate. What happens when you discover that? If you’ve put a lot of effort into your breakdown, then you’re likely to suffer from “Sunk Cost Fallacy” and “Anchoring Bias,” (see ​Cognitive Biases​) and try to maintain that original breakdown even in the face of evidence that another approach will be superior. It’s a natural human reaction to minimize the issues discovered with your initial approach, and magnify the risks of a different one. If this happens, it may be that your estimation technique leads us to build the wrong thing, or build it in a less effective way than you might. If so, it might be more expensive than a bad estimate.

On the other hand, if you don’t build the little pieces you used to make the estimate, then how worthwhile is the estimate? Will it be close enough for your purposes, or will it lead you astray? Should you reestimate the new pieces you discover? If so, should you add the cost of reestimating, this time and probable future times, to your estimate?