Estimating the Unknown
As you’ve already seen, there’s always the possibility of having items that you don’t know how to estimate. Some of these are difficult because they’re too large, and you’ve already looked at decomposition as a means to make them more amenable to estimation.
Sometimes, though, you don’t have reasonable historical data for comparison. We can still compare the parts with each other, but that gives us an ungrounded estimate. How do you turn that into something you can share with others?
Calibrating to Unknown Context
Perhaps the work is relatively well-known, but the conditions for doing it are not. You might have a new team that’s not used to working with one another. Or a team that’s not used to this type of work, or for this client. You can end up with an estimated pile of work, but no way to associate that with calendar time.
That’s one of the advantages of using Story Points (see Story Points), or, as they were alternatively named, Nebulous Units of Time (NUTS) or Gummi Bears of Complexity. These fanciful titles are a reminder that you can’t just add up the numbers and expect a precise and accurate prediction. But you can use them to plan your work and track our progress. If you start working and see how many of these you accomplish in a week, you can guess, using the concept of Yesterday’s Weather, that next week you’ll do about the same.
This gives a current rate of progress, and you can use that to calibrate your estimates. Be wary here. You don’t know the variability in your rate of progress. You don’t know the variability in the work. You’re putting a lot of faith into a small amount of data if you use this to look very far into the future. The use of Story Points works well for selecting how much work would fit into the next iteration, perhaps looking ahead two weeks, where the consequences for being wrong is very low. If you take on too little work, you can bring in the next prioritized story. If you take on too much, then you won’t get it all done this iteration. The wise advice was, and is, to have as few stories in progress as practical, so that it’s no big deal if one or two spill over. This avoids having them all “almost done” with no trustworthy indication of progress. This approach also supports projecting approximately how many iterations it will take to implement a particular pile of User Stories. This longer-term estimate is more reliable if the shorter-term errors tend to be wrong roughly equally in both directions.
Calibrating with Industry Data
Another way of estimating the unknown is to purchase the information we need. Sometimes historical information from some other context is available for purchase from a consulting agency, for example. They may be willing to share that information with us if we hire them for their advice. Be wary in such situations. The nature of such a relationship is that they need to make their information look as valuable as possible in order to induce us to buy. It may not be as applicable as we hope. Often their historical information is packaged in a high-priced estimation tool. These tools are created by analyzing a large number of projects and trying to isolate significant factors that affect the time and cost of them. Some mathematical curve-fitting gives an approximation based on that particular sample of projects. Is your project like them? In what ways might it be similar to the central measures of that population, and in what ways might it be an outlier?
Performing a Spike
You can also purchase information by running your own experiments. You may not know how long something might take to do if you don’t know whether it can be done at all. When you need some information, consider how you could learn that information. Often there’s some small but critical bit of info that you need. Articulate what that critical bit is. Devise the most inexpensive way that might possibly give you that information. How much are you willing to spend on that?
This is often called a spike solution, or just spike, evoking an image of a very slender solution driven through the heart of the problem. I recommend the following procedure for spikes:
-
Articulate the question to be answered. If it’s vague, it’s hard to tell whether you’ve answered it or not.
-
Decide how long you’re willing to work on it. Setting a budget is important to prevent a research project that dwarfs the importance of the question.
-
Work on a solution until either the question is answered or the time budget has been expended. If you still don’t have your answer, start over and thinking about it again. Is it worth spending a little more? Would it likely be more fruitful to try a different approach? Is there a way to bypass the need for this answer, at least, for now?
A spike is a handy tool for answering many questions:
-
Can this library do the job we need?
-
Can we develop an algorithm to do the job we need?
-
Can we calculate this to the needed precision?
-
If we have a data stream that looks like this, can we process it to extract that information?
-
If we offer users a way to do something, will they be interested in it?
You can also admit what you don’t know, and perhaps can’t know, given your current circumstances. It’s also worth knowing why you can’t know. That also gives you ideas about how you could go about learning what you would need to know. It can also give you credibility over people who are claiming to know the unknowable.[7]
How soon do you need to turn this into numerical information? If it’s a small part of the whole, it’s not likely to have a big impact. There’s enough noise in the process of estimation that this unknown may be unnoticeable. When you get to the point where it matters, perhaps you’ll have the information you need to estimate it.
Building a prototype is similar, though typically larger. Sometimes people estimate a prototype like they do a production project. Given the uncertainty that’s leading you to explore, I recommend setting a budget and seeing what value you can produce within that budget. You retain the option to explicitly add to that budget if it seems worthwhile.