68. Predicting Innovation

Photo by Gabriel Bulla

The team balances its need for innovation with its employer’s need for predictability.

If the systems you build are at all interesting, then you know that innovation is part of every project. Innovation means that your team will be doing something it hasn’t done before, and that the success of the effort depends upon your people solving problems in new and different ways. At the same time, you are nearly always required to predict, with a reasonable level of accuracy, when you will be finished. This puts you in the business of predicting the arrival of good ideas, and that is not easy.

You probably understand this puzzle all too well: You need to walk a narrow path that provides your developers with enough time to explore, discover, learn, and solve, while at the same time providing your employer and customers with reasonably accurate expectations for the completion of the development effort. It’s so easy to wander off this narrow path to one side or the other. If you set expectations too optimistically, you may very well pressure your developers to finish, and so limit their ability to get the right product. On the other hand, if you refuse to predict completion until very late in the development cycle, you may find your boss less than thrilled with your own performance.

The development planning process goes something like this: Even after you have a reasonable set of product feature requirements, you know that your developers may need several iterations before they can figure out what they need to build and how to build it. So you plan two or three early time-boxed iterations for prototyping and exploration. How long each iteration lasts depends on the nature and scale of the effort, but we typically see them lasting from one to four weeks each.

After two or three such iterations, most of the technical unknowns have become knowns, and development of final-state product features can begin. Here, too, iteration is important. Teams typically use anywhere from three to five additional iterations to complete the release. Once again, the duration of each iteration varies from team to team, but they often range from four to twelve weeks each. So, the overall development timeline for the release looks something like this:

In this example, the first three iterations are focused on exploration and prototyping of the entire feature space. This does not mean that the result of Itn 1 will be a prototype of all the features that will be in the shipping product. It just means that the developers will be free to work on any aspects of the product that they choose. It also means that the result of Itn 1 may be completely discarded at the start of Itn 2. It is more likely that some of the Itn 1 code will form the basis of the Itn 2 work, but that is not a necessity. The goal of Itn 1 (and of Itn 2, and if necessary, Itn 3) is to work out the unknowns and to reduce the remaining uncertainty in the development effort. Altogether, Itn 1 through Itn 3 might take anywhere from six to twelve weeks. Your mileage may vary.

The final iterations (there may be three or two or six of these, depending on your situation) constitute the construction of the shipping product, most probably using some of the code developed in the first three iterations. These iterations typically take more time than the exploratory ones; they are usually four-to-twelve weeks long, depending primarily on the development method used by the team.

How does this approach help you balance innovation with prediction? Balance comes from building into a fixed development period multiple passes through those areas of the problem space that require innovation. Does this guarantee that your team will generate the required good ideas, at the right time? Of course not. But this approach allows several creative iterations to be planned as part of the overall development time frame.

Projects thrive on innovation; they also need predictability. However, too much or too little of either can be crippling.