In spite of a long list of risks to watch for, Evolutionary Prototyping is a relatively low-risk practice. Of published case studies, 33 of the 39 have reported success (Gordon and Bieman 1995). Here are some risks to keep in mind.

Unrealistic schedule and budget expectations. Evolutionary Prototyping may create unfulfillable expectations about the overall development schedule. When end-users, managers, or marketers see rapid progress on a prototype, they sometimes make unrealistic assumptions about how quickly you can develop the final product. In some instances, the sales staff passes along unrealistic schedule expectations to customers. Customers later become upset when they hear that the software will take longer than they expected.

It's easy to complete the most visible work quickly, but a great deal of the work in a programming project isn't obvious to the customer or end-user. Low-visibility work includes robust database access, maintenance of database integrity, security, networking, data conversion between successive product versions, design for high-volume usage, multi-user support, and multi-platform support. The better this work is done, the less noticeable it will be.

Another problem with the effort expectations created when prototyping starts with the user interface is that some of the functionality that looks easiest from the end-user's point of view is the most difficult to implement from the developer's point of view. Difficult areas that look easy include cut-and-paste within an application, cut-and-paste from other applications, printing, print preview, and WYSIWYG screen displays.

Finally, prototypes are generally designed to handle only the nominal cases; they aren't expected to handle the exceptional cases. But as much as 90 percent of a normal program consists of exception handling (Shaw in Bentley 1982), which suggests that even if a prototype were completely functional for all nominal cases, it would still take up to 9 times as much effort to implement the exceptional cases as it took to implement the fully functioning, nominal-case prototype.

If management does not understand the difference between creating a limited prototype and a full-scale product, it will be at risk of seriously underbudgeting prototyping projects. Scott Gordon and James Bieman reported a case in which management bought a 2-year project as a 6-week project. Management believed that the prototyping would produce fully functional results in an amount of time that was out of touch with real-world needs by a factor of almost 20 (Gordon and Bieman 1995).

You can manage this risk by explicitly managing the expectations of end-users, managers, developers, and marketers. Be sure they interact with the prototype in a controlled setting in which you can explain the prototype's limitations, and be sure that they understand the difference between creating a prototype and creating completely functional software.

Diminished project control. With Evolutionary Prototyping, it's impossible to know at the outset of the project how long it will take to create an acceptable product. You don't know how many iterations it will take or how long each iteration will last.

This risk is mitigated somewhat by the fact that customers can see steady signs of progress and tend to be less nervous about eventually getting a product than with traditional development approaches. It's also possible to use an evolutionary-delivery approach that's strongly tilted toward Evolutionary Prototyping, which provides some of the control of Staged Delivery and some of the flexibility of Evolutionary Prototyping.

Poor end-user or customer feedback. Prototyping doesn't guarantee high-quality end-user or customer feedback. End-users and customers don't always know what they're looking at when they see a prototype. One common phenomenon is that they are so overwhelmed by the live software demonstration that they don't look past the glitz to understand what the prototype really represents. If they give you a rubber-stamp approval, you can be virtually certain that they don't fully understand what they're seeing. Be sure that end-users and customers study the prototype carefully enough to provide meaningful feedback.

Poor product performance. Several factors can contribute to poor product performance, including:

You can take three steps to minimize the risk of poor performance:

Unrealistic performance expectations. The risk of unrealistic performance expectations is related to the risk of poor performance. This risk arises when the prototype has much better performance than the final product. Because a prototype doesn't have to do all of the work that a final product does, it can sometimes be very fast compared to the final product. When customers see the final product, it can appear to be unacceptably slow.

This risk also has an evil twin. If a prototyping language is used that has much worse performance than the target language, customers may equate weaknesses in the prototype's performance with weaknesses in the final product. More than one case study has reported project failure as a result of low expectations created by a prototype (Gordon and Bieman 1995).

You can address this risk by explicitly managing the expectations that customers develop from interacting with the prototype. If the prototype is very fast, add artificial delays to the prototype so that the prototype's performance approximates the performance of the final product. If the prototype is very slow, explain to customers that the performance of the prototype isn't indicative of the performance of the final product.

Poor design. Many prototyping projects have better designs than traditional projects, but several factors contribute to the risk of poor design.

You can mitigate design-quality risks by striking at the root of the problem. Limit the scope of the prototype to specific areas of the product, for example, to the user interface. Develop this limited part of the product using a prototyping language, and evolve it; then develop the rest of the product using a traditional programming language and nonprototyping approaches.

When creating the overall system design, make a conscious effort not to overemphasize the user interface or any other aspect of the system that has been prototyped. Minimize the impact that design in one area can have on the design as a whole.

Include a design phase with each iteration of the prototype to ensure that you evolve the design and not just the code. Use a design checklist at each stage to check the quality of the evolving product.

Avoid using inexperienced developers on a prototyping project. Evolutionary Prototyping requires that developers make far-reaching design decisions much earlier in the development process than when other development approaches are used. Inexperienced developers are often poorly equipped to make good design decisions under such circumstances. Case studies of prototyping projects have reported projects that have failed because of the involvement of inexperienced developers. They have also reported projects that have succeeded only because of the involvement of experienced developers (Gordon and Bieman 1995).

Poor maintainability. Evolutionary Prototyping sometimes contributes to poor maintainability. The high degree of modularity needed for effective Evolutionary Prototyping is conducive to maintainable code. But when the prototype is developed extremely rapidly, it is also sometimes developed extremely sloppily—Evolutionary Prototyping can be used as a disguise for code-and-fix development. Moreover, if a special-purpose prototyping language is used, the language may be unfamiliar to maintenance programmers, which will make the program difficult to maintain. In the published literature on prototyping, more sources than not have observed worse maintainability with Evolutionary Prototyping than with traditional approaches (Gordon and Bieman 1995).

Minimize the maintainability risk by taking a few simple steps. Be sure that the design is adequate by using the guidelines listed in the discussion of the "Poor design" risk (in the preceding section). Use code-quality checklists at each stage to keep the prototype's code quality from slipping. Follow normal naming conventions for objects, methods, functions, data, database elements, libraries, and any other components that will be maintained as the prototype matures. Follow other coding conventions for good layout and commenting. Remind your team that they will have to use their own code throughout multiple product-release cycles—that will give them a strong incentive to make their code maintainable and to keep it that way.

Feature creep. Customers and end-users typically have direct access to the prototype during an evolutionary-prototyping project, and that can sometimes lead to an increased desire for features. In addition to managing their interactions with the prototype, use normal change-management practices to control feature creep.

Inefficient use of prototyping time. Prototyping is usually intended to shorten the development schedule. Paradoxically, projects often waste time during prototyping and don't save as much time as they could. Prototyping is an exploratory, iterative process, so it is tempting to treat it as a process that can't be managed carefully. Developers don't always know how far to develop a prototype, so they sometimes waste time fleshing out features that are summarily excluded from the product later. Developers sometimes give the prototype a robustness it doesn't need or waste time working on the prototype without moving it in any clear direction.

To spend prototyping time effectively, make project tracking and control a priority. Carefully schedule prototyping activities. Later in the project you might break the project into chunks of a few days or weeks, but during prototyping break it into hours or days. Be sure that you don't put a lot of work into prototyping and evolving any part of the system until you're certain it's going to remain part of the system.

Developers need to have a well-developed sense of how little they can do to investigate the risky areas a prototype is intended to explore. On one project I worked on, we needed to create a user-interface prototype to demonstrate our user-interface design to our sponsor. One of the other developers thought it would take a team of three developers 6 weeks to build the prototype. I thought I could do all the work myself in about 1 week, which turned out to be accurate. The difference in our estimates didn't arise because I was six times as fast as three developers; they arose because the other developer didn't have a good sense of all the things that could be left out of that kind of a prototype. Ergo, avoid using inexperienced developers on prototyping projects.