Customer or Market Surveys

Most companies conduct customer surveys of some sort. In consumer markets, this can be problematic simply because customers are unable to articulate their future needs. For example, Apple’s iPod was not the result of extensive market research or customer demand, but largely because of the vision and commitment of Steve Jobs. In industrial markets, customers tend to be better equipped to communicate their future requirements, and consequently, business-to-business innovations often originate from customers. Companies can also consult their direct sales force, but these may not always be the best guide to future customer requirements. Information is often filtered in terms of existing products and services and biased in terms of current sales performance rather than long-term development potential.

There is no “one best way” to identify novel niches, but rather a range of alternatives. For example, where new products or services are very novel or complex, potential users may not be aware of, or able to articulate, their needs. In such cases, traditional methods of market research are of little use, and there will be a greater burden on developers of radical new products and services to “educate” potential users.

Our own research confirms that different managerial processes, structures, and tools are appropriate for routine and novel development projects [3]. We discuss this in detail in Chapter 9, when we examine new product and service development. For example, in terms of frequency of use, the most common methods used for high-novelty projects are segmentation, prototyping, market experimentation, and industry experts, whereas for the less novel projects, the most common methods are partnering customers, trend extrapolation, and segmentation. The use of market experimentation and industry experts might be expected where market requirements or technologies are uncertain, but the common use of segmentation for such projects is harder to justify. However, in terms of usefulness, there are statistically significant differences in the ratings for segmentation, prototyping, industry experts, market surveys, and latent needs analysis. Segmentation is more effective for routine development projects; and prototyping, industry experts, focus groups, and latent needs analysis are all more effective for novel development projects [4]. Research Note 9.2 identifies the factors that influence the accuracy of predictions of new product sales.

Internal Analysis, for Example, Brainstorming

Structured idea generation, or brainstorming, aims to solve specific problems or to identify new products or services. Typically, a small group of experts are gathered together and allowed to interact. A chairman records all suggestions without comment or criticism. The aim is to identify, but not evaluate, as many opportunities or solutions as possible. Finally, members of the group vote on the different suggestions. The best results are obtained when representatives from different functions are present, but this can be difficult to manage. Brainstorming does not produce a forecast as such, but can provide useful input to other types of forecasting.

We discussed a range of approaches to creative problem-solving and idea generation in Chapter 3. Most of these are relevant here and include ways of [5]:

• Understanding the problem – the active construction by the individual or group through analyzing the task at hand (including outcomes, people, context, and methodological options) to determine whether and when deliberate problem-structuring efforts are needed. This stage includes constructing opportunities, exploring data, and framing problems.
• Generating ideas – to create options in answer to an open-ended problem. This includes generating and focusing phases. During the generating phase of this stage, the person or group produces many options (fluent thinking), a variety of possible options (flexible thinking), novel or unusual options (original thinking), or a number of detailed or refined options (elaborative thinking). The focusing phase provides an opportunity for examining, reviewing, clustering, and selecting promising options.
• Planning for action – is appropriate when a person or group recognizes a number of interesting or promising options that may not necessarily be useful, valuable, or valid. The aim is to make or develop effective choices and to prepare for successful implementation and social acceptance.

External Assessment, for Example, Delphi

The opinion of outside experts, or Delphi method, is useful where there is a great deal of uncertainty or for long-time horizons [6]. Delphi is used where a consensus of expert opinion is required on the timing, probability, and identification of future technological goals or consumer needs and the factors likely to affect their achievement. It is best used in making long-term forecasts and revealing how new technologies and other factors could trigger discontinuities in technological trajectories. The choice of experts and the identification of their level and area of expertise are important; the structuring of the questions is even more important. The relevant experts may include suppliers, dealers, customers, consultants, and academics. Experts in nontechnological fields can be included to ensure that trends in economic, social, and environmental fields are not overlooked.

The Delphi method begins with a postal survey of expert opinion on what the future key issues will be and the likelihood of the developments. The response is then analyzed, and the same sample of experts resurveyed with a new, more focused questionnaire. This procedure is repeated until some convergence of opinion is observed or, conversely, if no consensus is reached. The exercise usually consists of an iterative process of questionnaire and feedback among the respondents; this process finally yields a Delphi forecast of the range of experts’ opinions on the probabilities of certain events occurring by a quoted time. The method seeks to nullify the disadvantage of face-to-face meetings at which there could be deference to authority or reputation, a reluctance to admit error, a desire to conform or differences in persuasive ability. All of these could lead to an inaccurate consensus of opinion. The quality of the forecast is highly dependent on the expertise and caliber of the experts; how the experts are selected and how many should be consulted are important questions to be answered. If international experts are used, the exercise can take a considerable length of time, or the number of iterations may have to be curtailed. Although seeking a consensus may be important, adequate attention should be paid to views that differ radically “from the norm” as there may be important underlying reasons to justify such maverick views. With sufficient design, understanding, and resources, most of the shortcomings of the Delphi technique can be overcome, and it is a popular technique, particularly for national foresight programs.

In Europe, governments and transnational agencies use Delphi studies to help formulate policy, usually under the guise of “Foresight” exercises. In Japan, large companies and the government routinely survey expert opinion in order to reach some consensus in those areas with the greatest potential for long-term development. Used in this way, the Delphi method can, to a large extent, become a self-fulfilling prophecy.

Scenario Development

Scenarios are internally consistent descriptions of alternative possible futures, based upon different assumptions and interpretations of the driving forces of change [7]. Inputs include quantitative data and analysis and qualitative assumptions and assessments, such as societal, technological, economic, environmental, and political drivers. Scenario development is not strictly speaking prediction, as it assumes that the future is uncertain and that the path of current developments can range from the conventional to the revolutionary. It is particularly good at incorporating potential critical events, which might result in divergent paths or branches being pursued.

Scenario development can be normative or explorative. The normative perspective defines a preferred vision of the future and outlines different pathways from the goal to the present. For example, this is commonly used in energy futures and sustainable futures scenarios. The explorative approach defines the drivers of change and creates scenarios from these without explicit goals or agendas.

For scenarios to be effective, they need to inclusive, plausible, and compelling (as opposed to being exclusive, implausible, or obvious), as well as being challenging to the assumptions of the stakeholders. They should make the assumptions and inputs used explicit and form the basis of a process of discussion, debate, policy, strategy, and ultimately action. The output is typically two or three contrasting scenarios, but the process of development and discussion of scenarios is much more valuable.

Scenario development may involve many different forecasting techniques, including computer-based simulation. Typically, it begins with the identification of the critical indicators, which might include use of brainstorming and Delphi techniques. Next, the reasons for the behavior of these indicators is examined, perhaps using regression techniques. The future events that are likely to affect these indicators are identified. These are used to construct the best, worst, and most-likely future scenarios. Finally, the company assesses the impact of each scenario on its business. The goal is to plan for the outcome with the greatest impact or, better still, retain sufficient flexibility to respond to several different scenarios. Scenario development is a key part of the long-term planning process in those sectors characterized by high capital investment, long lead times, and significant environmental uncertainty, such as energy, aerospace, and telecommunications. Case Study 9.2 shows how a large IT company uses scenario planning.

Processes of Diffusion

Research on diffusion attempts to identify what influences the rate and direction of adoption of an innovation. The diffusion of an innovation is typically described by an S-shaped (logistic) curve, as shown in Figure 9.1. Initially, the rate of adoption is low, and adoption is confined to the so-called innovators. Next to adopt are the “early adopters,” then the “late majority,” and finally, the curve tails off as only the “laggards” remain. Such taxonomies are fine with the benefit of hindsight, but provide little guidance for future patterns of adoption [8].

Hundreds of marketing studies have attempted to fit the adoption of specific products to the S-curve, ranging from television sets to new drugs. In most cases, mathematical techniques can provide a relatively good fit with historical data, but research has so far failed to identify robust generic models of adoption. In practice, the precise pattern of adoption of an innovation will depend on the interaction of demand-side and supply-side factors:

• Demand-side factors – direct contact with or imitation of prior adopters, adopters with different perceptions of benefits and risk
• Supply-side factors – relative advantage of an innovation, availability of information, barriers to adoption, feedback between developers and users

The epidemic S-curve model is the earliest and is still the most commonly used. It assumes a homogeneous population of potential adopters and that innovations spread by information transmitted by personal contact, observation, and the geographical proximity of existing and potential adopters. This model suggests that the emphasis should be on communication and the provision of clear technical and economic information. However, the epidemic model has been criticized because it assumes that all potential adopters are similar and have the same needs, which is unrealistic.

The Probit model takes a more sophisticated approach to the population of potential adopters. It assumes that potential adopters have different threshold values for costs or benefits and will only adopt beyond some critical or threshold value. In this case, differences in threshold values are used to explain different rates of adoption. This suggests that the more similar potential adopters are, the faster the diffusion.

However, adopters are assumed to be relatively homogeneous, apart from some difference in progressiveness or threshold values. Supply-side models do not consider the possibility that the rationality and the profitability of adopting a particular innovation might be different for different adopters. For example, local “network externalities” such as the availability of trained skilled users, technical assistance and maintenance, or complementary technical or organizational innovations are likely to affect the cost of adoption and use, as distinct from the cost of purchase.

Also, it is unrealistic to assume that adopters will have perfect knowledge of the value of an innovation. Therefore, Bayesian models of diffusion introduce lack of information as a constraint to diffusion. Potential adopters are allowed to hold different beliefs regarding the value of the innovation, which they may revise according to the results of trials to test the innovation. Because these trials are private, imitation cannot take place and other potential adopters cannot learn from the trials. This suggests that better-informed potential adopters may not necessarily adopt an innovation earlier than the less well informed, which was an assumption of earlier models [9].

Slightly more realistic assumptions, such as those of the Bass model, include two different groups of potential adopters: innovators, who are not subject to social emulation; and imitators, for whom the diffusion process takes the epidemic form. This produces a skewed S-curve because of the early adoption by innovators and suggests that different marketing processes are needed for the innovators and subsequent imitators. The Bass model is highly influential in economics and marketing research, and the distinction between the two types of potential adopters is critical in understanding the different mechanisms involved in the two user segments.

Bandwagons may occur where an innovation is adopted because of pressure caused by the sheer number of those who have already adopted an innovation, rather than by individual assessments of the benefits of an innovation. In general, as soon as the number of adopters has reached a certain threshold level, the greater the level of ambiguity of the innovation’s benefits, the greater the subsequent number of adopters. This process allows technically inefficient innovations to be widely adopted or technically efficient innovations to be rejected. Examples include the QWERTY keyboard, originally designed to prevent professional typists from typing too fast and jamming typewriters, and the DOS operating system for personal computers, designed by and for computer enthusiasts.

Bandwagons occur due to a combination of competitive and institutional pressures [10]. Where competitors adopt an innovation, a firm may adopt because of the threat of lost competitiveness, rather than as a result of any rational evaluation of benefits. For example, many firms adopted flexible manufacturing systems (FMS) in the 1980s in response to increased competition, but most failed to achieve significant benefits. The main institutional pressure is the threat of lost legitimacy, for example, being considered by peers or customers as being less progressive or competent [11].

The critical difference between bandwagons and other types of diffusion is that they require only limited information to flow from early to later adopters. Indeed, the more ambiguous the benefits of an innovation, the more significant bandwagons are on rates of adoption. Therefore, the process of diffusion must be managed with as much care as the process of development. In short, better products do not necessarily result in more sales. Not everybody requires a better mousetrap.

Finally, there are more sociological and psychological models of adoption, which are based on interaction and feedback between the developers and potential adopters [12]. These perspectives consider how individual psychological characteristics such as attitude and perception affect adoption. Individual motivations, perceptions, likes, and dislikes determine what information is reacted to and how it is processed. Potential adopters will be guided and prejudiced by experience and will have “cognitive maps,” which filter information and guide behavior. Social context will also influence individual behavior. Social structures and meaning systems are locally constructed and therefore highly context-specific. These can distort the way in which information is interpreted and acted upon. Therefore, the perceived value of an innovation, and hence its subsequent adoption, is not some objective fact, but instead depends on individual psychology and social context. These factors are particularly important in the later stages of diffusion. For example, lifestyle aspirations, such as having more exercise and adopting an healthy diet, have created the opportunity for many new products and services.

Initially, the needs of early adopters or innovators dominate, and therefore, the characteristics of an innovation are most important. Innovations tend to evolve over time through improvements required by these early users, which may reduce the relative cost to later adopters. However, early adopters are almost by definition “atypical,” for example, they tend to have superior technical skills. As a result, the preferences of early adopters can have a disproportionate impact on the subsequent development of an innovation and result in the establishment of inferior technologies or abandonment of superior alternatives. Research Note 9.4 examines the roles of early adopters and opinion leaders in the adoption of innovations.

Factors Influencing Adoption

Numerous variables have been identified as affecting the diffusion and adoption of innovations, but these can be grouped into three clusters: characteristics of the innovation itself, characteristics of individual or organizational adopters, and the characteristics of the environment. Characteristics of an innovation found to influence adoption include relative advantage, compatibility, complexity, observability, and trialability. Individual characteristics include age, education, social status, and attitude to risk. Environmental and institutional characteristics include economic factors such as the market environment and sociological factors such as communications networks. However, while there is a general agreement regarding the relevant variables, there is very little consensus on the relative importance of the different variables and, in some cases, disagreements over the direction of relationships. Case Study 9.3 identifies factors that have promoted and hindered the adoption of the Internet in China.

Characteristics of an Innovation

A number of characteristics of an innovation have been found to affect diffusion and adoption:

• Relative advantage
• Compatibility
• Complexity
• Trialability
• Observability

Risk as Probability

Research indicates that 30–45% of all projects fail to be completed, and over half of the projects overrun their budgets or schedules by up 200%. Figure 9.2 presents the results of a survey of R&D managers. While most appear to be relatively confident when predicting technical issues such as the development time and costs, a much smaller proportion are confident when forecasting commercial aspects of the projects.

We examined how commonly different approaches to project assessment were used in practice. We surveyed 50 projects in 25 companies and assessed how often different criteria were used and how useful they were thought to be. Table 9.3 summarizes some of the results. Clearly, probabilistic estimates of technical and commercial success are near universal and considered to be of critical importance in all types of project assessment. These are usually combined with some form of financial assessment and fit with the company strategy and capabilities.

Given the complexities involved, the outcomes of investments in innovation are uncertain, so that the forecasts (of costs, prices, sales volume, etc.) that underlie project and program evaluations can be unreliable. According to Joseph Bower, management finds it easier, when appraising investment proposals, to make more accurate forecasts of reductions in production cost than of expansion in sales, while their ability to forecast the financial consequences of new product introductions is very limited indeed [23]. This last conclusion is confirmed by the study by Edwin Mansfield and his colleagues of project selection in large US firms [24]. By comparing project forecasts with outcomes, Mansfield showed that managers find it difficult to pick technological and commercial winners:

• Probability of technical success of projects (Pt) = 0.80
• Subsequent probability of commercial success (Pc) = 0.20
• Combined probability for all stages: 0.8 × 0.2 = 0.16

He also found that managers and technical managers cannot accurately predict the development costs, time periods, markets, and profits of R&D projects. On average, costs were greatly underestimated, and time periods overestimated by 140–280% in incremental product improvements and by 350–600% in major new products. Other studies have found the following:

• About half of the business R&D expenditures are on failed R&D projects. The higher rate of success in expenditures than in projects reflects the weeding out of unsuccessful projects at their early stages and before large-scale commercial commitments are made to them [25].
• R&D scientists and engineers are often deliberately overoptimistic in their estimates, in order to give the illusion of a high rate of return to accountants and managers [26].

Trying to get involved in the right projects is worth an effort, both to avoid wasting time and resources in meaningless activities and to improve the chances of success. Project appraisal and evaluation aim to:

1. Profile and gain an overall understanding of potential projects.
2. Prioritize a given set of projects, and where necessary, reject projects.
3. Monitor projects, for example, by following up the criteria chosen when the project was selected.
4. Where necessary, terminate a project.
5. Evaluate the results of completed projects.
6. Review successful and unsuccessful projects to gain insights and improve future project management, that is, learning.

Project evaluation usually assumes that there is a choice of projects to pursue, but where there is no choice project evaluation is still important to help to assess the opportunity costs and what might be expected from pursuing a project. Different situations and contexts demand different approaches to project evaluation. We argued earlier that complexity and uncertainty are two of the most important dimensions for assessing projects. Different types of project will demand specific techniques or at least different criteria for assessment.

A large number of techniques have been developed over the years and are still being developed and used today. Most of these can be described by means of some common elements that form the core of any project evaluation technique:

• Inputs into the assessment include likely costs and benefits in financial terms, probability of technical and market success, market attractiveness, and the strategic importance to the organization.
• Weighting gives certain data more relevance than other (e.g., of market inputs compared with technical factors), in order to reflect the company’s strategy or the company’s particular views. The data is then processed to arrive at the outcomes.
• Balancing a range of projects, as the relative value of a project with respect to other projects is an important factor in situations of competition for limited resources. Portfolio management techniques are specifically devoted to deal with this factor.

Economic and cost–benefit approaches are usually based on a combination of expected utility or Bayesian assumptions. Expected utility theory can take into account probabilistic estimates and subjective preferences, and therefore it deals well with risk aversion, but in practice, utility curves are almost impossible to construct, and individual preferences are different and highly subjective. Bayesian probability is excellent at incorporating the effects of new information, as we discussed earlier under the diffusion of innovations, but is very sensitive to the choice of relevant inputs and the weights attached to these.

As a result, no technique should be allowed to determine outcomes, as these decisions are a management responsibility. Many techniques used today are totally or partially software based, which have some additional benefits in automating the process. In any case, the most important issue, for any method, is the managers’ interpretation.

There is no single “best” technique. The extent to which different techniques for project evaluation can be used will depend upon the nature of the project, the information availability, the company’s culture, and several other factors. This is clear from the variety of techniques that are theoretically available and the extent to which they have been used in practice. In any case, no matter which technique is selected by a company, it should be implemented, and probably adapted, according to the particular needs of that organization. Most of the techniques in practical use incorporate a mixture of financial assessment and human judgment.

Perceptions of Risk

Probability estimates are only the starting point of risk assessment. Such relatively objective criteria are usually significantly moderated by psychological (cognitive) perceptions and bias, or overwhelmed altogether by sociological factors, such as peer pressure and cultural context. Studies suggest that different people (and animals) have different perceptions and tolerances for risk-taking. For example, a study comparing the behaviors of chimpanzees and bonobo apes found that the chimps were more prepared to gamble and take risks [27]. At first sight, this appears to support the personality explanation for risk-taking, but actually the two types of apes share more than 99% of their DNA. A more likely explanation is the very different environments in which they have evolved: in the chimp environment, food is scarce and uncertain, but in the bonobo habitats, food is plentiful. We are not suggesting that entrepreneurs are chimp-like, or accountants are ape-like, but rather that experience and context have a profound influence on the assessment of, and appetite for, risk.

At the individual, cognitive level, risk assessment is characterized by overconfidence, loss aversion, and bias [28]. Overconfidence in our ability to make accurate assessments is a common failing and results in unrealistic assumptions and uncritical assessment. Loss aversion is well documented in psychology and essentially means that we tend to prefer to avoid loss rather than to risk gain. Finally, cognitive bias is widespread and has profound implications for the identification and assessment of risk. Cognitive bias results in us seeking and overemphasizing evidence that supports our beliefs and reinforces our bias, but, at the same time, leads us to avoid and undervalue any information that contradicts our view [29]. Therefore, we need to be aware of and challenge our own biases and encourage others to debate and critique our data, methods, and decisions.

Studies of research and development confirm that measures of cognitive ability are associated with project performance. In particular, differences in reflection, reasoning, interpretation, and sense making influence the quality of problem formulation, evaluation and solution, and therefore, ultimately, the performance of research and development. A common weakness is the oversimplification of problems characterized by complexity or uncertainty and the simplification of problem framing and evaluation of alternatives [30]. This includes adopting a single prior hypothesis, selective use of information that supports this, and devaluing alternatives, and illusion of control and predictability. Similarly, marketing managers are likely to share similar cognitive maps and make the same assumptions concerning the relative importance of different factors contributing to new product success, such as the degree of customer orientation versus competitor orientation, and the implications of relationship between these factors, such as the degree of inter-functional coordination [31]. So the evidence indicates the importance of cognitive processes at the senior management, functional, group, and individual levels of an organization. More generally, problems of limited cognition include [32] the following:

• Reasoning by analogy, which oversimplifies complex problems
• Adopting a single, prior hypothesis bias, even where information and trails suggest that this is wrong
• Limited problem set, the repeated use of a narrow problem-solving strategy
• Single outcome calculation, which focuses on a simple single goal and a course of action to achieve it, and denying value trade-offs
• Illusion of control and predictability, based on an overconfidence in the chosen strategy, a partial understanding of the problem and limited appreciation of the uncertainty of the environment
• Devaluation of alternatives, emphasizing negative aspects of alternatives

At the group or social level, other factors also influence our perception and response to risk. How managers assess and manage risk is also a social and political process. It is influenced by prior experience of risk, perceptions of capability, status and authority, and the confidence and ability to communicate with relevant people at the appropriate times [33]. In the context of managing innovation, risk is less about personal propensity for risk-taking or rational assessments of probability and more about the interaction of experience, authority, and context. In practice, managers deal with risk in different ways in different situations. General strategies include delaying or delegating decisions or sharing risk and responsibilities. Generally, when mangers are performing well, and achieving their targets, they have less incentive to take risks. Conversely, when under pressure to perform, managers will often accept higher risks, unless these threaten survival.

In most organizations, risk has become a negative term, something that should be minimized or avoided, and implies hazard or failure. This view, particularly common in the policy domain, is enshrined in the “precautionary principle” and the many regulatory regimes it has spawned, which, as the title suggests, wherever possible, promotes the avoidance of risk-taking [34].

However, this interpretation perverts the nature of risk and opportunity, which are central to successful innovation, and promotes inaction and the status quo, rather than improvement or change. The term “risk” is derived from the Latin “to dare,” but has become associated with hazard or danger. We must also consider the “risk” of success or risks associated with not changing [35]. Berglund provides a good working definition of risk in the context of innovation, as “the pursuit of perceived opportunities under conditions of uncertainty” [36].

In a corporate context, he identifies three aspects of risk that need to be managed:

• Compliance with formal project and process requirements, rather than innovation outcomes
• Internal control and autonomy and influence and use of external expertise
• Flexibility of the business model and experimentation with alternative configurations and organization

In any large organization, there will be formal process and project requirements. However, these may conflict with the goals of innovation. Risk-taking requires a degree of tolerance of uncertainty and ambiguity in the workplace. In the high risk-taking climate, bold new initiatives can be taken even when the outcomes are unknown. People feel that they can “take a gamble” on some of their ideas. People will often “go out on a limb” and be first to put an idea forward. In a risk-avoiding climate, there is a cautious, hesitant mentality. People try to be on the “safe side.” They decide “to sleep on the matter.” They set up committees, and they cover themselves in many ways before making a decision. When risk-taking is too low, employees offer few new ideas or few ideas that are well outside of what is considered safe or ordinary. In risk-avoiding organizations, people complain about boring, low-energy jobs and are frustrated by a long, tedious process used to get ideas to action. These conditions can be caused by the organization not valuing new ideas or having an evaluation system that is bureaucratic, or people being punished for “drawing outside the lines.” It can be remedied by developing a company plan that would speed “ideas to action.” When risk-taking is too high, you will see that people are confused. There are many ideas floating around, but few are sanctioned. People are frustrated because nothing is getting done. There are many loners doing their own thing in the organization and no evidence of teamwork. These conditions can be caused by individuals not feeling they need a consensus or buy-in from others on their team in their department or organization. A remedy might include some team building and improving the reward system to encourage cooperation rather than individualism or competition [5].

A recent study of organizational innovation and performance confirms the need for this delicate balance between risk and stability. Risk-taking is associated with a higher relative novelty of innovation (how different it was to what the organization had done before), and absolute novelty (how different it was to what any organization had done before), and that both types of novelty are correlated with financial and customer benefits [37]. However, the same study concludes that “incremental, safe, widespread innovations may be better for internal considerations, but novel, disruptive innovations may be better for market considerations … absolute novelty benefits customers and quality of life, relative innovation benefits employee relations (but) risk is detrimental to employee relations.” In fact, many of the critical risks that need to be identified and managed are internal to organizations, rather than the more obviously anticipated external risks such as markets, competition, and regulation [38]. For example, at 3M, 100 years of successful innovation was almost reversed following a change of CEO and an emphasis on Six-Sigma quality processes, rather than maintaining an innovative climate and products.

The inherent uncertainty in some projects limits the ability of managers to predict the outcomes and benefits of projects. In such cases, changes to project plans and goals are commonplace, being driven by external factors, such as technological breakthroughs, or changes in markets, as well as internal factors, such as changes in organizational goals. Together, the impact of changes to project plans and goals can overwhelm the benefits of formal project planning and management, as shown in Table 9.4 [22].

This is consistent with the real options approach to investing in risky projects, because investments are sequential, and managers have some influence on the timing, resourcing, and continuation or abandonment of projects at different stages. By investing relatively small amounts in a wide range of projects, a greater range of opportunities can be explored. Once uncertainty has been reduced, only the most promising projects should be allowed to continue. For a given level of investment, this real options approach should increase the value of the project portfolio. However, because decisions and the options they create interact, a decision regarding one project can affect the option value of another project [39,40]. Nonetheless, the real options perspective remains a useful way of conceptualizing risk, particularly at the portfolio level. The goal is not to calculate or optimize, but rather to help to identify risks and payoffs, key uncertainties, decision points, and future opportunities that might be created [41]. Combined with other methods, such as decision trees, a real options approach can be particularly effective where high volatility demands flexibility, placing a premium on the certainty of information and timing of decisions.

Financial Assessment of Projects

As we showed earlier, financial methods are still the most commonly used method of assessing innovative projects, but usually in combination with other, often more qualitative approaches. The financial methods range from simple calculation of payback period or return on investment, to more complex assessments of net present value (NPV) through discounted cash flow (DCF).

Project appraisal by means of DCF is based on the concept that money today is worth more than money in the future. This is not because of the effect of inflation, but reflects the difference in potential investment earnings, that is, the opportunity cost of the capital invested.

The NPV of a project is calculated using:

where:

• P_t = Forecast cash flow in time period t
• T = Project life
• i = Expected rate of return on securities equivalent in risk to project being evaluated
• C = Cost of project at time t = 0.

In practice, rather than use this formula, it is easy to create standard NPV templates in a spreadsheet package such as Excel.

How to Evaluate Learning?

However, the potential benefits of innovative activities are twofold. First, extra profits are derived from increased sales and/or higher prices for superior products and from lower costs and/or increased sales from superior production processes. Conventional project appraisal methods can be used to compare the value of these benefits against their cost. Second, accumulated firm-specific knowledge (“learning,” “intangible assets”) that may be useful for the development of future innovations (e.g., new uses for solar batteries, carbon fiber, robots, word processing). This type of benefit is relatively more important in R&D projects that are more long-term, fundamental, and speculative.

Conventional techniques cannot be used to assess this second type of benefit, because it is an “option”—in other words, it creates the opportunity for the firm to invest in a potentially profitable investment, but the realization of the benefits still depends on a decision to commit further resources. Conventional project appraisal techniques cannot evaluate options, as shown in Research Note 9.6.

The inherent uncertainty in most R&D projects limits the ability of managers to predict the outcomes and benefits of projects. Research suggests that changes to R&D plans and goals are common, being driven by external factors, such as technological breakthroughs, as well as internal factors, such as changes in the project goals. Together, the impact of changes to project plans and goals overwhelms the effects of the quality of formal project planning and management [22]. This reality is consistent with the real options approach to investing in R&D, because investments are sequential, and managers have some influence on the timing, resourcing, and continuation or abandonment of projects at different stages. By investing relatively small amounts in a wide range of projects, a greater range of technological opportunities can be explored. Once uncertainty has been reduced, only the most promising projects are allowed to continue. For a given level of R&D investment, this real options approach should increase the value of the project portfolio. However, because options interact, a decision regarding one project can affect the option value of another project (unlike NPV calculations, which rarely include interaction effects). Therefore, the creation of further options through R&D projects may not increase the overall option value of the R&D portfolio, and conversely, the interaction of options arising from different projects can give rise to a nonlinear increase in the combined option value [39].

However, in almost all cases, it is impossible to calculate the value of R&D using real options, because unlike financial options, it is difficult to predict technological breakthroughs, estimate future sales from products flowing from the R&D (or project payoff), or identify and model project-specific risks and the time-varying volatilities of the processes and eventual values [40]. Nonetheless, the real options perspective remains a useful way of conceptualizing R&D investment, particularly at the portfolio level. It can help to make more explicit and to identify future growth options created by R&D, even when these are not related to the (current) goals of the R&D. Combined with decision trees, a real options approach can help to identify risks and payoffs, key uncertainties, decision points, and future branches (options) [41]. It is particularly effective where high volatility demands flexibility, placing a premium on the certainty of information and timing of decisions, as shown in Research Note 9.7.

In other words, the successful allocation of resources to innovation depends less on robustness of decision-making techniques than on the organizational processes in which they are embedded. According to Mitchell and Hamilton [43], there are three (overlapping) categories of innovation that large firms must finance. Each category has different objectives and criteria for selection, the implications of which are set out in Table 9.5.

• Knowledge building This is the early-stage and relatively inexpensive research for nurturing and maintaining expertise in fields that could lead to future opportunities or threats. It is often treated as a necessary overhead expense and sometimes viewed with suspicion (and even incomprehension) by senior management obsessed with short-term financial returns and exploiting existing markets, rather than creating new ones.
  With knowledge-building projects, the central question for the company is: “What are the potential costs and risks of not mastering or entering the field?” Thus, no successful large firm in manufacture can neglect to explore the implications of development in IT, even if IT is not a potential core competence. And no successful firm in pharmaceuticals could avoid exploring recent developments in biotechnology. Decisions about such projects should be taken solely by technical staff on the basis of technical judgments and especially those staff concerned with the longer term. Market analysis should not play any role. Outside financial linkages are likely to be with academic and other specialist groups and to take the form of a grant.
• Strategic positioning These activities are in between knowledge building and business investment, and an important – and often neglected – link between them. They involve applied R&D and feasibility demonstration, in order to reduce technical uncertainties, and to build in-house competence, so that the company is capable of transforming technical competence into profitable investment. For this type of R&D, the appropriate question is: “Is the project likely to create an option for a profitable investment at a later date?” Comparisons are sometimes made with financial stock options, where (for a relatively small sum) a firm can purchase the option to buy a stock at a specified price, before a specified date – in anticipation of increase in its value in future.

  Decisions about this category of project should involve divisions, R&D directors, and the chief executive, precisely because – as their description implies – these projects will help determine the strategic options open to the company at a later date. At this stage, market analysis should be broad (e.g., where could genetic engineering create new markets for vegetables in a food company?). A variety of evaluation methods may be used (e.g., the product–technology matrix), but they will be more judgmental than rigorously quantitative. Costs will be higher than those of knowledge building, but much lower than those of full-scale business investment. As with knowledge-building projects, both high volatility in predictions and expectations and long-time horizons are not unwelcome signs of unacceptably high risk, but welcome signs are rich possibilities and sufficient time to explore them. Outside linkages require tighter management than those related to knowledge building, probably through a contract or equity participation.

• Business investment This is the development, production, and marketing of new and better products, processes, and services. It involves relatively large-scale expenditures, evaluated with conventional financial tools such as net present value. In such projects, the appropriate question is: “What are the potential costs and benefits in continuing with the project?” Decisions should be taken at the level of the division bearing the costs and expecting the benefits. Success depends on meeting the precise requirements of specific groups of users and therefore depends on careful and targeted marketing. Financial commitments are high, so that volatility in technological and market conditions is unwelcome, since it increases risk. Long-time horizons are also financially unwelcome, since they increase the financial burden. Given the size and complexity of development and commercialization, external linkages need to be tightly controlled through majority ownership or a joint venture. Given the scale of resources involved, careful and close monitoring of progress against expectations is essential. For such projects, most firms rely on financial methods to evaluate their project portfolio – around 77% of firms according to a recent survey. However, the same survey revealed that only 36% of the best-performing firms rely on financial methods, compared to 39% that use strategic methods [37]. An explanation for the relatively poor performance of financial methods is that the sophistication of the models often far exceeds the quality of the data inputs, particularly at the early stages of a project’s life.

Checklists are a commonly used example of a simple qualitative technique. A checklist is simply a list of factors that are considered important in making a decision in a specific case. These criteria include technical and commercial details, legal and financial factors, company targets, and company strategy. Most useful criteria are essentially independent of the business field and the business strategy, but the precise criteria and their weights will differ in specific applications.

The requirements for the use of this technique are minimal, and the effort involved in using it is normally low. Another advantage of the technique is that it is very easily adaptable to the company’s way of doing things. However, checklists can be a starting point for more sophisticated methods where the basic information can be used for better focus. One simple and useful example is a SWOT analysis, where projects are assessed for their strengths, weaknesses, opportunities, and threats.

Therefore, this technique can be developed further, and the analysis interaction and feedback can be easily managed using simple information technology. Ways to make the technique more sophisticated include the following:

• To include some quantitative factors among the whole list of factors
• To assign different weights to different factors
• To develop a systematic way of arriving to an overall opinion on the project, such as a score or index

A simple checklist could be one made up of a range of factors that have been formed to affect the success of a project and that need to be considered at the outset. In the evaluation procedure, a project is evaluated against each of these factors using a linear scale, usually 1 to 5 or 1 to 10. The factors can be weighted to indicate their relative importance to the organization.

The value in this technique lies in its simplicity, but by the appropriate choice of factors, it is possible to ensure that the questions address, and are answered by, all functional areas. When used effectively, this guarantees a useful discussion, an identification and clarification of areas of disagreement, and a stronger commitment, by all involved, to the ultimate outcome. Table 9.6 shows an example of a checklist, developed by the Industrial Research Institute, which can be adapted to almost any type of project.

As with all techniques, there is a danger that project appraisal becomes a routine that a project has to suffer, rather than an aid to designing and selecting appropriate projects, as argued in Research Note 9.8. If this happens, people may fail to apply the techniques with the rigor and honesty required and can waste time and energy trying to “cheat” the system. Care needs to be taken to communicate the reasons behind the methods and criteria used, and where necessary, these should be adapted to different types of project and to changes in the environment [44].

Portfolio methods try to deal with the issue of reviewing across a set of projects and look for a balance of economic and nonfinancial risk/reward factors. A typical example is to construct some form of matrix measuring risk versus reward, for example, on a “costs of doing the project” versus expected returns, as shown in Figure 9.3.

Rather than reviewing projects just on these two criteria, it is possible to construct multiple charts to develop an overall picture, for example, comparing the relative familiarity of the market or technology – this would highlight the balance between projects that are in unexplored territory as opposed to those in familiar technical or market areas (and thus with a lower risk). Other possible axes include ease of entry versus market attractiveness (size or growth rate), the competitive position of the organization in the project area versus the attractiveness of the market or the expected time to reach the market versus the attractiveness of the market. Research Note 9.9 provides more information on matrix approaches to project portfolio assessment.

A useful variant on this set of portfolio methods is the “bubble chart” in which the different projects are plotted but represented by “bubbles” – circles whose diameter varies with the size of the project (e.g., in terms of costs). This approach gives a quick visual overview of the balance of different-sized projects against risk and reward criteria. Case Study 9.6 gives an example. However, it is important to recognize that even advanced and powerful screening tools will only work if the corporate will is present to implement the recommended decisions, for example, Cooper and Kleinschmidt found that the majority of firms studied (885) poorly performed at this stage and often failed to kill off weak concepts [45].

How Practicing Managers Cope

These two sets of difficulties – in evaluating the potential contributions of technological investments to firm-specific intangible assets and in dealing with uncertainty – are reflected in how successful managers allocate resources to technological activities. In particular, they:

• Encourage incrementalism – step-by-step modification of objectives and resources, in the light of new evidence.
• Use simple rules models for allocating resources, so that the implications of changes can be easily understood.
• Make explicit from the outset criteria for stopping the project or program.
• Use sensitivity analysis to explore if the outcome of the project is “robust” (unchanging) to a range of different assumptions (e.g., “What if the project costs twice as much, and takes twice as long, as the present estimates?”).
• Seek the reduction of key uncertainties (technical and – if possible – market) before any irreversible commitment to full-scale – and costly – commercialization.
• Recognize that different types of innovation should be evaluated by different criteria.

View 9.1 provides insights from a director of R&D on how, in practice, firms assess and value different types of research and development project.