Our formulation of the evolutionary problem-solving process contains three phases called Modeling, Exploration, and Consolidation. The process is supposed to be aligned with the organizational strategies and objectives such as the increase of competitiveness of products or the stronger penetration of a specific market. The planning criteria studied in Section 5.9 are considered a refinement of these global objectives. What comes out of the iterative process is a product release plan suggested for implementation.

Evolutionary problem solving refers to an effort that focuses on the process to formulate and solve the right problem. This capability is closely linked to a proper understanding and formulation of the problem. As this is never possible in a perfect way, we need a mechanism to evaluate the current progress in this process. Evolutionary problem solving simultaneously involves learning about the problem, its structure, the key parameters, and how things relate to each other.

Modeling as part of product release planning refers to all the effort involved in the definition of the right granularity and the most appropriate constraints for describing the problem as studied in Chapter 5. Models are formulated at the level of detail and accuracy that corresponds to the understanding of the real world. This also means that certain aspects need to be intentionally left out from the model to keep it tractable.

In the process of modeling, there are problem aspects which are almost impossible to be formulated explicitly. These aspects are also called implicit concerns. EVOLVE II is designed to allow consideration of both explicit constraints (as formulated in Chapter 5) and implicit concerns. While explicit constraints can be handled formally, implicit concerns need the involvement of the human experts to be properly addressed. As an example of an implicit concern, the cohesiveness of the set of features offered for a specific release was introduced in Chapter 4. While is it extremely hard to model and evaluate the cohesiveness of 2ⁿ possible combinations of n given features for one release, human experts might be able to evaluate the degree of cohesiveness of a given set of features of a specific release or to select the one being most cohesive among a given number of alternative plans.

For product release planning, Modeling includes the definition of a list of candidate features as well as the description of their dependencies and constraints. Modeling also includes the description of what is, or contributes to, the “goodness” of a solution and which resource or budget constraints need to be fulfilled. Other data, such as stakeholder evaluation of features, are considered part of modeling as well. It is important to also consider features from the perspective of their resource consumption. In reality, the resources necessary to implement all features within a given time frame are seldom completely available.

A formalized description of the problem as obtained from Phase 1 allows the application of rigorous methods and techniques. The main goal of Exploration is to explore the solution space in order to obtain deeper insight into the problem and its solution structure.

At all iterations of the exploration phase, a set of the most promising alternative solutions is determined. These alternative solutions are intended to be (i) highly qualified for solving the stated problem, and (ii) diversified in relation to each other. As explained in more detail in Section 5.10, diversification among a set of qualified solutions is a mechanism for approaching the inherent uncertainty in understanding the problem and for addressing implicit concerns.

For a relatively simple problem comprising 20 features and looking at three release periods, the number of alternatives is already in the order of 10¹². This means that it is impossible to explore the solution space just “intuitively”, without qualified computer support in place. This process, including the subsequent reduction of the space of solution alternatives, is illustrated in Figure 6.1. From the original set of 10¹² solutions, a set of about 100 solutions is created from the associated optimization algorithms (implemented in ReleasePlanner™). All of these 100 solutions are considered qualified in terms of the stated objective (i.e., maximize the total number of stakeholder feature points). The diversification principle now suggests offering a subset of plans being as different as possible among each other. Analysis of the five qualified and diversified plans gives a higher chance to find one that also addresses implicit concerns. This is another formulation of saying that the plan solves the right problem. The number of five diversified plans (as described in Figure 6.1) is implementation-specific.

For performing the phase of Exploration, a wide range of methods and techniques exists that have proved themselves useful in very different contexts. The different optimization techniques including exact as well as heuristic approaches fit into this category. Meta-heuristics such as the ones based on genetic algorithms or simulated annealing are receiving more and more attention because of their broad application success. For complex problems, these techniques have proven to provide sufficiently good solutions in reasonable time [Harman ‘07]. But Exploration is not limited to these techniques. If appropriate, reasoning, machine learning or simulation is applicable whenever it contributes to problem solving.

In the case that proposed alternative solutions are accepted by the human expert to address the right problem, which one from the set of candidate solutions should finally be taken? While the emphasis of the Consolidation phase is on human expert involvement, there are a number of techniques and considerations that might be useful in this context.

Consolidation is relying on the knowledge and experience of the human experts. This involvement is aimed at addressing implicit concerns, which are hard or impossible to handle in a completely formalized manner. Consolidation refers to the process of evaluating the qualified solution alternatives, selecting the best solution or making suggestions for refinement of the underlying model.

For prioritization among plan alternative, the AHP is applicable (see Section 3.6). Another possible method refers to ELECTRE [Figuera et al. ‘05]. ELECTRE is one of a family of individual methods for multi-criteria decision support. ELECTRE methods in general comprise two main procedures: (i) construction of one or several outranking relations followed by (ii) an exploitation procedure. Each individual order relation (derived from a criterion) is assigned a weight representing the relative importance of the criterion in the overall comparison. The method then aggregates these relations using their weights to achieve a final outranking relation from which one alternative can be identified as not outranked by any other one.

Facing a small set of qualified solutions, the product manager and the associated stakeholders can get further insight into the problem structure and dependencies and move to the next iteration with a refined (better) model. This is illustrated in Figure 6.1 by the link back to Modeling. Refinements can be related to any of the parameters, constraints and formulated objectives, in an attempt to qualify the model and to better match reality. Stakeholders can finally be asked to prioritize among the proposed (qualified) alternative solutions.

This section provides an operational description of the underlying process of the EVOLVE II method. The process is subdivided into 13 steps. Besides the product manager, stakeholders are involved in different ways at the different steps. An overview of all the steps and their dependencies is given in Figure 6.2. Some of steps are optional in the sense that their execution is not mandatory for achieving the final plan decision in Step 13. While the individual steps are largely arranged in sequential order, there are feedback links between some of them. For example, as a result of the quality and resource analysis (Step 9), it might be necessary to return to Step 7 to reformulate technology (including resource) constraints. More general, from any of the steps 9 to 12, there is the possibility to return to one of the steps 2 to 7.

In what follows, all the steps of EVOLVE II are described by applying the same template to all of them. The entries of the template are:

• Description: What is supposed to be done at this step?

• Roles involved: Which roles are involved in this process step?

• Input: Which artifacts are used as input for this process step?

• Output: Which artifacts are generated from this process step?

• Entry criteria: Which conditions must be satisfied for starting the step?

• Exit criteria: Which conditions must be satisfied to terminate the step?

The first step of the process is devoted to preparation. This step defines the actors and key parameters for the subsequent planning activities. Preparation includes the definition of planning criteria. The content of process Step 1 is shown in Table 6.1.

The content of EVOLVE II process Step 2 is shown in Table 6.2. The definition of weights for the different criteria can be done by the product manager alone or as a result of a collective discussion. The latter can be achieved by prioritization performed by the selected critical stakeholders. The final weights can then be defined as the averages of this scoring, eventually followed by a subsequent discussion and negotiation of controversial issues. Alternatively AHP can be applied at this point.

The content of EVOLVE II process Step 3 is shown in Table 6.3. In the case of a large set of candidate features (especially when including enhancements of existing features and bug fixes) it is worth performing a pre-selection of features [Heikkilä et al. ‘10]. This is to keep the workload for both prioritization and effort estimation focused on features that are candidates to be released. The pre-selection process is performed by a set of nominated stakeholders who are asked to provide their preference.

Prioritization here means application of the Multi-score method described in Section 3.8. The content of process Step 4 is shown in Table 6.4. Stakeholders can be assigned to specific criteria only to reflect their specific role and background. Optionally, if features are subdivided into feature groups, the stakeholder assignment can be further differentiated by groups. That way, not all the stakeholders are supposed to prioritize all the features under consideration. This reduces their effort and allows them to focus on the features they have most competence about. In addition to the assignment to groups of features, the degree of importance of the respective scores (per criterion and per group) can be adjusted as well.

Once all the stakeholders have provided their priority scores, analysis of the scoring can be done. This is called voice-of-the-stakeholder analysis. For creating optimized planning results, this is an optional process step which can or cannot be performed. It gives an overview of the ranking of features and the degree of agreement in stakeholder opinions. This has proven to be important information when individual features need to be discussed more specifically as part of the whole decision-making process. The content of Step 5 is shown in Table 6.5.

The critical task of providing reliable effort estimates is addressed in Step 6. For that purpose, collective knowledge and experience can be utilized by asking multiple experts for their opinion. For effort estimation, candidate stakeholders would be developers, domain experts or project managers. Instead of a single point-wise estimate, three-point estimation can be applied. This is defined as the optimistic, most likely, and pessimistic estimates. For feature f(n) and stakeholder stake(p), these estimates are denoted by opt_effort(n,p), likely_effort(n,p), and pess_ effort(n,p), respectively. Their integration into an overall effort estimate effort(n,p) per feature is done following the Program Evaluation and Review Technique (PERT) [Elmaghraby ‘77]:

• (6.1)       effort(n,p) =
  [opt_effort(n,p) + 4•likely_effort(n,p) + pess_effort(n,p)]/6

The integration of the individual stakeholder effort estimates into an overall effort estimate Effort(n) is done by calculation of the weighted average:

• (6.2)      Effort(n) = Σ_p=1…P weight_stake(p)•effort(n,p))/(Σ_p=1…P weight_stake(p))

The estimation explained for effort estimates in general is applied similarly for estimates related to specific resource types. From the integration of these multiple estimates, on average a more reliable estimate can be expected compared to estimation by just a single person. These deviations can be discussed and resolved from meetings as suggested by the Delphi-technique [Rowe and Wright ‘99]. The difference to the traditional techniques is that now the focus can be on the most controversial features with no obligation to look at all of them. The range of the estimation interval per single expert and the variation of intervals between experts are indicators about the (un-) certainty of the estimations. The content of Step 6 is shown in Table 6.6.

The art of modeling is addressed in process Step 7. Its content is shown in Table 6.7. Modeling is done by the product manager (decision-maker) in consultation with domain experts. Pre-assignment of features in conjunction with resource and technological dependencies might create violation of feasibility. In that case, up-front analysis and resolution of the feasibility conflict needs to be done.

Product release planning has been formulated in Chapter 5 as an optimization problem. On top of applying branch-and-bound and specialized integer programming procedures, a heuristic is implemented for determining upper bounds of the objective function value at each step of the branching process. See [Ngo-The and Ruhe ‘08] for a description of the algorithms. The content of Step 8 is shown in Table 6.8.

Step 9 is optional. It aims at the detection of differences and commonalities in resource consumption and quality between plans. Higher stability in the level of resource consumption implies higher stability in the amount of personnel needed. The quality of plans describes its (guaranteed) level of optimality for utilizing the resources in a best possible way to archive a maximum total number of stakeholder feature points. This information is a possible input for making the final release plan decision in Step 13. The content of Step 9 is shown in Table 6.9.

In this optional process step, predictions are made on the individual excitement or disappointment reaction of stakeholders for a proposed plan. Related to a given plan, a stakeholder excitement profile is defined based upon the stakeholder’s scoring. Excitement (or disappointment) can be defined for one specified planning criterion or as a cumulative measure over all criteria. This helps the human expert select between the set of diversified alternative solutions. The decision can be based upon different strategies such as maximizing excitement or minimizing disappointment for one or all stakeholders. The overall content of Step 10 is shown in Table 6.10.

To describe the different possible reactions on proposed plans, seven different categories are introduced. Excitement (disappointment) of a stakeholder refers to the degree of conformance between the feature-related score of the stakeholder and the assignment of the feature to a release. Besides excitement and disappointment, there is another category called surprise. (Very) Surprised refers to the situation where a feature is scored (very) low, but is offered in an early (next) release. The formal notation of the excitement categories is shown in Table 6.11. This notation is then used in Tables 6.12 and 6.13 to define excitement responses to given plans.

The stakeholder excitement table assigns one of the seven values for each possible combination between stakeholder scores and release numbers proposed by the plan under consideration. In Table 6.12 and Table 6.13, projected excitement tables are defined for the case of two and three releases, respectively. These tables can be customized for a specific project or company context. In case of more than three releases, tables are specified accordingly.

For example, in the case of K = 2 releases (see Table 6.12), a stakeholder having scored 9 for some feature (and some criterion) will be very excited (VE) to see this feature in the next release (Release 1). However, the same stakeholder is likely to be very disappointed (VD) if this feature would be postponed, and at least disappointed (D) to see it in release 2. On the other hand, having scored 1 for a feature and finding it in release 1 would probably create a very surprised (VS) reaction, while postponing this feature is expected to result in a neutral (N) position.

Proactive what-if analysis is one way to study the impact of possible scenarios of the future. This supports current decision-making because some of the projected results of the scenario runs can be taken into account. These future scenarios need a reference point (baseline) in terms of project data and a (set of) solutions generated based upon the data. Changes in the problem space can be related to a number of parameters:

• Modify planning criteria weights

• Modify stakeholder weights

• Modify resource capacities

• Modify release weights

• Add or remove feature dependencies

• Add or modify resource constraints

• Add or modify features and their resource consumption

The content of the optional Step 11 is shown in Table 6.14.

The whole process of decision-making can be seen as a (intelligent) search in the solution space where the set of candidates is reduced more and more toward selection of the top ones. The information collected in Steps 9 through 11 was all intended to further differentiate between the five pre-selected qualified plans. In the optional Step 12, decision-critical stakeholders are explicitly asked to evaluate the proposed plans from their overall attractiveness point of view. With the same nine-point scale as applied before, the scores express the range from Very low to Very high attractiveness. For that purpose, the stakeholders can use or not use information gained from former steps. The content of Step 12 is shown in Table 6.15.

Based upon all the information gained from Steps 9 through 12, the final decision needs to be made about which plan will be selected. This selected plan is the result of an evolutionary problem-solving process. Explicitly formulated constraints have been taken into account, eventually after some refinements. In addition, implicit concerns have been addressed as well. For the generation of the plan, stakeholders have played an essential role at the different stages of the process. This increases the likelihood of getting their acceptance for the implementation of the plan. The content of Step 13 is shown in Table 6.16.