According to ISO/IEC 42010 (ISO/IEC, 2007), the notion of system can be defined as a set of components that accomplishes a specific function or set of functions. Each system has an architecture, which is defined as “the fundamental organization of a system embodied in its components, their relationships to each other, and to the environment, and the principles guiding its design and evolution.” When reuse is an important concern, a system can be built based on the PL approach. For very large systems, the scope of the PL can be extended further, and the product can be built using subproducts from MPLs. The notion of MPLs has been addressed earlier by different authors including Aoyama et al. (2003), Archer et al. (2010), Fritsch and Hahn (2004), van der Linden and Wijnstra (2001), van Ommering (2002), and Rosenmüller and Siegmund (2010). In this context, the terms MPLs, nested PLs, or PLs of PLs have been used to denote the same concept. Rosenmüller and Siegmund (2010) define MPLs as “a set of interacting and interdependent SPLs.”

In principle, we can consider the composition of PLs as the application of a composite pattern as shown in Figure 10.1. PL could be either a flat SPL or a Composite Product Line (CPL). CPL itself could contain other PLs; likewise, the PL can be built in a nested manner. Alternatively, the CPL could include only flat PLs, leading to an MPL consisting of independent PLs. In each CPL the separate PLs could use other PLs. A PL (CPL or PL) can include other reusable assets that are not PLs themselves (e.g., libraries).

The pattern in Figure 10.1 appears to be general and can model different configurations of MPLs. It should be noted that each PL is defined by a two-life cycle process including domain engineering (with product management) and application engineering. Based on these two separate processes, the notion of architecture is usually specialized into a PL architecture and an application architecture (Pohl et al., 2005). A PL architecture represents the common and variant structures of a set of products of the selected PL; an application architecture represents the architecture of a single system. The architects who define these architectures can be called PL architects and application architects.

10.1.2 Software architecture analysis methods

The architecture forms one of the key artifacts in the entire software development life cycle because it embodies the earliest design decisions and includes the gross-level components that directly impact the subsequent analysis, design, and implementation. The key concerns of an architecture are defined by stakeholders: an individual, team, or organization with interests in, or concerns relative to, a system. Each of the stakeholder’s concerns impacts the early design decisions that the architect makes. As architecture is critical for the success of a project, different architectural evaluation approaches have been introduced to evaluate the stakeholders’ concerns. A comprehensive overview and comparison of architecture analysis methods have been given by, for example, Dobrica and Niemela (2002) and Babar et al. (2004). Kazman et al. (2005) have provided a set of criteria for comparing the foundations underlying different methods, the effectiveness, and usability of methods.

Figure 10.2 provides a conceptual model that we have defined to describe the architecture evaluation approach. Although different architecture evaluation approaches have been proposed in the literature, we can state that most of these follow the model in Figure 10.2. In essence, each architecture evaluation approach takes as input stakeholder concerns, environmental issues, and architecture description. Based on these inputs, the evaluation results in an Architecture Evaluation Report, which is used to adapt the architecture.

The proposed architecture evaluation approaches usually differ with respect to, for example, the goal of the approach, the type of inputs, the evaluation techniques, the addressed quality attributes, the stakeholders’ involvement, the ordering of activities, and the output results (Babar et al., 2004; Kazman et al., 2005). It appears that no explicit approach seems to have been provided to analyze architecture within an MPL engineering context. In the following sections we show the need for a specific analysis approach for MPL engineering and the relation to the existing architecture analysis approaches.

10.4.1 Preparation phase

During the Preparation Phase, first the stakeholders and the evaluation team (step 1) are selected. The stakeholders are typically a subset of the stakeholders (including project decision makers) listed above. After the stakeholders are selected, the schedule for evaluation is planned (step 2). In general, the complete evaluation of the MPL will take more time than for a single architecture evaluation. Hence, for defining the schedule a larger timeframe than usual is adopted.

10.4.2 Selection of feasible MPL decomposition

In this phase the different MPL architecture design alternatives are provided (step 3), and the feasible alternative is selected (step 4). The MPL alternatives are described using the MPL decomposition and uses viewpoints. Representation of the MPL architecture in step 3 is necessary to ensure that the proper input is provided to the analysis in step 4. At this stage, no detailed design of the MPL is necessary. This is because designing an MPL is a time-consuming process. Only after the feasible decomposition is found in step 4 will the design documentation be completed in step 5.

For the selection of the feasible PL architecture in step 4 we adopt the Goal-Question-Metric (GQM) approach, a measurement model promoted by Basili and others (Roy and Graham, 2008). The GQM approach is based upon the assumption that for an organization to measure in a purposeful way, the goals of the projects need to be specified first. Subsequently, a set of questions must be defined for each goal, and finally a set of metrics associated with each question is defined to answer each one in a measurable way. For applying the GQM, usually a six-step process is recommended where the first three steps are about using business goals to drive the identification of the right metrics, and the last three steps are about gathering the measurement data and making effective use of the measurement results to drive decision making and improvements. The six steps are usually defined as follows (Roy and Graham, 2008; Solingen and Berghout, 1999):

1. Develop a set of corporate, division, and project business goals and associated measurement goals for productivity and quality.

2. Generate questions (based on models) that define those goals as completely as possible in a quantifiable way.

3. Specify the measures needed to be collected to answer those questions and track process and product conformance to the goals.

4. Develop mechanisms for data collection.

5. Collect, validate, and analyze the data in real time to provide feedback to projects for corrective action.

6. Analyze the data in a post mortem fashion to assess conformance to the goals and to make recommendations for future improvements.

10.4.3 Evaluation of selected MPL design alternative

Step 4 focuses on selecting a feasible MPL decomposition alternative. An MPL consists of several PLs and thus multiple architectures. Likewise, in step 5 of Archample, we focus on refined analysis of the selected MPL alternative. In fact, the selected alternative can be a single PL architecture or different MPL architectures. In case the alternative is a CPL, we apply a staged-evaluation approach in which the MPL units (PLs or CPLs) are recursively evaluated. From this perspective, we distinguish among the following two types of evaluations: (a) top-down product evaluations and (b) bottom-up product evaluations.

In the top-down evaluation, first the higher level PLs are evaluated. This is illustrated in Figure 10.7. Here, the evaluation order is indicated through the numbers in the filled circles. The evaluation starts with evaluation the top-level decomposition of the MPL architecture and continues with the subelements of the MPL, which can be again CPL or single PL.

In the bottom-up approach first the leaf PLs are evaluated, then the higher level architectures. An example bottom-up specialization is shown in Figure 10.8. Obviously, other hybrid specialization approaches that fall between top-down and bottom-up strategy can be applied. The selection of the particular evaluation strategy (top-down, bottom-up, or hybrid) depends on the particular constraints and requirements of the project. A hybrid approach can be preferred by considering the dependency relations among PLs, which are modeled in the PL dependency view.

The evaluation of the architecture can be done using any architecture evaluation method (including GQM again). Over the last decade several different architecture analysis approaches have been proposed to analyze candidate architectures with respect to desired quality attributes (Babar et al., 2004; Dobrica and Niemela, 2002; Kazman et al., 2005). The architecture evaluation methods can be categorized in different ways. Early evaluation methods evaluate the architecture before its implementation while late architecture evaluation methods require the implementation to perform the evaluation. In principle, within Archample we do not restrict the selection of any method.

10.4.4 Reporting and workshop

In the last phase of Archample, a report of the evaluation results is provided and a workshop with the stakeholders is organized. The stakeholders are typically a subset of the list as defined in Section 10.4. A template for the report is given in Table 10.2.

The first Chapters 1–3 of the report provide the background information about the company and its business goals and describe the Archample method. Chapter 4 defines the different MPL architecture alternatives. Chapter 5 analyzes the MPL design alternatives and selects a feasible alternative. Chapter 6 presents the documentation of the selected alternative. In Chapter 7, the evaluation of the alternative is described using staged-evaluation approach (top-down, bottom-up, hybrid) and the evaluation results. Chapter 8 presents the overall recommendations, and Chapter 9 concludes the report. An appendix can consist of several sections and include, for example, the glossary for the project, explanation about standards, viewpoints, or other pertinent factors. After the first complete draft of the report, a workshop is organized to discuss the results. The discussions during the workshop are used to adapt the report and define the final version.

10.5.1 Preparation phase

In this phase, we identified the stakeholders and the evaluation team(s) as defined in Section 10.4. These included the project decision makers, three MPL architects, a PL architect for each PL, PL evaluation team, and other stakeholders required for each PL (such as developers, testers, and customers).

10.5.2 Selection of feasible MPL decomposition

Within the REFoRM project of Aselsan REHİS four different MPL architecture alternatives were identified:

1. One PL: Defining the system as one PL as shown in Figure 10.9.

2. Four PLs: Defining the system as four independent PLs as shown in Figure 10.10.

3. AD PLs: Defining only the application domains as PLs as shown in Figure 10.11.

4. CPLs: Defining a CPL as it was shown earlier in Figure 10.4.

Before the analysis, the MPL architecture was not designed using the viewpoints as defined in Section 10.3. Thus, we took some time to provide a proper design of each alternative.

Evaluating four different alternatives using the GQM evaluation approach, as part of Archample, was carried out. Table 10.3 shows the GQM results as defined during the evaluation. The goals represent high-level business goals that were found important from the project decision makers. These goals are listed below:

– Optimize Reuse: MPL architecture should supply maximum reuse within radar and electronic warfare projects; no functionality should be repeated in different PLs.

– Increase Productivity: The MPL architecture should need minimum manpower. Where possible the need for hiring new personnel should be minimized.

– Manage Complexity: Due to the large domain and huge sets of features, the complexity of variability management of the PL is very high. MPL architecture should help to cope with this complexity.

– Ease Organizational Management: Aselsan is working in well-defined domains and internal business units are currently organized around these units. The MPL architecture should be in alignment with these domains and not disrupt the structure too much.

– Ease PL Management: It should be easy to add/remove/update product features within and across business units. Also, the evolution of MPL architecture should be manageable.

– Ease Composition of Products: The PL decomposition should enable combinations of products from different domains managed by related business units in the company.

Once the goals, questions, and metrics were defined, we could start the actual evaluation. The result of the evaluation of each of the four alternatives according to the presented goals is shown in Table 10.4. It was decided that all the six goals should have equal weight. For each criterion the evaluation scale includes the values − −, −, +−, ++, and ++, defining a very negative evaluation to a very positive evaluation. The goals were evaluated with respect to the corresponding questions and the metrics. Below, we provide a short discussion of the alternatives.

Table 10.4

Evaluation Matrix for Design Alternatives of the Product Line Decomposition

Goals	One PL	Four PLs	AD PLs	CPLs
Optimize Reuse	++	− −	−(−)	++
Increase Productivity	+(+)	−	−	++
Manage Complexity	− −	+	+	+
Ease Organizational Management	− −	++	++	+−
Ease Product Line Management	+(+)	+	− −	++
Ease Composition of Products	+(+)	− −	−	++

1. One PL: As shown in Table 10.4, defining the system as one complete PL (Figure 10.9) is not favorable from the complexity management perspective and the ability to manage the organization for the resulting very large PL. In addition, the Mission Domain Applications level of the system was currently defined across different departments and from an organization point of view. Putting all the product divisions together was considered not feasible. On the other hand, this alternative was valued as positive because it would probably not include overlapping functionality, and the manpower needed would be optimized. Further, this alternative would also be beneficial for managing products and composing new products.

2. Four PLs: The second alternative of defining the system as four independent PLs (Figure 10.10) has been positively evaluated with respect to alignment with the domains because it supports the four Mission Domain Applications level. The organization management would also be distributed over the multiple divisions, thus the problems of a heavy central organization would be avoided. The product management would be easy but the composition of new products across MPLs would be severely impeded. Moreover, this alternative required that the different alternative PLs include development of similar functionality, and due to the overlapping functionality, reuse would not be optimized. Consequently, this alternative would also require more additional manpower than the other alternatives.

3. AD PLs: The third alternative (Figure 10.11) is the MPL with four separate PLs for the Mission Domain Applications level and an additional platform layer with reusable assets as libraries. For the first four criteria this alternative was evaluated like the second alternative. The motivations for the evaluations were also similar. In contrast to the second alternative, this alternative is not considered feasible for product management. The reason for this is that some critical domains are not developed as PLs but remain as libraries whose variability and architecture were not defined. Thus the composition of new products will be impeded. Regarding complexity of variability management, this was considered also similar to the second alternative. There will be less variability in the four domains because part of the functionality will reside in libraries. On the other hand, new variations would be harder to define.

4. CPLs: Defining the system as a CPL (Figure 10.4) was positively evaluated for almost all the defined criteria. The MPL could on the one hand include a hierarchical structure while still keeping the separation of domains and organizational management in the company. Overlapping functionality could be reduced or eliminated. Due to the hierarchical and composite structure, the development of new products would be supported. The only neutral result for this choice is the optimization of manpower; the manpower needed to develop the assets would be optimized as the reuse is optimized, but the manpower needed for the management of the PLs will be higher than a single PL because each PL and the CPL will need separate management.

From Table 10.4, it can be observed that defining “four PLs” and “application domain PLs” are the worst choices because they have the most negatives. Despite the increased management complexity in the PL, the alternative with one PL was more positively evaluated than was expected. After careful thought and discussion with the stakeholders, the most feasible alternative for the Aselsan REHİS case was determined to be the composite MPL alternatives. It should be noted that the evaluation of each of the alternatives was actually only possible after modeling each alternative using the architectural viewpoints in the previous section. Without the explicit architectural views, one had to rely on discussions, informal sketches, or feature modeling approaches. None of these were considered as powerful as explicitly documenting the architectural views.

10.5.3 Evaluation of the selected MPL design alternative

After the composite MPL was selected as the most feasible design alternative, the refined evaluation was necessary to evaluate the PL architectures in the MPL. For the refined evaluation, we adopted a top-down evaluation strategy. That is, we decided to analyze the top-level MPL first and then the sub-PLs. For some PLs it was decided not to perform an evaluation yet due to the time constraints. The strategy together with the selected evaluation methods for the various PLs are shown in Figure 10.12. The figure shows the order of the evaluation of the PLs.

As shown in Figure 10.12, it was decided to do an ATAM for the RadEW and a GQM for Radar and ComEW. For the PLs of RadEW an ATAM was also performed. Each ATAM evaluation took about 1 week. For the PLs of Radar, the team decided to apply SAAM, while the CPLs of ComEW were not evaluated yet. The separate PLs, on the right part of the figure, were either not evaluated or a quick SAAM process was applied (about two days).

10.5.4 Reporting and workshop

The complete design and the corresponding evaluation of the MPL alternatives were reported and written as a technical report (Tekinerdogan et al., 2012). Interactive workshops were held in step 4 (select feasible MPL using GQM) and step 6 (evaluate MPL). For the latter step, we needed more than one workshop meeting to do the evaluation for the separate PLs. The overall evaluation was completed using a final two-day workshop in which the process for selecting the MPL, the modeling of the MPL, and the modeling and evaluation of the required PLs was discussed.