24. Architecture Competence

If software architecture is worth “doing,” surely it’s worth doing well. Most of the literature about architecture concentrates on the technical aspects. This is not surprising; it is a deeply technical discipline. There is little information that speaks to the fact that architectures are created by architects working in organizations, full of actual human beings. Dealing with these humans is decidedly nontechnical. What can be done to help architects, especially architects-in-training, be better at this important dimension of their job? And what can be done to help architecture organizations do a better job in fostering their architects to produce their best work?

An organization’s ability to routinely produce high-quality architectures that are aligned with its business goals well cannot be understood simply through examination of past architectures and measurement of their deficiencies. The organizational and human causes of those deficiencies also need to be understood.

This chapter is about the competence of individual architects and the organizations that wish to produce high-quality architects. We define the architecture competence of an organization as follows:

The architecture competence of an organization is the ability of that organization to grow, use, and sustain the skills and knowledge necessary to effectively carry out architecture-centric practices at the individual, team, and organizational levels to produce architectures with acceptable cost that lead to systems aligned with the organization’s business goals.

Because the architecture competence of an organization depends, in part, on the competence of architects, we begin by asking what it is that architects are expected to do, know, and be skilled at. Then we’ll look at what organizations can and should do to help their architects produce better architectures. Individual and organizational competencies are intertwined. Studying only one or the other won’t do.

24.1. Competence of Individuals: Duties, Skills, and Knowledge of Architects

Architects perform many activities beyond directly producing an architecture. These activities, which we call duties, form the backbone of an individual’s architecture competence. We surveyed a broad body of information aimed at architects (such as websites, courses, books, and position descriptions for architects). We also surveyed practicing architects. These surveys tell us that duties are but one aspect of individual competence. Writers about architects also speak of skills and knowledge. For example, the ability to communicate ideas clearly and to negotiate effectively are skills often ascribed to competent architects. In addition, architects need to have up-to-date knowledge about patterns, database platforms, web services standards, quality attributes, and a host of other topics.

Duties, skills, and knowledge¹ form a triad upon which architecture competence for individuals rests. The relationship among these three is shown in Figure 24.1—namely, skills and knowledge support the ability to perform the required duties. Infinitely talented architects are of no use if they cannot (for whatever reason) perform the duties required of the position; we would not say they were competent.

Figure 24.1. Skills and knowledge support the execution of duties.

To give examples of these concepts:

• “Design the architecture” is a duty.

• “Ability to think abstractly” is a skill.

• “Patterns and tactics” is a part of the body of knowledge.

This example purposely illustrates that skills and knowledge are important (only) for supporting the ability to carry out duties effectively. As another example, “documenting the architecture” is a duty, “ability to write clearly” is a skill, and “ISO Standard 42010” is part of the related body of knowledge. Of course, a skill or knowledge area can support more than one duty.

Knowing the duties, skills, and knowledge of architects (or, more precisely, the duties, skills, and knowledge that are needed of architects in a particular organizational setting) can help establish measurement and improvement strategies for individual architects. If you want to improve your individual architectural competence, you should do the following:

1. Gain experience carrying out the duties. Apprenticeship is a productive path to achieving experience. Education alone is not enough, because education without on-the-job application merely enhances knowledge.

2. Improve your nontechnical skills. This dimension of improvement involves taking professional development courses, for example, in leadership or time management. Some people will never become truly great leaders or communicators, but we can all improve on these skills.

3. Master the body of knowledge. One of the most important things a competent architect must do is master the body of knowledge and remain up to date on it. To emphasize the importance of remaining up to date, consider the advances in knowledge required for architects that have emerged in just the last few years. For example, the cloud and edge computing that we discuss in Chapters 26 and 27 were not important topics several years ago. Taking courses, becoming certified, reading books and journals, visiting websites and portals, reading blogs, attending architecture-oriented conferences, joining professional societies, and meeting with other architects are all useful ways to improve knowledge.

Duties

This section summarizes a wide variety of an architect’s duties. Not every architect in every organization will perform every one of these duties on every project. But a competent architect should not be surprised to find himself or herself engaged in any of the activities listed here. We divide the duties into technical duties (Table 24.1) and nontechnical duties (Table 24.2). One immediate observation you should make is how many nontechnical duties there are. An obvious implication, for those of you who wish to be architects, is that you must pay adequate attention to the nontechnical aspects of your education and your professional activities.

Table 24.1. The Technical Duties of a Software Architect

Table 24.2. The Nontechnical Duties of a Software Architect

Skills

Given the wide range of duties enumerated in the previous section, what skills (beyond mastery of the technical body of knowledge) does an architect need to possess? Much has been written about the architect’s special role of leadership in a project; the ideal architect is an effective communicator, manager, team builder, visionary, and mentor. Some certificate or certification programs emphasize nontechnical skills. Common to these certification programs are nontechnical assessment areas of leadership, organization dynamics, and communication.

Table 24.3 enumerates the set of nontechnical skills most useful to an architect.

Table 24.3. The Nontechnical Skills of a Software Architect

Knowledge

A competent architect has an intimate familiarity with the architectural body of knowledge. The software architect should

• Be comfortable with all branches of software engineering from requirements definition to implementation, development, verification and validation, and deployment

• Be familiar with supporting disciplines such as configuration management and project management

• Understand current design and implementation tools and technologies

Knowledge and experience in one or more application domains is also necessary.

Table 24.4 is the set of knowledge areas for an architect.

Table 24.4. The Knowledge Areas of a Software Architect

24.2. Competence of a Software Architecture Organization

Organizations by their practices and structure can help or hinder architects in performing their duties. For example, if an organization has a career path for architects, that will motivate employees to become architects. If an organization has a standing architecture review board, then the project architect will know how and with whom to schedule a review. Lack of these items will mean that an architect has to fight battles with the organization or determine how to carry out a review without internal guidance. It makes sense, therefore, to ask whether a particular organization is architecturally competent and to develop instruments whose goal is measuring the architectural competence of an organization. The architectural competence of organizations is the topic of this section.

Activities Carried Out by a Competent Organization

Organizations have duties, skills, and knowledge for architecture as well. For example, adequately funding the architecture effort is an organizational duty, as is effectively using the available architecture workforce (by appropriate teaming, and so on). These are organizational duties because they are outside the control of individual architects. An organization-level skill might be effective knowledge management or human resource management as applied to architects. An example of organizational knowledge is the composition of an architecture-based life-cycle model that software projects may employ.

Here are some things—duties—that an organization can perform to help improve the success of its architecture efforts:

• Hire talented architects.

• Establish a career track for architects.

• Make the position of architect highly regarded through visibility, reward, and prestige.

• Establish a clear statement of responsibilities and authority for architects.

• Establish a mentoring program for architects.

• Establish an architecture training and education program.

• Establish an architect certification program.

• Have architects receive external architect certifications.

• Measure architects’ performance.

• Establish a forum for architects to communicate and share information and experience.

• Establish a repository of reusable architectures and architecture-based artifacts.

• Develop reusable reference architectures.

• Establish organization-wide architecture practices.

• Establish an architecture review board.

• Measure the quality of architectures produced.

• Provide a centralized resource to analyze and help with architecture tools.

• Hold an organization-wide architecture conference.

• Have architects join professional organizations.

• Bring in outside expert consultants on architecture.

• Include architecture milestones in project plans.

• Have architects provide input into product definition.

• Have architects advise on the development team structure.

• Give architects influence throughout the entire project life cycle.

• Reward or penalize architects based on project success or failure.

Assessment Goals

The activities enumerated above can be assessed. What are the potential goals from an assessment of an organization’s architecture competence? There are at least four sets of reasons for assessing organizational architectural competence:

1. There are goals relevant to any business that wishes to improve their architectural competence. Businesses regularly assess their own performance in a variety of means—technical, fiscal, operational (for example, consider the widespread use of multi-criteria techniques such as the Balanced Scorecard or Business/IT Alignment in industry)—for a variety of reasons. These include determining whether they are meeting industry norms and gauging their progress over time in meeting organizational goals.

2. There are goals relevant to an acquisition organization. For example, an organization can use an assessment of architecture competence to assess a contractor in much the same way that contractors are scrutinized with respect to their CMMI level. Or an organization might use an assessment of architecture competence to aid in deciding among competing bids from contractors. All other things being equal, an acquiring organization would prefer a contractor with a higher level of architectural competence because this typically means fewer downstream problems and rework. An acquisition organization might assess the contractors directly, or hire a third party to do the assessment.

3. There are goals relevant to service organizations: such organizations might be motivated to maintain, measure, and advertise their architectural competence as a means of attracting and retaining customers. In such a case they would typically rely on outside organizations to assess their level of competence in an objective fashion.

4. Finally, there are goals that are relevant to product builders: these organizations would be motivated to assess, monitor (and, over time, increase) their level of architectural competence as it would (1) aid in advertising the quality of their products and (2) aid in their internal productivity and predictability. In fact, in these ways their motivations are aligned with those of service organizations.

Assessing an Organization’s Competence

In addition to duties, skills, and knowledge, there are other models of individual and organizational competence that are helpful in building an instrument for assessing an organization’s competence. They are the following:

• The Human Performance Technology model, which measures the value of an individual or department’s output and the cost of producing that output. It holds that competent people produce the most value for every organizational dollar spent.

• The Organizational Coordination model, which measures how teams in multiple sites developing a single product or related set of products cooperate to produce a functioning product. An organization that is architecturally competent will have more effective and efficient coordination mechanisms than an organization that is not architecturally competent.

• The Organizational Learning model, which measures how well an organization’s learning processes transform experience into knowledge, moderated by context.

We have created a framework for organizational architecture competence that forms the foundation for a competence assessment procedure. A small team of trained assessors can use the framework to conduct interviews (with architects, developers, and technical and organizational managers), examine current practices, read documents and evidentiary artifacts (such as organizational standards), and investigate architecture-based successes and failures in the recent past. They can use their findings to identify systemic trouble spots and recommend improvement strategies.

Table 24.5 shows the framework. For convenience, it is divided into practice areas that relate to software and system engineering, technical management (which is by and large the management of single projects or small numbers of related projects), and organizational management (which is management at a scope more broad than that of projects).

Table 24.5. Framework for Organizational Architecture Competence

The framework is populated by questions inspired by the four models of competence that we previously described. Each question has a set of answers that we might expect to see in a competent organization. For example, a question associated with the practice area “Hire talented architects” deals with how a candidate architect’s experience and capabilities are assessed. Expected answers might include having the architect take a test, requiring that candidates possess an architecture certification, or examining previous architectures designed by the candidate. (Our expected answers grow as we visit more and more organizations. It’s a pleasant surprise when we find an organization carrying out a practice area in a clever way that we hadn’t thought of.)

Questions Based on the Duties, Skills, and Knowledge Model

Our assessment framework contains dozens of questions related to duties, skills, and knowledge. The questions are posed in terms of the organization: The questions ask how the organization ensures that the architectural duties are carried out in a competent manner, and how the organization measures and nurtures its architects’ skills and knowledge. Here is a small set of example questions based on the Duties, Skills, and Knowledge model (chosen from among the dozens that populate our assessment framework) that we use in an architecture competence assessment exercise with an organization.

Duty: Creating an Architecture

Question: How do you create an architecture?

• How do you ensure that the architecture is aligned with the business goals?

• What is the input into the architecture creation process? What inputs are provided to the architect?

• How does the architect validate the information provided? What does the architect do in case the input is insufficient or inadequate?

Duty: Architecture Evaluation and Analysis

Question: How do you evaluate and analyze an architecture?

• Are evaluations part of the normal software development life cycle or are they done when problems are encountered?

• Is the evaluation incremental or “big bang”? How is the timing determined?

• Does the evaluation include an explicit activity relating architecture to business goals?

• What are the inputs to the evaluation? How are they validated?

• What are the outputs from an evaluation? How are the outputs of the evaluation utilized? Are the outputs differentiated according to impact or importance? How are the outputs validated? Who is communicated what outputs?

Knowledge: Architecture Concepts

Question: How does your organization ensure that its architects have adequate architectural knowledge?

• How are architects trained in general knowledge of architecture?

• How do architects learn about architectural frameworks, patterns, tactics, standards, documentation notations, and architecture description languages?

• How do architects learn about new or emerging architectural technologies (e.g., multi-core processors)?

• How do architects learn about analysis and evaluation techniques and methods?

• How do architects learn quality attribute-specific knowledge, such as techniques for analyzing and managing availability, performance, modifiability, and security?

• How are architects tested to ensure that their level of knowledge is adequate, and remains adequate, for the tasks that they face?

Questions Based on the Organizational Coordination Model

Questions based on the Organizational Coordination model focus on how the organization establishes its teams and what support it provides for those teams to coordinate effectively. Here are a couple of example questions:

Question: How is the architecture designed with distribution of work to teams in mind?

• How available or broadly shared is the architecture to various teams?

• How do you manage the evolution of architecture during development?

• Is the work assigned to the teams before or after the architecture is defined, and with due consideration of the architectural structure?

Question: Are the aspects of the architecture that will require a lot of interteam coordination supported by the organization’s coordination/communication infrastructure?

• Do you co-locate teams with high coordination? Or at least put them in the same time zone?

• Must all coordination among teams go through the architecture team?

Questions Based on the Human Performance Technology Model

The Human Performance Technology questions deal with the value and cost of the organization’s architectural activities. Here are examples of questions based on the Human Performance Technology model:

Question: Do you track how much the architecture effort costs, and how it impacts overall project cost and schedule?

• How do you track the end of architecture activities?

• How do you track the impact of architecture activities?

Question: Do you track the value or benefits of the architecture?

• How do you measure stakeholder satisfaction?

• How do you measure quality?

Questions Based on the Organizational Learning Model

Finally, a set of example questions, based on the Organizational Learning model, which deal with how the organization systematically internalizes knowledge to its advantage:

Question: How do you capture and share experiences, lessons learned, technological decisions, techniques and methods, and knowledge about available tooling?

• Do you use any knowledge management tools?

• Is capture and use of architectural knowledge embedded in your processes?

• Where is the information about “who knows what” captured and how is this information maintained?

• How complete and up to date is your architecture documentation? How widely disseminated is it?

Performing an Assessment

Our organizational competence assessment is carried out using a team of three to four assessors. The exercise is set up by establishing the scope of the review: Are we assessing the entire company? One of its divisions? Or perhaps a single important project?

After we establish the scope, we identify the groups we wish to interview. Of course, we’ll want to interview the architecture team(s) within the scope. From there we identify groups both upstream and downstream of the architects. Upstream are groups that manage the architects or provide organization-wide architecture training. Downstream, we interview the “consumers” of the architectures, such as developers, integrators, testers, and maintainers. We interview small groups, making sure that no members of an interview group have reporting relationships with each other.

We try hard to establish an informal atmosphere in the group interviews, to avoid inhibiting the participants. The tone is conversational, not inquisitional. We begin each interview by reminding the participants of the purpose of the exercise, and to assure them that nothing they say will be quoted to anyone outside the group in any way that could identify them.

For each group, we have planned which parts of the framework we wish to discuss with that group. We won’t ask testers, for example, questions intended for managers, and vice versa. We use the questions as a guide for the conversation, but not a rigid script. Whenever we pose a question in the assessment instrument, there are a number of meta-questions that automatically accompany it. For example:

• What evidence could you show us to support your answer? Supporting evidence might include a software development plan that lays out the role of architecture in a project, an organization’s architecture-based training curriculum, or many other kinds of documentation.

• How sustainable is your answer over time, over different systems, and across different architects? For example, we might ask how an answer might change if a different architect came on board the project.

The outcome of an assessment is organized by the practice areas of the framework. For each practice, we assign one of three values that correspond to “you’re doing this well,” “you could be doing this better (and experiencing more benefit),” and “this is an area of high risk.” Graphically, we show this as green light, yellow light, red light. We have found that this simple metric provides organizations with enough granularity to turn their attention to problem areas, which is the whole point of the assessment. We do not give an overall rating. Thus, we closely mirror the “continuous representation” option of maturity models such as CMMI, in which the result is a vector rather than a scalar.

We present the findings in a written report and a slide presentation. In both, we describe and justify each finding, based on what we were told in the interviews and/or read in provided documents.

24.3. Summary

The vast majority of work on software and systems architecture (including our own) has focused on the technical aspects. But an architecture is much more than a technical “blueprint” for a system. This has led us to try to understand, in a more holistic way, what an architect and an architecture-centric organization must do to succeed. To this end, we have developed a framework that aids us in assessing an organization for competence.

We use the framework to ask questions about an organization’s practices. We can also ask about recent architecture successes and failures, and investigate the causes of each. The output of this exercise is a formal report that assesses competence at organization, team, and individual levels. Along with this report we make improvement recommendations based on assessment results; these, too, are tied to the underlying competence models.

You can do the same sort of evaluation on your own organization. The key to the process is in understanding the various models and in creating questions based on these models that aid you in assessing how well you are doing in those areas that you care about. Given this knowledge, you can create your own improvement plan, as an individual architect or for an entire organization.

24.4. For Further Reading

The four models that underlie the assessment framework presented here are described in more detail in the Technical Note “Models for Evaluating and Improving Architecture Competence” [Bass 08]. These models are the following:

• Duties, Skills, and Knowledge (DSK) model of competence. This model is predicated on the belief that architects and architecture-producing organizations are useful sources for understanding the tasks necessary to the job of architecting. To assemble a comprehensive set of duties, skills, and knowledge for architects, we surveyed approximately 200 sources of information targeted to professional architects—books, websites, blogs, position descriptions, and more. The results of this survey can be found in [Clements 07].

• Human Performance model of competence. This model is based on the human performance engineering work of Thomas Gilbert [Gilbert 07]. This model is predicated on the belief that competent individuals in any profession are the ones who produce the most valuable results at a reasonable cost. Using this model will involve figuring out how to measure the value and cost of the outputs of architecture efforts, finding areas where that ratio can be improved, and crafting improvement strategies based on environmental and behavioral factors.

• Organizational Coordination model of competence. The focus of this model is on creating an interteam coordination model for teams developing a single product or a closely related set of products. The architecture for the product induces a requirement for teams to coordinate during the realization or refinement of architectural decisions. The organizational structure, practices, and tool environment of the teams allow for particular types of coordination with a particular interteam communication bandwidth. The coordination model of competence compares the requirements for coordination that the architecture induces with the bandwidth for coordination supported by the organizational structure, practices, and tool environment [Cataldo 07].

• Organizational Learning model of competence. This model is based on the concept that organizations, and not just individuals, can learn. Organizational learning is a change in the organization that occurs as a function of experience. This change can occur in the organization’s cognitions or knowledge (e.g., as presented by Fiol and Lyles [Fiol 85]), its routines or practices (e.g., as demonstrated by Levitt and March [Levitt 88]), or its performance (e.g., as presented by Dutton and Thomas [Dutton 84]). Although individuals are the medium through which organizational learning generally occurs, learning by individuals within the organization does not necessarily imply that organizational learning has occurred. For learning to be organizational, it has to have a supra-individual component [Levitt 88]. There are three approaches to measure organizational learning: (1) measure knowledge directly through questionnaires, interviews, and verbal protocols; (2) treat changes in routines and practices as indicators of changes in knowledge; or (3) view changes in organizational performance indicators associated with experience as reflecting changes in knowledge [Argote 07].

The Open Group offers a certification program for qualifying the skills, knowledge, and experience of IT, business, and enterprise architects, which is related to measuring and certifying an individual architect’s competence. Visit opengroup.org for details. The International Association of Software Architects (IASA) offers a similar certification; see iasahome.org.

Dana Bredemeyer and Ruth Malan have written many articles on the role of the software architect (www.bredemeyer.com/who.htm), including their duties and skills [Bredemeyer 11] (www.bredemeyer.com/Architect/ArchitectSkillsLinks.htm). They have their own competence framework as well as a skills development program.

The U.K. Chapter of the International Council on Systems Engineering (INCOSE) maintains a “Core Competencies Framework” for systems engineers that includes a “Basic Skills and Behaviours” section listing “the usual common attributes required by any professional engineer” [INCOSE 05]. The list includes coaching, communication, negotiation and influencing, and “team working.”

The classic work on the Balanced Scorecard was created by Kaplan and Norton [Kaplan 92] and the classic work on Business/IT Alignment was originally created by Luftman [Luftman 00], although this has been updated to explicitly consider the role of architecture in alignment [Chen 10].

Boehm, Valerdi, and Honour [Boehm 07] provide one of the few empirical studies of systems engineering and how an investment in “better” engineering pays off (or doesn’t) in the future.

24.5. Discussion Questions

1. In which skills and knowledge discussed in this chapter do you think you might be deficient? How would you reduce these deficiencies?

2. Which duties, skills, or knowledge do you think are the most important or cost-effective to improve in an individual architect? Justify your answer.

3. How would you measure the specific value of architecture in a project? How would you distinguish the value added by architecture from the value added by other activities such as quality assurance or configuration management?

4. How do you measure items such as “customer satisfaction” or “negotiation skills”? How would you validate such measurements?

5. How would you distinguish benefits caused by systematic organizational learning from the benefits due to heroic efforts by individuals within the organization?

6. Section 24.2 listed a number of practices of an architecturally competent organization. Prioritize that list based on expected benefit over expected cost.

7. Suppose you are in charge of hiring an architect for an important system in your company. How would you go about it? What would you ask the candidates in an interview? Would you ask them to produce anything? If so, what? Would you have them take a test of some kind? If so, what? Who in your company would you have interview them? Why?