Chapter 5. Architects and Their Roles
Business Processes and Organizational Silos
Business processes frequently span multiple systems and their organizational silos (Figure 5-1). In the past, most projects making changes to these business processes have tended to focus on individual activities associated with a single application silo. The design focus for these projects was system-centric. Later, another project would tackle another activity, and so on. For the most part, business processes evolved incrementally through a succession of system-focused projects.
Figure 5-1. Business Processes and Organizational Silos
This picture, however, has changed. These days, projects often need to make coordinated changes to multiple systems. One driver for this is the demand for larger-scale improvements to business processes, such as shrinking the overall time frame for process execution. Another driver is the introduction of new architectural styles such as service-oriented architecture (SOA) and business process management (BPM). Services are often new abstracted interfaces wrapped around the functionality of existing systems. Business process management introduces a process coordinator to direct the activities of both people and systems. SOA and BPM projects thus involve coordinated changes to multiple systems. And this presents a challenge.
Development Processes
When projects focus on changes to a single system, the development process is fairly simple (Figure 5-2). The requirements are assembled and given to the development team responsible for the system. The development team determines the required changes and implements them. The result is run through quality assurance (QA) and then placed in production.
Figure 5-2. System-Centric Development
This type of development process may be adequate for making changes to a single system, but it is deficient when it comes to making coordinated changes to multiple systems. For that you need the kind of process-centric development shown in Figure 5-3. The key difference is the reintroduction of a conscious architecture step into the process. We say reintroduction because classical software engineering has always had an architecture step here. However, in many development shops this step has atrophied to the point where it no longer exists. If so, it needs to be reintroduced.
Figure 5-3. Process-Centric Development
For those of you working with agile development, you need to pay particular attention to the manner in which architecture is addressed in your development process. It is all too easy to focus only on the needs of the current build (scrum) when it comes to making architecture decisions. The decisions you make, and the development investment in implementing those decisions, can back you into a corner later on when you encounter more complex architectural challenges. Make sure that you keep the downstream architectural challenges in focus when you make your design decisions. For a more detailed discussion of the issues and tradeoffs, see Boehm and Turner’s Balancing Agility and Discipline: A Guide for the Perplexed.1
The Architecture Step
So what’s going on in this architecture step? It is simply the examination of the end-to-end business process along with the end-to-end system dialog that supports it, rationalizing the changes that are required, and then determining what changes need to be made to individual organizations and systems. The architects are leading the development process and determining what each of the development teams needs to do.
The total architecture synthesis methodology described in Implementing SOA2 describes a comprehensive and efficient approach to developing an architecture in far more detail than can be covered here. Nevertheless, it is worth standing back and getting a feel for the kinds of decisions that an architect typically faces.
Figure 5-4 illustrates some of the questions that might arise in a service-oriented architecture. If a service is expected to provide functionality to different front-end systems, the architect needs to consider whether a single operation can support the multiple front-end systems. When back-end systems need to be accessed, should the service directly access the systems via their proprietary interfaces, or should those interfaces themselves be wrapped with services? Should the service be a monolith, or should it, in turn, be composed of other services?
Figure 5-4. Architectural Decisions
But the most telling question is what functionality belongs in the front-end systems versus the service versus the back-end systems. It is in considering this question that the full scope of architecture responsibility becomes apparent, for this question cannot be answered from the perspective of a single system or service. It requires an overview of the full set of business process participants.
For historical reasons, the question of where functionality belongs is particularly important. Prior to service-oriented architectures, most front-end systems interacted directly with back-end systems. The front-end systems ended up containing a mixture of two types of logic. One type of logic dealt with managing the interface and the other type dealt with managing the business process. The presence of business logic complicated the addition of new interfaces, for the business logic needed to be replicated. The presence of multiple interfaces complicated the maintenance of business logic, for it needed to be updated in multiple front-end systems.
Introducing a service layer or a business process manager affords the opportunity to improve the situation by moving the business process logic out of the front-end systems into shared services or business process models. The resulting division of responsibilities is similar to that found in the old IBM 3270 green-screen interfaces. The 3270 presented a collection of fields to be filled in and managed the navigation through these fields. The user filled in the fields and, when done, pressed a “do it” key. At this point the 3270 gathered all the data and submitted it to a back-end transaction that executed the business logic. Replicating this kind of separation of responsibilities between the front-end systems and the services or business processes that lie underneath can significantly simplify the evolution of the enterprise architecture.
However, it is important for you to recognize that this architecture pattern only affords the opportunity to separate interface and business process logic. Someone (namely the architect) must take advantage of this opportunity in order for the enterprise to realize the benefits. This should be done proactively, defining the roles and responsibilities of the front-end systems and the individual services as well as defining the interfaces between them before investing in the design of those systems and services.
The Project Charter
The project charter can have a significant impact on the success of the project. This charter should address three key issues:
- Quantifying business expectations
- Establishing cost and schedule expectations
- Quantifying business process risk
Quantifying Business Expectations
It is good practice to challenge the business to quantify their expectations regarding the project. Ask them directly how they are going to measure success. The answer brings focus to the design effort and can radically alter the architecture.
Consider a project that seeks to improve the productivity of the people involved in the business process. But by how much? A 5% productivity improvement requires automating some of the busywork that consumes people’s time. A 50% productivity improvement will require automating half the work that people currently do. A 95% productivity improvement calls for completely automating the process and using the few remaining people for exception handling.
As this illustrates, the quantification of expectations can have a significant impact on the approach taken to the architecture and design. Consequently, it is a good practice to not only quantify the expectations but also build the measurements into the revised business process. The measurements will establish how successful the project was in achieving the goals and indicate whether further work is warranted.
Establishing Cost and Schedule Expectations
It is important to clearly understand the business expectations regarding the project cost and schedule right from the beginning, and it is equally important to understand what these represent. We have all heard project teams complain on day one of the project, “Where did those estimates come from?” This reflects a serious misunderstanding. Although the cost and schedule numbers may have started out as estimates, by the time the project is chartered they represent business expectations that the quantified business benefits can be achieved within the project’s cost and schedule guidelines.
The import is that the first task of the architects should be to determine whether the quantified business benefits can, indeed, be achieved within the cost and schedule guidelines. This determination should be made as quickly and efficiently as possible so that if the answer is no, the bulk of the time and resources are left unspent. This affords maximum flexibility to the business in deciding what to do next, whether it is altering the project scope, adding time and resources, or cancelling the project and applying the resources to achieving other goals. The total architecture synthesis (TAS) methodology described Implementing SOA3 provides an efficient approach that answers this question with minimal time and resources.
Quantifying Business Process Risks
The third key understanding required from the project charter is that of risk to the business. Not risk associated with the present project, but rather risk associated with the business processes that are impacted by the project. In particular, you want to understand two things:
• What is the business impact if a single execution of the business process fails?
• What is the business impact if the business process becomes unavailable for some period of time?
Understanding these risks helps the architects to determine where business continuity is an issue. This is important because the techniques used to establish business continuity (fault tolerance, high availability, and site disaster recovery) significantly increase the cost of the resulting systems. A fully fault-tolerant, site-disaster-recovery-enabled solution will require a minimum of a four-fold increase in hardware and software over simply implementing the required business functionality. Such investments should only be undertaken when there is appropriate business justification. A more detailed discussion of business risks can be found in Succeeding with SOA.4
The Integration Test Step
The third step that may be missing from your present development process is that of systems integration. Although it is common practice in a system-centric design to simply deploy the changes and begin testing, this type of approach for a distributed system can lead to chaos. The reason is that it is often difficult to determine the source of a problem in a distributed system.
The system integration test step is nothing more than a well-thought-out order of assembly for the elements of the distributed system. Put parts A and B together and make sure they are communicating properly. Then add parts C and D and make sure they are talking properly. Continue adding parts incrementally until the entire solution is functioning on some basic level. Then begin more advanced testing.
This integration test step sounds so simple that you may be asking yourself why it is such a big deal. The answer is that in large-scale enterprise systems the time differential between spending a few days on a well-considered integration process and simply turning everything on at once can be as extreme as several months. It’s well worth the effort.
Architecture Improves Project Schedules
Architecture isn’t just a nice thing to do. It has a measurable impact on the duration of projects. Figure 5-5 presents a summary of data from 160 projects showing the impact of architecture and risk resolution (architecture analysis) on the amount of rework that occurs in a project. There are three sets of curves on the graph, one representing data from small projects (10K lines of code), medium projects (100K lines of code), and very large projects (10 million lines of code). If you think about it, your enterprise architecture comprises a large-scale system of systems whose total complexity likely exceeds this very large project threshold by a wide margin. Projects that seek to make significant changes to the enterprise architecture are likely to be in this very large category.
Figure 5-5. Architecture and Project Duration5
So what is this graph telling us? It says that in large projects, with little or no architecture effort, the time added to the project for rework approaches 100%—doubling the duration of the project. It also shows that as time is added to the project for architecture, the amount of rework plummets. This effect is so dramatic that adding 40% for architecture reduces the time spent doing rework to 5% or so. Combining the time added for architecture with this small amount of rework gives us a 50% reduction in the time added to the project and a net 25% reduction in overall project duration. Architecture isn’t just nice to have—it has real measurable benefits.
The fact that architecture work can have such a dramatic impact on project duration makes common sense. You are defining and then analyzing your high-level design while it is largely a paper exercise, doing development work only in controlled experiments to resolve feasibility questions. You’re making, finding, and fixing mistakes while it is still a paper design and easily modified. In contrast, finding and fixing these mistakes later in the project requires more rework. The moral is: Do your due diligence on architecture. It will more than pay for itself.
The Roles of Project and Enterprise Architects
When considering architecture, one important fact stands out: You don’t build an entire enterprise architecture in a single project. Instead, the enterprise architecture is constructed piecemeal, one project at a time. For this reason, there are two important architectural roles: project architect and enterprise architect. The project architect’s role is to rationalize the total architecture of the present project including both business process and systems. The result is the architectural vision for what this particular project needs to build. The enterprise architect establishes the architecture vision for the enterprise and guides individual projects to contribute to realizing that vision. Creating and sharing architecture vision is critical to both roles.
Project Architect Responsibilities
The project architect is responsible for defining and communicating the overall architecture of the project’s work—the big-picture design. This includes:
• Defining the end-to-end business process and supporting systems dialog
• Identifying and applying appropriate reference architectures
• Identifying existing services that can be used and incorporating them into the design
Defining the End-to-End Business Process and Systems Dialog
The primary responsibility of the project architect is to define the changes to the business process and systems dialog required by the current project. Doing this requires understanding the present end-to-end business processes that are impacted and the end-to-end system dialogs that support them. Given this perspective, the changes required to achieve the project goals are defined and the specific changes required to each of the involved systems are identified.
An important aspect of this responsibility is documenting the business process and systems architecture so that it can be understood and reviewed by the other stakeholders in the project. This affords the business community the opportunity to review and validate the proposed changes before the design proceeds any further. It also helps the design teams responsible for the individual systems clearly understand the requirements for their particular systems and how they fit into the overall picture. The techniques described in Chapter 3 are central to this communication.
Identifying and Applying Reference Architectures
A challenging aspect of project architecture is to ensure that the project architecture contributes to the realization of the enterprise architecture vision. An important instrument in this endeavor is the reference architecture. Enterprise architects create reference architectures to illustrate how specific patterns of work should be implemented in the intended enterprise architecture.
The project architect’s responsibility in this regard is to recognize the work patterns found in the current project and identify the appropriate reference architectures to be applied. An equally important responsibility is to determine when there are work patterns for which there are no reference architectures at present. If these work patterns appear likely to be found again, then the enterprise architects should be engaged to define the appropriate reference architecture.
Identifying and Applying Existing Services
Realizing the benefits of a service-oriented architecture requires reusing existing services, where appropriate, to avoid building unnecessary interfaces. This requires the project architect to examine the business processes impacted by the project, determine whether there are existing services that could be used, and incorporate those services into the architecture.
This exercise frequently requires a bit of creativity. Often the existing service will not fit cleanly into the initial conceptualization of the business process. Here the architect must experiment with the business process definition to determine whether a variation can be found that will both satisfy the business requirements and allow the use of the existing service operations. Failing that, the architect should consider whether a new or modified service operation would be appropriate. The enterprise architects should be engaged in this latter discussion.
Identifying New Service Opportunities
The project architect is in an ideal position to identify new service opportunities. This typically occurs when the project architect identifies functionality that looks like it might be usable in other contexts. However, it is typically beyond the charter of the project to validate the applicability of the proposed service in those other contexts. For that, the enterprise architects must be engaged and their involvement continued through the specification of the service. After that, the project team can implement the service as just another component of the project.
Enterprise Architect Responsibilities
The responsibilities of the enterprise architect are broad, and encompass:
• Defining the target architecture for the enterprise
• Defining a practical evolution strategy
• Defining reference architecture(s) consistent with the target architecture
• Guiding project teams in evolving toward the enterprise architecture
• Participating directly in projects requiring complex designs
• Training and mentoring project architects
Defining the Target Architecture for the Enterprise
The primary responsibility for the enterprise architect is to define the target architecture for the enterprise. This effort requires understanding the business drivers for the enterprise and determining the architecture changes required to be responsive to the business needs. Beyond simply defining the target architecture, the enterprise architects must articulate the manner in which the architecture changes contribute to the enterprise’s ability to respond to business conditions.
Communicating this architecture vision is as important as defining it. In reality, the architecture will be realized through incremental changes being made by individual projects. Thus it is important that the project teams understand the rationale behind the architecture vision so that they can incorporate this thinking into their day-to-day decisions.
Defining a Practical Evolution Strategy
No matter how wonderful the new architecture vision, the reality is that the business is presently operating with the old architecture and must continue to operate during the transition. Thus it is important for the enterprise architects to define a practical, affordable migration strategy that keeps the enterprise operating during the transition. This is often a bigger challenge than defining the new architecture.
For example, the enterprise may settle on a service-oriented architecture, but on the first day there are no services. What is a project to do? It is unlikely that the project, if it is to justify its investment in terms of returned business benefit, will be able to afford to build every service it can identify. In such cases the enterprise architects must provide guidelines for determining which services should actually be built.
Defining Reference Architecture(s) Consistent with the Target Architecture
Enterprise architectures often focus on the architecture pattern—the arrangement of components, the communications channels between them, and the guidelines governing their use. However, this alone is not sufficient to guide project architects in the appropriate implementation of this architecture. There are many ways in which work functionality can be mapped onto the architecture pattern. Some of these are good and others are bad.
The purpose of a reference architecture is to define how a work process should be mapped onto an architecture pattern. Whether reference architecture is organized as one reference architecture, mapping many different work processes onto a single architecture pattern or as a series of reference architectures each mapping a single work process onto a subset of the overall architecture pattern, reference architecture serves to illustrate, by abstract example, the intended use of the architecture pattern.
Enterprise architects must pay attention to how closely their reference architectures meet the needs of the individual projects. Do the reference architectures match the work processes being found by the projects? Are they, in fact, being applied? Are there common work processes for which reference architectures have not yet been defined? Paying attention to the usage of reference architectures is one way to ensure that the architecture vision is appropriately realized and delivers value to the enterprise.
Guiding Project Teams in Evolving toward the Enterprise Architecture
Regardless of the quality of documentation surrounding the architecture vision and the reference architectures, there are always situations in which some interpretation is required to apply the vision to the problem at hand. Enterprise architects must be prepared to provide leadership to projects in such circumstances. The alternative, reviewing project-level designs after the fact, is a recipe for rework and project delay. Worst case, the architecture guidelines will be set aside because the degree of rework, and delay cannot be tolerated. There is no substitute for proactive leadership when it comes to architecture!
Directly Participating in Projects Requiring Complex Designs
This is sort of an extension of the previous point. When you encounter complex design situations that are not covered by existing reference architectures it may be far from obvious how the enterprise architecture vision can be applied to the situation. In such cases the most senior architects (namely the enterprise architects) should step in and assess the situation. It is, after all, their vision, and they are in the best position to determine how the present situation should be addressed.
One of two outcomes (or maybe a combination) will emerge from this situation: an architecture (and possibly a reference architecture) that describes the solution in a manner consistent with the present enterprise architecture vision, or a modified enterprise architecture vision that now encompasses the situation presented by the current project. Either way, it is important that the visionaries (namely the enterprise architects) be the ones deciding which route to take.
Train and Mentor Project Architects
Good architects don’t grow on trees: They must be trained and groomed for the role. This is particularly true at the project level. Here you will find individuals from the technical side who are interested in higher-level design (i.e., architecture) but may not have the familiarity with business processes required for a balanced total architecture approach. Similarly, you will find business analysts who understand business process design but may not have the required familiarity with the technical aspects. Both make good candidates for project architects.
Whether starting from the business or technical side, project architects need to be shown how to address the full scope of an architecture: the work processes, the architectural pattern, and the mapping of the processes onto the pattern. They must be given the tools and introduced to the methodology. Most importantly, they need to have a mentor—someone they can call when it may not be obvious what to do in a particular situation. This training and mentoring are important facets of the enterprise architect’s role.
The Importance of Vision
The importance of communicating vision should not be underestimated. Consider the following thought experiment. You are planning a picnic for your team and their families. To avoid overburdening anyone, you decide to split up the responsibilities, asking one person to select the beverages, another to choose the food, and a third to determine the location for the picnic. Given these assignments, the three people make the incompatible choices shown in Figure 5-6.
Figure 5-6. Planning a Picnic without Vision
What is missing from this exercise is a shared vision of the end result—in this case, the picnic. Imagine now the outcome would be different if the image of Figure 5-7 were presented first. Here you see the vision of the intended event, including the type of food, the type of beverage, and the setting.
Figure 5-7. A Vision for the Picnic
Architecture is as much about vision as it is about good design. The project architect defines the vision for the project and the enterprise architect the vision for the enterprise as a whole. It is not sufficient for these architects to have their visions in their heads. They must be effectively shared or the rest of the participants in the process will not be able to make decisions with confidence that they are contributing positively to the realization of the vision.
Summary
The job of an architect is high-level design (i.e., architecture), whether it be of an individual project’s efforts or an entire enterprise. These designs characterize the manner in which the enterprise performs its functions (the business processes), the structure and organization of the systems that support these business processes (the architecture pattern), and the mapping of the business processes onto the architecture pattern.
Almost by definition, the work of an architect spans multiple systems and organizations. As such, the architect is effectively determining the changes that are required to those systems and organizations in order to realize the project goals. Doing this job effectively requires a clear understanding of the project charter: the quantified expectations of the project’s results, the project’s cost and schedule constraints, and the business impact of business processes that fail to execute properly. The architect can significantly reduce integration time by defining an efficient order of assembly for the project components and then overseeing the actual integration test step.
The work of the architect is best accomplished by leading the design effort, not critiquing the designs of others. An analysis of data from real projects shows that this type of leadership can reduce the duration of complex projects by up to 25%—a figure that business and IT ought to find hard to ignore.
Enterprise architectures are not built all at once. Instead, they are realized, piece by piece, through a series of projects. This gives rise to two coordinated architecture roles: the project architect and the enterprise architect. The role of the project architect is to rationalize the changes to business processes and systems within the scope of the current project. The role of the enterprise architect is to define the overall architecture vision and then coordinate the work of the individual projects to contribute to realizing that vision.
Reference architectures play a key role in this dialog between project and enterprise architects. They provide an efficient and effective mechanism for the enterprise architect to document how specific patterns of work ought to be carried out in the intended architecture. When project architects recognize those work patterns, they apply the appropriate reference architectures. This saves time and ensures that the project work is done in a manner consistent with the enterprise architecture vision.
It is important that the architecture vision be shared widely both at the project and enterprise level. This allows business stakeholders to validate the appropriateness of the proposed architecture. It also helps technical stakeholders to understand the origins of the requirements for their components and how those components are expected to participate in the larger design.