5. Architectural Design

Architect

“Every system has an architecture, encompassing the key abstractions and mechanisms that define that system’s structure and behavior as seen from the perspective of different stakeholders, each with a different set of concerns. In every case—from idioms to mechanisms to architectures—these patterns are either intentional or accidental, but insofar as they are visible, such patterns reflect the style and inner beauty of each system.”¹

—Grady Booch, Handbook of Software Architecture

Figure 5.1 Every system has an architecture. The honeycomb of a beehive is a great example of an architecture whose conceptual integrity allows and applies patterns, continual evolution, flexibility, scaling, and reuse.

Architectural design transforms the requirements, as captured in scenarios and qualities of service into designs that can be built to satisfy and delight users. The analogy between the work of software architects and civil architects has been made countless times to illustrate the point. Civil architecture, however, consists not only of the work of professional architects but also of thousands of years of craftsmanship in the application of patterns that work.²

A good architecture reflects a conceptual integrity that makes the architecture easy to understand, use, maintain, and evolve. This conceptual integrity is typically achieved through the choice, application, and specialization of the right patterns to the broad qualities of service and scenarios defined for a system.

Although there is broad agreement on the characteristics of a good architecture, there is controversy about the process of architectural design. The extremes range from the concept that architectural design should be practiced independently of implementation³ to the idea that architecture should emerge from implementation.⁴ Accordingly, architecture is sometimes an explicit process, whereas at other times it is a set of activities combined closely with development.

A Value-Up View of Architecture

A value-up approach asserts that architecture needs to be explicit but can emerge from the implementation, and it absolutely needs to be current with the implementation. VSTS follows this view. In VSTS, architectural designs are views on source code at whatever state of implementation the code happens to be. Architecturally significant updates to diagrams, source code, or configuration simultaneously update the architecture and the implementation.

VSTS supports the architectural design process by enabling an architect to design the architecture of software to be built, and it exposes to the architect aspects of the architecture of software that already exists. VSTS focuses on the deployment structure, which is expressed in terms of configurations of applications, which are defined as discrete packages of resource that are deployed together.

Service-Oriented Architecture

Conceptual integrity is a key part of good architecture. Service-oriented architecture (SOA), the predominant modern conceptual framework for software and systems architecture, provides that integrity. The rise of SOA has been driven by two overriding business concerns—the need for business agility and the desire for maximum reuse of existing assets. Together, these two collapse into the twenty-first century edict: “Do more with less.”

Business agility is all about “doing more” faster. IT systems have historically suffered from much less flexibility than the business demanded. As the pace of business change has increased in the last couple decades (ironically, often spurred by IT), the gap between the evolving business needs and IT’s ability to adapt to them has widened. The promise of SOA is that IT capabilities can be exposed as services that are closely aligned with business services, flexibly recombined, and changed as needed in small increments.

A business process can be seen as a thread of control flowing through the capabilities of the organization. Exposing the organization’s capabilities as discrete granular services allows flexibility in the way they can be combined into business processes and increases the ease with which the processes can be changed and the capabilities or their implementation can be altered.

This style contrasts with the historic approach of monolithic systems that attempted to circumscribe large problems, anticipate all requirements in advance, and address them in very large projects. In many ways, SOA is an architectural paradigm aligned with continuous improvement.

Maximum reuse is about doing it “with less.” First, almost every enterprise has large sunk costs in legacy systems. These systems were not often built for maintainability or flexibility, and the costs and business risks of replacement are often high. SOA enables these legacy systems to be wrapped with facades that expose them as web services so that they can be recomposed flexibly into larger systems. Additionally, SOA enables interoperability among diverse operating systems and technical platforms. The WS-^* standards make it possible to orchestrate distributed applications across Windows, Linux, and mainframes, spanning .NET and J2EE implementations.⁵

Another business trend favoring SOA is the need for geographic and organizational flexibility in locating and composing systems. Increasingly, what appears to the users as one system may be developed and managed as many separate services, and over time, those services may be migrated or replaced as business needs warrant. Consider the typical web experience of looking up directions and a map. There are only a handful of GIS mapping services, whose interfaces are exposed through web services to hundreds of thousands of web sites that provide the appearance of customized maps. The visitors to those sites couldn’t care less whose engine is drawing the map and directions—the web site owner makes a simple business choice and can change the choice as business conditions evolve.

The same flexibility applies to proprietary services. By using SOA and making services self-contained and granular, organizations can retain significant choices in how to locate, outsource, or contract for the development and management of the business capabilities on which they rely. Similarly, choices about maintenance, upgrade, and tuning can be made at the level of the services, affording a flexibility not found with more monolithic systems.

Web Services and SOA

In theory, SOA does not require web services, but in practice, the technology for implementing web services as been almost completely aligned with the WS-^* standards, if only for the reason that this enables technical interoperability. When developing with Visual Studio, the Microsoft .NET Framework makes it easy to implement the web services. Of course, there is more to interoperability than just the standards.

Contract-First Design

The key to interoperability is that services describe themselves in terms of interface contracts. For web services, these interfaces are expressed in the Web Services Description Language (WSDL).⁶

A valuable SOA practice is contract-first design, or in other words, specifying the message formats and the WSDL among participating services before being concerned with implementation details. Contract-first design can ensure loose coupling and prevent decisions about how to implement services from creeping into the overall distributed system design.

Practices for contract-first design are still evolving. WSDL and XSD do not yet describe message sequence or pre- and post-conditions as examples. Fuller contracts need to specify the messages that the services publish and consume, the sequence of handling, and the constraints in order to isolate the public contracts from the private details of the implementation. One approach to improving contracts can be found in the Windows Communication Foundation Services (WCF), previously known as project “Indigo,” part of Windows Vista.⁷

Constraints with Degrees of Freedom

A value-up approach requires an explicit architecture, but it may emerge from the implementation. This creates an apparent contradiction. On one hand, you must consider all the QoS and deployment constraints in the context of the scenarios. On the other hand, value-up considers running tested code to be the best measure of system progress, and the act of developing is the act of designing, albeit at a very detailed level. It is exactly the act of development, in small fixed-scope iterations, that enables design to emerge from the implementation.

The answer lies in the creation of a baseline architecture early in the project lifecycle.

Baseline Architecture

A baseline architecture is a executable skeletal application designed expressly to mitigate technical risk and provide a solid, stable basis for project iterations. Some agile developers refer to this as “iteration zero” when platform infrastructure components are installed and a “thin vertical slice” of the system is built, exercising a very simple scenario from end to end (for example, from the user interface to the database). The goal here is to flush out and validate the project’s architecturally significant mechanisms unambiguously. Removing ambiguity mandates that the baseline architecture be executable.

As an example, consider a popular application style—a web application using a relational database. A number of fundamental questions need to be answered early to determine the best fit based on good citizenship and all the perspectives of the advocacy groups described in Chapter 2, “Value-Up Processes”. Should this application be bought or built? Should the application be a smart client or a web application? If it is a web application, what processing should be done on the client using technologies such as Asynchronous Java Script and XML (AJAX), and what processing should be done on the server? As for persistence (assuming a relational database), what are the organizational standards for the choice of a relational database? Should a new database instance be created, or should tables be added to an existing database instance? What is the source of reference data for this application? How does this application integrate with other applications in the enterprise? How many business transactions per day must this system be able to execute to support the project’s economic justification? Will this application be deployed, operated, and maintained in a corporate datacenter? If so, what are the technical constraints of the datacenter? As these fundamental kinds of decisions are made early in the project lifecycle, they result in constraints that guide and stabilize the project under development.

There is a delicate balance to strike. As you work through these technical issues, try to defer decisions to the “last responsible moment.” The last responsible moment, however, will vary depending on project complexity. Pull together all the advocacy groups from Chapter 2 early in the project lifecycle to agree on the high-level decision policy. Often, you will let detailed design issues, such as interfaces and method factoring, evolve from implementation. In more complex projects with many dependencies, you may need to pin down interfaces and mitigate architecturally significant risks early to avoid later rework. Then, within the agreed policy, build an executable baseline that implements these high-level design decisions so that they can become architectural constraints on the project.

Defining this baseline early benefits developers by providing stable platform infrastructure components and protocols for application development. Defining these constraints early also enables predictability and planning for other stakeholders. For example, Release/Operations can plan for the deployment and management of the application in production and can ensure datacenter readiness. Product management in the line-of-business can create marketing plans designed to attract levels of usage at transaction rates the system can support. Business analysts can design business process monitoring reports to analyze the application’s return on investment, which will feed planning for the next version. Enterprise architects can design and implement reusable services across the enterprise, migrating toward an enterprise-level SOA. Defining these constraints early enables a stable, predictable basis for multiple stakeholders.

Validate Architectural Decisions

This process is not just top-down. You need to validate these constraints bottom-up by building an executable skeletal architecture. VSTS enables you to model your “as-is” datacenter using the Logical Datacenter Designer (shown in Figure 5.5) and your “to-be” application using the System and Application Designers (shown in Figures 5.3 and 5.4) and then to run a validation report to identify conflicts between the two. Then your System and Application models can be used to generate skeletal code that can become part of your executable baseline.

Doing so provides many benefits. The running code is unambiguous—should areas of design seem unclear, the source code can be consulted. The running code can be performance tested to validate initial transaction rates of the system. System settings can be verified against datacenter policies to identify potential conflicts. Also, building an executable system forces lower-level design decisions to become explicit, further refining the baseline.

Refining the Baseline

As you build out the skeletal architecture to run end-to-end, you are forced to consider lower-level design decisions. How will you structure your application logic? What is your data access approach? Will you deploy the business logic on the same server as the web server, or will you deploy them on separate application servers? What authentication and authorization mechanisms will you use? What protocols will you use between servers?

Don’t try to address all the concerns yet. Choose the areas of highest technical risk to validate and let the other decisions emerge through implementation. Think about building the simplest skeletal application that will meet your constraints, mitigate technical risks, and still provide a sound, stable basis for development iterations.

You should also leverage the design knowledge and experience of others who may have also solved similar problems before and captured the learning in patterns.⁸ Patterns also give you a language with which you can concisely describe solutions, mitigate risk by reusing known good designs, and enable future maintenance and reuse of your work.

Reference Architectures

In many cases, you can use a reference architecture as a baseline. For example, Microsoft has created a reference application called Applied Integration Baseline (AIB), which uses a pattern-based approach to build an executable baseline (see Figure 5.2). This application ships with more than twelve Visual Studio projects and is based on ASP.NET, BizTalk Server, SQL Server, Host Integration Server, and VSTS.

Figure 5.2 The Application Integration Baseline provides an interactive “Narrator” to visualize the baseline architecture. The visual models include VSTS Application Designer and Logical Datacenter Designer models as well as a pattern-based description of the system. Here the narrator is illuminating a scenario one step at a time by rendering each step on the Application Designer diagram.

In a similar way, the Windows Server System Reference Architecture (WSSRA) offers the infrastructure architect a baseline for the configuration of a datacenter. Obviously, actual circumstances will vary, but the WSSRA baseline provides a preferred model to set up the server environments.

VSTS and Service-Oriented Architecture

For all the following reasons, VSTS focuses on the practical aspects of implementing a distributed system using a SOA. VSTS provides four designers that handle the major activities involved:

1. The Application Designer (see Figure 5.3) enables you to design the application components that expose and consume web services.

Figure 5.3 Use the Application Designer to describe the components that will communicate with web services. Note that you can add your own custom types to the toolbox.

2. The System Designer (see Figure 5.4) enables you to compose and configure the applications into systems and potentially reusable subsystems.

Figure 5.4 Use the System Designer to compose these applications into whole systems or reusable subsystems.

3. The Logical Datacenter Designer (see Figure 5.5) captures servers (such as IIS), their configurations, and network trust zones as you use them in a data-center into which one or more systems will be deployed.

Figure 5.5 Use the Logical Datacenter Designer to capture the configuration of the servers and the network trust zones that are in operation or planned for the datacenter.

4. The Deployment Designer (see Figure 5.6) enables you to map each component in a system to the servers in a logical datacenter to specify how the distributed parts of the system need to be deployed.

Figure 5.6 Use the Deployment Designer to map each application component to the corresponding server and thereby specify how the distributed application needs to be deployed.

In VSTS, the designs generate source code and XML files and therefore become live views into the source code and configuration files. As architecturally significant changes are made in source code and configuration files, these diagrams update automatically.

Quality of Service Mindset

MSF describes a QoS mindset as follows:

The idea is that qualities of service such as performance and security should not be considered late in the project but throughout it. When ignored, these qualities of service are ultimately consumer dissatisfiers.⁹

Architectural design needs to reflect this QoS mindset. Often the architect is the team member most able to consider the implications of QoS requirements, implicit or explicit. In turn, the decisions with the greatest impact on QoS are typically made during design.

Security

Security is a primary architectural concern. The primary architectural analysis technique for security is threat modeling, which looks for potential vulnerabilities of the planned system.¹⁰

Fortunately for web services, the security profiles are implemented in the Web Services Enhancements (WSE) Microsoft .NET Framework.¹¹ WSE enables you to sign and encrypt web services and set trust domains.

Performance

Performance is another leading architectural concern. A typical trap in architecture is that performance problems usually are not detected until late in product development, leading to costly redesign or rework. VSTS mitigates this risk in two ways. Because the SOA designers generate code, you can create application skeletons early that can be submitted to trial deployment and performance testing, well before the system is complete. And because the systems are made of autonomous services, reconfiguration is straightforward, and tuning can be optimized to the problem areas.

Citizenship Mindset

The shift from self-contained applications to services requires a change in mindset, which MSF calls Citizenship. As applications move from being self-contained projects to consumers and publishers of services, the project team needs to think differently about its approach. Consumers need to look for services to consume and not bypass the infrastructure that is available to them. Publishers need to realize that many of their consumers are unknown. Published services need to act reliably for any consumer adhering to the service contract and protect against malicious hackers who are looking to exploit the service with ill-formed messages or other attacks.

This mindset presumes a transparency and trust that are typical of high-performing organizations. Here’s a description from Philip Evans’ and Bob Wolf’s “Collaboration Rules” that plays equally to the services of an SOA as to the appropriate culture for developing one:

Where trust is the currency, reputation is a source of power. In a dense network . . . . there is more power to being an information source than an information sink. Consequently, individuals are motivated to maximize both the visibility of their work and their connections to those who are themselves broadly connected.¹²

Design for Operations

A typical problem that most organizations face is the complexity of moving a system as designed into a datacenter as configured. The development and operations teams typically have different vocabulary, different staffs, and different physical locations (sometimes on different continents or in different companies). The information flow between development and operations can be very dissatisfying, often inaccurate and incomplete.

Particularly with distributed systems, there are many parts to deploy on many servers, each with its own configuration requirements. For example, applications that run in test environments don’t run in production environments because the applications violate policies that developers were not aware of. In this situation, architectural design changes are sometimes required to bring applications into conformance, or conversely, applications need to make requirements of the datacenter, such as specific versions or service packs. These operational requirements may conflict with the existing deployments of other applications. The consequence of these communication mismatches may be a delay of weeks or months between the time that architects, developers, and testers announce that an application is ready to deploy and the readiness of operations to actually deploy it.

In the value-up approach of embracing change and enabling business agility, these delays are a huge barrier. VSTS starts to remove these disconnects by enabling Design for Operations. The principle is simple. Rather than waiting until an application is implemented to attempt deployment, VSTS tests the deployability of the application during its design. The solution architect and infrastructure architect can then resolve any incompatibility between the application requirements and the datacenter constraints well ahead of actual deployment and significantly reduce the bake time needed for successful deployment.

Design for Operations is made possible by the System Definition Model (SDM):

At the heart of SDM is the notion of a system. In its most basic form, a system is an independently deployable configuration of resources. For software systems, resources are ultimately directories and files, such as binaries, XML files, configuration files, SQL script files, and so on. For hardware systems, such as graphics systems, motherboards, network interface cards and power supplies, resources might include the boards, processor chips, memory chips, fans, and other lower-level components.

If a system enables access to its resources, or if its resources access services offered by resources in other systems, it exposes those resources via endpoints. For example, a motherboard might provide an IDE end-point for peripherals; web service endpoints provide a means for an application to expose or consume web services; HTTP endpoints provide a means for a server to enable access using the HTTP protocol.¹³

The SDM is visualized on the Deployment Designer (as shown in Figure 5.6). It marries the application design (typically from the solution architect) with the logical datacenter design (from the infrastructure architect). Directly on this diagram, you can validate the architecture—that is, you can determine whether the application as designed will deploy in the datacenter as configured. Any exceptions appear directly as warnings with glyphs on the diagram as needed (see Figure 5.7).

Figure 5.7 Validation of the Deployment Designer can identify conflicts between the application as designed and the datacenter as configured. In this example, the application requires Windows Server 2003, but it is intended for deployment on a server with an earlier OS.

The SDM defines a model of the system and a model of the datacenter and enables a deployment to be defined and validated. Models allow validation of a design, even before coding has begun. After the application has been implemented, changes to the application’s configuration to resolve errors are directly synchronized with code and configuration files. The fact that models reflect real code is very important for deployment validation. Models also describe how configuration can be overridden.

In addition to enabling the validation at design time, VSTS facilitates the verification at test time by enabling test labs to run in virtual machines (which is discussed in Chapter 7, “Testing”).

Summary

In this chapter, I briefly discussed a value-up approach to architectural design, the process of creating a solution architecture. It is highly iterative, working both top-down and bottom-up, emerging from choices about implementation and driven by the scenarios and qualities of service that define user value and contract boundaries.

SOA is the dominant super-pattern that governs overall design of modern distributed systems because it enables business alignment, agility, interoperability, and reuse. The MSF mindsets for Qualities of Service and Citizenship bring essential points of view to SOA—the consideration of all aspects of experience and creation or application of reuse wherever possible. Contract-first design is an important practice for successful SOA.

The web services standards are the main enabler of SOA. VSTS supports architectural design of web services and composition of systems directly based on these services. VSTS introduces Design for Operations to reduce the complexity of preparing distributed systems for deployment. VSTS provides a Deployment Designer that marries the logical datacenter design to the system design, enabling you to see whether the application as designed will deploy in the datacenter as it is configured.

The next chapter looks at the value-up practices of development.

Endnotes