Design Principles

Principles are propositions. They assert a set of beliefs about the world. They act as a substrate to a system of values. They serve to guide decision making across your entire organization.

Principles are important in architecture and design because they help to scale your team. They provide guidance to developers making a local decision. Many small local decisions made by the implementation teams will tend to add up over time to produce something that has imperceptibly skewed over a period of months to accrue toward a design result that looks little like you intended.

As propositions, principles are abstract, but they should precipitate actions on the part of your teams that support them. Presumably, the principles are subsets or decompositions of your overarching corporate vision or the strategy that you’ve identified. If they’re not, your teams and department will suffer from a lack of alignment. You’ll be doing stuff that doesn’t matter. This is mistaking activity for progress.

Well-established principles will also allow you to negotiate with the product managers, executives, and other stakeholders. If you state your principles, and they follow logically from the stated business vision and strategy, you can more quickly resolve disputes as they arise in the project. If your CIO believes that they should be moving compute and storage to the cloud, and you believe that you must run your own datacenters, you have a mismatch of principles that must be reconciled to ensure that your project doesn’t blow up.

On the other hand, if you state your principles and publish, circulate, and reference them in other architecture artifacts, you have a token for conversation with these stakeholders. If you can get them to agree on the principles, you can help guide the teams with confidence that you’re all rowing the boat in the same direction. This makes decision making less fraught with friction; the answers become more obvious. They help you to gain alignment and clarity.

An obvious example of a set of principles is the Ten Commandments. Specifically, the commandment to “honor thy mother and father” works well here. You want your principles to be at this level. It is not certain what it means to “honor” a person; it is abstract. The discrete set of actions to carry out are not obvious or stated. You then must consider, when left to your own devices, how you should proceed in order to enact the concrete action of “honoring.” Perhaps you decide that not blaming them for forgetting your birthday is the honorable thing to do. Perhaps you think it means that you must buy them a house. Because, like a parent, the architect cannot be there to direct when every local decision is being made. These are the sorts of things that will shape the overall quality and adherence to the timeline and budget, and the kinds of things developers decide all the time: whether to inline the stylesheet or not, or take an extra few minutes to code this to an interface instead of an implementation, or take the time to store the application’s database passwords in a key management store instead of in plain text. If you have principles that they can consider, they will act more in accordance and the system will be closer to the desired image.

There’s an old adage I’m sure you’re familiar with that states, “You can have it good, fast, cheap: pick two.” That is applicable here. If the quality and craftsmanship are high and you deliver it inexpensively, it won’t be done fast (and so on). It’s OK to want it fast and cheap; this is the reason fast food restaurants exist. But stating what you value up front over other also desirable qualities is making an actual decision that can truly guide behavior.

The Open Group Architecture Framework (TOGAF), which was my architecture training and certification many years ago, publishes an in-depth way of approaching technology principles. You can also take a shortcut and read Digital Principles or IBM’s old published principles.

Those sets of principles are in no way intended to be something to just copy. Take a look at them. They are just a sample so that you can see the level at which they’re written. You can adopt them for use as a jumping-off point in creating your own.

Let’s take one example: “Data is an asset.” The opposite is “Data is not an asset.” If you thought that your only job was to make applications for end users to accomplish tasks quickly and you wanted to keep costs, management, and liability down, that would be a reasonable conclusion. However, if you state that data is an asset for your organization, the applications might be window dressing over the data. You might spend considerable time gathering data, storing it, protecting it, organizing it, making it available to other applications, finding ways to market and sell it, finding new applications to build on top of it, foregrounding machine learning, and the like.

Your principles should be broad without being empty slogans. Guidelines (covered in an upcoming section) will be more local and specific. Your principles should be thoughtfully constructed, publicly stated, and often referred to. They will help guide your teams toward more efficient decision making, reduce churn, and create a sense of alignment.

Pair Designing

There are a few things you can do to help you capture the concept for the architecture that you have in your mind, which is still forming.

After you have an idea, you need to begin shaping it with others. When you’re in the stage of forming the idea for what the software should be, what it can be, and what the general contours are, it’s natural and important to express these in collaborative conversation.

To capture your ideas and test their validity, their boundaries, and their value, try pair architecting.

In 1999, the venerable Kent Beck, perhaps most famous for JUnit, wrote a book called Extreme Programming Explained. In it, he proposed the idea of pair programming, that two programmers should share one screen and one keyboard, each taking a different view into the partnership. Each could catch the other’s bugs as they happen, learn the software, and think at different levels. It’s mentally taxing to write software: you’re essentially solving a logic puzzle in a foreign language. So when one programmer becomes tired, they could switch the keyboard to let the other drive and change roles.

I love the practice of pair programming, and it has served me and my teams quite well over the years. I don’t see it used as much any more, because perhaps the people with the purse strings think it’s a form of socializing. I’ve heard one executive complain that her pair programming teams were getting half as much done. I don’t believe she was ever a programmer, because this is laughably far from the case.

As an architect designer, try carrying this practice forward. Set aside a time to solve a particular design challenge on some specific aspect of the system, especially at the idea stage. This does a variety of wonderful things:

• You’ll get a clearer, more developed idea because your partner will prompt you to clarify what you mean. If you need to express it to someone else who is present, you’ll be forced to shine a light on certain shadowy areas of your idea. You might not solve everything at that moment, but you’ll know which areas you need to refine.

• Your partner will have better ideas than you about some things. Maybe they excel at hyperthreading models, whereas you’re weaker, but you excel at design patterns. Together, your idea will take richer shape.

• Two people now understand the idea and can represent it. You’re scaling the idea quickly. This provides better avenues into an eventual project planning session.

If you’re not geolocated together, you might put this in practice by having a Webex or Hangout where one of you shares your screen, and you begin sketching your parti, your lookbook, subsystem components, and interface outlines. You might do this on a very specific piece of the puzzle. Then, you can zoom your camera out a bit and have a discussion about how this part relates to the broader concept and how it will support or shape related ideas.

Your work is to define the concept, agreeing on what populates this world, what their names are, how they interact, and where the boundaries are. I have used this technique a lot with my team; it results in more nuanced and thoughtful design, closer personal relationships, and more common understanding, and the best idea always wins.

The artifact result will be a few key sections of code that help clarify unknowns and reduce the risk to the project.

Murals

While you are in the archetypal and conceptual realm of design and are transitioning into the developmental stage, you might want to use a mural to help organize your design concepts into something you can begin to work on.

A mural can be a collage that might cover an entire conference room wall. Use the wall and sticky notes and pens to capture your ideas. The elements of your wall could include the following items that you would have created during Design Thinking:

Personas

Who will use this work?

Customer Journey Map

Why are they using this and how do they interact with it?

Pains

What about their current process is problematic? Where are they thwarted?

Gains

What new innovations can you offer them that might help them take a leap forward in productivity, delight, options, opportunities?

Outcomes

What is the change that would make a difference to your customers and other users? What gumdrop do they want at the end of the journey?

Metrics

How will your customers measure the difference that was made for them? How will they know they got great value from your system? Note that these should be rough and general at this stage.

Invite peers, executives, team members, and others to walk the wall with you. Guide them through how you are thinking about the project. This allows others to know where you’re headed and to contribute their ideas before things are implemented. This is about ensuring that you’re building the right thing.

Capturing the aforementioned items in the mural will help give you the shape of the work on which you will soon base your architecture definitions and your project plan.

You can make this a bit of a collage. It’s OK, and even encouraged, to add elements to your collage that capture some of the extra-synthetic conceptual work you did in Chapter 4. What painting, image, piece of music, building, motto, sketch, fabric, sculpture, or texture is inspiring to you and helps brings your concept to life? You can capture these in your lookbook.

Consider the following example. When architect Frank Gehry was designing the tower at 8 Spruce Street in New York City, his inspiration for the building was the Bernini sculpture The Ecstasy of Saint Teresa (1652), shown in Figure 5-1.

Figure 5-1. Bernini’s Ecstasy of Saint Teresa in Rome (photo by Alvesgaspar, Wikipedia CCA)

Notice how the building at 8 Spruce Street, depicted in Figure 5-2, incorporates and echoes the folds in Saint Teresa’s robes.

Figure 5-2. The tower at 8 Spruce Street, New York City, incorporating Bernini’s folds (photo by Jim.henderson, Wikipedia CCA)

Pushing yourself outside of what is comfortable and familiar to do this kind of work at this stage can mean the difference between making something mundane and forgettable and something truly special, useful, and exciting to your customers.

It’s one thing to see an example of how a marble sculptor inspired the lines for a steel building; even though one is art and one is architecture, both are in the realm of the physical, plastic arts. But what would this mean in our less tangible universe of software?

Is it a painting like Edward Hopper’s “Automat,” or Piet Mondrain’s “Composition II”? A Volkswagen Beetle, a Bugatti, a Tesla, or a Mack truck? Is the system Glenn Gould’s Bach: The Goldberg Variations or ABBA? Bootsy Collins of Parliament-Funkadelic is one kind of bass player, and Jaco Pastorius another. A sharp, classic Prada blue wool suit suggests one sense of style, whereas an Alexander McQueen patchwork paisley suit suggests another. Is the system more like a wild Jean Paul Gaultier or a buttoned-down Brooks Brothers? The venerable Chanel No. 5 perfume is appropriate for one kind of occasion, whereas Tom Ford’s scent “Fucking Fabulous” represents another attitude altogether. Which informs your concept of your software? Add sketches.

After you feel you have the mural complete, you can invite stakeholders to visit it and vote on their level of confidence that it represents the correct direction and has the vital emphases.

The mural is a temporary, transitional artifact. It serves as an organizing mechanism for all of these disparate elements and eventually gives way into traces in other artifacts. Like a butterfly, it won’t likely last more than a few weeks.

Vision Box

I will not deny, but possibly it might be reduced to a narrower Compass than it is; and that some Parts of it might be contracted.... But to confess the Truth, I am now too lazy, or too busy, to make it shorter.

John Locke, An Essay Concerning Human Understanding

Making your concept expressable in a single sentence that represents it is the mark of a clear thinker and a clear concept. Allowing three to five phrases to further describe it will help advance your idea with executives and other stakeholders. You must be able to take your broad canvas and reduce it to the value proposition succinctly.

Software used to come in cardboard boxes that had disks in them. Although not as large, these boxes were colorful, featured images and slogans, and represented the product’s promise. They told you what the name of the software was, what it was for, and what you could become by using it.

Even though software rarely comes in boxes anymore, you can adopt this idea to help create your executive summary and communicate quickly to stakeholders. The point is to work with your team in a couple of hours to produce the statement of value to customers and get away from the technical implementation underpinnings.

Jim Highsmith¹ introduced the idea of the product vision box, similar to a cereal box, which you can use to describe the main features of what you’re building. The vision box acts as a kind of executive summary. The small space forces you to pare down the concept to the three to five most impactful things from your customer’s view. Figure 5-3 shows a version of this.

Figure 5-3. The product vision box

To structure this short activity with your team, ask of your system the standard reporter questions: who, what, when, where, why? Your answers to these should help you quickly put together your box.

Mind Maps

A mind map is a way to organize and classify ideas or information. You start with a single box in the middle and allow subtopics to branch out from it. In turn, these subtopics each have subtopics. In a sense, it’s a visual representation of an outline. It acts as an aid in information architecture, and you should consider a mind map a necessary first step before any work is done on designing a user interface. It will act as a method for organizing the areas of functionality you will expose. It can serve as a method for determining the placement of your reveals (see Chapter 2 on the parti and reveals).

Mind maps are wonderful to create in collaboration with your team. Figure 5-4 shows an example.

Two great tools for making these are Lucidchart and XMind. You can use both free of charge. Lucidchart is online and works well when you want to collaborate. XMind is locally installed, and the paid version allows many more advanced features, such as saving to PowerPoint.

Figure 5-4. Mind map image from Lucidchart

Use Cases

At this point, you can use any mural, mind map, product vision box, and material generated in Design Thinking to begin to make your use cases. A use case captures a user role who performs an action to derive some clear value. These are captured two ways: in the form of a Unified Modeling Language (UML) diagram, and in a written list of steps.

Here is a template for use cases that you can use:

Capture all of your use cases together in a single list. These will become user stories or functional requirements. The Design Definition Document, which we cover in a moment, captures the nonfunctional requirements. Together, these are converged and refined into user stories, with acceptance criteria capturing both functional and nonfunctional requirements from these two sources.

Guidelines and Conventions

Principles are broad statements to guide decision making when no direct, specific guideline exists. Guidelines and conventions consist of a set of unambiguous directives for development teams to follow. So they are more specific and concrete direction on the management of certain processes, the use of certain tools, or the management of people than principles are.

You should publish guidelines on focused topics. They should state how you want people to use a particular tool, practice, methodology, or process. You might create guidelines on the Use of Security Groups in the Azure Cloud, how to use the Deployment Pipeline, best practices on writing integration tests, and so forth.

Publish conventions so that everyone does something the same way. You might have conventions on how to format source files to make them more readable and more readily understandable. The value in conventions is that your code is easier to maintain because it is quicker to read and understand, and tooling can be more predictable. It’s not that camelCase is inherently better than InitialCaps for naming methods (as in Java versus .NET), but if everyone does it the same way, you can eschew obfuscation. When 80% of the cost of software is in the maintenance, making it easier to read and understand and find issues and fix them is a valuable long-term investment.

You don’t need to invent many of these convention or style guides from whole cloth. For common languages and frameworks, first search to see whether someone has already written a convention guide that you can borrow. Here are some I’ve used, and you can, too:

Java

Follow the Google Java coding guidelines.

JSON

Follow the Google JSON coding guidelines.

React

Follow the Airbnb coding standards for React.

These can often be checked as part of your automated build process.

You should create and publish guidelines and conventions for certain critical things in your project that you know must be correct and will make a material difference in the success of the architecture and strategy. For example, suppose that you have developers who don’t get along and organize code differently in their own silos, and extensibility really matters in your project. You want to be sure to call out how to organize the codebase so that they can support your aim. You might write a guideline document stating the structure for the projects under the service, as shown in Example 5-1.

+---<application name>
: +---<application name>-utils
: +---<application name>-domain
: +---<application name>-service-api
: +---<application name>-service-impl
: +---<application name>-service-client
: +---<application name>-web

Then, you can go the extra mile and explain the purpose of each of these subprojects, to help justify why you’re asking people to do the extra work to separate them out. The subsections that follow provide some examples that can be used in any language type (but Java is assumed here).

Approaches

Approaches are short versions of Design Definition Documents (discussed in the following chapters). These are perhaps 3 to 10 pages long and cover a specific engagement.

For example, we had a customer who wanted to integrate with our APIs in a certain unusual way because of constraints of their legacy system. So, the architect wrote an Approach document describing how the integration should be performed, the protocols to use, where to get certain data, the security implementation, and so forth.

Approaches have a loose structure, other than the expected front matter. The remainder has headers for the sections you need to draw attention to. Make sure they’re MECE (Mutually Exclusive and Collectively Exhaustive).  

Approaches are meant to be written quickly and focused on a specific local problem. They are not used when designing an entire new system or process or strategy.

When the development teams or sales teams come across one of the following situations, you have something of architectural significance on your hands and you should create an Approach:

• The requirement is associated with high business value and changes could break mission-critical/high-revenue components (such as Shopping), or there is a large new opportunity.

• Has high technical risk (our organization has little real experience with a proposed new technology).

• Has high business risk (could jeopardize our reputation, change intellectual property, mergers, and acquisitions).

• The requirement is “net new" or first-of-a-kind for us: none of the responsibilities of already existing components in the architecture addresses it.

• The requirement would cross service boundaries/create a new orchestration.

• The proposal requires crossing datacenter/cloud boundaries.

• The requirement is a concern/on the radar of a particularly important/influential stakeholder (big customer, President, etc.).

• The requirement has Quality of Service/Service-Level Agreement (SLA) characteristics that exceed current ones.

• The requirement has caused budget overruns or client dissatisfaction in a previous project with a similar context, suggesting that it needs more senior attention.

• The component breaks backward compatibility of an in-use production service

Design Definition Document

The Design Definition Document (or “D3”) represents one of the crucial elements of a successful semantic design practice. This is a template that I have used with great success, and therefore little modification, for more than a decade. Here, I’ll give you the basic outline of the template, which you can adopt and adapt for your purposes.

All of the work you have done in the conceptual stage has been captured in a variety of models, including your lookbook, parti, personas, use cases, guidelines, and more. These have helped you define your own terms and think about what you’re doing, how you want to do it, and how it should be structured a bit. Some of these are internal, just for you. These help you to figure out where and what your semantic field is; they don’t so much help others. It’s not user-friendly to expect someone to sort through that collection of material, which is purposefully idiomatic.

But all of that concept work is not useful if it’s not communicated to others in a way that is clearly executable. You must represent your concept in an act of translation in a way that is comprehensive in scope, captures the relevant elements in a single place, and is testable and measurable.  The D3 is crucial to our practice because it acts as the translation of your concept into an executable, testable specification.

In this section, we present the template with the questions that you need to answer. In the next four chapters, we cover each of its main areas (business, application/services, data, and infrastructure) from a comprehensive view. Those chapters will help you formulate the content, the answers to the questions in this template.

Your audience for the D3 are the analysts who will write detailed requirements, developers who will write the code that realizes your design, testers, the operations or “run” team, compliance officers, “architects” in your organization, and the occasional executive who might need to understand how all of your quiet and contemplative concept work is going to help realize actual business value.

This is not a blueprint or specification exactly. It is at a technical and precise level. Its scope is the nonfunctional requirements of the system. The functional requirements are expected to be expressed elsewhere, say in the form of user stories derived from your use cases.

Here is the template:

Position Papers

These are comments that your architecture/design team makes on a particular tool, framework, style, or trend. When blockchain reached the common developer’s minds recently, everyone wanted to make a proof of concept and look for places to implement it. When senior developers and development managers and directors stop you in the hall and ask you about a trendy new technology, you know it’s time to write a position paper. You need to make a statement to the broad organization on your views on the value of this technology, and what applicability (if any) you see for it.

For example, when the Gartner hype cycle sees a technology (such as, say, blockchain) heading for the apex, it’s probably time to write a position paper.

I’ve also had to do so in the past for warring internal factions: some thought we should use Python for machine learning, and others not; some thought we should use JavaSpaces, and others not; some thought we should use Hibernate, and others not; some thought we should always use stored procedures, and others never; and some thought we should use aspect-oriented programming; and so on. Or, consider a scenario where we all like a certain rules engine, but people do not know when or how to use it consistently.

Position papers tend to be oriented around a specific framework, and you’re stating for the entire organization what your position is so that the organization can consider it further. Consider the objects of your position papers through the lens of people and process, as well as technology.

Position papers tend to be needed in two situations:

• When there is some emerging technology that people are excited about and you need to investigate to help keep entropy contained, lest developers begin randomly downloading some 0.01 version tool and marrying the organization to something that might be insecure, unsupported, inappropriately licensed, or wrong for some other reason. You want to state your reasons to help nurture the entire organization along.

• When there are two warring factions in your organization and you need to help clarify and set a direction for them.

You can use a ThoughtWorks Radar here to help you see when things might be coming around in your own organizations as solutions looking for a problem or when a turf war or religious war is about to break out. Your position papers should go a long way to solving this.

RAID

The RAID document comes from the world of project management, and its name stands for Risks, Assumptions, Issues, Dependencies.

This is a document that will be owned and carried through the life of the project by the project or program manager. But you must start it off properly. You as the architect are the first person to be able to see and understand the risks, assumptions, issues, and dependencies that will matter most as you begin your project.

Let’s define the words that make up the RAID.

Risks

Risks are something that could possibly occur in the course of your project, and if they did, they would have a detrimental impact. Consider risks across the areas of people, process, and technology. Second, characterize the likelihood of this event occurring as well as the anticipated impact or severity. Third, ensure you have metadata such as owner, status, and date raised.

Assumptions

This is a log of the factors that you anticipate will be in place to contribute to the success of your project. For example, you might assume that the CIO will accept your decision to deploy in the Microsoft Azure cloud, or that you will be able to hire three scrum teams. Or, you might assume that you will meet your first project milestone phase gate in order to unlock a new tranche of funding.

Include the date raised, a short name, and a description along with the reason for having made this assumption as well as the action to validate that the assumption is true. You can also include a necessary action or response to remediate, if the assumption proves false. Always include a status column with the values “open” or “closed.”

Issues

An issue is something currently presenting a problem in the project. These must be logged, foregrounded, and actively managed. Your RAID document’s issues tab will have columns for ID, short name, description, impact description, impact level (high, medium, low), management priority (high, medium, low), mitigation plan, owner, and status. At the outset, you might not have any of these.

Dependencies

The columns include ID, date raised, short name, description, whether it is an internal or external dependency, the date it must be resolved, priority (high, medium, low), and status (open, closed).

This artifact is a spreadsheet, with four separate tabs, one for each letter. Each item should have its own ID for easy reference.

The RAID is a log, so you do not delete items as they are resolved; change their status to closed. This will help you in later projects to anticipate what kinds of concerns you will see. It will also help you in today’s project to see how much progress you’re making or whether you need to change your management: if you are opening as many new items as you’re closing, your project is in churn and you need to get the management team together and evaluate what’s going on, why information is not flowing or how you can better anticipate challenges, or how you can reduce areas of unknowns. Identify the source of the chaos.

Perhaps it seems unusual to have this document started at this stage, before you’ve created the architecture document. But you already should be able to anticipate and identify many of the risks, assumptions, and dependencies at this point, even if there are no issues yet before the building work has begun.

Even if you are not the development manager or project manager, the creative architect is the spiritual leader of the endeavor. You help conceive and create it, and should assist in identifying in an actionable, formal manner what can go wrong so that you can help add effort and care in those areas.

Review the RAID regularly, such as at project status meetings, to ensure that you are actively managing the project toward success.

Presentations and Multiple Viewpoints

The building architect has the paying client for whom they are working. You have the same thing: the executives running the show. Perhaps you identified a need that no one else saw and are proposing some work. More often, someone in charge will ask you to do something that requires architecture work. Either way, someone is paying for it. Eventually, they will want to know how their money is being spent, and will expect the dots to be connected for them on how your proposal solves their problem.

Architecture changes by definition are not small, simple, local changes. You must have executive support at the highest level you can get: the COO, president, even the CEO for large projects.

To help you gain this support, put yourself in the executives’ shoes and with empathy for their position, their context, and their charge, consider what pains them and what keeps them up at night. You must do two basic things:

• Map your ideas to their pains and concerns

• Illustrate these in a way that speaks their “love language”

I know one very senior executive who loves spreadsheets. He adores spreadsheets. I believe in the holiday cards he sends his friends and family, there’s a stocking hung right next to those for children on his chimney reserved for Excel. Giving him a presentation that is something other than a decorated spreadsheet illustrating how you’re managing costs gets you thrown out of the room and your project cancelled. He doesn’t want to hear the ideas as much as read the numbers. He definitely doesn’t want to see any text. Spreadsheets are his “love language.” They are the way he sees, thinks, feels, understands, and interprets work.

He doesn’t think in terms of project milestones, the points at which you ship along the way. He doesn’t think in terms of quarters. He thinks only in terms of annual budget cycles.

Working for this client, you need to express the architecture in a way that maps to these concerns and speaks in this language.

You can’t give them 200, or even 20, architecture slides. They won’t read it. Then they won’t have the right expectations about what you’re doing or why. Confusion, frustration, and heartbreak ensues. Prepare a short-pitch deck that summarizes your project based on your Concept Canvas. It should include what you’re doing, why, timeline and budget estimate, and what the state of the world will be (what value or new capabilities the executive gets) at the end of the project. Keep it to 4–8 slides. You’ll be in good shape to earn their support for your project.

Summary

In this chapter, we reviewed the kinds of documents that you can create to represent architectural concepts and practical designs as well as how to present them. Following these will help your work be clear and actionable by many teams perhaps in a variety of time zones.

As you make your documents, prepare them in a way that they serve as a pool from which you can quickly and easily draw as you build the 5 or 10 or 15 decks for the presentations that you’ll need to make to various executives, peers, teams, and other stakeholder groups.

In the coming chapters, we explore each of the major areas of traditional architecture, taking an enterprise view. Your work will vary between the enterprise and executive level, and also at the local application level. When you design one system, whether it’s a new application or a modified business process, you must consider all the viewpoints of business, application and services, data, and infrastructure. For this reason, each of the next four chapters is devoted to those respective areas, and in some technical depth. We’re switching gears.