Every project has objectives—goals or outcomes toward which the work is directed. Every project also has requirements that define what is needed to satisfy a business need or fill a product niche in the marketplace. Most projects begin with considerable uncertainty about requirements details. Those details become clearer incrementally as customers learn more and provide feedback on the project team’s initial work of investigating the problem and exploring solutions. The requirements might be recorded with precision, or they could exist only in stakeholders’ heads. Either way, without a clear and shared understanding of requirements, it’s unlikely that the team will achieve its objectives.
The team will discover all (or at least most) of the customer’s requirements eventually. It’s cheaper and less painful to discover them early—before the team thinks the development work is complete—rather than late.
Exploring requirements is far more involved than simply asking users what they want. (See Lesson #11, “People don’t simply gather requirements.”). The first challenge is that not everyone has the same idea of what a requirement is. The software literature contains numerous definitions; this one is broadly inclusive (Wiegers and Beatty, 2013):
A statement of a customer need or objective, or of a condition or capability that a product must possess to satisfy such a need or objective. A property that a product must have to provide value to a stakeholder.
The term requirements encompasses numerous categories of information; several are defined in Table 2.1 (Wiegers and Beatty, 2013; IIBA, 2015). Software practitioners don’t uniformly agree on what to call each type of information. What you call them is less important than recognizing the need to explore, record, and communicate these diverse classes of information to the people whose work depends on them.
Concisely put, business requirements describe why the organization is undertaking a project. User requirements describe what users will be able to do with the product. Functional requirements tell developers what to build. Aligning business, user, and functional requirements is an essential component of planning for success.
The core of requirements knowledge is a set of product—or solution—requirements that describe the product’s features, functional behaviors, and characteristics. Projects often have additional transition requirements, which describe activities the project must complete beyond building the product itself (IIBA, 2015). Examples of transition requirements include creating and delivering training materials, creating documentation for product certification, writing support documentation, migrating data, and other actions needed to help users move from the current state to a future state that includes the system.
We can divide the broad domain of requirements engineering into the major subdomains of requirements development and requirements management. These subdomains encompass the five major sets of activities identified in Table 2.2 (Wiegers and Beatty, 2013). Software teams don’t perform the various requirements development activities sequentially. They are incremental and interwoven.
One of the central challenges of software requirements engineering is to ensure that the team understands and addresses the real problem, which might differ from the problem the customer initially presents. Customers often present solution ideas instead of needs. Those solution ideas can mask the actual problem and lead to implementing a solution that misses the mark.
Unlike most software work, requirements work is less about technical computing and more about interpersonal communication. Because requirements development is challenging, it’s not realistic to expect every project team member to be fully proficient at it. Many organizations develop a cadre of people who are highly skilled at requirements activities: trained and experienced business analysts (BAs), product managers, or—on projects using agile development approaches—product owners. The term business analyst has largely replaced other terms for people who perform requirements functions on information technology (IT) projects, such as requirements engineer, requirements analyst, systems analyst, and simply analyst. Unless the role distinction is significant, I’ll use business analyst to refer to whoever is conducting requirements activities on a project, regardless of their job title or other responsibilities.
In recent years, the importance of business analysis as a specialized project role has been recognized through the establishment of professional organizations, such as the International Institute of Business Analysis or IIBA (https://www.iiba.org). These organizations have developed bodies of knowledge and certification programs (IIBA, 2015). Even if a project team doesn’t include a dedicated BA, other team members who work with stakeholders to understand requirements and define solutions do perform the BA role.
Skillful BAs uncover the real stakeholder needs and craft specifications to guide the work of designers, developers, testers, and others. A dedicated BA has the system- or enterprise-level view necessary to evaluate requirements in a broad business context. When customers communicate their needs directly to developers, both parties have only a siloed view of the system from their limited perspectives. A BA offers a higher-level view that spans all the developers and their customers.
Different organizations task their BAs to perform various project functions. The BA typically leads requirements development and management activities for the project. They guide discussions with user representatives to elicit requirements through a variety of activities. Knowledgeable stakeholders provide most of the input, and the BA organizes, records, and disseminates the information.
Requirements serve as the foundation for all projects. There’s no single “right” way to handle requirements. Software development projects can choose among numerous life cycles and development models, which advocate various ways to represent requirements. The essential point is that developers need the same information to correctly build the right software regardless of the project team’s development approach. (See Lesson #6, “Agile requirements aren’t different from other requirements.”) Not all project teams create written requirements specifications. Nonetheless, they all accumulate requirements knowledge of various kinds and store it in some container, which I’ll call a requirements document or a set of requirements for convenience.
Several of my consulting clients have asked me, “How do companies that are really good at requirements handle them?” My answer is, “I don’t know; they don’t call me.” It’s hard to learn what organizations that have mastered the requirements process do unless they make their experiences public through publications or presentations. I’ve also had more than one client tell me, “You’re here because the pain has become too great.” More often than not, shortcomings in requirements are a major source of that pain.
All project teams should take requirements seriously, adopting and adapting established requirements engineering techniques to suit their project’s nature and team culture. Software teams that neglect requirements increase their risk of project failure. I’ve been interested in improving how software and systems development projects handle their requirements since about 1985. This chapter describes sixteen valuable lessons I’ve learned in that time.
I suggest you spend a few minutes on the following activities before reading the requirements-related lessons in this chapter. As you read the lessons, contemplate to what extent each of them applies to your organization or project team.
1. List requirements-related practices that your organization is especially good at. Is information about those practices documented to remind team members about them and make it easy to apply them?
2. Identify any problems—points of pain—that you can attribute to shortcomings in how project teams deal with requirements.
3. State the impacts that each problem has on your ability to complete projects successfully. How do the problems impede achieving business success for both the development organization and its customers? The problems could lead to tangible and intangible costs from unplanned rework, schedule delays, product support and maintenance, uncomplimentary product reviews, and customer dissatisfaction.
4. For each problem from Step #2, identify the root causes that trigger the problem or make it worse. Some root causes are internal to the project team or organization; others arise from sources outside the team that are beyond your control. Problems, impacts, and root causes can blur together, so try to tease them apart and see their connections. You might find multiple root causes that contribute to the same problem or several problems that arise from a single root cause.
5. As you read this chapter, list any practices that would be useful to your team.
A BA at one of my consulting clients related an unfortunate project experience. Their IT department was building a replacement information system for use within their company. The development team believed that they already understood the system’s requirements without obtaining any additional user input. They weren’t arrogant, just confident. However, when the developers presented the completed system to the users, their reaction was, “But seriously, folks, where’s our application?” The users rejected the system as completely unacceptable. The development team was shocked; they thought they were working in good faith to build the right product. However, neglecting to engage with the users to ensure that the development team understood the requirements was a serious oversight.
When you proudly present your new baby to the world, you do not want to be told, “Your baby is ugly.” But that’s what happened in this case. So, what did the company do? They rebuilt the system, this time with adequate user input. (See Lesson #45, “Organizations never have time to build software right, yet they find the resources to fix it later”) That was an expensive lesson in the importance of customer involvement in getting the requirements right.
Whether you’re building a new product or enhancing an existing one, requirements are the basis for all subsequent project work. Design, construction, testing, documentation, training, and migration from one system or operating environment to another all depend on having the right requirements. Numerous studies have found that effectively developing and communicating requirements are critical success factors for any project. Conversely, common contributors to troubled projects include inadequate project vision, incomplete and inaccurate requirements, and changing requirements and project objectives (PMI, 2017). Getting the requirements right is core to ensuring that the solution aligns with the developing organization’s product vision and business strategy (Stretton, 2018). If you don’t get the requirements right, you will fail.
I’m not saying that you need a complete set of requirements before you commence implementation. That’s not realistic for any but the smallest and most stable products. New ideas, changes, and corrections will always come along that you must fold into your development plans. But for any portion of the system that you’re building—whether it’s a single development iteration, a specific release, or the full product—you need to have the requirements as nearly correct as possible. Otherwise, plan on performing rework after you think you’re done. Agile projects use development iterations to validate the requirements that fed into the iteration. The farther away those initial requirements are from what customers actually need, the more rework that will be needed.
Some people claim that you’ll never get the requirements right. They say customers always think of more to add, there are always worthwhile changes to make, and the environment evolves continuously. That may be true, but I counter with, “In that case, you might never finish the project” (which may also be true). From the perspective that there’s always something you could add, you might never nail the requirements perfectly. But for the agreed-upon scope of a given development portion, you’ve got to get them right, or success will elude you. The more uncertainty about what the customer needs, you can proceed with development, the more rework you should anticipate after discovering the real needs.
The situation is a little different if you’re building a highly innovative product. If no one has ever made anything like it before, you’re unlikely to get it right on the first try. Your first attempt is essentially a plan to test hypotheses and determine the requirements through experimentation. Ultimately, though, your explorations will lead to an understanding of your novel product’s capabilities and characteristics—its requirements.
There’s no substitute for ongoing customer engagement to develop a set of accurate, clear, and timely requirements. (See Lesson #12, “Requirements elicitation must bring the customer’s voice close to the developer’s ear.”). You can’t just hold a workshop early on and then tell the customers, “We’ll call you when we’re done.” Ideally, the team will have customer representatives available throughout the project. The team will have many questions to ask and points that require clarification. They’ll need to elaborate high-level requirements from early explorations into appropriate detail at the right time. The team needs frequent feedback from users and other stakeholders to validate their understanding of requirements and the solutions they conceive.
It can be challenging to get customers to sign up for this extensive level of participation. They have their own work to do; their managers might not want some of their best people to spend a lot of time on the project. “You can go to a workshop or two,” the manager might say, “but I don’t want those software people interrupting you all the time with questions.”
One way to sell the case for ongoing engagement is to point out problems the organization has experienced because of inadequate customer participation. Even better, cite local experiences where customer engagement paid off. Another persuasion technique is to propose a structured framework for the engagement instead of making it completely open-ended. This framework might include some combination of informal discussions, elicitation workshops, requirements reviews, and working with screen sketches, prototypes, and incremental releases.
Customers are more likely to be excited about the project and willing to contribute if they see signs of tangible progress, such as through periodic releases of working software. They’ll also be more enthusiastic if they see that their input truly influences the project’s direction. It’s sometimes a struggle to persuade users to accept new and replacement software systems. User representatives who worked with the IT team and understand the new system and its rationale can greatly smooth the transition.
I’ve worked with several customer representatives who had an outsize impact on the project’s success. Besides providing input on requirements, some of them also provided user interface sketches and tests to verify that portions of the software were implemented properly. I can’t overstate the contribution such committed customers made to help the development team get the requirements right and deliver the right solution.
Without high-quality requirements, stakeholders can be surprised at what the development team delivers. In my experience, software surprises are usually bad news. When they see the product, the reaction I want from my customers is, “Wow, Karl, this is better than I ever imagined. Thank you!” That’s the kind of software surprise we can all live with.
The tangible output from requirements development is a record of the findings in some persistent form. That form is often a written document, commonly called a software requirements specification, business requirements document, or market requirements document. Alternatively, you could represent requirements in the form of index cards, sticky notes on a wall, diagrams, acceptance tests, prototypes, or combinations of these. All of those artifacts are useful deliverables.
The most significant requirements development outcome, though, is a shared understanding and agreement among stakeholders regarding the solution the project team will deliver. That understanding provides a reality check on whether the project’s proposed scope and budget align with the solution’s needed features and characteristics.
Expectation management is an important aspect of project management. Requirements development strives to craft a shared expectation—a common vision—among the project stakeholders regarding the solution. The requirements artifacts mentioned above communicate the specifics of the agreement. That vision aligns all project activities (Davis, 2005):
• The work that the project sponsor is funding.
• The solution that customers expect will let them achieve their business objectives.
• The software that testers verify.
• The product that the marketing and sales teams are offering to the world.
• The plans and task lists that project managers and development teams create.
It’s hard to determine whether multiple people share a common understanding of something as complex as a software development project. I’ve been in meetings where a group of people reached some agreement, but later we realized that we understood certain aspects of the agreement—and hence the outcome—differently. Those differences could lead the participants to work at cross-purposes.
A vision statement helps to achieve shared understanding and aligned expectations. I’ve used the vision keyword template below to focus the thinking of project stakeholders (Wiegers and Beatty, 2013; Moore, 2014):
As a simple example, below is the vision statement I wrote for a website that I built to support a book I wrote. Even though that tiny project was all in my brain, writing a vision statement at the outset brought clarity to what I hoped to accomplish with the website.
For readers who are interested in the book Pearls from Sand, PearlsFromSand.com is a website that will provide information about the book and its author, allow site visitors to buy copies in various formats, and facilitate building a community of people interested in sharing their life lessons. Unlike websites that merely describe and promote a book, PearlsFromSand.com will allow visitors to contribute their own life lessons and to read and comment on life lessons posted by others.
If your project doesn’t have a vision statement, it’s never too late to write one. In the training classes I teach on software requirements, I ask the students to write a vision statement for their current project using this keyword template. I’m always impressed by the concise summaries they generate in only five minutes. I can quickly understand what each project is about from their vision statements.
When several people from a project team take a class together, sometimes their vision statements have significant differences. I suggest that multiple stakeholders who represent diverse perspectives write vision statements separately. Comparing those statements reveals whether the stakeholders share a common understanding, at the top level, of where the project is heading. Disconnects suggest that team members need to work on aligning their expectations.
A consultant friend had the same experience on a client project. She said: “I asked the four major stakeholders to write their own vision statements while we were all in the same room. The results were highly varied and incompatible in some ways. It’s better to find that out early on.”
A vision statement provides a common strategic target toward which all project participants should direct their efforts. If the vision changes during the course of the project, the project sponsor must communicate those changes to everyone affected, so they retain a common focus. A vision statement doesn’t replace requirements analysis and specification. It provides a reference point to ensure that the team’s solution requirements align to achieve that vision and, consequently, success.
Consultant and author Tim Lister defines project success as “meeting the set of all requirements and constraints held as expectations by key stakeholders.” This statement implies that a project team must identify its stakeholders and determine how to engage with them to understand those requirements and constraints.
A stakeholder is any individual or group that is actively involved in a project, is affected by the project, or can influence the project’s direction. The relationships between stakeholders and the project span a broad spectrum. Some stakeholders simply have the project outcome imposed on them; others profoundly shape the requirements. And there will be someone who can change the project’s direction or even terminate it.
Stakeholders can be internal to the project team, internal to the developing organization, or external to the organization. Figure 2.1 shows some typical stakeholder communities that most software projects need to consider. There could be others, depending on the product category: corporate information system, commercial software app, government system, or a physical product that contains embedded software.
The project team should cast a wide net early on to identify potential stakeholder groups. Don’t be surprised if you generate a frighteningly long list. It will take some work to identify and understand your stakeholders. However, that’s much better than overlooking a critical community and having to make adjustments late in the project.
Acquiring customers and users are core sources of product requirements. A customer who specifies, selects, or pays for a product doesn’t always use it—and might have misconceptions of what the user community needs to do their job. Many products have a diverse array of end users. To simplify requirements exploration, group your users into user classes that have distinct sets of needs (Wiegers and Beatty, 2013). Users might not even be humans but rather hardware devices or other software systems that interface to yours. You’ll need to identify people who can provide requirements on behalf of those nonhuman components.
We usually think of direct users who will interact with the product hands-on, but you might also have indirect users. Indirect users could provide data that feeds into an information system, or they might receive the system’s outputs even if they don’t generate those outputs themselves. I was involved with a corporate project metrics system that consolidated data from dozens of projects and produced monthly reports that were distributed to many managers. Those managers were indirect users—they didn’t touch the metrics application itself. As the audience for the system’s reports, though, they were core stakeholders.
A colleague described an indirect user succinctly: “Your customer once removed is still your customer.” To identify indirect users, you’ll need to go out one or two layers beyond the application’s immediate context to see what groups of people and other systems require representation. Also, identify disfavored user classes that you don’t want using the system, such as hackers. They aren’t stakeholders—they won’t provide requirements or constraints—but you need to anticipate and thwart their evil intentions.
Consider the following questions about each stakeholder group you identify.
Who are they? Describe each stakeholder group so that all project participants know who they are. Stakeholder descriptions might be reusable across several of the organization’s projects.
How interested are they? Think about how strongly the project’s outcome will affect the group and how much involvement they want with the project. You’ll need to learn about each stakeholder group’s expectations, interests, concerns, constraints, and fears.
What influence do they have over the project? Determine what decisions each stakeholder can and cannot make. Which groups hold the greatest power over the project? What are their attitudes and priorities? You’ll particularly need to engage those groups that have both high interest and a high degree of control over the project (Lucidchart, 2021).
Who would be the best people to talk to? Identify appropriate representatives of each community with whom to work. They should be sources of definitive information.
Where are they? Elicitation is an iterative process that requires multiple encounters. It will be easiest to gather input from a stakeholder group if you have direct access to individual members. If you don’t, you’ll need to establish long-distance communication mechanisms and protocols.
What do I need from them? Determine the information, decisions, and data you’ll need from each group. This understanding will help you choose the best ways to get that information at the right time. For user classes, you’ll need to understand their user requirements—the things the product must let them do—and their quality expectations. Some stakeholder groups will impose constraints the project team must respect. Constraints could fall into several categories, including:
• Financial, schedule, and resource limitations
• Applicable policies, regulations, and standards (business rules)
• Compatibility with other products, systems, or interfaces
• Legal or contractual restrictions
• Certification requirements
• Limitations in the product’s capabilities (that is, functionality not to include)
What do they need from me? Some stakeholders merely need to be informed about significant issues that affect them, so you’ll have to know what project information is relevant for each group. Others might need to review the requirements to ensure they don’t clash with applicable policies or restrictions. Work with your stakeholders to understand what they expect from you, just as you communicate your expectations to them. A big part of successful collaboration is to build and maintain mutual trust through effective communication.
How and when should I engage with them? Once you understand which stakeholder representatives you can interact with, consider the best ways to work with them to exchange the information you both need. If you can’t access real representatives of a particular user class, consider creating personas, imaginary people to serve as stand-ins for the real human beings (Cooper et al., 2014).
Which stakeholders are most important when resolving conflicts? When resolving conflicting requirements and making priority decisions, assess which outcome aligns most strongly with the project’s business objectives. Certain user classes may be favored over others; meeting their needs contributes more to business success than satisfying requirements from other user classes. Use early stakeholder analysis to understand these priority impacts instead of waiting until you confront your first problem.
Identifying the decision makers helps to lay the foundation for a successful project. In some cases, the decision maker could be a single person, such as the project sponsor. That’s the most efficient method, provided the individual has the information to make appropriate decisions and is accessible to make them quickly when needed. More often, you’ll need to identify the right group of people to make each category of decisions. Group decisions take longer to resolve, but they better reflect a composite set of interests toward achieving the project’s objectives.
Those responsible for making decisions that span multiple stakeholder communities should base their decisions on the project’s business objectives. Objectives, a vision statement, project constraints, and other business requirements typically are recorded in a project’s vision and scope document or a project charter (Wiegers, 2007; Wiegers and Beatty, 2013). A project without clear business requirements has little basis for making—and justifying—significant decisions.
It’s not always possible to thrill all stakeholders with a project’s outcome. Tensions between stakeholders can devolve into an adversarial situation, with people working at cross purposes to protect their interests. Building a collaborative relationship with your key stakeholders goes a long way toward achieving project success. Since you might have to work with those same people in the future, it pays to establish communication pathways and mutual respect from the beginning.
One of our internal corporate users asked my team to add a new feature to an application his group used. He stressed how necessary the feature was, so we dutifully built in the capability. To the best of our knowledge, though, no one ever used that feature. I wouldn’t have been very receptive to that customer’s next request for an enhancement.
There’s folklore in the software industry that—depending on the source you read—50 to 80 percent of software features are rarely or never used (The Standish Group, 2014). Regardless of the exact figures, considerable delivered software functionality does provide little value to end users. Does your personal usage fully exploit every application’s features? Mine doesn’t. I’ve written numerous books and many articles using Microsoft Word, yet Word has many features I’ve never touched and never will. The same is true for other apps I use. Unless I’m an anomaly (which is always a possibility), the software industry puts considerable effort into delivering rarely-used features that sit there idle, patiently waiting for someone to call on them.
Focusing requirements explorations on features—the product itself—contributes to this proliferation of dormant functionality. Soliciting an open-ended list of functions from customers invites feature bloat. A feature-centric perspective can also lead to a product that seems to have the right capabilities yet doesn’t let users accomplish their tasks.
I recommend shifting requirements conversations away from the product and toward what users need to do with the product. We change the emphasis from functionality to usage, from the solution to the need. A usage-centric strategy helps the BA and development team quickly understand the user’s context and objectives. From that knowledge, the BA can better identify what capabilities a solution must have, for whom, why, and when.
Both feature- and usage-centric approaches lead to identifying the functional requirements that developers must implement. However, focusing on usage helps ensure that we include all the functionality users need to perform their tasks. It reduces the problem of building excess functionality that seems like a good idea but doesn’t help users achieve specific goals. The usage-centric perspective enhances usability because developers can thoughtfully integrate each bit of functionality into a task flow or a user objective (Constantine and Lockwood, 1999).
Usage-centric requirements exploration involves a small but significant shift in the questions a BA might ask during elicitation activity. Rather than asking, “What do you want?” or “What do you want the system to do?” the BA asks, “What do you need to do with the system?” The resulting conversation identifies tasks or goals the users need to accomplish with the help of the system.
Use cases are a good way to represent those tasks (Kulak and Guiney, 2004). Users don’t launch an app to use a particular feature; they launch an app to accomplish an objective. Each time I open my business accounting software, I have one or more goals in mind. Maybe I want to reconcile my credit card, transfer funds to my personal bank account, pay a bill, or record a deposit. Each of those goals is a use case—literally, a case of usage. I open the app with that intent in mind, and I follow a sequence of steps that invokes the functionality needed to accomplish the task. If all goes well, I complete my objective successfully and close the app—mission accomplished.
Use cases are appealing for several reasons. They’re a natural way for user representatives to think about their needs. It’s hard for users to articulate just the right set of functionality for a product, but it’s easy for them to talk about usage scenarios from their daily lives. Use cases provide a structured way to organize descriptions of related functionality. A use case template has slots in which you can record information to provide as rich—or as slender—a use case description as your team finds valuable (Wiegers and Beatty, 2013). The related functionality includes a description of the most typical or default interaction sequence for the task (the normal flow) and any variations on that typical sequence (alternative flows). The BA or developers will deduce the functionality the solution must provide so that users can perform those tasks. A proper use case description also identifies error conditions that could arise and how the system should handle them (exception flows).
Usage-centric analysis aids prioritization. The highest priority functional requirements are those that enable the highest priority user tasks. Some use cases will be more important and timelier than others, so implement those first. Within a single use case, the normal flow takes top priority, along with its accompanying exceptions. The alternative flows will have lower priorities and often can be implemented later, or perhaps never. An operational profile assessment can help you determine which use cases are performed most frequently and should have the highest priority. (See Lesson #15, “Avoid decibel prioritization when deciding which features to include,” for more about operational profiles.)
Putting yourself in the user’s shoes leads to better user experience designs. It can provide insights about limitations of an implementation that you might not get from a product- or feature-centric mindset. If users can’t do what they need to do with the product or they don’t enjoy using it, adding more features won’t increase their satisfaction.
Many agile projects record requirements in the form of user stories. According to agile expert Mike Cohn (2004), “A user story describes functionality that will be valuable to either a user or purchaser of the system or software.” User stories generally are written according to a simple template:
As a <type of user>, I want to <perform some task> so that I can <achieve some goal>.
As a <type of user>, I want <some goal> so that <some reason>.
User stories are intended to be brief placeholders that remind team members to have conversations to fill in missing details shortly before implementing the story.
One concern I have with user stories is that they possess no intrinsic organizational scheme. Merely collecting a bunch of user stories, even if written per this pattern, is not much different from the age-old elicitation method of asking users, “What do you want?” You get many significant but random bits of information, all mixed up together, along with extraneous content.
One large project collected several thousand user stories from many stakeholders on yellow sticky notes. Some of the user stories were incomprehensible; many conflicted with each other. Some looked like probable duplicates, and still others were tantalizingly suggestive but incomplete. The project had a big pile of unorganized information of many different kinds, all labeled as user stories. It was hard to tell from the pile which bits of functionality related to user tasks and aligned with the project’s business objectives and which stories were just thoughts that people had contributed.
Some of these user stories were usage-centric; others were not. The stories spanned a wide range of detail, size, and importance. They ranged from “As a user, I want the screen font to be sans-serif so that I can easily read it” to “As a payroll supervisor, I want the system to calculate state unemployment taxes for every state where we have employees so that we can pay the unemployment taxes correctly.” Neither of these stories addresses users and their usage, but rather the system’s features and properties.
Amassing an extensive collection of stories containing isolated functionality fragments requires that someone aggregate them from the bottom up to recognize themes related to user tasks. Organizing any pile of information from the bottom up is rather like assembling a jigsaw puzzle by picking up one piece at a time and asking, “I wonder where this piece goes?” My brain works better from the top down. I prefer to begin with broad strokes like identifying user tasks and then progressively refine and elaborate them into their details. I’m less likely to miss something significant that way, and it’s less work than assembling the whole puzzle one isolated piece at a time.
On the surface, the simple user story template seems like a reasonable way to record user requirements. It’s good to know what user class is requesting each bit of functionality, so you know whom to talk with to flesh out the story details at the right time. Deferring that exploration of details until the information is needed is an efficient way to allocate limited time. Statements written in this form can be usage-oriented, describing a task and stating the goal the user wishes to achieve. However, author Raj Nagappan (2020a) points out some limitations of how people might misapply the user story pattern, such as by writing stories that focus more on solutions than on the problem, and the user story’s deliberate lack of needed detail. An alternative is a job story template in the following form, which emphasizes the problem (situation) more clearly (Klement, 2013):
When <situation>, I want to <perform some task>, so I can <achieve an outcome>.
I began taking a usage-centric approach to requirements development in 1994. I quickly recognized how focused and efficient it was compared to my previous elicitation approach. If properly written with a perspective on tasks and goals, use cases, user stories, and job stories all invite requirements participants to focus on how people will use the product instead of just its functional properties. Even if you take a lean approach, rather than fully populating a rich use case template, usage-centered thinking will lead to solutions that do a superior job of meeting customer needs.
Early in my programming days, I would often begin coding with just a fuzzy notion of what I wanted the program to do. Sometimes I would begin to flail, writing and rewriting code, making lots of changes but little progress. I’d start to panic, knowing I was spinning my wheels. Eventually, I recognized the problem: I hadn’t thought through the requirements for the program sufficiently. My false starts resulted from iterating on the code, instead of iterating in my brain first; the latter is far faster. Once this light bulb came on, I always took the time to explore requirements carefully before diving into the source code editor. I never panicked again.
Later, when I was writing software for other people, I would walk out of a customer discussion feeling that I understood what they had told me and that I had the information I needed to proceed. But as I worked with our initial requirements, questions came up, and knowledge gaps revealed themselves. I often had to circle back to the customers to clarify issues, refresh my memory, and close those gaps. Sometimes they weren’t happy to see me again, but we discovered that requirements development demands an iterative and incremental process.
Once I appreciated the importance of understanding where I was headed before starting to code, I realized that I could never discover all of the requirements up front, even for a small application. Nor could I think of all the pertinent details for each requirement at the outset. But then I understood: that’s okay. I didn’t need all the details right away. I just needed enough knowledge to get my thinking started.
Effective requirements development involves progressively refining your set of requirements, their details, and their clarity. You can’t get all the requirements right with the first discussion. However, you do need to get enough accurate requirements information before you build each chunk of the product, or you’re going to build it again. Here’s a process I found to work well once the project’s business requirements have been established:
Step 1. Develop a preliminary list of user requirements (use cases or user stories). Learn enough about each to understand their scope, magnitude, and relative importance.
Step 2. Allocate the user requirements to upcoming development cycles based on their priority. Some are up first; others can wait.
Step 3. Further elicit and refine the details of those user requirements that are planned for the upcoming development cycle.
Step 4. Reprioritize—including any new requirements that have come to your attention—and then move down the priority list as development continues.
Step 5. Return to Step 2 and continue.
Ongoing prioritization is essential because there’s little value in diving into the details of requirements whose implementation isn’t imminent. As the project goes on, some anticipated needs may be deferred into the far future or disappear entirely. As Mike Cohn (2010) points out, “Teams should be careful to make sure there is a real need to better understand an item before putting more early effort into it than would otherwise be warranted...”
As people use a software application, they get ideas: “Wouldn’t it be nice if…” or “What if I could…?” Maybe a user thinks of an easier way to perform some action or discovers that, while performing action A, they’d like to drop into action B briefly. If those ideas are important enough, you’ll need to modify the system to incorporate them. Those bits of functionality that you can’t identify in advance are emergent requirements (Cohn, 2010). Regardless of the development life cycle you’re following, project plans must accommodate this natural—and beneficial—requirements growth.
Now, this doesn’t mean we need to build a complete system—or even a complete iteration—based on what we already know and then patch in all the extra functionality that pops up. We can use various techniques to reveal some of these emergent requirements. One approach is to create multiple views of requirements. Instead of only writing use cases, functional requirements, or user stories, draw some pictures. Visual analysis models describe requirements at a higher level of abstraction that lets people step back from the details and see a bigger picture of workflow and interconnections.
Writing tests flips a requirement on its head. Tests define how to tell if a system functions as we expect. Writing tests involves a different thought process from describing how we expect the system to behave under certain conditions. If we conceive tests early on, we’ll find ambiguities and errors in our requirements and discover missing requirements, such as unhandled exceptions. We might even realize that we don’t need certain requirements if we can’t think of any tests that imply their existence. The concept of early test thinking was the foundation of the agile approach of test-driven development (Beck, 2003).
Prototypes are a powerful way to bring requirements to life. They put something more tangible in front of users than a list of functional requirements or a stack of story cards. Prototypes can be simple or elaborate, paper or executable (Wiegers and Beatty, 2013). Iterative prototyping advances the requirements conversation and helps users find requirements errors and omissions before you’ve invested much effort in building the product. See Lesson #17, “Design demands iteration,” for more about prototyping.
As with functionality, parameters for some nonfunctional requirements also are hard to specify early on. Precisely what levels of availability, reliability, or usability does the application need? Plan on some learning cycles to home in on realistically achievable, cost-effective, and quantifiable target values for each significant quality attribute.
Don’t expect a meaningful answer the first time you ask some user, “What are your usability requirements?” You’ll start with the basic understanding that usability is important. Over time, you’ll extend that into understanding the various pertinent aspects of usability and ultimately determine each of those usability aspects’ target goals. With quality attributes, though, the trick is to get sufficient information early enough that the team can make architectural decisions to achieve each attribute’s objectives. It’s a lot easier to paste in new functionality than to remedy foundational architectural flaws. (See Lesson #20, “You can’t optimize all desirable quality attributes.”)
Developing a useful set of requirements of any kind requires the patience to iterate, progressively acquiring more knowledge at the appropriate time so developers can build the right product. I don’t know of any shortcuts.
Many software organizations use agile development methods on at least some of their projects. Business analysts and product owners sometimes use the term agile requirements to describe their work (Leffingwell, 2011). This term implies that an agile project’s requirements are somehow qualitatively different from those for projects following other life cycles. In my opinion, they are not (Wiegers and Beatty, 2016).
The critical point is that a developer needs the same information to correctly implement the right functionality no matter what development or project management process the project uses. Agile and traditional projects handle requirements differently in various respects. Nonetheless, most established requirements engineering and business analysis practices are useful on agile projects when thoughtfully applied.
Agile approaches strive to adapt to the inevitable change that occurs, rather than imagining that all requirements can be well understood early on and remain stable throughout the project. However, all projects require the same fundamental requirements activities. Someone still needs to analyze stakeholders, elicit requirements from various sources, and validate that a solution based on those requirements will achieve the project’s business objectives. The major differences between how agile and traditional projects handle requirements activities fall into several categories.
Most traditional project teams include one or more dedicated BAs who perform or lead the project’s requirements elicitation, analysis, specification, validation, and management activities. Many agile projects lack an official BA role. Instead, a product owner defines the project’s scope and boundaries, creates and maintains the product backlog of pending work, and gets the user stories ready to be implemented (Cohn, 2010; McGreal and Jocham, 2018). Requirements development is a collaborative process among the product owner, suitable user representatives, and other stakeholders. (See Lesson #12, “Requirements elicitation must bring the customer’s voice close to the developer’s ear.”) Developers, not BAs, are responsible for ensuring the stories contain enough information before accepting them for development.
Traditional project teams generally employ use cases and functional requirements. Instead of talking about requirements, most agile teams refer to user stories, epics (high-level features or tasks that are broken down into smaller user stories), acceptance tests, and other product backlog items, all of which represent pending work (Cohn, 2004). But it’s the same requirements knowledge, by any name. No matter how you represent it or what you call it, the team must produce and communicate that knowledge so everyone can do their jobs effectively.
Agile methods embrace the principles of lightweight and just-in-time documentation. Customers’ close collaboration with developers on agile projects generally means that requirements need less detail than they do on traditional projects. Stakeholders will develop the needed precision when they need it through conversations and appropriate documentation. Certain user stories might have little detail provided. Riskier, complex, or high-impact functionality might be elaborated in more detail.
However, relying primarily on verbal communication poses risks. Our memories are incomplete, inconsistent, and impermanent. People come and go on project teams. And, of course, the system continues to exist long after the initial development work is done. Someone must maintain it, update it, provide production support (sometimes in the middle of the night), and eventually decommission it. Each project team needs to create sufficient documentation to manage those issues without wasting time recording information that no one will use. (See Lesson #7, “The cost of recording knowledge is small compared to the cost of acquiring knowledge.”)
Starting with a broad exploration of requirements at a high level of abstraction lets you do some initial estimation, prioritization, and sequencing in any software development approach. Traditional approaches aim to create a fairly complete requirements specification early in the project. That works fine in some situations; it fails miserably in others.
In contrast, agile teams plan to generate the requirement details just before implementing a particular bit of functionality. This method reduces the risk of the information being obsolete or the requirement being unnecessary when developers and testers act on it. As shown in Figure 2.2, stakeholders and the product owner perform some initial elicitation and analysis. The product owner then allocates user stories and other backlog items to specific iterations for implementation. The product owner, developers, and customers will further clarify each story’s details through the usual requirements development activities, creating no more written documentation than necessary. The team will continue to accept and implement requirements throughout the project.
A just-in-time approach reduces unnecessary effort, but it makes it harder to recognize requirement dependencies and architectural implications that should be addressed early to facilitate stable product growth. To mitigate the risk of a flawed architecture, agile teams should look at the broad scope in early iterations and consider what major architectural decisions might be necessary. Similarly, the team should begin exploring nonfunctional requirements early so the design achieves critical performance, availability, and other quality goals that span the product.
At the top level, user stories are analogous to use cases. The distinction is in how thoroughly you detail them and whether you record the information. The BA on a traditional project might work from use cases to develop a set of functional requirements. Many agile project teams flesh out each user story’s specifics by writing acceptance criteria and tests that will indicate whether developers correctly implemented the story. If your project uses tests to represent requirement details, then as you read the other lessons about requirements in this chapter, think about how they might apply to your tests.
In reality, functional requirements and their corresponding tests are alternative ways to represent the same information. Requirements specify what to build; tests describe how to tell whether the system behaves as expected. Writing both requirements and tests provides a powerful combination. It works best when different people write requirements and tests from the same source of information, such as a use case.
Every time I’ve created and compared those two requirements views, I’ve found gaps, ambiguities, and interpretation differences. Correcting those errors during requirements exploration is far cheaper than finding them in the implemented software. An alternative strategy is to have developers record the user story details in the form of acceptance tests, which a tester then reviews. As with writing both requirements and tests, you’re engaging a pair of brains and thought processes to look for problems.
If you create multiple views of requirements, disconnects between them reveal problems. If you create just a single view—no matter which technique you choose—you must trust it to be accurate.
Prioritization considers the relative value that each requirement contributes to the customer versus the effort, risk, and cost of implementing it. Traditional projects might prioritize requirements early and rarely look back. On an agile project, prioritizing the product backlog contents is an ongoing activity. You need to select which items go into upcoming iterations and which ones ultimately to discard from the backlog. The teams are always asking, “What’s the most important thing to work on next?” In reality, all project teams, not just agile teams, ought to manage their remaining work priorities to deliver the maximum customer value as quickly as possible.
For the most part, customers don’t care how you build software applications. Customers just want the products to meet their needs, be efficient and easy to use and extend, and satisfy their other quality expectations. Most requirements development and management techniques that traditional projects employ are equally applicable to agile projects. As always, the team needs to adapt practices to best suit their objectives, culture, environment, and constraints.
In agile development, each bit of incremental change provides a standalone, usable piece of functionality. Your product is a system that you improve iteratively, learning from an unsuccessful change and rapidly pivoting to a better solution for that piece. Focusing on small requirements, each of which provides a complete, usable small change, is a different analysis process than chopping a large, integrated solution into little just-in-time bites that fit into a development iteration.
The business analysis process is somewhat different on agile projects as well. While legacy BA techniques still apply—stakeholder and business rules analysis, process modeling, and many more—how you integrate them into an incremental agile process poses a challenge for many BAs (Podeswa, 2021). As agile business analysis expert Howard Podeswa points out:
Part of the transition is developing a new mindset. The BA role on an agile project is not as much about predetermining what will be done, but about continuously renegotiating during the development process. There’s an ongoing assessment of what the work team should and should not do—the tradeoffs needed to maximize the delivered value.
Fundamentally, though, the requirements knowledge used on an agile project is not qualitatively different from that on a traditional project. Requirements development still boils down to discovering and clearly communicating the information to allow all project participants to build some portion of the right product well.
When I teach a requirements class, I ask if any of the students ever had to reverse engineer information from an existing system to figure out how to modify it or add new functionality. Nearly everyone raises their hand. Then I ask if they recorded what they learned for future reference. Few hands go up. That means if anyone has to go into that same part of the system in the future to make another change, they’ll have to repeat the reverse engineering process.
Recovering knowledge through reverse engineering is tedious. Doing it repeatedly is inefficient. If you write down what you learn, that information is available if you or anyone else needs to go back to it. This recording is a way to incrementally accumulate knowledge about an ill-documented system as you work on it. It always takes less time to write down what you learned through reverse engineering than it took you to learn it.
I would decide not to record that newly-recovered knowledge only if I were certain that no one—including myself—would ever need to work with that part of the system again. As I’m not skilled at predicting the future, I prefer to represent knowledge in some shareable form. That’s better than keeping it in my brain, where it will fade over time.
One of my consulting clients reverse-engineered a full set of use cases from their poorly-documented, mature flagship product. Then they developed a comprehensive set of test cases from those use cases, which allowed them to do thorough regression testing as the product continued to evolve. They found it was well worth their time to record that reverse-engineered knowledge.
Some people balk at taking the time to document requirements. But the hard part isn’t writing requirements—it’s figuring out what they are. Similarly, people are sometimes reluctant to write a project plan. Again, the hard part is thinking through all the activities needed to complete the project: identifying deliverables, tasks, dependencies, resources needed, schedules, and so forth. Writing a plan is transcription. It takes time, yes. However, I argue that it takes less time than trying to communicate the same information to multiple people in a consistent way throughout the course of the project. It’s also less error-prone than having to remember all that information accurately.
Perhaps teams that are reluctant to document their requirements are concerned about succumbing to analysis paralysis. A symptom of this trap is that requirements development seems to go on forever, yet construction cannot begin until the requirements are complete. Analysis paralysis is a potential risk, but good judgment will let you avoid it. Don’t use the fear of paralysis as a rationale for not recording essential requirements information.
I once was the lead BA on the third team to undertake an essential corporate project. The two previous efforts had stalled out for some reason; I never learned why. When my fellow BAs and I asked the key customer if we could talk with him about requirements, he resisted. “I gave my requirements to your predecessors,” he said. “I don’t have time to talk about requirements anymore. Build me a system!” That became our BA team’s mantra: “Build me a system!”
Unfortunately, the previous teams hadn’t documented any of the information they’d acquired. We had to start from scratch. Our customer wasn’t happy, but he cooperated when we promised to use requirements techniques that we knew worked well and to write down what we learned. That project was highly successful. My company decided to outsource the system’s implementation, and the contracting company found that our requirements provided a solid foundation for development. Had the first two teams recorded the knowledge they’d acquired, we would have had a big head start on the project.
Early in my software career, I led a project with two other developers. Two of us worked in the same building; the third sat a quarter-mile away. We didn’t have a written project plan or requirements, though we had a good shared understanding of where we were heading. We met weekly to go over progress and plans for the next week. On two occasions, one developer walked out of our weekly meeting with a different interpretation of what we had decided than the other two of us held. He worked in the wrong direction for a week, and then he had to redo that work. It would have cost little for us to write down our plans and requirements as we went along to avoid wasting time like that. I never made that mistake again.
Agile teams following Scrum hold a brief meeting called the daily scrum to update status, identify impediments, and keep the team aligned on their project goal for the next twenty-four hours (Visual Paradigm, 2020). That’s easier to do when everyone is colocated or well-connected electronically. However, even frequent touchpoints like a daily scrum don’t retain any historical record and don’t facilitate planning beyond the immediate future. If you’re confident no one will ever need to revisit the decisions or information exchanged in a meeting, then there’s no reason to write it down. Otherwise, the time needed to document those useful bits of knowledge is a modest investment.
If every project stakeholder were privy to every discussion, interpreted information identically, and had a perfect memory, you’d never need to write anything down. Alas, this is not reality. Documentation serves as a persistent group memory, a resource to which team members can refer across space and time (Rettig, 1990). During a retrospective, project records can refresh participants’ memories about what was intended and what actually happened. If people other than the original developers need to modify a product, project documentation can save them time.
Instead of waiting until users can see working software to provide useful feedback for developers, written requirements reviews invite domain experts to spot problems before they’re cast into code. Many projects invoke complex logic or business rules that are best represented in decision tables or mathematical formulas. Record that information in persistent forms so you can validate its accuracy and completeness. That documentation feeds nicely into test design, as well.
If someone has gone to the trouble of preparing documentation, it behooves those who work on the system to exploit that knowledge. One project team developed a good set of requirements, but the contract team brought in to implement the system ignored them. The contractors elected to talk to the users again about their needs, thereby annoying them and wasting time.
One of my consulting clients hired a team of contract BAs who created multiple three-ring binders of requirements for a very large project. The company then brought in a second contract team to build the product. That implementation team saw the binders and said, “We don’t have time to read all those requirements. We’ve got software to build!” So they built the system based on what they thought it ought to be. Then they built it again, based on the actual requirements that the earlier BA team had created. No matter how pressed you are for time, it will be quicker to learn from a good set of requirements than to build the product twice.
Sometimes it’s in the developer’s best interest to have certain information documented. For instance, if a team is writing software for an organization that hasn’t documented its business rules, the software code that applies or enforces those rules becomes the definitive source of that business-level knowledge. The developer who implemented the code around those business rules then becomes the subject matter expert to whom the business must turn when policies change. People shouldn’t have to extract business-level knowledge from a particular application’s code.
Documents have their benefits and their limitations. Even the best written requirements can’t replace human dialog, but they certainly help. Writing something down doesn’t guarantee that it’s accurate, complete, or immutable. It does increase the chance that people who access the information will reach the same understanding of it and can refresh their knowledge in the future. Documentation must be current, accurate, and accessible to those who need it. If readers can’t readily find what they’re seeking, it doesn’t matter how good the documentation is.
Some people misinterpret the agile development philosophy and resist creating documentation. The Manifesto for Agile Software Development states, “We have come to value…working software over comprehensive documentation” (Beck et al., 2001). It does not state, “We don’t need no stinkin’ documentation.” Agile expert Mike Cohn (2010) pointed out the shortcomings of written documents but advised, “Don’t throw out the baby with the documentation:”
These weaknesses of written communication are not to say we should abandon written requirements documents—absolutely not. Rather, we should use documents where appropriate.… The goal in agile development is to find the right balance between documentation and discussion. In the past we’ve often been skewed way too far toward the side of documents.
Cohn’s advice suggests that BAs, project managers, product owners, and developers should create written documentation judiciously. They should record information at an appropriate—not necessarily minimal—level of detail. When details are known and precision is necessary, write it down. This approach is more realistic than attempting to cast in stone volatile or preliminary information that someone must maintain over time. And it’s safer than relying totally on human memories.
With experience, you learn that the world is drawn in shades of gray. In almost any situation, any extreme position is silly. The two extremes of either writing every piece of project information in great detail or having no written documentation are both silly. If you remember that it’s less expensive to record knowledge than to discover—or rediscover—it, you can decide what information is worth writing down.
Software development is partly about computing and partly about communication. Requirements engineering, though, is entirely about communication. In general, we’re better at the technical side of software development than the human side. Those team members who lead requirements activities—who I’m calling business analysts, regardless of their job title—sit at the hub of a project communication network, as shown in Figure 2.3. They coordinate the exchange of requirements knowledge among all project participants.
The communication links in Figure 2.3 are all two-way arrows. Some participants primarily supply requirements input to the project from the customer domain: project sponsor, marketing, key customers, and user representatives. Other participants lie in the implementer domain and consume the requirements process outputs: architects, software and user experience designers, developers, and testers. If your product contains both software and hardware components, electrical and mechanical engineers also will be involved.
The BA must keep all participants informed about the body of requirements knowledge, priorities, status, and changes. Everyone could participate in requirements reviews; they’ll see different kinds of issues from their diverse perspectives. Some people will make the many requirements-related decisions that every project faces. And everyone involved should contribute comments and ideas regarding the requirements to help the team reach and sustain a holistic understanding.
The BA has a tough job. The other participants will communicate requirements input primarily verbally. The BA also might refer to documents, such as sources of business rules or information about related products. The BA must assess, classify, and record all that information in appropriate written forms. I say “written” because the BA needs to communicate the information back to the sources for verification and to those who will base their work on the requirements. With so many audiences who have diverse backgrounds and vocabularies, the BA must think carefully about the best ways to communicate with each of them. The people who receive requirements knowledge differ in several respects:
• What information they need to see
• When they need to see the information
• How much detail they need
• How they prefer to receive information
• How they want written information to be organized
Anyone who ever tried to create a single requirements deliverable that tells all project participants everything they need to know quickly learned that it’s impossible. There are many kinds of requirements information, as we saw in Table 2.1. The BA must determine how to represent each type of information at the appropriate level of detail and organize it meaningfully for each audience.
The BA should ask the various audiences what information they need and how they want to receive it. For instance, developers and testers need the details about each requirement they work with, but the project sponsor doesn’t care about those specifics. People who want only the broad strokes might favor pictures over words, so they don’t get mired in details. A shared repository, such as a requirements management tool, can serve as the definitive location of requirements information for those who need all the specifics.
It helps to use standard templates for certain sets of information, so readers know where to find what they need. I’ve found templates for a vision and scope document, use case document, and software requirements specification particularly helpful (Wiegers and Beatty, 2013). Document creators should work with the recipients of each deliverable to tailor a standard template that will best suit their needs. (See Lesson #57, “Adopt a shrink-to-fit philosophy with document templates.”)
Document authors choose the vocabulary, level of detail, and organizational scheme that make sense to them. But maybe their choice isn’t the most effective for some recipients. A senior systems engineer described the challenge of using language that speaks to the major requirements audiences:
The implementer is the most critical customer of requirements because their interpretation of what they’ve read dictates what the product will be. A requirement written in the business customer’s language might be full of terms and processes unfamiliar to the average implementer. The implementer will have a hard time understanding exactly what is being requested. Requirements are often written in the business customer’s language to get contractual buy-off. Translating these perspectives into requirements that the implementer understands is an important step.
Consider creating a glossary so that everyone involved with the project shares the same understanding of relevant business terms, technical terms, abbreviations, and acronyms. A glossary can be reused across multiple projects in the same application domain, thereby enhancing consistency.
The most obvious way to represent requirements information is in natural language text. However, that gets bulky for a sizable system, whether you’re writing requirements in a document, on index cards or sticky notes, or in a requirements management tool. Readers can get the details of the requirements, but it’s hard to visualize an overview from all those details and see how the pieces fit together. Natural language is prone to ambiguity, fuzzy words that leave too much to the reader’s imagination, and mistranslation. Still, natural language is how human beings communicate, so writing requirements that way is logical.
You can write requirements using various techniques. Some teams employ use cases and lists of functional requirements. Others rely on user stories, feature descriptions, and/or acceptance tests. The method you choose doesn’t matter, provided the technique achieves the goal of clear and effective communication with the audience.
Of course, you’re not limited to using natural language. You can supplement—though not replace—written requirements with different representations or views: visual models, prototypes, screen designs, tables, and mathematical expressions. Each view depicts part of what people need to know about requirements; combining multiple views provides a richer understanding (Wiegers, 2006a). There are even formal requirements notations that can be proven to be correct, although they are rarely used other than on life-critical systems.
For those audiences who need only an overview, consider emphasizing diagrams rather than text. I took an excellent class on analysis and design modeling long ago that completely altered how I approached software development. Diagrams can illustrate process flows, data relationships, user interface navigation, system states and transitions between them, decision logic, and more. I enthusiastically incorporated visual modeling into my development practices.
To my disappointment, I soon learned that no single diagram would show everything I needed to know about a software system (Davis, 1995). Instead, each model shows portions of system knowledge from a particular perspective. Business analysts have to choose appropriate models based on the information their audiences need to see.
Standard vocabularies and notations are central to clear communication. We can’t work together if we don’t share the same understanding of words and symbols. A consulting client once asked me to review a model one of their BAs had drawn. I understood the diagram, but the BA had used some unconventional arrow notations. I had no idea what those arrows signified—there was no legend—or why they differed from the standard arrows used in other places.
Software methodologists have developed numerous standard notations for drawing analysis and design models, including these:
• Structured analysis (DeMarco, 1979)
• IDEF0 (Feldmann, 1998)
• Unified Modeling Language or UML (Booch, Rumbaugh, and Jacobson, 1999)
• Requirements Modeling Language or RML (Beatty and Chen, 2012)
I strongly advise using these standard models. Don’t invent your personal notation unless you’ve concluded that no models exist that show what you want, which is unlikely. You’ll need to educate the people who see the diagrams about how to read the notations; legends on the diagrams that explain the symbols used can help. Keep the models as simple as you can—focus on communicating clearly with your audience.
High-quality software development is based on high-quality requirements, elicited from the right people, crafted into usable forms, and communicated to everyone who needs to know. An effective BA is skilled at many forms of communication: listening, questioning, restating, writing, modeling, presenting, facilitating, and reading nonverbal cues. If you’re functioning in the BA role, you’ll need all these skills—plus the wisdom to know how and when to apply them—to unite all the project participants toward their common objective.
Just as beauty is in the eye of the beholder, so too is quality. Software requirements deliverables have an audience of people who will use them to do their parts of the project work. Those recipients—not the people who produce the deliverables—are the right ones to assess their quality.
I could create a set of requirements that seems perfect to me. It contains everything it should and nothing it shouldn’t, the contents are organized logically, and all of the statements seem clear and understandable—to me. But if someone finds problems with my requirements, it doesn’t matter how good I think they are. The creators (BAs) and recipients (architects, designers, developers, testers, and others) of these bodies of knowledge should agree on their contents, format, organization, style, and level of detail.
The BA’s challenge is that there are so many audiences for requirements information, as the previous lesson described. Those audience members have different notions of quality and what they want to see because they use the information for different purposes. They come from different backgrounds, have differing perspectives, make various assumptions, and might prefer different communication mediums. This diversity makes it hard for a BA to meet everyone’s needs.
A good way to tell if your requirements are of high quality is to invite people who represent multiple reader perspectives to review them. (See Lesson #48, “Strive to have a peer, rather than a customer, find a defect.”) Those reviewers will look for different kinds of problems. Table 2.3 lists some quality issues that various requirements audiences can look for in a review; they want to be able to answer “yes” to each question. The type of formal peer review called inspection is particularly good at finding certain classes of requirement errors (Wiegers, 2002a). During an inspection, one participant describes each requirement in their own words. The other participants can compare that interpretation to their understanding of the requirement. If the interpretations don’t all match, they’ve found an ambiguity.
Business analysts should strive to build the following characteristics into their deliverables as they pursue high-quality requirements (Davis, 2005; Wiegers and Beatty, 2013).
• Complete. No requirements are missing. Each requirement includes all the information the reader needs to do their job. Any known omissions are flagged as TBD, To Be Determined. In practice, there’s no way to be sure that you’ve found all the requirements. If you deliberately opt to write partial requirements, expecting readers to acquire further details when needed, make sure the readers know the requirements are incomplete.
• Consistent. A solution that satisfies any requirement would not be incompatible with any other requirement. Catching inconsistencies is difficult. It’s hard to spot inconsistencies between different requirement types, such as a functional requirement that violates a business rule or clashes with a user requirement, if the two types of information are stored in different places.
• Correct. Each requirement accurately states a need expressed by a user or other stakeholder. Only the pertinent stakeholders can assess this property.
• Feasible. Developers can implement a solution to address this requirement within known technical, schedule, and resource constraints.
• Necessary. Each requirement describes a capability or characteristic some stakeholder really needs.
• Prioritized. Requirements are classified as to their relative importance and the timeliness of their inclusion in the product.
• Traceable. Each requirement is uniquely identified so it can be linked back to its origin and forward to designs, code, tests, and any other items created because of that requirement. Knowing each requirement’s source adds context and shows who to consult for clarification.
• Unambiguous. All readers will interpret each requirement in the same way and in only one way. If a requirement is ambiguous, you can’t determine whether it is complete, correct, feasible, necessary, or verifiable because you don’t know precisely what it means. You can’t remove all the ambiguities of natural language, but avoiding words like these can help: best, etc., fast, flexible, for instance, i.e. and e.g., improved, including, maximize, optionally, several, sufficient, support, and usually (Wiegers and Beatty, 2013).
• Verifiable. There is some objective, unambiguous, and cost-effective way to determine whether the solution satisfies the requirement. Testing is the most common verification technique, so some people refer to this characteristic more narrowly as testable.
You’re never going to create a perfect set of requirements. You don’t need to, provided the project’s development process includes mechanisms to quickly detect and rectify requirements errors before the team implements them. Soliciting feedback from the multiple requirements audiences helps you avoid excessive rework costs because of requirement defects.
As I mentioned, you’re never going to get a perfect set of requirements. Some requirements could be incomplete, incorrect, unnecessary, infeasible, ambiguous, or missing entirely. Requirements sometimes conflict with each other. But you still need to build software based on the available requirements information.
Practically speaking, your goal is to develop requirements that are good enough to allow the next development stage to proceed. It’s a question of risk. You should invest enough effort into requirements development to reduce the risk of performing excessive unplanned rework because of requirements problems. Unfortunately, no green light comes on when your requirements are good enough. It’s hard for a BA to judge if they’ve elicited all the pertinent requirements and stated them accurately. Yet, someone must decide when the next portion of the product’s requirements provides a suitable foundation for construction. System architects, designers, and developers can help make this judgment.
Being good enough encompasses both the quantity of information presented and its quality. A minimal set of perfectly written requirements could lack details that developers and testers need, yet a comprehensive set of poorly written, inaccurate requirements is worthless. Requirements expert Alan Davis (2005) nicely stated the goal of requirements specification: “To specify the desired behavior of the system, in sufficient detail that system developers, marketing, customers, users, and management are closely aligned in their interpretation.” The key phrase here is “sufficient detail.” We can think of three dimensions of requirements completeness: the types of information included, the breadth of knowledge, and the depth of detail for each item.
• Types of information. Project participants naturally focus on the functionality that users need to accomplish their goals, but a useful set of requirements goes well beyond that. Developers also need to know about quality attribute requirements, design and implementation constraints, business rules, external interface requirements, and data types and sources. A simple collection of functional requirements or user stories isn’t sufficient.
• Breadth of knowledge. This dimension encompasses the scope of requirements information contained in a specification. Does it include every user requirement known to be in scope or just the high-priority ones? Are all relevant quality attributes addressed or just those that are paramount? Would a reader know whether this is the complete spectrum of expectations or if they’ll need to close gaps? If it’s not complete, will all readers see the same gaps? Implied and assumed requirements that no one writes down pose a high risk of being overlooked. (See Lesson #13, “Two commonly used requirements elicitation practices are telepathy and clairvoyance. They don’t work.”)
• Depth of detail. The third dimension addresses how much detail and precision is provided for each requirement. Do the requirements identify possible exceptions (errors) and specify how the system should handle them? Or do they cover only the happy paths of normal behavior? If the specification addresses a nonfunctional requirement such as installability, does it also cover uninstallation, reinstallation, repairing an installation, and installing updates and patches? Both functional and nonfunctional requirements must be precise enough to be verifiable in the implemented solution.
There’s no single right answer to the question of how much information is enough in a given situation. However, anywhere that knowledge gaps exist, someone will have to close them. Part of deciding when the specified requirements are good enough is determining how much detail is needed, who will acquire it, and when. The Business Analysis Body of Knowledge from the IIBA contains a definition of solution requirements that states in part: “They provide the appropriate level of detail to allow for the development and implementation of the solution” (IIBA, 2015). What constitutes “appropriate” is a judgment call whose interpretation will vary from person to person.
Many agile software development teams don’t specify written requirements in detail, but that doesn’t mean the developers and testers don’t need those details. As we saw in Lesson #6, they do. If written information isn’t available at implementation time, someone must track it down from the right source. Otherwise, the software team members must fill in the blanks themselves, possibly missing the mark from the customer’s perspective. If that happens, the requirements weren’t quite ready for prime time.
People often speak of gathering requirements on a software project, but this conveys an inaccurate impression. The word gathering suggests that the requirements are lying around out there somewhere, just waiting to be collected. When I hear someone say “gathering requirements,” I conjure a mental image of picking flowers or hunting for Easter eggs. Alas, it’s not that simple.
Requirements rarely exist fully formed in users’ minds, ready to be passed on to a BA or development team on demand. Assembling a set of requirements does involve some collection, but it also involves discovery and invention. The term requirements elicitation more accurately conveys how software people collaborate with project stakeholders to explore how they work now and determine what capabilities a future software system should provide. Requirements experts Suzanne and James Robertson (2013) vividly refer to this process as “trawling for requirements:”
We use the term trawling to describe the activity of investigating the business. This term evokes the nature of what we are doing here: fishing. Not idly dangling a line while hoping a fish might come by, but rather methodically running a net through the business to catch every possible requirement.
According to The American Heritage Dictionary of the English Language (2020), elicitation means calling forth, drawing out, or provoking. The sense of calling forth and drawing out requirements describes the process better than merely gathering. (Business analysts aren’t trying to provoke the stakeholders with whom they work, though I fear that happens inadvertently sometimes.) A big part of the BA’s function during elicitation is to ask the right questions to stimulate the stakeholders’ thinking and get beyond superficial and obvious.
The least useful questions to ask when exploring requirements are “What do you want?” and “What are your requirements?” Such vague questions trigger a lot of random—yet important—input, blended with extraneous information and seasoned with unstated assumptions. A BA is not a mere scribe, writing down whatever stakeholders tell them. A skillful BA facilitates elicitation conversations, guiding the participants to discover the pertinent knowledge in a structured fashion. The BA needs to analyze and organize the information collected—without the irrelevant bits—and then present it in useful forms to developers and other project participants.
As we saw in Lesson #5, elicitation calls for an iterative and incremental approach with cycles of refinement, clarification, and adjustment. The discussions could move from fuzzy high-level concepts into details, or they could begin with specific functionality fragments that the BA must then synthesize to a higher level of abstraction. Information from one source might conflict with that from another. Sometimes the BA obtains new input that forces them to revisit what the group thought was a resolved issue. Cycling back like this can be frustrating for participants (“Haven’t we had this conversation already?”), but it’s the nature of nonlinear human communication and exploration. Requirements elicitation is like peeling away the layers of an onion, revealing the interior as you go along—except that the more you peel, the bigger the onion appears.
In the hypothetical, pure waterfall life cycle, elicitation is performed only at the project’s beginning. Ideally, a BA could assemble all the requirements up front, and they would remain stable throughout system development. Projects for which that approach works well do exist, but the time they must invest in the requirements phase is significant. Even traditional project teams know that requirements written early on must be revised and elaborated throughout the project’s course.
Agile development projects deliberately address requirements in smaller chunks and expect the requirements set to grow and evolve during development. Each development iteration includes elicitation activities. The project begins with some requirements exploration, but there’s no expectation of gaining a complete and detailed understanding at that point. Instead, the team accumulates just enough knowledge to prioritize and allocate requirements to the early development iterations. During each iteration, the team then refines its allocated requirements—typically represented as user stories and acceptance tests—to whatever level of detail the developers and testers need.
A project’s charter or vision and scope document sets the stage for elicitation. It establishes the project’s business objectives, scope—what’s explicitly in—and limitations—what’s explicitly out. To begin the elicitation process, identify stakeholders who might be sources of valuable information. Those stakeholders could have veto power over a requirement (“You can’t do that”) or the power to add a requirement (“You must also do this”). You’ll work with those stakeholders to understand their business, their needs and concerns, and what they expect a new or modified system to do for them.
If you’re the BA, plan your elicitation strategy before diving in. The interaction techniques you choose will depend on the access you have to the stakeholders, where they’re located, whether group or individual discussions are most appropriate, and how much time they can spend. Plan each interaction to make sure you get the information the development team needs. You might need to adjust the interaction techniques depending on the level of engagement you’re getting from your stakeholders.
You can find requirements knowledge about the business or the project in many places and ways. Following are several elicitation techniques that most projects would find useful (Davis, 2005; Robertson and Robertson, 2013; Wiegers and Beatty, 2013).
One-on-one stakeholder interviews are efficient and focused, letting you drill down into details without distracting side trips. However, they lack the synergistic interactions that often stimulate new ideas in group discussions. For both individual and group interviews, the BA should prepare a list of areas to explore and questions to ask (Podeswa, 2009).
Workshops in engages a BA meets with several user representatives and other stakeholders are a common elicitation practice. Workshops usually explore user requirements to understand the tasks a system must let users perform. All group activities risk being diverted by discussions that are out of scope for the meeting’s objectives. It’s easy for the group to head down a rat hole, mired in details when they should be thinking at a higher level. A skillful facilitator keeps the participants on-topic so that the workshop produces useful information. (See Lesson #14, “A large group of people can’t agree to leave a burning room, let alone agree on exactly how to word some requirement.”)
Observing users as they work in their native environment yields information that they might not think to share if a BA simply asked questions about what they do. An observant BA can identify issues and bottlenecks that the new system could address to make the business processes more efficient. Users often compensate for shortcomings in a software system through workarounds, so observation can reveal improvements to make in a replacement system. User experience designers—who also perform the BA role on some projects—also find it valuable to observe users as they work.
Documentation on existing systems, products, and business processes can be a rich source of potential requirements. Studying such documentation helps a BA get up to speed on a new application domain. Documents provide information about relevant business rules: corporate policies, government regulations, and industry standards. Suzanne and James Robertson (2013) aptly term the process of reverse engineering new requirements from existing records “document archaeology.” Information gleaned from historical sources needs to be validated to ensure it isn’t out of date.
In-person interviews and workshops can engage only a limited number of participants. Surveys let you elicit needs and attitudes towards current products from a larger population. Online surveys are useful for commercial products where the development team might not have direct access to representative users. There’s an art to creating surveys that elicit the information you seek and increase the chance that users will complete them (Colorado State University, n.d.). Surveys should ask the fewest questions that will elicit the desired information.
Online surveys often force the user to answer every question to enable submitting them. I sometimes abandon surveys when I realize how long they are. I’m willing to share my opinions, but I don’t want to spend a lot of time wading through pages of questions.
Wikis and other collaborative tools let you collect input and ideas from more people than you could fit into a workshop. One person’s post prompts another’s endorsement, revision, extension, or dissent. The drawback to this freewheeling approach is that the BA will need to filter through the discussion threads to search for gems of knowledge.
It’s hard for people to visualize what a proposed solution might be like from abstract discussions and requirement lists. A prototype makes requirements more tangible. Even simple screen sketches can help workshop participants crystallize their thinking. It’s risky to build prototypes too early during requirements exploration because people can fixate prematurely on a particular—and maybe not ideal—solution.
Elicitation is the core requirements engineering practice on a software or systems development project. Without a solid foundation of requirements knowledge acquired through effective elicitation, a project is on shaky ground.
During one of my most productive periods as a software developer, I created some applications for a scientist named Sean in Kodak’s research laboratories. Sean was the only user; I was the entire software team. I performed all the activities needed to create an application: requirements development, user interface and program design, coding, testing, and documentation. One application was a complex spreadsheet tool that let Sean simulate the photographic results from numerous camera and film parameters. Another was a mainframe-based application to analyze his experimental data.
Sean and I sat ten feet apart. I was highly productive because Sean and I could interact frequently, informally, and quickly. I could show him what I was doing, get my questions answered, and obtain his feedback on user interface ideas. This proximity, and the fact that only Sean had input to the project, let us collaborate in tiny, fast cycles. We didn’t need written requirements because I could get the details I needed to adjust quickly.
My work with Sean was the ideal software development environment: one developer plus one customer, sitting close together. That’s also rare. Most projects have many customers grouped into several user classes, numerous sources of requirements, and multiple decision makers. They have a development team that ranges from a few people in one place to hundreds in multiple locations. These vastly more challenging projects need other ways to get the voice of the customer close to the ear of the developer to elicit requirements, establish priorities, communicate changes, and make decisions.
Unless you’re building software for yourself, you’ll always confront a gap between customers who have needs and developers who build solutions. Each project team needs to establish effective communication pathways between the two communities early in the project. Your options depend on how many participants are involved, who and where they are, how well the communities understand each other, and the software team’s skill set.
Figure 2.4 shows several communication models to connect the customer’s voice to the developer’s ear. My situation with Sean illustrates model A, a direct connection between users and developers. This direct connection presents the least need for detailed written requirements and the fewest miscommunication opportunities, provided the developer and user understand each other’s terminology. More often, though, some intermediaries will be involved.
When you have many users with diverse needs, they can’t all talk directly to developers. That’s a recipe for chaos, as developers are bombarded with input without knowing which sources are authoritative. It burdens the developers to resolve conflicting input from multiple sources and establish priorities. To cope with that diversity, many of my consulting clients and I have successfully used the model labeled B in Figure 2.4.
Stakeholder analysis generally reveals multiple user classes that have largely distinct needs. Members of different user classes could use different product features, perform different tasks, vary in their frequency of use, or have other distinctions. Once you’ve identified your user classes, you need to determine who will serve as the literal voice of the customer for each group.
In model B, the primary conduit for requirements information involves one or more key user representatives, called product champions, who collaborate with one or more BAs (Wiegers and Beatty, 2013). The product champions have the domain knowledge and understand the project’s business objectives. They interact with their fellow user-class members to solicit requirements input and feedback on ideas and to inform the other users of project progress. The BA facilitates bridging the communication gap between product champions and the development team.
Note that communications also take place along the reverse direction of the arrows shown in Figure 2.4. As developers—or anyone else involved in the pathway—have questions or require clarification, they’ll need to go back to the source of the requirement to resolve them. It’s helpful to record where each requirement came from so that developers can quickly get the answers they need.
Companies that build commercial products often use communication pathway C in Figure 2.4. The marketing department assesses the market needs and sales potential for a new or improved product. Marketing may work with a product manager who has the lead responsibility for defining the product characteristics that will lead to business success. The organization can partition the responsibilities for defining the product in various ways across the marketing and product management functions.
The product manager performs the functions that a BA handles on an IT project. Here’s a concise statement of the product manager role (280 Group, 2021):
The Product Manager is responsible for delivering a differentiated product to market that addresses a market need and represents a viable business opportunity. A key component of the Product Manager role is ensuring that the product supports the company’s overall strategy and goals.
Projects that follow an agile development approach, particularly Scrum, often follow communication pathway D in Figure 2.4. The product owner establishes the product’s vision and goal, and they create and communicate the contents of the product backlog (Cohn, 2010). The product owner defines a roadmap by which the product will evolve incrementally from concept, through early releases, to a mature product that delivers customer value (McGreal and Jocham, 2018). The product owner thus serves as the voice of the customer. Unless they’re already domain experts in all the areas the users represent, responsible product owners will seek input from people like the product champions I mentioned earlier. In Scrum, the product owner—a single individual—has sole accountability for managing the product backlog contents, even if they delegate some of the work to other people (Schwaber and Sutherland, 2020). The product owner also interacts with marketing and business managers and takes their input into account in the prioritized story backlog.
As you can see, the product owner fulfills much of the role that a BA might otherwise perform on an IT project, and more. However, some agile project teams recognize the value of having a skilled BA on the team to work with the product owner. When both roles are present, they can collaborate in various ways. The BA often functions as an extension of—or surrogate for—the product owner. The product owner might delegate some areas of responsibility, such as working with particular user classes, to the BA. The BA then is responsible for everything about that area except the prioritization against requirements from other areas. Prioritization remains the product owner’s purview.
Sometimes the product owner is more product- and market-facing, whereas the BA is more technical-facing, crafting solution requirements from user requirements (Datt, 2020a). This model appears as pathway E in Figure 2.4. In other cases, the opposite is true. The nature of their collaboration boils down to however the product owner thinks the BA can add the most value to the project.
Whether the title is business analyst, requirements engineer, product manager, product owner, or something else—or even if developers perform requirements activities themselves—all projects need to connect a product’s users with its creators. The job title is less important than the presence of the role and a clear definition of its responsibilities and authority. The individuals who perform this function need the right knowledge, skills, experience, and personality to work with both customers and developers. They must establish a relationship based on mutual trust and respect with both communities.
Effective requirements development ensures that developers hear the customers’ voices loud and clear. Those connections can determine whether the project is a grand success or never makes it to the finish line.
According to The American Heritage Dictionary of the English Language (2020), telepathy is “the supposed process of communicating through means other than the senses, as by the direct exchange of thoughts.” Clairvoyance is “the supposed power to see objects or events that cannot be perceived by the senses.” Those skills certainly would make software development far easier—if they were real. Although they don’t exist, telepathy and clairvoyance appear to be the technical foundation for some projects.
People sometimes think that certain requirements are so obvious they need not be stated. Some users might be wary of coming across as condescending, not wanting to insult a BA’s intelligence by stating something they think the BA already knows. I’d rather hear something that I already know a second or third time—thereby reinforcing its validity and my understanding of it—than to have someone assume that information is in my brain when it’s not.
Some busy users don’t want to spend the time discussing requirements with a BA, product owner, or developer. Their attitude seems to be, “You should already know what I need. Call me when you’re done.” That attitude assumes both telepathy and clairvoyance on the BA’s part.
The two big risk areas are assumed requirements and implied requirements. Assumed requirements are those that people expect without having to say so. Implied requirements are needed because another requirement exists, but they aren’t explicitly stated either. It’s not reasonable to expect a BA to read minds or see beyond the horizon to acquire that hidden knowledge. As we’ve seen, you’ll never have an entirely complete set of requirements, but the project participants must exercise judgment regarding what can safely remain unsaid.
I prefer to communicate the expectations we know about explicitly instead of hoping that someone else knows what I’m thinking. It’s great if all project stakeholders have a close enough mind meld to yield the right product without having to record requirement details. The more that those people have worked together, and the more the development team knows about the application domain, the easier that mind meld is to achieve. But my general philosophy is that if the requirements don’t describe a particular capability or characteristic, no one should expect to find it in the product.
We sometimes express requirements informally because we assume the reader has a “sensibility filter” similar to our own, yet people might interpret the same statements differently. That ambiguity leads to mismatched expectations and surprises upon delivery. You might be operating under a different set of assumptions than I am. An assumption is a statement we regard as being true in the absence of definitive knowledge that it is true. The BA should try to reveal and confirm unstated assumptions, as sometimes they’re invalid or obsolete.
The risk of miscommunication increases when system implementation is outsourced. I once reviewed a requirements document for a project that was planning to outsource the development work. The document contained many requirements that began, “The system shall support…” I asked the document’s author how the contracting company’s developers would know precisely what functionality the word support implied in each instance. After a moment of thought, she gave the correct answer: “I guess they won’t.” She wisely decided to clarify just what he meant by support throughout the document to eliminate the ambiguities. That’s much better than relying on telepathy or clairvoyance.
Here’s an example of an implied requirement. You request an undo function for some application feature. The developer implements the undo function, and you test it. It works fine. You ask the developer where the redo operation is.
“You didn’t ask for redo,” the developer replies.
“I thought that asking for an undo function would obviously imply a redo capability as well. Could you put that in, please?” you ask. The developer adds the redo function, and it works. But then you wonder why there’s only a one-level redo. That leads to further discussion: How many levels of redo do you want? Do you want to be able to jump to any point in the undo sequence and redo all of the undone actions from that point onward? When should the undo history queue be cleared? And on and on.
If the developers and users are in close contact, they could start with that simple undo requirement and discuss it to agree on exactly how the undo/redo function should behave. That way, it doesn’t take multiple development iterations to rework your way to what the customer had in mind. However, if you’re outsourcing development, you’d better do that thinking beforehand and include all the specifics in the requirements. Otherwise, don’t be surprised if the faraway developer’s interpretation of a scantily written requirement doesn’t match the customer’s expectation. The contractor might even detect this implied functionality in a proposal but base their bid on only the original requirement, anticipating that you’ll come back with the additional request. Then the contractor can ask for more money and time to accommodate your “scope creep.”
You can’t work out the nuances of all functionality just through thinking and discussion. Sometimes only development cycles or prototypes will let users figure out just what they need. Assumed requirements and the resulting design choices can lead to expensive rework, though. I recently read about a design decision that engineers made regarding the control stick in the F-16 Fighting Falcon fighter jet (Aleshire, 2005):
Originally, the engineers made the stick itself solid, unmoving, since the computer could translate the pressure on the stick from the pilot’s hands as easily as the stick’s actual movement. But the pilots hated that. They wanted to move the stick, to gain that sense of control.
There’s no substitute for getting input on both requirements and proposed solutions from people who will use the product. And there’s no excuse for not writing down the resulting knowledge to ensure that the product designers can satisfy the customer’s needs.
I once was the lead BA on a medium-sized information system project. Two other BAs and I worked with members of different user classes to understand their requirements. One day my colleague Lynette called me, concerned. Her first elicitation workshop had covered far less ground than planned. The frustrated participants were concerned about how long this process would take. Lynette sought advice.
I asked her how many people were in the workshop. “Twelve,” she replied. Ah, therein lay the problem. A large group of people has a hard time reaching agreements and making decisions. Group members are easily distracted by pairwise side conversations. Multiple people might have something to say on each topic, leading to longer—but not necessarily more fruitful—discussions. It’s easy to get dragged into one participant’s pet topic in considerable detail, which might not advance the day’s objectives. Disagreements can escalate into lengthy debates. Certain individuals can dominate the discussion, while others might tune out entirely.
I suggested that Lynette cut the size of the group in half. She didn’t need six user representatives, just two or three. Some people who were there as observers or to protect their interests weren’t adding value to the requirements exploration. I recommend including people with software development and testing experience in elicitation discussions. They bring insights regarding the feasibility and testability of proposed requirements. In this case, Lynette’s personal experience let her represent those perspectives herself. Lynette shrunk the group for subsequent workshops, and everyone felt much better about their quicker progress.
Four people can have a productive discussion without getting distracted by sidebar conversations. As Figure 2.5 illustrates, there are few pairwise interactions in a group of just four people. However, the number of such connections snowballs with larger groups. Figure 2.6 shows how many more two-way connections there are in a group of ten. It’s not surprising that some people will slip into private discussions, especially if they aren’t interested in the current topic. The idea of keeping groups relatively small to make rapid progress applies to requirements elicitation workshops, peer reviews, project retrospectives, and similar collective activities.
One strategy we found to help control group size was to hold separate workshops with members of different user classes. Different user classes have largely distinct requirements. If you collect all project stakeholder representatives into a single workshop, any topic that comes up will be of interest to only a subset of the group. The others may well be bored and feel that they’re wasting their time. Conducting workshops with separate user classes helps ensure that all participants are interested in the agenda items. That said, judiciously combining members of different user classes can synergistically reveal connections, questions, and innovations that neither group might come up with independently.
Keeping a large group on track requires skillful facilitation. Sometimes facilitation is ad hoc: someone in the group takes the initiative to stand at the whiteboard with a marker in hand and bring order out of chaos. The BA could serve as the facilitator, or the group could bring in an impartial outside facilitator. A facilitator should come prepared with objectives and an agenda for each session. Timeboxing discussions sustains progress, so the group doesn’t get too tangled up with one topic and neglect others due to lack of time.
A facilitator can judge when to let a discussion go on longer than planned because of the value it’s adding. They can assess when it’s appropriate to drill down into more detail and when it’s time to move on to the next topic. A good facilitator will notice if a discussion about requirements has slipped into solution explorations and bring the group back on track. Ellen Gottesdiener’s book Requirements by Collaboration (2002) provides extensive guidance on planning and leading requirements workshops.
One of the biggest challenges with leading a large workshop is keeping discussions in scope. Figure 2.7 shows that the facilitator must first consider the horizontal scope, the subset of possible user requirements (use cases, user stories, or features) that the group should plan to discuss in a specific workshop. There’s also a vertical scope, the depth into which the group should explore each of the selected user requirements.
A workshop is not the time to try to define all the details of every requirement. Early in requirements exploration, you want to learn just enough about each one to let the team estimate their relative size and prioritize them for implementation. Spend only enough time discussing the wording to ensure that all the participants share a common understanding of what each requirement means. The BA can flesh out the details offline with users at the right time, such as just before implementing a requirement that was allocated to a specific development iteration.
The facilitator must respect the input that workshop participants offer. While the facilitator needs to keep the discussion focused on the day’s topic, they also must honor the basic human need to be heard. If someone raises an issue outside the discussion’s scope, the facilitator should record the idea so that it’s there for future reference and then quickly bring the discussion back in scope.
You do miss something by having only a small group of workshop participants. Keeping the group small is not intended to exclude input from others, just to accelerate progress. To close the gaps, supplement your elicitation workshops with asynchronous methods, such as wikis, to solicit input from more people. I found it helpful to write up the information from each workshop and distribute it promptly to the participants for review and to a larger group for their comments, corrections, and elaborations. Engaging other stakeholders in the process broadens your knowledge, validates the workshop results, informs the community about the project’s direction, and promotes buy-in.
As you collect information from various stakeholders through multiple channels, someone has to resolve conflicting requirements. Even terminology can conflict. Two people might use different words to mean the same thing or overload a single term with multiple definitions that lead to confusion. Someone will also need to reconcile priorities across the requirements obtained from various user classes and decide which proposed changes to accept. Every project needs to determine who its decision makers are for these matters and how they’ll make the call—their decision rules (Gottesdiener, 2002). Some decision-making processes take longer than others. Negotiating to reach consensus requires more discussion than a simple majority vote, delegating the decision to a single individual, or flipping a coin.
Elicitation workshops are an effective way to explore requirements collaboratively. As a BA, it’s rewarding to lead a team of engaged user representatives to a shared understanding of their requirements and a solution. If the group is too big, though, you can wind up in the proverbial herding-cats scenario, doing your best to keep people engaged and on-topic, and walking out tired and frustrated at the end. Small groups move much faster. And they’ll always be able to agree on which fire exit to use should it become necessary.
You’ve probably heard the saying, “The squeaky wheel gets the grease.” The software analogy suggests that the person who advocates most loudly for their requirements gets top priority. I call that decibel prioritization. It’s not the best strategy.
Most teams confront a mass of functionality that’s too large for the project box. Designated decision makers must choose what to include and what to defer or discard. Even if you can handle all of the requested work, you need to determine the most appropriate implementation sequence. Some requested capabilities are both important and urgent; they go first. Others are important but can wait for later implementation, and still others are neither important nor urgent. An essential aspect of project planning, prioritization guides allocating items from the backlog of remaining work to the most appropriate iteration, build, or release.
Every stakeholder likes to think that their needs are the most profound. Influential, strong-willed, or highly vocal managers and customers can exert a lot of pressure to address their needs first. However, the loudest customers aren’t necessarily demanding the most significant features from a business perspective. The objective of all prioritization exercises is to deliver the maximum customer value quickly and cheaply. You don’t want to waste effort on features that don’t contribute much to the product’s success. Thoughtful prioritization, therefore, should consider numerous factors besides speaking volume.
People have developed many methods for prioritizing software requirements. Several researchers have studied which requirements prioritization techniques are most practicable and effective (Hasan et al., 2010; Kukreja et al., 2012; Achimugu et al., 2014). Commonly used techniques include these:
• Three-level classifications, which all boil down to high–medium–low.
• MoSCoW (Must, Should, Could, Won’t) classification.
• Pairwise comparisons of individual features, functional requirements, use cases, or stories to sort the items by priority.
• Rank ordering the priorities of a set of similar requirement items.
• Distributing 100 points among individual requirements, giving the highest priority items the most points.
• Planning game, commonly used on agile projects, in which customers and developers collaborate to list a set of user stories in order of their relative priority (Shore, 2010).
• Analytical methods that rate requirements based on the value they contribute to the product and their implementation cost (Wiegers and Beatty, 2013).
The same thought process applies whether you’re prioritizing use cases, individual use case flows, features and subfeatures, functional requirements, or user stories. Consider the following factors as you decide which of your requirements are imperative, which desirable, and which optional.
The most decisive criterion is the extent to which each requirement will help the organization achieve its business objectives. The reference for making this judgment lies in the project’s business requirements, which the project sponsor should establish early in the project. They describe the business opportunity being created or exploited and quantify the business’s objectives for the project. Without clear objectives, it’s hard to make rational decisions about which functionality to implement and when.
Not all user classes are created equal. Satisfying the requirements from favored user classes contributes more to business success than meeting other groups’ needs. Stakeholder analysis includes determining which user classes are most important—favored—so you can grant their requirements a higher priority.
Understanding how often users will use certain functions helps to judge which capabilities to implement first. One way to assess the frequency of usage is to develop an operational profile for the application. An operational profile describes what percentage of the time users will perform each operation during a usage session (Musa, 1993). For instance, what percentage of user sessions at an airline’s website involves making a flight reservation, modifying or canceling a reservation, checking flight status, or tracking missing luggage? All other things being equal—and they aren’t always— the most frequently used operations will have the highest implementation priority.
Requirements that allow the product to achieve regulatory compliance or certification must have high priority. It doesn’t matter if the right user functionality is present if you’re not permitted to sell or use the product. A related consideration affects functionality that certain stakeholders need but which isn’t visible to most users. Examples include security requirements, recording access history, and building audit trails. Some of these capabilities could have high priority even if no end user is clamoring for them.
Certain functionality should be implemented early, even if it doesn’t provide immediate user value, because it’s foundational for later functionality. That is, you need to consider the dependencies between requirements when you determine the implementation sequence. Some functionality might establish a complex product’s architectural soundness. It can be disruptive to shoehorn in this kind of functionality later in the project, so build it earlier rather than later.
Requirements that present high risk from an implementation point of view should be implemented as soon as possible to verify their feasibility and lower the overall technical risk of the project.
Whichever prioritization technique you select, you’ll be better off if you consider the factors above in your analysis than if you just listen to the loudest voice in the room. The squeaky wheel shouldn’t necessarily be first in line for the grease.
When I ask students in my training courses how many of them have worked on a project that suffered from scope creep, nearly everyone raises their hand. Then I ask how many of those projects had a defined scope statement: practically none. What does the idea of scope creep even mean if a project’s scope was never clearly defined?
Scope creep—the ongoing and uncontrolled growth of functionality—has plagued software projects since time immemorial. Scope creep often is cited as a reason why a project failed to meet its planned schedule. But you can’t even know if scope creep is taking place unless there’s some agreed-upon reference point that states, “Here’s what we plan to do in this period of time.”
We can define scope as a set of capabilities that stakeholders agree to be delivered in a specific iteration, build, or product release. Scope prescribes the boundary between what’s in and what’s out for a body of work. The scope for any portion of the project represents a subset of the ultimate product vision, a stepping stone on the pathway from project initiation to ultimate product delivery. Every planned chunk of work begins with a baseline of functionality that the team intends to implement during that chunk. That baseline is the reference point for scope change.
People traditionally regard scope creep as a bad thing. It suggests that requirements elicitation was incomplete or inaccurate, thereby leading to an ongoing stream of added and amplified requirements. As we’ve seen, it’s impossible to fully define all the requirements for any sizable project up front, and it’s unrealistic to expect them to remain static. Every project must anticipate some requirements churn and growth as users try out early releases and get both new ideas and a better understanding of the problem.
Stifling change because of scope creep fear (creepophobia?) can lead to products that achieve their initial vision yet don’t meet customer needs. However, continuous scope growth can drive a project off the rails if plans and schedules don’t accommodate changes through contingency buffers and ongoing prioritization. (See Lesson #25, “Icebergs are always larger than they first appear.”) Incorporating every new requirement that comes along guarantees schedule and budget overruns.
Agile projects deliberately don’t attempt to define the scope for an entire project. Instead, they define each iteration’s scope based on which items from the product backlog the product owner allocated to it for implementation. New requirements or other work items are added to the backlog. The product owner prioritizes new backlog items against the rest of the remaining work so they can allocate the items to a future iteration at the right time. This approach helps to target user needs accurately, but it also can cause an uncertain back end on the ultimate delivery schedule.
The simplest scope-representation technique lists the requirements, features, or backlog items scheduled for implementation during a particular development cycle, as in an agile release plan (Thomas, 2008a). Other useful techniques for representing scope at various levels of detail include these:
• Context diagram, which reveals nothing about the system internals but identifies entities outside the system—users, other software systems, hardware devices—that connect to it across the system boundary (Wiegers and Beatty, 2013).
• Use case diagram, which depicts actors outside the system boundary and the use cases through which they interact with the system (Ambler, 2005).
• Ecosystem diagram, which shows how multiple systems are interconnected so you can judge the ripple effect of a change on systems that don’t directly interface to yours (Beatty and Chen, 2012).
• Iteration backlog, which identifies the set of product backlog items that an agile team plans to complete during a single iteration (Scaled Agile, 2021a). Called a sprint backlog on a Scrum project.
• User story map, which shows the activities, steps, and details of user stories that define the scope of an entire product, an iteration, or a specific feature or portion of the user experience (Kaley, 2021).
• Feature roadmap, in which you define several levels of increasing functionality for each feature and then describe the scope of a particular release by listing the specific enrichment levels of individual features that the release includes (Wiegers, 2006a).
• Feature tree, which visually decomposes major features into subfeatures that a planner can group to define each development cycle’s scope (Beatty and Chen, 2012).
• Event list, which identifies the external events that each release will handle (Wiegers and Beatty, 2013).
In each case, the purpose of defining scope is to define the capabilities that a particular portion of the project is to deliver. That distinction establishes the scope boundary and serves as the reference point for considering scope modifications while development is underway.
Since the scope will be somewhat volatile, every project needs a practical change management process. Simply throwing proposed items into the backlog unfiltered isn’t helpful. There should be some well-communicated mechanism by which stakeholders can request changes so the right people can assess their impact and decide whether to include them. A formal change process doesn’t kick in until the team establishes a baseline for a particular body of work. Before that point, we know that requirements are dynamic and the scope is evolving. During each increment of work, change control should become more viscous as time progresses to increase the chance of delivering on the baseline as scheduled.
The question to ask when someone proposes a new requirement is, “Is this in scope?” There are three possible answers:
• Yes, it’s clearly in scope. The proposed functionality could be something no one had thought of before but is needed to achieve our objectives for the current development cycle. Therefore, we must add it to our work in progress.
• No, it’s clearly out of scope. This requirement doesn’t contribute to our business objectives for the current body of work, so we need not address it now. We could place it into the backlog of pending work for future consideration, or we might reject it.
• It’s not in scope as the scope is currently defined, but it ought to be. The project sponsor must make a business decision whether to increase the project’s scope to accommodate the new capability. If the requested change would add sufficient business value, then increasing scope is the right choice. But it always comes at a price. If scope increases, something else must move to accommodate it: other functionality, schedule, cost, staff, or quality. Change is never free.
How people write requirements can lead to an uncertain understanding of scope. Ambiguity might lead one party to think that a particular function is obviously in scope, whereas someone else disagrees. Vague terms like support, etc., for example, and including (even worse, including but not limited to) are intrinsically uncertain. I don’t like to see them in requirements statements. Here are some examples.
• “The system shall process characters A, B, C, etc.” Will all readers agree on where that list ends and what all it contains? The open-ended etc. makes me nervous.
• “The system shall support Microsoft Word documents.” Readers could have very different ideas of exactly what functions support refers to. Suppose a project manager or product owner makes plans based on a restricted interpretation of support. Later on, you find that the customer who requested this function has a much broader expectation. Is that scope creep? Or is it just a refinement of the original expectation into specifics, thereby revealing the rest of the iceberg? Writing requirements carefully before making commitments helps to avoid such problems.
Ambiguous and incomplete requirements can cause problems when scope debates arise on contracted projects. One project included a requirement that a vendor migrate several sets of data from the client’s existing information system into the vendor’s new software package (Wiegers, 2003). After the project was underway, the client identified several more data sets to be converted. The client considered these to lie within the original project scope and balked at paying extra for the new conversions. The vendor disagreed. Between that and other problems, the eventual outcome was a canceled project and an expensive lawsuit. A consultant friend was hired as an expert witness for five contracted projects that resulted in lawsuits and multimillion-dollar settlements. Four of the project failures involved poorly defined requirements; two of those involved scope issues.
Wise contractors will include contingency buffers to accommodate a certain amount of growth and scope fuzziness. However, quoting a price that includes such buffers might be off-putting to a prospective client. Contracts need to explicitly address how to handle scope increases and who will pay for them. The more crisply you can distinguish what’s included from what is not, the less painful these debates will be.
I have a friend who makes sure the project’s purpose can be stated concisely enough to fit on a T-shirt. He keeps that purpose visible on meeting agendas and notes, documents, and other public locations. I’ve also known people who wrote their vision and scope statements on a poster board and brought it to requirements discussions. That visibility helps people answer the question, “Is this new requirement necessary to meet that purpose?” That’s the core issue behind scope management.
Change happens, although excessive change suggests that nobody understood the problem adequately in the first place. Clearly defining the scope of a planned body of work focuses the team on creating a valuable deliverable within the schedule and budget constraints and permits sensible business decisions when change requests come along.
1. Identify which of the lessons described in this chapter are relevant to your experiences with requirements development and management.
2. Can you think of any other requirements-related lessons from your own experience that are worth sharing with your colleagues?
3. Identify any practices described in this chapter that might be solutions to the requirements-related problems you identified in the First Steps at the beginning of the chapter. How could each practice improve how your project teams handle requirements?
4. How could you tell if each practice from Step #3 was yielding the desired results? What would those results be worth to you?
5. Identify any barriers that might make it difficult to apply the practices from Step #3. How could you break down those barriers or enlist allies to help you implement the practices?
6. Put into place process descriptions, templates, guidance documents, checklists, and other aids to help future project teams apply your local requirements best practices effectively.