2

Writing Great Feature Specifications

“How can I know what I think till I see what I say?”

– Attributed to Graham Wallas, E. M. Forster, and W. H. Auden, et al.

In the previous chapter, we saw the importance of exploratory testing to map out a feature. With that experience, you’re now ready to prepare a first draft of the feature specification. This is a vital document that will guide all future testing. It is also known as the Software Requirements Specification (SRS), but here, I will refer to it as the feature specification.

Writing a clear and precise feature specification gives you the best chance of releasing a feature quickly and successfully. Without a detailed specification, you risk surprises throughout the feature's development, even after it has gone live.

In an ideal project, the feature would be comprehensively specified before any work by the development and test teams begins. The product owners should clearly state what they want from the feature; then, the development and test teams can refine it so that everyone is clear about what they will deliver. If that is not the case, push for it in your organization. Again, early planning prevents later surprises.

But even if a feature has a great initial specification, it is worth revisiting it after the initial design. Due to timing or technical issues, the implementation may have deviated from the plan, the plan may have changed, or exploratory testing may have identified behavior that needs to be specified. This chapter will show you what to aim for in a feature specification, whether it’s part of that initial plan or after exploratory testing.

You’ll see how to write requirements to cover the external functionality but not the implementation in clear, independent, testable statements. You’ll learn the ideal format of the document to make it easy to reference and to ensure it covers the core concepts. This chapter will then describe how to write requirements that cover all the different testing areas and how to write a test plan based on those requirements.

In this chapter, we will cover the following topics:

• Advantages, disadvantages, and alternatives to feature specifications
• Improving the handover from the product owner
• Understanding the requirements format
• Requirement priority
• Improving requirement statements
• Improving requirement content
• Completing the specification
• The first draft of a feature specification
• Turning a specification into a test plan
• Countering arguments against specifications

First, we will consider the advantages and disadvantages of functional specifications and alternative forms of documentation that perform similar roles.

Advantages, disadvantages, and alternatives to feature specifications

Precisely describing your features has many advantages, but specifications can be neglected due to the time they take to write. The onus can fall on testers to prepare these specifications since they need to know how the product will work in practice. If no one else has written a description by the time the feature reaches you, it may be up to you to prepare it. Despite its downsides, preparing a feature specification is well worthwhile, as described in the following table:

Advantages	Disadvantages
Early chance to find bugs in the requirements	Takes time to prepare
Avoids surprises for developers and testers	No immediate payback
Saves time by avoiding misunderstandings
Great source of information for development, test, support, and documentation teams

Explicitly writing out the feature specification is a chance to catch bugs early in the development cycle. If your team is organized and the specifications are written upfront, you can head off issues before they are even coded. More often, in my experience, the detailed specification is prepared during or after coding, at which point you’ll need to spend time fixing any problems you find.

Some of my proudest moments as a tester have been when I’ve highlighted issues at the specification stage. I didn’t click a button or run a single test. A well-placed question was enough to send the feature back to the drawing board, as these simplified exchanges show:

• Product Owner: “This new feature will let administrators disable users instead of just deleting them.”
• Me (Tester): “Okay. External integrations are a kind of user, so can we disable them too?”
• Product Owner: “We’ll need to think about that…”

The following example shows an alternative:

• Lead Developer: “This feature will read external meetings into our service’s calendar so that we can display them all together.”
• Me (Tester): “What happens if a user reads in an external meeting, then adds it to our service’s calendar too?”
• Lead Developer: “That would be a problem, yes…”

While it’s a shame that the feature will take more time to develop, it’s vastly quicker than discovering those issues later in the process. That’s always the way with testing: it’d be faster not to find any bugs at all, but given that there are bugs, it’s always better to find them today rather than tomorrow.

A specification also lets you avoid surprises later in the project. It’s slow and costly for the product owner to spot an enhancement they need during beta testing when the feature is already implemented. Imagining exactly how a feature will look and behave is very difficult, so it’s very easy for mistakes and misunderstandings to creep in. There’s no simple way to solve that but thinking through the behavior and explicitly describing it gives you the best chance.

The specification also saves time by avoiding misunderstandings. Testing can go very quickly when the tests are passing but for that, you need to be clear on the product’s behavior. As soon as a test goes wrong, testers need to stop to investigate, double-check their config, and be sure of their results before checking the results with the development team. If you expect one behavior but see another, you will have to investigate that discrepancy, such as a bug. You can save that investigation time if you know what to expect, thanks to precise requirements.

The specification also lets other teams understand what a feature will do. When a customer has a specific question about how your product works or the documentation team need to write up a description, they can read the specification. It’s a massive help to speed up others’ work and empowers them to find answers for themselves without checking the product behavior or asking for help.

All those benefits accrue over time. The problem with writing feature specifications is the effort they take upfront. They take thought from the test and development teams, as well as the product owner when they have many other tasks to keep the project moving. It takes their time, but they are mainly the source of information, and it doesn’t directly help them. Because of that, it can fall to the test team to drive the process of getting specifications created. In my experience, everyone is happy to help and appreciates them when they are done, but it takes a push to make it happen. If specifications aren’t written in your organization, then it’s up to you to make that change. All the testing described in this book relies on detailed knowledge of how the product should behave. This is a necessary step for releasing quality code.

Alternatives to feature specifications

There are many alternatives to having a feature specification that are worth describing. Whatever its form, you need some documentation telling you how the feature should behave. This cannot be the code itself because the whole point is to test that against some independent measure. In the ISTQB, the source of your tests is known as the test basis, a deliberately vague term whose only requirement is that it forms a base from which to design your test plans. This section considers some related documents, as well as possibilities for that basis, aside from a feature specification:

• Business Requirement Document (BRD)
• Technical specifications
• User stories
• Implicit specifications
• User interface mockups
• Diagrams
• Test plans

You can use some of these methods to complement a specification, such as UI mockups, while others could be a replacement. This section considers each of those document types in more detail.

Business Requirement Document

Before you write a feature specification, there needs to be a BRD. This lays out the commercial case for developing this feature or product. The size of this document will vary in size, roughly correlating with the size of the development effort required.

The BRD does not contain details of the product behavior, so it isn’t suitable for writing tests. It will contain the business need for the change by analyzing competitors’ products, the likely commercial returns, the scope of the change, and potential risks the project will face. It will identify stakeholders and list the user stories that this change will satisfy.

I assume the product owner has done all this before the project begins, so I won’t consider it here. The product owner should have taken those business requirements and worked with the development leads to propose a solution they intend to implement. That solution is documented in the feature specification.

Technical specifications

As we shall see, the feature specification should say what the product will do, but not how it should do it. Those implementation details can be recorded in a technical specification, which describes how the product or feature works. This might include descriptions of the following:

• Functions and classes that need to be written
• Modules and how they will interact
• Database design
• Program flow
• Internal APIs

While internal APIs should be part of the technical specification, public APIs are testable and should be recorded in the feature specification instead.

The technical specification can be helpful when designing white-box tests (see Chapter 6, White-Box Functional Testing), and you need to know the workings of the code. However, like the BRD, technical specifications aren’t an alternative to the feature specification but provide different details. These documents work together to describe the change, but the functional requirements are most valuable to you, the tester.

User stories

User stories are a great way to work out the problems your users are facing and how to solve them. For example, “As an e-commerce business owner, I want to see my bank account balance to check that recent transfers have cleared.”

User stories should always state who has the need and a persona with a brief history to understand why they have this need. There should be a specific problem they want to solve – for instance, knowing their bank balance – and each user story should tie into a larger goal. Why do they have this need? That will let you judge how important it is and how well different solutions might work.

User stories describe a problem, not a solution. In the preceding example, there are many ways the balance of a bank account could be shown: in a title bar, in an overview list, or on the details page for that account, for instance. The user story only states the customer’s needs; another layer of detail is needed to say how that need will be met. That layer of detail is the feature specification, which states what the product should do.

For instance, this is not a good user story because it contains too much detail: “As an e-commerce business owner, I want to see my bank account balance on the top left of the Accounts Overview page.”

User stories are a powerful tool for determining the requirements, but for this book, I assume they have already been decided. As a tester, you need a detailed description of the product’s behavior, so you need a specification rather than the user stories that helped generate it.

Implicit specifications

The most successful onboarding of a new tester I ever achieved was by using implicit specifications. He found 10 genuine bugs in his first 2 days in a product still under development. I’ve never seen anyone get up to speed and be so productive so quickly. While he did turn out to be a great engineer, using implicit specifications made that success possible.

The trick was saying that our new product had to do everything our old product did, but on new and more powerful hardware. He could use all the menus and functions of the old product and see what was still missing or behaved differently on the new product. Having that detailed information at his fingertips made finding bugs vastly easier.

The problem with implicit specifications is that you rarely have anything to compare against. Unless you are running a project to reimplement a feature or product, for instance, using a new software framework or hardware platform, you won’t have anything to implicitly specify your requirements. Most features are unique, providing a function that has never been done before or never done in that way, so there is nothing to compare with.

Look out for the opportunity to use implicit specifications. Use them when you can, but unfortunately, you cannot rely on them being available. For most features, you need to explicitly state the requirements yourself.

User interface mockups

A picture says a thousand words, as they say. You should use UI mockups to specify how user-facing sections of your product will look, in addition to written feature specifications. The color and position of UI elements are best shown in a picture instead of an extended description. A single picture is what both the developers and testers need to work from.

Of course, a picture doesn’t give all the details. Should an element be of a fixed width from the edge or a percentage of the distance across the screen? A single screenshot won’t illustrate that. The precise color of UI elements needs to be documented, as well as shown. And most importantly, the behavior should be described. What are valid inputs? What is the effect of the buttons and links? What are the outputs? All of these need to be captured in the specification.

So, a UI mockup does not remove the need for a feature specification but is a vital addition for user-facing changes.

Diagrams

As with UI mocks up, diagrams are a great addition to a feature specification. I love having visual representations of information to make it clearer. This might show the code architecture, the information flow, user journeys through a feature, or anything else associated with your product. It all helps stakeholders understand the feature better to see any issues or changes they want to make.

They are particularly useful when detailing APIs and data flows within an application to give a clear and instantly understandable view of the behavior that might be difficult to discern from a paragraph of text.

However, a diagram isn’t a replacement for the specification. As well as being shown, you should also write about each step of your information flows or user journeys so that the implementation and test plans can cover them. Specifications serve as valuable checklists for developers and testers, so they should be a complete list of the behavior. Diagrams help illustrate that but shouldn’t be used as a replacement.

Test plans

Rather than putting your effort into writing and reviewing a feature specification, why not write the test plan directly? For testing, the test plan is the outcome we need to work from, so why add the extra complexity of a feature specification? After writing the specification, you will need an additional step to generate the test plan from it, so writing the test plan directly avoids that step.

I recommend writing a feature specification because it is accessible and valuable to more teams. Test plans are not very readable and are intended only for testers. On the other hand, product owners can use the specification to ensure the feature will work as expected. Developers can use it as a checklist to ensure they have implemented all the necessary functionality. The documentation team can use the specification to understand the feature and describe it in the help section. The support team can use it to look up the details of product behavior when they get questions from customers. That’s in addition to you being able to use it for your test plan.

As well as being beneficial to the whole development team, specifications provide a useful intermediate step to help you think through the test plan. Some requirements may need a single test to cover them, but others may require several. For instance, you need to try a whole range of values in text input fields (see Chapter 5, Black-Box Functional Testing). Having a specification lets you know if you have covered everything by dividing it into clear sections.

So, while a test plan is a final goal we’re working toward, writing a specification is the best way to get there. See the Turning a specification into a test plan section later in this chapter for more details on generating the test plan.

Round-up of feature specification alternatives

You can use many other documents and sources of information to guide your test plan, but a single feature specification is the most important. With that as a guide, you can flush out any issues and describe all the behavior precisely. Diagrams, UI mockups, and other visual aids are helpful but should always be in addition to the feature specification. Implicit specifications can likewise be a great source of detailed product behavior but should complement the specification, not replace it.

The next section describes how to prepare your specification, starting with improving the handover from the product owner.

Improving the handover from the product owner

I have seen features handed to the test team with nothing more than a name and a few notes tracking the required code changes. I was supposed to pick the testing up with no details about the feature’s function, appearance, or implementation. If you’re in that state, then have a word with the product owner and developers. Throwing features over the fence to test in that way is massively counter-productive; the communication needs to be much better.

A note on naming

I refer to the person who prioritizes the features and describes their functionality as the product owner. They go by different names in different organizations, such as product managers or project managers, so you’ll have to translate it for your situation. The important point is that this is the person who advocates for the customer’s needs, prioritizes features, and decides their behavior. That may be an individual or an entire team, depending on the size of your company. Whoever that is in your organization, that’s whom I’m referring to here.

Most of the time, the situation for me was much better, and the product owner did provide a feature specification of new features. However, those specifications rarely had sufficient details for complete testing. Encourage your product owner to provide all the information they can, but always be ready to add to it.

After the product owner’s draft, you should follow the same philosophy, adding as much description as possible while knowing that your draft is not the final article. The drafts are just that – drafts – until the product owner, the development team, and the test team have sat down together to review them (see Chapter 3, How to Run Successful Specification Reviews). Until then, there will always be gaps. Your job is to get the specification sufficiently detailed – to identify all the requirements you can – that the review meeting can provide the remaining answers.

To illustrate this, we will consider the example of some requirements written for us by a product owner for a new feature they want to add to a website – the ability to sign in. For version 1, signing in won’t allow any extra functionality, and users will just register to enable other functions in the future. The feature is a simple username and password combination with no Two-Factor Authentication (2FA) or temporary code. Here is the initial specification that they provide:

Login feature specification – draft 1

Users can log into the example.com website

By default, when upgrading to a version that supports logging in, logging in is disabled

Users enter their email address and will be sent an email to verify it is valid

When users click a link in the verification email, they will be sent to a page to choose a password

When users have entered a password, they can log in

This seems simple enough and covers the functionality. You can see what this feature is supposed to do, especially since I’ve chosen an example that we’ve all used dozens of times for dozens of different services. You’ve been a user, and there’s a good chance your company has a signup page of some kind in its product too.

If you received this level of detail from your product owner, you’d probably be grateful for it, but I’m about to massively expand it to produce a comprehensive document covering all testing aspects. It is already pretty good – it’s a list of specific and testable statements, which, as we’ll see, is just what we need.

One exception is the first requirement, which is an umbrella statement describing the feature overall. It is too vague to be tested on its own because logging in involves a whole sequence of steps. Having an umbrella requirement like that is acceptable for readability, but only if it is followed by specific statements describing how the product will meet the general requirement.

Notice that I just referred to the first requirement. You’ll discuss these statements, refer to them, and link them to tasks and bugs, so the first thing to change in this example is to make that referencing easier, as shown in the next section.

Understanding the requirements format

I don’t care how you format the feature specification document. Feel free to have an overview and an introduction, state the scope and the stakeholders, or add versioning tracking every change. If you find those useful for the time they take to prepare, add that information. If not, don’t.

Many tools let you track requirements more flexibly than having a single document. They have features such as reusing requirements between projects and mapping requirements onto the test cases that cover them. While I’ll refer to the specification as a document here, I recommend that you use these tools since they give you greater control to move requirements around and tie them into your development team’s processes.

If you’re stuck with wiki pages or shared documents until you persuade everyone to change, a few formatting rules will make everyone’s lives easier. The other improvements listed here apply to the text itself, and should be used with whatever tracking system you are using.

Numbering

The requirements in Login feature specification – draft 1 are in a bullet point list, which makes them easy to read and separates different statements. If you’re presented with a paragraph of text as a specification, your first job is to turn it into a list of statements instead.

Then, whatever requirement tracking tool you are using, make sure you uniquely number all your requirements. The previous example uses bullet points, which don’t identify the different statements. That’s no good and needs changing. As with many of the suggestions here, this is a minor point but makes everyone’s lives easier:

Login feature specification – draft 2

Section 1 – Main functionality
1.1	Users can log into the example.com website
1.2	By default, when upgrading to a version that supports logging in, logging in is disabled
1.3	Users enter their email address and will be sent an email to verify it is valid
1.4	When users click a link in the verification email, they will be sent to a page to choose a password
1.5	When users have entered a password, they can log in

This description lets everyone refer to the requirements easily. You can read requirement 1.2 to see if you have tested it and raised bugs against it. I have heard objections to numbering the requirements because the statements aren’t ordered, and numbering artificially imposes an order. In the preceding case, there is an ordering at the moment – users will work through them chronologically as they sign up – but even without that, it’s important to be able to refer to the requirements easily. Even if the statements jump from topic to topic, with no connection from one line to the next, give them numbers so that you can reference them.

Now that we have a numbered list of statements, we can group them into relevant sections to make them easier to write and read.

Sections

The specification needs to be split into different sections to track the different aspects of the feature. Currently, our entire specification is only five lines long, but as we expand it, we will need to group requirements. For instance, we will need a section about how we enable the feature, which deserves a section on its own.

Dividing the specification into sections helps keep the document clear for everyone to read and means renumbering has less effect. The specification is a living document and should be improved by developers, testers, and product owners throughout the development cycle, so you may need to renumber requirements as you go along. To fully reference a requirement, you not only need to state which number it is but also which version of the specification it is from. Edits and changes should be encouraged, even if they require renumbering requirements. Usually, mentioning the version number of the specification is unnecessary, but changing requirements can cause confusion, so that is the way to resolve ambiguities.

This isn’t necessary when you only have a few requirements, but if you have a section with dozens of requirements, always split them up to make it clearer.

Now that the requirements are easy to reference and read, we need to consider their relative importance and responsibility for tracking each requirement’s priority.

Requirement priority

All requirements are not equal, and the product owner should assign a priority to each one. That will let the development team know their relative importance so that they can make informed choices when they have to make inevitable tradeoffs.

I won’t describe that process here; instead, I focus on what you need for testing. Once the priorities have been decided and the technical limitations and opportunities have been explored, you need a final description of what will be shipped in this release. As a tester, your job is to make the specification binary – whether a requirement is included or not. Product owners and developers work in the gray areas of deadlines and feature estimates, so it’s your job to make the outcome black and white.

Feature prioritization and estimation are skills in their own right, but they are beyond the scope of this book. Here, I will concentrate on describing the features that made the cut by being clear on the behavior of this release. We will do that by looking at the style you should write requirements in and some common pitfalls to avoid.

Improving requirement statements

There is an art to writing requirement statements. It’s easy to write vague or useless requirements, so you need to be strict and follow a particular style. Specifically, your requirements should have the following properties:

• Specific
• Measurable
• Agreed
• Realistic
• Complete
• Consistent
• Independent
• No implementation details

This section will examine each of those qualities in turn. The first four come from the SMART acronym for management objectives and are a great guide to writing goals in general. The T in SMART stands for timed, and that’s one aspect that these requirements don’t need to cover. Project planning is a skill that is beyond the scope of this book, but it isn’t necessary for the functional requirements. It only records what the product should do, not when. Let’s compare two requirements to illustrate the qualities that requirements need to have:

A. Reading the database tables should be fast enough

B. The User Details page fully loads within 3 seconds

I hope you can already see which one is superior, and here, we’ll describe exactly why.

Specific

First, requirements need to be specific. Which screen do they apply to? Under what conditions? Both of the preceding requirements imply they always apply, but there are many different conditions. Requirement A is very vague. Which database table does it mean? All of them? And how fast is fast enough? It’s a poor requirement and needs improving. Requirement B is better because it refers to a specific page and a specific time. Even then, there are many different values the User Details page might show, so you’ll need to pick a worst-case scenario to gain confidence that it will always load within the time limit.

Look out for the word all or every in requirements, which can be a tester’s nightmare. Consider the following requirements:

C. Every event is rendered correctly in the reporting screen

D. All API calls correctly return error messages

For requirement C, does anyone have a list of all the events? And how is a tester supposed to generate them all? Even worse, for requirement D, the tester needs to generate every error for every API call. There are a lot of possible errors!

Keep the requirements specific. Requirement B refers to a single screen, which you can turn into a single test case within a more extensive test plan. It’s clear where the requirement starts and stops, what it refers to, and what it leaves out. If you want to make a blanket statement such as C or D, be aware that you need to turn the word all into a list of testable cases.

Watch out for comparative requirements, for example, being better than a competitor. Better in what way? Again, that lacks specificity, and your requirements need greater clarity.

However, your requirements needing to be specific presents a problem. What if you don’t know what the behavior should be? If you ask detailed questions, possibly no one has worked out what should happen in those scenarios. Is it better to write down potentially incorrect requirements or leave out those sections? I propose always making your assumptions explicit, even if they may be wrong. It’s much easier to correct a precise statement than improve a vague one or add in a requirement that wasn’t specified before.

Be precisely wrong rather than vaguely right

Let’s return to our example feature specification:

Login feature specification – draft 2

Section 1 – main functionality
1.1	Users can log into the example.com website
1.2	By default, when upgrading to a version that supports logging in, logging in is disabled
1.3	Users enter their email address and will be sent an email to verify it is valid
1.4	When users click a link in the verification email, they will be sent to a page to choose a password
1.5	When users have entered a password, they can log in

Requirement 1.2 states that login in is disabled after the upgrade. This implies there is a feature flag to control that configuration, and feature flags can be turned both on and off. So, what happens if we enable the feature, users sign in, and then we find a critical bug and have to turn it off again? The specification doesn’t say, so we need to document and test it.

This presents a problem because we don’t know what the behavior is if we disable the feature flag. This is another area where product owners are sometimes remiss. While they are very keen on turning a feature on, they may not specify what happens if it is turned off.

You could add a requirement to say, The login function can be disabled via a config option. This might be true but isn’t useful. What exactly happens? Are current users logged out? Are the login details of everyone who’s signed up saved? Hopefully, they aren’t all deleted and are ready to be used again when the feature is re-enabled in the future. However, there aren’t any requirements about that at present, so who knows what the development team has implemented. They may not have thought through that case themselves and may not know the behavior without checking.

If you find yourself unsure of the behavior, don’t write a vague statement that fudges the issue. All the requirements in the specification need to be, as the name suggests, specific. Only then are they testable, and the statements you add should be no exception.

In this case, if you’re not sure what the behavior is, make it up. Invent the behavior out of thin air. What do you assume should happen? Testers constantly test against their assumptions about what a feature should do, so make those assumptions explicit.

If you add a requirement that you are unsure of, note that it needs to be reviewed. That means it will be discussed in the specification review meeting, as described in Chapter 3, How to Run Successful Specification Reviews. In effect, these statements are questions, but instead, they propose a specific, testable behavior. That lets other stakeholders disagree. Importantly, there’s no escaping the question – the system will do something, so it must be specified.

You could ask the other stakeholders these questions as you go along, but there are likely to be many of them. For now, note that this needs checking, but fill the specification document with the detail it needs.

Isn’t that a terrible way to work, you may ask, to write down statements that aren’t correct and which you have made up? No. It turns out it’s much easier to fix requirements that are precisely wrong than those that are vaguely right. By being specific, you can highlight issues and decide what the behavior should be in consultation with other teams.

Here is our specification with those details added in. You can see the complete version of the feature specification in Appendix, Example Feature Specification. In this section, we can see that Section 1 - enabling login is now how to enable login, with the other requirements following in Section 2 – main functionality:

Login feature specification – draft 3

Section 1 – Enabling login
1.1	Users can log into the example.com website
1.2	By default, when upgrading to a version that supports logging in, logging in is disabled
1.3	Logging in can be enabled via a configuration option
1.4	Logging in can be disabled via a configuration option
1.5	When logging in is disabled, the login button disappears from the web page
1.6	When logging in is disabled, all user details are saved
1.7	When logging in is re-enabled, users who had previously signed up can log in again
Section 2 – Main functionality
	(Section 2 requirements continue here)

Requirement 1.7 is the important one. If you enable the feature, sign up, disable the feature, then re-enable the feature again, you can still log in. You don’t lose any user details when you disable the feature. Has the development team implemented it that way? Is that what the product management team wants? These are questions for the specification review. By adding a clear, specific statement, you can now precisely decide what the feature should do.

Measurable

As well as being specific, each requirement should be measurable:

A. Reading the database tables should be fast enough

B. The User Details page fully loads within 3 seconds

Requirement A is too vague. What counts as fast enough? What is a pass, and what is a failure for that case? Watch out for words such as sufficient, large, or quick, which describe a measurable number without actually giving the number to measure against. Requirement B says that 3 seconds is the acceptable limit. That’s great: you can test against it and get a clear pass or fail result.

It isn’t easy or obvious how to make those decisions. What should count as fast enough, or what capacity is large enough? These are decisions for the product owner, and it’s part of what makes that role so challenging. Feel free to give your opinions to help them decide, but in the end, it is up to them.

Requirement A is also too detailed to be easily measurable. It refers to the speed of database reads, which you’ll need tools or logs to check. Sometimes, code is being refactored, and you can only measure low-level changes like those but, wherever possible, relate requirements to user-visible changes. Requirement B is better because it refers to the whole page loading. It’s no good database reads being quick if some other bottleneck slows the page loading. Requirement B is also easier for testers and test tools to check.

Agreed

All the requirements need to be agreed upon by both the product owner and the development team. It’s no good for the product owner to state that a page should load within 3 seconds if the developers know that that will never happen in practice. Conversely, the developers aren’t allowed to sneak in caveats and exceptions they haven’t been able to deliver without explicit signoff from the product owner. Chapter 3, How to Run Successful Specification Reviews, covers this in more detail.

Realistic

After being agreed upon, the requirements need to be realistic. Maybe the product owners and developers would both like to implement some functionality but, given the time and the team available, it’s highly unlikely to happen. The specification needs to describe exactly what you’ll deliver in this project. You can update it in each cycle as you add functionality, and you can have a future section where you list the plans of features you will add later. However, you need a realistic description of the current implementation to test against.

Complete

The requirement document needs to be complete. That is the real trick of requirements and finding all the subtleties and interactions is the skill of writing a great specification. That will depend on the specifics of your feature, of course, but the main categories are described later in the Completing the specification section.

Independent

Each requirement should be independent of the others. It should be possible to pick a requirement, read it in isolation, and check that the product fulfills it. The aim is that they can fail individually, that any of them could fail independently of the others. So long as each describes an atomic unit of functionality, they can be developed and tested separately.

This means that the requirements will be somewhat repetitive, but that is a small price to pay for making them clear. Consider the following requirements:

A. The User Details page fully loads within 3 seconds

B. It displays the username, email address, and profile picture

Requirement B only makes sense if you read it along with requirement A. Instead, make sure that each requirement is a statement on its own:

A. The User Details page fully loads within 3 seconds

B. The User Details page displays the username, email address, and profile picture

This means extra writing but helps avoid any possible ambiguities. Ambiguities lead to misunderstandings, which lead to bugs, which lead to late surprises, project delays, and unhappy customers. This is your chance to stamp them out, even if it makes your document sounds like a 5-year-old wrote it: The house is big. The house is strong. The house is made of red bricks. Such writing isn’t elegant, but it is very clear, and that’s what you need for your requirements.

Keeping requirements independent isn’t always possible. Even in our simple example, the later stage depends on the earlier ones. Section 2 – main functionality of our specification states this:

Section 2 – Main functionality
2.1	Users can log into the example.com website
2.2	By default, when upgrading to a version that supports logging in, logging in is disabled
2.3	Users enter their email address and will be sent an email to verify it is valid

If you don’t receive your login email, for instance, you can’t set up a password. In that case, you can still make the dependencies as loose as possible – the requirement to send a login email doesn’t depend on setting up the password, so the dependency is only in one direction. While each requirement needs the previous one to succeed, most describe separate steps. One exception is this requirement:

2.3	Users enter their email address and will be sent an email to verify it is valid

It seems innocuous enough but needs improving. Both the step to enter your email address and send emails have many different conditions to consider, so this needs to be split up and expanded:

2.3	Users enter the email address they want to sign up with
2.4	An email is sent to the entered email address with a link to verify their authenticity

This way, you can track a failure on the text box for entering address information separately from failures in the mechanism to send emails. Watch out for the word and in your requirements because it probably indicates a statement that needs splitting up.

If you have a list, add that in as a sub-requirement. Consider this hypothetical section later in the specification. Here, we have the following:

10.1	The User Details page displays the username, email address, and profile picture

Instead of writing that, we can write this:

10.1 (revised)

The User Details page displays the following details: A. Username B. Email address C. Profile picture

This keeps each requirement entirely separate, and you can reference any fields that aren’t available. If the username and email address are present but the profile picture is missing, the system doesn’t fulfill requirement 10.1 (revised) C, for instance.

Consistent

The requirements also need to be consistent with each other. This is obvious to say but harder to achieve in practice, depending on the size of your specification.

Real-world example – The duplicated requirement

I once had the task of responding to Requests For Quotations (RFQs) for large projects. Telecom network providers would produce vast specifications of systems they wanted to buy. Different hardware vendors would propose their products and say how many of those requirements their system could meet. We were one of those vendors, trying to make our product match up to the thousands of requirements in the RFQs.

Having spent a week preparing and refining our answers for one RFQ, I noticed that requirement 371 was the same as 824 (I don’t remember the actual numbers!) Worse, the requirements were in different contexts. In one, we had said we were partially compliant, but in the other, we’d claimed to be fully compliant. Claiming too much for the product meant urgent feature requests and emergency projects to deliver functionality. Once we had responded, we were contractually obliged to provide everything we’d promised, so we had to be incredibly careful what we said.

In specifications with tens of requirements rather than hundreds, spotting inconsistencies is much easier. Still, look out for general requirements overlapping with specific ones:

11.1	The User Details page fully loads within 3 seconds

Then, later in the same specification, you may have the following:

11.17

All user pages load within 2 seconds

The User Details page has a laxer specification, so the product may not meet the more general requirement if you just test that.

Implementation-free

Finally, the requirements shouldn’t contain any implementation details. The users don’t care what database technology you choose or the web framework you are using; they don’t care about the code modules or functions involved. The specification is a black box because it only describes the result.

The feature specification should only state what the behavior is, not how it will be implemented. That is up to the development team. Sometimes, there is a clear route to achieving the functionality you want, which the product owners have in mind, and maybe that is the best solution. Product owners should also know at least one way to achieve what they want, to be confident they’re not asking for something impossible or at least incredibly costly to implement. However, it is the development team’s responsibility to decide how to implement a feature. They have complete discretion to achieve these requirements in whatever way they like. They will think it through, considering technologies and options that product owners don’t need to worry about.

The specification document should reflect that. If you find any implementation details appearing, remove them. This document should be silent on how functionality is achieved; it should only list what is necessary. This applies both to technical details to deliver the feature and user interface decisions, which are best left to the user experience (UX) team.

For instance, take a look at the following requirement:

12.1	The user state is read out of the user_details table

Don’t mention database tables in the specification. The development team will choose where the data should be stored. The important point is, what user-visible effect does having the user state cause?

12.2	A table shows the values for each product over the last 12 months

Maybe the user experience team wants to have a diagram or a chart instead of a table. The only requirement is that the information is available, not how it should be shown:

12.3	The system sends a message to initiate a recording of the session

Maybe there’s one message, or maybe the API involves several. The requirements shouldn’t prejudge the messages the application sends. If you find yourself writing requirements like this, stop and think about how you can rephrase them as a behavior rather than implementation details. For instance, take a look at the following requirements:

12.1 (revised)	The user state is displayed on the User Details page, including the following information: Name Email
12.2 (revised)	The Product History page displays the values for each product over the last 12 months
12.3 (revised)	The system can initiate a recording of the session

The changes are subtle but important. These requirements are much better because they say exactly what the system should do but don’t mention any user interfaces, APIs, databases, or other implementation details that will make that possible.

However, some implementation details are important, not least where the option is to turn this feature on, for example. You should add these as notes to the requirements. They are not requirements in themselves, but they are necessary to document for the test team:

Login feature specification – draft 4

Req no.	Requirement	Notes
Section 1 – Enabling login
1.1	Users can log into the example.com website
1.2	By default, when upgrading to a version that supports logging in, logging in is disabled
1.3	Logging in can be enabled via a configuration option	Set the enable_login option to True in the login_configuration.conf file

This version separates the implementation from the functional requirements. The implementation can change without affecting the requirement to have a configuration option available somewhere.

Round-up of requirement statement improvements

There’s a lot to remember when writing specifications. These meta-requirements for the requirement statements mean there are many ways to go wrong. Instead of trying to remember all these rules, it’s best to practice lots, then edit them to make sure they meet all the criteria:

• Specific
• Measurable
• Agreed
• Realistic
• Complete
• Consistent
• Independent
• No implementation details

In practice, you develop a style that makes it easier to see when you’ve strayed into describing implementation or become too vague. To help, you can start by writing clear statements about the feature’s basic functionality before adding more detail. The following section describes those first, obvious requirements and the actual content of your feature specification.

Improving requirement content

Now that we’ve seen how to format the requirements and the best style to write them in, we can consider what they should say. The rest of this book describes the different aspects of testing you should consider and capture within the specification, while this section covers the general principles you should apply to the content of the requirements.

Keep it obvious, cover everything

The first requirements to track are the most obvious ones: the main functionality of this new feature. The product owner should cover these requirements, but even here, there can be gaps. There is an art to identifying your assumptions and saying them out loud. Never be shy about being obvious – every so often, something that was obvious to you will be the source of disagreement and confusion. You can only resolve that confusion by asking precise questions, starting with the easy ones.

Real-world example – The vote counter

I once worked on a large project for a national mobile phone network. We provided the network hardware for SMS, text message, delivery, and value-added services. One of these was a system for voting by text message to provide the service you regularly see on television programs. The specification document was vast, stating the connections and inputs to other networks, the rate of SMSs arriving we had to sustain, the different events we could host simultaneously, and the output statistics we had to support.

Looking through the document, I realized one requirement was missing. The system went straight from describing all the inputs and messages arriving to describing all the outputs and how they would be displayed. However, there was no requirement to actually count the votes. In that case, the requirement was so obvious that it didn’t cause problems. Of course, our system supported that. Other projects with missing requirements aren’t so lucky.

So, start simple and cover everything. Don’t be shy about stating the obvious before moving on to the more detailed requirements.

Version requirements

The specification needs to list all the prerequisites for this feature to work. This detail is difficult for product owners to add – they may know what they want, but it is up to the development team which version they implement it in. For testing, that is the first piece of information you need. Which versions have this new feature, and how do we turn it on?

If you have performed exploratory testing (see Chapter 1, Making the Most of Exploratory Testing), you already have this information and can add it. However, ideally, the specification document comes first to tell you what you need:

Login feature specification – draft 5

Req no.	Requirement	Notes
Section 1 – enabling login
1.1	In version 5.7.1 and later, users can log into the example.com website
1.2	By default, when upgrading to a version that supports logging in, logging in is disabled
1.3	Logging in can be enabled via a configuration option in version 5.7.1 and later	Set the enable_login option to True in the login_configuration.conf file

Adding the version number to the umbrella requirement shows that all the subsequent requirements rely on that version. An alternative is including the version in the description of the entire document. You can say this is the feature specification for version 5.7.1, then list the behavior.

Including the version is a gray area because it is an implementation detail and, as shown in the previous section, the feature specification generally shouldn’t describe the implementation. However, the product version also determines the user-facing behavior of the system, so it is valid to include it as a requirement. It’s essential since it decides whether the feature will work at all.

Describing configuration

What about settings? While this feature is present in version 5.7.1, it is turned off by default. You also need to change a setting to make this behavior appear, so you must add that to the specification. This is another vital detail for the test team that the product owner may miss.

Having feature flags and options to enable new functionality is hugely beneficial. It lets you separate rolling out new code from changing its behavior (see Chapter 10, Maintainability). Feature flags mean you can get important bug fixes and other improvements out to your customers without being dependent on new features working. They mean you can disable problematic features without requiring a downgrade that might reintroduce old bugs and remove other new features. If your development team isn’t using this method already, I highly recommend it.

Your specification needs to record all the settings associated with this feature, starting with those that enable it. That is why this requirement is so important:

1.2

By default, when upgrading to a version that supports logging in, logging in is disabled

We now know that to see this new feature and start testing, we need version 5.7.1, and we need to have enabled it. That’s great, but how do we turn it on? As we saw, there is a big difference between documenting implementation and behavior in feature specifications. While you should record that detail, it is a note, not a requirement itself.

So far, the improvements to the initial test plan have been around its presentation – the numbering and sections, the wording, and how to divide up requirements. The following section concentrates on the content – what requirements should cover. As with exploratory testing, this will be a miniature tour of the remainder of this book.

Completing the specification

To make the specification complete, we need to tour all the different areas of testing once more:

• Functional tests
• User experience tests
• Security tests
• Error case testing
• Maintainability
• Non-functional tests – performance and load requirements

This time, you need to think them through in detail to capture all the product’s behavior. Later chapters describe these more thoroughly, but here, we will examine how they relate to the feature specification, ready to be fleshed out later. First, we will cover the functional requirements that describe the main use cases.

Functional test requirements

The feature specification needs to describe all the feature’s designed behavior, starting with the happy-path cases of the functionality it is supposed to provide. State machines are an excellent way to step through the different points on a user’s journey, tracking the various stages. At each one, list what should happen based on all their possible actions and all the outputs that they should see.

In our example of a signup feature, a user first sees the screen to enter their email address. This has a text field, so there are many possible inputs. Those will be considered in more detail in the next section, as will the restrictions on email addresses. But what happens if that email address is already in use? What happens if that email address has been entered but not verified yet? You need to identify all the possible system states to choose which tests you need to run in each. Here, we have up to five:

1. A new email address.
2. An email address that has been entered by not verified.
3. An email address that has been verified but that doesn’t have a password set up.
4. A verified email address with a password but that has never logged in.
5. An email address that has been used to log in at least once.

Possibly states 4 and 5 are equivalent from the database’s point of view. Only by identifying those states can you work with the developers on the necessary tests. For the feature specification, all these different states need statements describing the behavior:

Req no.	Requirement	Notes
Section 4 – sign up
4.1	If the user signs up with a new email address that has never been used before, then an email is sent to that address to verify the account
4.2	If the user signs up with an email address that has been entered but not verified, then another email is sent to the address to be verified
4.3	When a user signs up, no feedback should be given to the requester as to whether the email already exists	This avoids leaking information about any email addresses already in use on the system
4.4	If the user signs up with an email address that has been verified but doesn’t have a password set up, then another email is sent to the user with a new link to the password page
4.5	If the user signs up with a verified email address with a password set, then they are sent another email with a link to the page to reset their password

Note that requirement 4.5 covers both states 4 and 5. The behavior doesn’t differ depending on whether the user has logged in, so we don’t need to spend time trying both cases.

Designing the functional test part of the specification involves carefully stepping through all possible situations the code could encounter and its inputs. For each, ask what could happen and record the planned behavior. It sounds so easy! Of course, it is not in practice and will require your skill to do well. After the happy-path cases, you can move straight to documenting what should happen with invalid data.

Error cases

What happens when things go wrong? Again, this is an area that can be overlooked by product owners, in my experience. They mainly concentrate on the behavior they want to have rather than what should happen when things go wrong. This area also requires lots of input from the user experience team, and you may need more text to cover failures than for the working cases since there are more of them.

Failures should be graceful, and errors should guide users, where possible, to resolve their issues. In our example, we have to cover users entering invalid email addresses or mismatched passwords, email delivery failing, or users clicking out-of-date links.

More severe internal failures are considered as part of non-functional testing. Otherwise, this section follows the same principles as that of the rest of functional testing – identifying the different states and specifying what should happen when things go wrong in each one. See Chapter 7, Testing of Error Cases, for more details.

User experience specification

One area the requirements in our preceding example did not cover was the user experience. We stated that the user should be able to log on, but not how that screen should look. If you are in a company of any size, a dedicated user experience team should decide this. They will specify the font, placement, colors, and exact text of all the user-visible elements. These should be produced in a tool for quickly mocking up user interfaces, and there are many to choose from. The UX team should then share those mockups with the test team to compare against the actual implementation. This is one of the more accessible areas of testing, similar to playing spot the difference. Does the implementation match the request?

The other important aspect of user experience is the location and flow of user tasks. Maybe there should be a button to change your password, but if it is buried in four layers of menus, then no user is likely to find it. That is also a bug, although a problem with the specification rather than the implementation.

As well as the ideal cases, the user experience section should list the full range of systems your application or web page supports. You need to list all the operating systems and web browsers your product supports, the lowest screen resolution, and the maximum. Those must all be specified, ready to become part of the test plan.

Security test requirements

Security considerations will also be covered in more detail later, including covering the cases of authentication and authorization, and the triad of Confidentiality, Integrity, and data Availability (CIA). In our login example, security testing should describe timeouts on email links and logins, the generation of secure keys, and rate limits on password attempts and form submissions.

Notice the preceding requirement:

2.3	Users enter the email address they want to sign up with

You fill in the signup form with anyone’s email in the world, which will fire off an email that could be used maliciously to annoy someone you dislike if they use this service. While that’s not so different from emailing them from any other account, this time, it’ll have your company’s name on the spam messages. To block that, there should at least be a limit on how often you email to one account.

In this example, users choose passwords, so you must specify the complexity requirements you will enforce. You also need to check that those passwords are stored securely. On one website that shall remain nameless, when I requested a password reminder, they emailed my current password to me. Passwords should be stored hashed so that even those with access to the database cannot read them.

This applies to all passwords and PINs, not just those protecting user accounts. In a company I worked for, the main passwords were hashed, but users could assign passwords or PINs to protect recordings. They were stored in plain text, so we could see what people had chosen. Wherever there is a password, it must be unreadable, even to the development team. It’s your job to check that.

Security requirements are often an anomaly because they effectively limit the behavior of features rather than stating available functionality. The specification needs to capture all those restrictions and protections.

Maintainability

This section is always close to my heart, even if it is not always a priority in projects. How easy will this be to debug? Specifications rarely go down to the level of what logging the system will provide, but that is necessary to work with the feature long-term. Are the log lines understandable? There should be a clear line recording each user interaction, including the contents of the messages. If a user with a particular email address had trouble signing up, you have to be able to search for that email address to find the actions associated with it. Are errors and warnings usefully logged, and can you easily aggregate and view them? Again, add those requirements to the specification because no one else may.

The specification also needs a dedicated section to record the events and instrumentation. How will you measure how well this code is performing, how much your users are using it, and in what ways? While not part of the main functionality, this is vital information for your company, and it needs to be specified along with the feature itself.

None of these requirements help your customers when they first start using your new product, but it will make your life a lot easier, in the long run, to specify the behavior you need while everyone is thinking about this feature.

Non-functional tests

As Rex Black notes in his book, Advanced Software Testing, non-functional areas of software testing tend to be underspecified, so they may need particular attention from you.

What’s the maximum number of users that can sign up for the service? When I have asked this, I sometimes got blank looks. There is often no limit in the code – it is only limited by the memory and disk space of the system it’s running on. If user records take a few bytes of data each, on a system with gigabytes of storage, that will be huge. In that situation, the solution is to pick an arbitrary limit and test up to that. Verify that the system works successfully with up to a million users, for instance. That gives you confidence for the foreseeable future, and everyone knows when you are approaching the dangerous ground of untested behavior. Most importantly, as a tester, you have an actionable test and either pass or fail.

Often, with non-functional tests, the pass criterion isn’t as simple as a behavior. Maybe the system can support a million users, but at that size, it takes over an hour to send out emails to verify people’s addresses. It works, but far too slowly. All functionality is assumed to work within a specific time, so make that explicit in the specification: All the pages load within 3 seconds, for instance. Then, you know what time to set as a time out in automated tests, and if responses get slow when the system is in use, you know precisely what a pass or a fail is.

What rate of signups can your web page support? Excessive usage is a good problem to have, but it’s still a problem. If your next marketing video goes viral and suddenly you are inundated with eager users, you don’t want their first impression of your fantastic company to be a signup page that doesn’t load. As with other load limits, it’s possible that product management hasn’t thought of this number, and the development team doesn’t know what the system can support in practice. So, again, choose a limit and test to that. This form of testing is vital as you, the tester, determine the system’s performance. See Part 3, Non-Functional Testing, of this book for more details.

Documentation

Finally, the specification needs to describe what documentation is provided for the customer. Are there tooltips or help boxes, manuals, or instructional videos? Ideally, a product’s user interface is clear enough that little guidance is required, but it’s always useful to provide help if people need it.

This is technically a subsection of the user-interface testing but can be worthwhile pulling out into its own section within a specification to ensure it is considered in detail. As with the rest of the user interface, ideally, the documentation needs to be specified by a dedicated user experience team who will finalize its wording and format.

When they provide those details to you, the testing just involves working through the interface to ensure all the planned fields have been implemented correctly. While testing the presence of help files is black and white, the content of the help will benefit from feedback and suggestions. If there is anything you struggle to understand or have to read twice, report that back to the UX team. If others face similar difficulties, that area should be changed.

Feature flags are useful for rolling out features but can make documenting interfaces harder. Often, feature flags control menu systems or other visible elements, removing them altogether when the feature is disabled. As well as making sure the configuration is correct for your testing, the documentation team needs to ensure all screenshots and videos show the correct features.

The documentation needs to be reviewed regularly. Even a tiny change on a single page might render many screenshots and videos out of date. It can be hard to tell where changes need to be made, so the documentation team will need to record which pages appear in which screenshots so that they know when to update them.

Round-up of specification sections

This section contained brief notes about the different areas and what you will need to specify and test. They all need more detail, which will be added later in this book, and you need to apply them to your particular products and features. When testing a feature, your main task is to work out what it should do and the questions to ask about its interactions, so if you complete this section thoroughly, your testing is well on the way. The following section describes taking this specification, refining it, reviewing it, and using it as a basis for future testing.

The first draft of a feature specification

Remember the five requirements we received from the product owner? While they seemed to describe the feature well enough, they are nowhere near sufficient to cover all the previous considerations. Remember, this is for three new pages – login, signup, and password reset – that provide no additional functionality. After considering all the aspects of the feature described previously, we can complete a draft of the specification. This is included in Appendix, Example Feature Specification, which shows a good first draft for a feature specification. Note how the requirements are independent, implementation agnostic, and specific.

This book aims to help you quickly produce specifications like this for the features you are testing. The same considerations crop up repeatedly in different forms, so it’s a case of learning them and applying them to your system while watching out for special cases.

Even after expanding the requirements to 10 times their initial size, you’re not finished. There are still many gaps in this feature specification that need to be reviewed and corrected by the developers and product owners. Remember all those times you chose to be precisely wrong rather than vaguely right? You need to get an agreement for them. You need to check all those performance numbers you plucked out of the air. By choosing those options yourself, you speed up the process and have the best chance to have a clear, helpful conversation in the feature specification review. Preparing the specification as you see it is only the first step. The next chapter covers the review in more detail, but here, we will skip straight to the stage after – that is, producing a test plan from the requirements.

Turning a specification into a test plan

This book is about designing great system test plans; however, I propose that those test plans are secondary. They are an outcome of having a great feature specification.

You only need to put in the effort once to design and create a comprehensive document describing the feature. You could put that effort into creating the test plan directly, then get that reviewed by the various stakeholders. However, as we saw in the Advantages, disadvantages, and alternatives to feature specifications section, a feature specification is more valuable to many different teams. The product management team can read a specification and agree that they want those requirements. The documentation team can use it to describe and document behavior and the support team can read it to check on details to answer customer questions. The development team can use it to track the progress of their implementation, and, of course, the test team can use it to prepare the test plan. Because of that, I recommend that the document you spend the time on is the feature specification, although it leaves you an extra step to produce a test plan from it.

The relationship between the test plan and the feature specification should be straightforward. So long as you have precise, declarative, independent statements in your specification, each can become one or more tests in your test plan. Let’s look back on the main functionality section of our specification:

Login feature specification – draft 5

Req no.	Requirement	Notes
Section 2 – Main functionality
2.1	Users can log into the example.com website
2.2	By default, when upgrading to a version that supports logging in, logging in is disabled
2.3	Users enter their email address and will be sent an email to verify it is valid

Requirement 2.1 is an umbrella requirement fulfilled by the statements that follow it, so you don’t need to test it separately. Requirement 2.2 is testable – when you first upgrade, there should be no change in functionality. Requirement 2.3 lets users enter their email addresses. That is a text box that needs many tests to verify, which will be described in Chapter 5, Black-Box Functional Testing. From the previous Error cases section, we had the requirement that the application should reject invalid email addresses with a suitable error message. Email addresses can be incorrect in many ways, however, so those all need to have test cases added. That one requirement needs many test cases to cover it completely.

Other parts of the specification also require unpacking, and most of the rest of this book will be devoted to describing common patterns of requirements and the tests you need to run to check them.

While feature specifications have many benefits, it can be hard to convince a development team to fully embrace them. The following section considers some arguments against feature specifications, and how to counter them.

Countering arguments against specifications

I’ve encountered various reactions to specification documents over my years in the software industry. Usually, people’s reactions are positive, or at least hopeful. Most people would love to have a clear description of the feature under development, but between changing priorities and a lack of time, specifications can be challenging to produce in practice. Other people are more actively negative and argue for spending valuable project time elsewhere. As you can tell, I am firmly in favor of specifications, so here are some anti-specifications arguments and some counterarguments against them.

“This feature is too small to be specified”

Sometimes, a feature is so small that it hardly seems worth writing a specification for it. If a feature is obvious and everyone seems to know what it does, why spend the time writing that down? Some changes are so small that there seems to be nowhere for surprises to hide.

To counter that, consider the many different areas of testing listed previously. Have all the error cases been thought through? Have its maximum load limits been specified and tested? What events and logging does it generate? What documentation changes does it need? Even the simplest change can render multiple screenshots and videos out of date, as we mentioned previously. And even seemingly simple changes can hide issues.

Real-world example – The two tick boxes

There was once a minor feature request at a company where I worked to expose two tick boxes on our customer-facing portal. These controls were already present in our internal tools, but we got so many requests to change them from customers that the support burden was significant enough to justify a little development time to make them public. Two tick boxes – what could go wrong?

Two binary options produce four possible states in total, and it turned out that one of those four states was invalid. Our internal system detected that arrangement and blocked it, but the public-facing pages used an API that bypassed that check. Should we have used a dropdown instead of two independent options when only three states were valid? Absolutely! But we had not.

Such a minor change received a cursory test to check that both options worked, and they did, individually. But we didn’t write a specification and didn’t methodically step through the four possible choices. After all, they already worked internally, didn’t they? We released the broken code so our customers were the first to find that the invalid configuration didn’t work, and I learned that even the smallest features can hide critical bugs.

On the plus side, small changes in functionality only require a small specification. It doesn’t take a significant investment of time to write a specification for a minor modification, so it scales in proportion to the project. Even small features can hide surprises, so it’s always worth spending time upfront finding all the details you can.

“We don’t know that behavior”

While you are busy as a tester asking difficult questions, other team members have the even more challenging job of answering them. Another argument against a specification is that no one knows what the current desired behavior is, with the implication that it isn’t worth the time to find out.

This objection comes in two different versions:

• The first one is temporary – we don’t know that yet. As with exploratory testing, it’s possible to begin the specification too early when the feature is still in flux. The product owner may still need to make decisions, or different design options may result in different outcomes. If that is genuinely the case, keep talking to people and wait for the feature to be ready. Just make sure it’s not a stalling tactic; you will need that specification sooner or later.
• The second version of this objection is that some questions can’t be answered. This is generally around non-functional specifications or interoperability. Does the application work on old browsers and operating systems? What is its maximum throughput? Questions like those can become tasks for you, the tester, to answer. If you pick them up as a task or track them as unanswered questions, they are no objection to getting the rest of the feature fully specified.

“We don’t have time to write a specification”

The most common argument against writing a specification is that there isn’t sufficient time. Yes, they’re important; yes, they’re a good investment of time; but the deadline is looming, and if the choice is between delivering the product and writing about it, you should choose to deliver it.

This is another area where you can help. Do as much as you can on the specification yourself. It may not feel like part of your job. Ideally, the specification would be an input into the test team: you would be given a feature and a description of what it should do, ready to test one against another. However, in practice, since you need the specification, it’s sometimes up to you to produce it. While it will also need time from the development team and product owner to review, you can do the heavy lifting by writing it out. All they have to do is agree, and that is far quicker.

That is why I have spent so long in a book about testing talking about how best to write requirements. Clarity of thought is vital both for specifications and testing, and being able to clearly describe the feature and ask questions about it. The real reason was practical: testers need to know how to write specifications because it will often fall to you to get it done.

This book aims to let you write better test plans faster. If you are better at designing specifications, it takes less time and is less of a drain on the project. By reading this book, you will learn about the important considerations so that you can rapidly think through how they affect each feature. Producing a specification faster is the best argument against anyone who says they take too long.

Summary

In this chapter, you learned how to turn a short, basic description of a feature into a comprehensive feature specification. That is one small step away from the test plan you need to guide all your testing.

You’ve seen the strengths and weaknesses of a specification and the other types of documents in a project. Many of those complement the feature specification, but none fully replace it, and it’s the feature specification you need to base your test plan on.

We’ve covered the journey of a specification, from the initial handover from the product owner to how to structure the document in terms of numbering and sections, then the vital guidelines to writing requirement statements. They need to be specific, measurable, agreed, realistic, complete, consistent, independent, and contain no implementation details.

There’s a lot to remember, but once you’ve mastered the style, it’s simple enough to work through the many areas of a feature that you need to specify. They mirror the chapters of this book, from functional, security, and usability testing to non-functional tests and documentation.

Finally, we looked at how the specification relates to the test plan and countered some arguments against writing specifications. As you have seen, even seemingly small and straightforward features can hide blocking bugs.

Your specification is just a first draft, despite all your work turning the brief requirements from the product owner into a comprehensive description. It will still have many holes until you review it with the product owner and the development team to answer the many uncertainties and fill in all the areas that you haven’t realized that you’ve missed. How to run that review successfully is the topic of the next chapter.