C H A P T E R  4

Requirements

The hardest single part of building a software system is deciding what to build. No other part of the conceptual work is as difficult in establishing the detailed technical requirements, including the interfaces to people, to machines, and to other software systems. No other part of the work so cripples the results if done wrong. No other part is more difficult to rectify later. Therefore, the most important function that the software builder performs for the client is the iterative extraction and refinement of the product requirements.

—Fred Brooks¹

Before you start coding – yes, before you start coding – you need to know what it is you're going to build. That's what requirements are: a list of stuff you have to implement in order to create your terrific program. Most developers hate requirements. Really, all we'd like to do is sit down and start coding. All of us have that super-programmer mentality; just give me the problem and I can sit down and design and code it on the fly. Not! If you want to be a productive developer and make fewer errors and come up with a good, clean design, you need requirements – the more detailed the better. A good set of requirements tells you just what the program is supposed to do. It gives you the scaffolding around which you'll hang your design. You'll do requirements anyway – it's one of those steps in a standard development lifecycle that you can't avoid, but if you don't make room for it in your project, you'll create a program that is pretty crappy. Being intentional about requirements forces you to think about the details of the program, and it also lets you listen to the users so you have a better idea of what they really want. So let's talk about requirements.

What Types of Requirements Are We Talking About Here?

We're really talking about functional requirements. That is, the list of features the user will see and be able to use when they fire up your program. These are the “black box” requirements that show the external behavior of your program. They will certainly lead to lower-level requirements that talk more about how your program works, rather than what it does. The output of this process of identifying requirements is a functional specification of what the software system is supposed to do.

____________________________

Functional Specification?

A functional specification describes what the program will do entirely from the user's perspective. It doesn't care how the software is implemented. It talks about the features of the program and specifies screens, menus, dialogs, and the like. Think of it as a badly written user manual. A second kind of spec can be called a technical specification. The technical specification describes the internal implementation details of the program. That is, it talks about data structures, algorithms used, database models, choice of programming language, and so on. We're not going to talk about technical specs in this chapter, just functional specs.

“Wait,” you say. “What about all those agile methodologies we talked about in Chapter 2? They don't write functional specs. So there! I'm off the hook.” Well, in fact, agile methodologies do write functional specifications. They're just in a different format from the 300-page single-spaced requirements document that some plan-driven methodologies require. XP requires that together with the customer representative you write user stories that lay out what the program will do. That's a spec. The important part and the idea behind this entire chapter is to write down what your program is supposed to do before you start coding.

But I Don't Like Writing!

A standard argument made by software developers is that they can't write. Nonsense! Everyone can learn to write functional specs. But writing is work. You have to get in there and practice writing before you'll be any good at it. If you're still in school (be it undergrad or graduate school), take a course in writing, one where you've got to write essays or journal entries or stories or poetry every single week. You should also have to read other works critically; reading other people's writing, whether good or bad, is a great way to learn how to write better.

That Natural Language Thing

Functional specifications should always be written in a natural language. Why? Well, it's the Sapir-Whorf linguistic relativity hypothesis, don't you know?² In a nutshell, language not only determines what you do say, it determines what you can say (and think). That is, the language you use determines what kinds of thoughts you are able to have, and thus what you can think about and how you express your thoughts. If the language doesn't have room for certain kinds of thoughts, you are much less likely to think them. Natural languages are much more expressive and varied than programming languages, so you want to do your designs in natural languages and save the programming languages for implementation later. Whether you believe the Sapir-Whorf hypothesis or not, it's nearly always a good idea to develop your functional specification in a natural language so you don't get bogged down in the syntactic and semantic details of a programming language before you need to. This doesn't mean that you can't think about implementation while you're doing the functional specification (you will, trust me), but just shunt those thoughts over into a “technical note” sidebar of your specification.³ You might also look at Kenneth Iverson's Turing Award lecture, “Notation as a Tool of Thought,” for a similar discussion.⁴

____________________________

Outline of a Functional Specification

Every functional specification is different, just as every software development project is different. So take this outline with a grain of salt and just use the parts that apply to your project. Lots of the ideas here are from.⁵ Every function specification should have the elements discussed in the following sections.

Overview

This is your executive summary. A paragraph or at most two of what the program is supposed to do. “This program runs your microwave oven. It interfaces to a keypad and an LCD display that provides user input and output functionality. Its functions are limited to those that a standard microwave would have, with the addition of single buttons for pizza and coffee reheating. It also will run a time of day clock and a standalone countdown timer. It doesn't control the light. It has a safety interlock that will prevent the microwave from starting if the door is open.”

Disclaimer

You should always put it a statement right at the beginning that “This specification isn't done yet. If you think something is missing or wrong, just sent me an email.” That helps keep all the marketing guys off your back and lets you file new feature requests in your mail trash bin. Lots of people will put a big, black DRAFT in the header or footer of the document. That can work as well, but folks tend to ignore it. Some people will use a big DRAFT watermark on their specs, so that every page has the word embedded behind the text. This doesn't stop people from yelling at you either. At some point your disclaimer should change to something like “This specification is as complete as it will be for this release. If you think something is missing or wrong, just sent an email to the author and we'll consider it for the next release.”

Author's Name

Somebody needs to be responsible for the functional specification. Not a committee, not the development team, one person. This is usually either the development manager, the project manager, or the chief architect, depending on how your company sets up development projects. There are pros and cons to all the different organizational arrangements.

If the development manager (the person to whom the developers report) is in charge of the functional spec, then that person is usually up to speed on all the technical aspects of the project. That's good. On the other hand, if your boss writes the functional spec, it might be harder to tell her that there's something wrong with the specification, or that you don't agree with the design. Also, development managers were probably developers at one time and so they may not have the people skills (read: charm and schmoozing skills) necessary to talk to marketing, the customer, documentation, testing, and so on.

____________________________

If your company uses project managers that are in charge of the specification, design, and schedule, but don't have developers directly reporting to them, then you run the risk of getting someone that isn't as technically astute as a former developer. On the other hand, these folks can usually charm the socks off the other teams, so negotiations are a lot smoother. Project managers need to have some technical skills and to be very good at getting all the stakeholders to reach consensus on the contents of the functional specification.

The chief architect model is like the project manager model except that the architect is a developer and so is more technically competent and is usually in charge of the functional specification and all the program design issues. This is like the chief programmer model in Brooks' The Mythical Man-Month. Someone else is in charge of interfacing with the other teams, the schedule, and doing all that people-interaction stuff. This can be good if the architect is disciplined and doesn't let any requirements creep get in the way of a good design.

Scenarios of Typical Usage

A great way to get customers to respond to your requirements list is to present several scenarios of typical usage of the program to them as part of the specification. This has a couple of advantages.

• First, if you write the scenarios as if they're user stories, the customer is more likely to read them.
• Second, customers are more likely to understand what you're doing and come up with ideas for things you've missed or gotten wrong. This is always a good thing, because the more customer input you get early in the process, the more likely you'll actually create something they want.

In many agile methodologies, including XP, user stories are often written like scenarios. And in XP, the customer is part of the project team, so you get constant feedback on user stories and daily program builds. In the Unified Modeling Language (UML, see www.uml.org), there is an entire notation used to create use cases (another word for scenarios). But as we discussed above, nothing beats natural language for describing usage scenarios. We'll come back to use cases later, in Chapter 8.

Detailed Screen-By-Screen Specifications

Once you've written a couple of scenarios, you will have a much better idea of how your program will flow, and what screens, dialog boxes, menus, and so on you'll need. This lets you go through each one of those screens and flesh out the details of how they're laid out, what buttons, text boxes, icons, graphics, and so on they'll have, and what other screens they connect to. Use pictures! A picture of a screen or a dialog box is worth way more than a thousand words. It gives the reader something to react to and it gets them thinking about program flow and user interface issues. Don't expect anyone to read these except for the developers who will implement them and the tech writers who will write the user manual (that no one will read).

Non-requirements

This may be the most important section of the functional specification. This tells the world what you're not going to do. Really, you need to put this in because after laying out what the program will do, the most important thing the functional specification does is manage expectations. One of the worst phrases a customer can utter at that final demo before you release is, “But I thought it was going to do ....” You need to tell all the stakeholders in a project what the program is going to do and also what it's not going to do. “This microwave software will not balance your checkbook.” Well, okay, that's a little over the top, but you do need to let them know that there are requirements that won't be implemented – at least not in the current release. “Only one countdown timer may run at a time.” “There will not be a defrost cycle that allows defrost modes to be selected by food type.” It's likely that your customer won't read this section, but at least you can point to it when they ask.

Open Issues

When you first write the functional specification, there will be one or two (thousand) things you don't know. That's okay. Just put them in the “Open Issues” section. Then every time you meet with the customer, point to this section and try to get answers. Some of these questions will move to requirements sections and some will end up in the “Non-requirements” section, after you get those answers. By the end of the project, though, this section should be empty. If it's not, well, you've got issues that will haunt you.

Design and Feature Ideas

If you're like me, you'll be trying to design and code the program all the time you're doing your requirements gathering and analysis. That's just what developers do. So to avoid having your head explode from all these fantastic design and implementation ideas that you can't write down because you're writing requirements after all, write a separate notebook. This notebook is just a separate document – keep a text document open on your desktop for it – that contains a note for later. The two types of notes I typically create are technical notes containing design or coding ideas for developers, and marketing notes containing feature ideas for the marketing folks and the customer.

Backlog

As your project proceeds through development, new requirements will surface. Get used to it; this always happens. But if you want to keep to a schedule and deliver a working product, you just cannot implement everything that will come up. That's what the “Backlog” section is for, all the requirements you are going to consider for the next release of the product. Most functional specifications don't have a “Backlog” section, but if you want your functional spec to be a living document, you need a place to put all the tasks you will do later. This does a couple of good things for you. It tells the customer you haven't forgotten these features, and that by moving them to the next release you are committed to delivering the current release as close to the published schedule as possible. And it tells the developers that you're not out of control and that the project has a good shot at being done with high quality and on time. For more information on backlogs, take a look any of the Scrum agile methodology descriptions.⁶

____________________________

One More Thing

One more thing about the functional specification – don't obsess. Chances are that you'll do a good job of picking out requirements and writing them down in the functional spec, but that it won't be as detailed as you like and it won't be complete. Don't worry, be happy. The only time a functional specification is complete is when you ship the release. Don't spend time trying to get every single detail correct, don't spend time trying to tease every requirement out of your customer. It just won't happen. Set a time limit, do your best, and let it go. You don't want to have a bunch of developers sitting around twiddling their thumbs with nothing to do waiting for the spec do you?

Types of Requirements

In a functional specification you'll usually see four different types of requirements: user requirements, domain requirements, non-functional requirements, and non-requirements.

User Requirements

User requirements are nearly always expressed in natural language. They are the details of what the user expects to see as she uses the program. They also include descriptions of screen layouts, dialog boxes, and menus. Any interaction element in the program should be described in the user requirements. For example:

Logging into the System: When Gloria clicks on the Login button on the main page, a login dialog box appears in the middle of the screen. The login dialog must contain two text boxes, labeled “Username” and “Password.” There must also be two buttons in the dialog box, labeled “Submit” and “Cancel.” If at any time Gloria presses the Cancel button, the dialog box shall disappear and she will be taken back to the previous screen. In normal usage, she will click in the Username text box and type in her user name, and then click (or tab) in the Password text box and type in her password. The text typed in the Password text box must be hidden. Once Gloria is finished typing in her user name and password she must press the Submit button. If she has entered a correct user name/password combination she will then be taken to the main menu page. If Gloria's user name/password combination is incorrect, an “Invalid user name or password, please try again” message shall appear in the dialog box, the text boxes shall be cleared and she will be given the opportunity to login again.

As seen in this section, you can express user requirements as scenarios, and as detailed screen-by-screen descriptions. Remember to use pictures as much as you can when you're doing user requirements. If your program is web-based, do lots of quick and dirty html pages and paste them into the spec. If it's not web-based, use a drawing program to create pictures of what the user will see.

Domain Requirements

These are requirements that are imposed on you by the application domain of the program. If you're writing a new version of TurboTax®, you will be constrained by the latest IRS regulations. A general ledger program will have to abide by the latest edition of the Generally Accepted Accounting Principles (GAAP), and an AT&T-branded iPhone will need to implement the latest GSM protocols. You don't need to write down all these requirements, just refer to them. A set of detailed domain requirements give the developers information they will need during their design of the program. Domain requirements are usually considered “middle layer” software because they are the heart of the application, below the user interface and above the operating system, networking, or database software. Lots of domain requirements will get implemented as separate classes and libraries with their own APIs.

Non-functional Requirements

Non-functional requirements are constraints on the services and functions of the program and also expectations about performance. They can include target platform specifications, timing constraints, performance requirements, memory usage requirements, file access privileges, security requirements, response times, minimum number of transactions per second, and so on. These are usually requirements that may not be visible to the user, but which do effect the user experience.

Non-requirements

These are the things you're not going to do. See the previous section for a description of non-requirements.

Requirements Digging

Most software engineering texts use the phrase “requirements elicitation” to talk about the process of getting your users to tell you what they want. Hunt and Thomas, in their book The Pragmatic Programmer use the much more descriptive phrase “requirements digging” to emphasize the point that what you're really doing is digging for all those requirements that your customer doesn't know they want yet.⁷ Hunt and Thomas also make the terrific distinction between requirements, policies, and implementations as a way to illustrate the requirements digging process.

For example, “The system must let the user choose a loan term” is a nice succinct requirement. It says that there's something you have to do. It isn't specific enough for implementation yet, but it tells the developer something concrete that must be built.

“Loan terms must be between 6 months and 30 years” is not a requirement, although it kind of looks like one. This statement is an example of a business policy. When statements like this are presented to developers as requirements they have a tendency to hard-code the statement in the program. Wrong, wrong, wrong. Policies like this can change, so you need to be very careful about putting business policies in your requirements. It is almost always the case that you need to implement a more general version of the business policy than is stated. The real requirement is probably something like, “Loan terms are of finite length but the length of the loan will vary by type of loan.” This tells you that you probably need to build a table-driven subsystem to handle this feature. That way, the loan term for a particular type of loan can be changed by making a single change in a data table and the code doesn't need to change at all.

“The user must be able to select a loan term using a drop-down list box” isn't a requirement either, although, again, it may look like one. This is only a requirement if the customer absolutely must have a drop-down menu to choose their loan term. Otherwise, this is an example of the implementation that the customer would like to see, and it may not be a requirement. As Hunt and Thomas state in their book, “It's important to discover the underlying reason why users do a particular thing, rather than just the way they currently do it. At the end of the day, your development has to solve their business problem, not just meet their stated requirements. Documenting the reasons behind requirements will give your team invaluable information when making daily implementation decisions.”

____________________________

Why Requirements Digging Is Hard

There are several reasons why pulling requirements out of your customer is a really hard exercise. We'll look at a few.

Problems of Scope

Lots of times the actual boundaries of what your program is supposed to do are fuzzy. This can be because of several things. The program may be part of a larger system and the integration of the parts is ill-defined. The customer may not have thought through exactly what they want the program to do, so they start throwing out all sorts of ideas, many of which may not even apply to the problem at hand. Finally, the customer may have dropped into implementation-land and provides unnecessary levels of detail.

It takes lots of patience, discipline, repeatedly saying the word “no,” and repeatedly asking, “why does this need to be part of the program?” in order to overcome problems of scope. Scope is directly related to requirements creep, so beware.

Problems of Understanding

Let's face it; the customer and you as the developer speak different languages. Your customer is the domain expert and they speak the domain language (accounts receivable, accounts payable, reconciliation, general ledger, and so on). You speak the design and implementation language (class, object, method, use case, recursion, activation record, and the like). This is usually worse than an American in Paris; at least there, both sides can pantomime their needs and figure things out. With problems of domain understanding, the best you can usually do is order drinks together.

There are usually two ways to overcome problems of understanding. The first is to have someone in the middle who has lived in both worlds and who can translate between the two. Some companies have folks called system engineers or technical marketers who fulfill this role. These folks have done development and have also worked the customer side of things so they can speak both languages. Good system engineers are worth their weight in use-cases. The second way to promote understanding is to have the customer as part of the development team. This is the approach taken by some agile methodologies, notably XP. When the customer is part of the development team you get to talk to them every day, ask them questions, teach them technical stuff. Both sides benefit. And because the on-site customer sees intermediate product builds as soon as they pop out of your build machine, you get immediate feedback. Win, win, win.

Problems of Volatility

Things change. This is by far the hardest part of requirements gathering and analysis and the biggest reason why schedules slip. You can't do anything about it. Get used to it. As Kent Beck says, “Embrace change.” What you can do is manage change. Create a backlog of new features that get added as they arrive. In the Scrum methodology, new requirements are always added to the release backlog, they are not added to the current iteration; this allows the current sprint to proceed normally and the requirements are all reviewed at the end of the sprint. Another way to manage change is to push the decision onto the user; give the user a choice - “If we implement this new feature it will add 6 weeks to the schedule. Do you still want it?” Alternatively, “If you want to keep to the original schedule we can only implement and test one of A, B, or C. You pick the one you want most.” This is one of the things that the agile folks mean by “courage;”⁸ sometimes you have to take a stand and choose what is best for the project as a whole.

Non-technical Problems

From a developer's perspective, non-technical problems with requirements are the worst ones you will see. In fact, these are problems developers should never see; their managers should shield them from non-technical problems. Non-technical requirements problems are fundamentally political. Examples abound. One group of customers in an organization has a different view of the program requirements than another group. Or worse, one manager has a different view than another manager. The program being developed will reduce the influence of one department by automating a function that they used to be the sole source of. The program will distribute data processing across several departments where it was once centralized in a single department. The list goes on and on. The best advice for non-technical problems is to run away – quickly. Let your vice-president deal with it; that's why she is paid the big bucks.

Analyzing the Requirements

Once you've written down a set of requirements you need to make sure that these are the right requirements for the program; you need to analyze them. Analysis has three basic parts.

First, you categorize the requirements and organize them into related areas. This will help the designers a lot.

Second, you prioritize them based on customer input. This is critical because you won't be able to implement all the requirements in the first product release (trust me, you won't). So this prioritized list will be what you'll use to set the contents of each interim release.

Lastly, you need to examine each requirement in relation to all the others to make sure they fit into a coherent whole. Ask yourself a series of questions:

1. Is each requirement consistent with the overall project objective? If your program is supposed to sell your users books, it doesn't also have to compute their golf handicap.
2. Is this requirement really necessary? Have you added something that can be removed without impairing the essential functionality of the program? If your first release is supposed to allow users to buy books, then you probably don't need to also allow them to buy sailboats.
3. Is this requirement testable? This is probably the most important question when you're doing requirements analysis. If you cannot figure out how to test a requirement, then you cannot know that you've implemented it correctly or that you are finished. All requirements must be testable, or else they are not requirements. In most agile methodologies, the rule is to write the test first, then write the code.
4. Is this requirement doable in the technical environment you've got to work in? This question normally applies to those non-functional requirements mentioned previously. Are your requirements feasible given the particular target platform or set of hardware constraints you must work under for this project? For example, if your target platform is a Macintosh running OS X, a requirement that the DirectX graphics library be used is not doable because DirectX is a Windows only library.
5. Is this requirement unambiguous? Your requirements need to be as precise as possible (refer to the previous testable questions), because as sure as you're sitting here reading this, someone will misinterpret an ambiguous requirement and you'll discover the error the day after you ship. Your requirements should never contain the words “or” or “may.”

____________________________

Conclusion

Once you're done with your functional spec and the analysis of your requirements you're done with the requirements phase, right. Well, of course not. As we've said before – requirements change. So relax, don't obsess about the requirements, do the best you can to get an initial list of clear, testable requirements and then move on to design. You'll always come back here later.

References

Beck, K. Extreme Programming Explained: Embrace Change. (Boston, MA: Addison-Wesley, 2000.)

Brooks, F. P. The Mythical Man-Month : Essays on Software Engineering, Silver Anniversary Edition. (Boston, MA: Addison-Wesley, 1995.)

Hunt, A. and D. Thomas. The Pragmatic Programmer: From Journeyman to Master. (Boston, MA: Addison-Wesley, 2000.)

Iverson, K. E. “Notation as a Tool of Thought.” Communications of the ACM 23(8): 444–465. (1980.)

Schwaber, K. and M. Beedle. Agile software development with Scrum. (Upper Saddle River, NJ: Prentice Hall, 1980.)

Spolsky, J., Joel on Software. (Berkeley, CA: Apress, 2004.)Wikipedia, Sapir-Whorf Linguistic Relativity Hypothesis, http://en.wikipedia.org/wiki/Linguistic_relativity retrieved, September 15, 2009.