In this chapter, the authors wax poetic on practical software development methodologies, and attempt to introduce a simple, or maybe simplistic, example application for use in future chapters.
We want to share with you some thoughts on practical software development. We are not covering anything specific to Linux or Java in this chapter; there will be no commands to type or syntax to learn here. You may skip ahead to the next chapter—but at your own peril. Those who know and use good software process won’t need this chapter, but many programmers don’t fall in to that category. In fact, some who think they are using good process may be using too much or too little for their actual situation.
If you are relatively new to the corporate world, or have only worked for one or two companies in your career to date, you may get a taste of how software is done in other corporate cultures. If you are a seasoned programmer with many such experiences, see if this doesn’t sound all too familiar.
You will also see the requirements for a simple budget application that will be used in succeeding chapters. It has little to do with real budgets, but lots to do with a simple application that we can use to demonstrate various Java technologies.
Financial planning is something that everyone does. The basic tool of financial planning is the budget. But unlike the home budget, the budget of a large corporation is managed at multiple levels within the organization. Sure, at some level, the board, the CEO, and the CFO decide that “we will spend X million dollars for operations for this quarter,” but that is the start of the process, not the end.
From there, the feeding frenzy of middle management begins. And keeping track of things becomes an important aspect of financial control and corporate governance.
Realistically, any large business will already have processes and tools in place that meet this need. We are not expecting that what we develop here will be anything that a business will actually adopt to manage this process. Rather, our goal is to illustrate some methods of software development that actually work to build real Java application. The outline of methods and code could be used to address many classes of enterprise software.
There is a science of Software Engineering. The development of software can be made an engineering discipline, with mathematical rules and metrics of success. Every aspect of a complete system can be worked out in detail beforehand so that you know, well before a line of code is written, what the outcome will be.
That’s not what we’re talking about here.
We are talking about software development not as it happens at NASA, medical device companies, and in factories where nuclear missiles are made. In those contexts, the potential costs of error are extremely high, ranging from the multibillion dollar loss (and public embarrassment) of crashing a spacecraft into Mars, on through having heart pacemakers fire off incorrectly, right to ending life as we know it on this planet. In such cases, no matter how much the correct software costs, you pay it because the consequences of not doing it perfectly are far too high.
Tip
Our discussion is not meant to be scholarship on the topic of software development methodology; instead, it is meant to show simple, basic processes that can bring a reasonable amount of control to a software development project. These steps are, to name a few, requirements gathering, specification, object analysis, database design, development iteration (code, unit test, repeat), and so on.
But most of us who write software do not deal with such consequences. Most of us are keeping track of purchases and payments. We’re recording production data. We’re tracking manifests and updating inventories. We are the great unwashed mass of MIS software developers. Here we, too, want to do it perfectly right. But every time we go to management and tell them how much it will cost and how long it will take, the little “mass layoff” vein throbs in their foreheads. We are always being told to do it faster and cheaper. And so we find ourselves, again and again, tilting at the windmill of quality.
So where does that leave us? When we go to management with the textbooks of software engineering, they either laugh or scowl. Clearly, the money people are not prepared to support the cost of doing it right. So what do you do? The best that you can. The one thing we can tell you for certain is that the formula for success is not “start writing code and trust to luck.”
It is fair to say that even the minimal software development method should include the following steps:
Requirements gathering
Use case specification
Class discovery and problem domain decomposition
Technical requirements specification (architecturing)
Testing
Code and release management
Production and operations support
Bug and enhancement tracking
This list, when done in that order, has been referred to as the classic “waterfall” model—each step is done in its entirety (or largely so) before proceeding on to the next step.
Or at least that’s the ideal which programmers have often pursued.
The problem is that the process involves people, and people, especially those responsible for the requirements, a) are sometimes unimaginative and 2) keep changing their minds. They start out with some requirements, based on what they think they’re going to need. But they just aren’t imaginative enough to think of how terrible their system will be for the average user. They also keep changing their minds as to what they want.[1]
The “iterative” approach has been tried as a way to address this problem. Rather than wait for all requirements to be spelled out perfectly, with the iterative approach you jump right in with what you do know, build that, but expect changes to come. The sooner you get a working product or prototype into the hands of the users, the sooner you’ll get feedback on what works, what doesn’t, and what is really wanted (“what works” is used here not in the testing sense, but in the usability sense).
Note, however, that in the iterative approach, one still gathers requirements, develops designs for the code and the tests, develops them, tests (and fixes) the code, and releases it. It’s just that one does that on a much smaller and more rapid basis. You get something runnable sooner, and continue to modify it.
Some people will complain that this makes for more expensive rework, but we (and others) would disagree. You are refining the process. Your reworks are less expensive than if you went to the work of building the entire system only to have some key requirement(s) change—there can be a lot more “wasteage” there.
Be aware, however, that the iterative approach is not just “whipping the horses to run faster.” It is not just the waterfall model run at high speed. Rather, it is using the early iterations of the product as a sort of a “living” requirements specification, one that you can show to people and that they can try out, in real-world scenarios, and on which they can give feedback. Don’t expect to be able to compile complete requirements, but don’t give up on talking to your end users and other stakeholders either. Requirements are still key to delivering a solution.
So with either approach, you’ll start with requirements. Let’s look at the art of requirements.
A good requirement is one that states a need but not a solution. Sounds simple, but it’s easier said than done—especially with solution-oriented technical types.
A typical first cut at a requirement might be something like “Our budget application should store its data in the database.” While it sounds reasonable, it is really a solution posing as a requirement.
The first step in refining such a requirement is to ask the simple question: “Why?” The answer we’re looking for is not “Because we’ve paid so much for our database software,” nor is it “Because we all know SQL.” Rather, it should be something dealing with reliability, fault tolerance, the need for transactional integrity, and so on.
Sometimes you may have to ask the “why” question more than once, to refine the requirement(s). “Transactional integrity” is, in a way, a solution. You could ask, “Why do we need that?” For some projects it may be appropriate to ask this, because there may not be a real need for it after all.
But don’t overdo it. Push any requirement in a business setting far enough, and you could get something like “To make money.” That’s not a helpful requirement. You’ve gone too far. Part of the art of requirements is recognizing when to stop asking why.
A more detailed description of a requirement is that it should be SMART—Specific, Measurable, Attainable, Repeatable, and Testable. Consider the following.
A common concern among users of almost any application is that it be “fast” or “responsive.” While we can sympathize with the concern, it will need some refinement before it can be considered a (good) requirement. Applying the “Specific” and the “Measurable” aspects of SMART, we need to specify what constitutes “fast enough.”
We can try “No button press in the GUI will delay more than .1 second before providing some evidence of activity to the user, or more than .5 second before completing its operation.”
Sounds more formal, and more specific, but is it realistic (i.e., attainable)? If the “button press” is one that updates a database across a network, what effect will network traffic have? What about the size of the operation? If the button press starts an operation that is dependent on the size of some data set, what’s the largest it could be and how long will that take?
Depending on how obsessive you or some colleague will be in enforcing these requirements, you would do well to add a few “weasel words” to give you some flexibility in the requirements. Phrases like “on average” or “most” will help. Notice, though, that such words are also the cause of much ambiguity, working against the “Specific” and “Measurable” aspects of good requirements. Use them sparingly, if at all.
We should also consider the “testable” aspect of our requirement for speed. Will we be able to measure this? Can we do so repeatedly? Consider the effect of network traffic on response times. Under what network load will the tests be done and the real usage occur? If you want to test under “normal” network loads, how can you control this (for the sake of repeatability)?
It really is an art to craft good requirements. Moreover, a good requirement for one organization may not work well for another. Some teams, groups, or companies want to be very precise in their use of requirements, viewing them almost like legal contracts for what will be delivered. Such requirements, however, would be greeted with derision in other, more informal, organizations. It’s not that the one will produce good software and the other garbage (well, they might). It’s more a matter of style. Excessively formal organizations will drown in the details and spend way too much time (and money) arguing over the minutiae of the requirements. Overly informal groups will get sloppy with their requirements and not reap the benefits of building the right thing the first time. As is so often the case in life, the answer lies in striking a balance between two forces, one pushing for exactitude and the other pulling you to get going and do something.
So let’s keep going.
There are many people to ask about the requirements for a software project or product. Ask yourself the following questions:
Who is going to use the software that you develop?
Who is going to use the data that comes from the use of the software (i.e., who will read the reports generated from the data collected either directly or indirectly from the running of the software)?
Who is going to support the software and who will support the machines on which it will run?
All these people can be considered “stakeholders” in the project.
So where do you start? That’s a political more than a technical question. Start with your boss and with whoever is the major backer of the project. Then ask your customers. For in-house IT projects, the “customers” are usually very accessible; for software products, the customer’s point of view may need to be represented by marketing and/or customer support people who have had direct contact with the customer base.
Let’s take a look at how such requirements might evolve. We’ll look at the situation through the eyes of a fictional IT guy named Bob.[2]
Bob gets called in to the office of his manager, Ellen. The conversation goes something like this:
Bob: Yes, Ellen, you wanted to see me?
Ellen: Come in, Bob. Yes. We’re just about to enter another budget planning cycle. We’ve got to propose our next year’s budget to the VP by the end of the quarter, and I got to thinking ...
Bob: Uh-oh.
Ellen: ... on my way to work today, I got to thinking that we ought to be able to develop a software tool that would help us do a better job of this process.
Bob: We’ve used a spreadsheet these past few years to do our budgets. You want us to develop another spreadsheet application?
Ellen: No, I want a whole new application.
Bob: You want us to reinvent the spreadsheet?
Ellen: No, I want something simpler and more specific to the budgeting process.
Bob: Tell me more. What are the key features that you see in this application?
Ellen: Well, first of all it needs to be able to work concurrently with all the users. With our spreadsheet, we’d have to take turns with the data entry or we’d risk loosing each other’s changes.
Bob: It may just be that we’re not using our spreadsheet’s advanced features. Shouldn’t we investigate that first?
Ellen: No, I’d rather have us invest our time in building the tool we know that we need. At the end of the day your investigation may only show that we still need the tool, and by then it might be too late to build it.
Bob: I hear you saying that the deadline is rapidly approaching.
Ellen: Yes—I want to be able to use it for the budget planning at the end of this quarter. How long do you think it will take you to build it?
Bob: Build what?
Ellen: Haven’t you been listening? The budget tool!
Bob: I know that you mean the budget tool—but you haven’t really given me enough requirements upon which to base an estimate. Tell me more about how you envision this tool being used.
Ellen: Well, in the past we’ve taken last year’s numbers and just bumped them up by a few percent. Then we look at each category and tweak them. I want a different approach this year. I’m going to take our department’s budget, give it a bump, then assign a chunk to each of my reports. I want you to take those discretionary dollars and spell out how you would spend them.
Bob: Shouldn’t we be providing you with estimates of what we need for the coming year, rather than you telling us what we have to spend?
Ellen: In theory, perhaps so. But in practice we can only grow the budget by so much. I’d rather skip the charade and jump right to allocating the dollars we will likely be able to spend. Then as the year progresses, I’d like to use this tool to track our spending against this plan.
Bob: But isn’t that why we have that big SAP application?
Ellen: Have you ever tried to use it?! Please! The CFO thought it looked great—and on paper it did. But that user interface makes it almost impossible to be productive. And it’s as slow as molasses.[3]
Bob: But back to this new application ... I’m assuming you’ll want a GUI on this?
Ellen: Of course. Give it a standard, simple GUI. Something like this. (She begins to draw on her whiteboard.)
For any given department there will be a “pool” of money. Those dollars are displayed and can be subdivided into smaller pools of money by creating subaccounts.
But as the money is subdivided those new accounts and associated dollars should become visible by others. And as dollars are spent during the year, we’ll want to track those dollars, so those amounts should be visible, too, and subtracted from the overall pool of available dollars.
Bob: Wait ... back up. What needs to be entered to subdivide an account?
Ellen: The user just picks an account, then chooses to subdivide it, entering the amount to put in each account ... or even just a percent of the larger pot of money.
Bob: So if he picks one account to subdivide, does it split into two, or three or how many?
Ellen: Let the user choose, but maybe two as a default.
Bob: OK, but we may need to take a harder look at that interaction.
Ellen: So how long will that take? Can you have it ready by the end of this month?
Bob: I’d like to try the “spiral” approach on this project. I can have something for you by the end of this week—from which you can tell me if I’m heading in the right direction. It will just be a beginning, but you’ll be able to see something run. By the way, is this tool only for our group?
Ellen: For now it is, but I could see other departments wanting to use it some day. Who knows how far it could go?
Bob is now back at his desk pondering the conversation he had with Ellen. “These are not like the requirements we learned about in my software engineering courses,” he muses. “I’ve got that sketch of the UI and a brief description of its functionality. But there seem to be so many unanswered questions.”
So what is Bob supposed to do? He could go back and try to get more “face time” with Ellen, and ask lots more questions. Sometimes that’s a smart thing to do. Other times such repetition is seen as annoying and a sign of a slow-witted analyst, never mind how obscure the initial discussions were or how many times someone changed their mind about what they want. You will have to judge each situation as you encounter it. At some point, though, you have to deal with whatever information you’ve been given, and try to make the best of it.
So where do you turn? The next best things to do are to begin to document the requirements as you understand them, to prototype a solution, and to start getting buy-in from other stakeholders. Each of these activities may help bring out more requirements, but that’s not a bad side effect.
Once a project is started, the design must be documented. A prototype may be built to validate and refine the design. Finally, everyone with a stake in the success of the design has to be brought up to speed and needs to agree on what is to be built.
After such a conversation, it’s smart to try to get your thoughts down on paper as soon as possible. Some of what gets said will fade with time, so work quickly to capture what you can of the requirements that were spoken. Even if you have to leave lots of blanks, keep moving and get as much of the major requirements written down as you can, even if they don’t sound very formal or polished. Then go back, revise and edit your statements, filling in the blanks where you can. Sometimes you will need to ask others to get the answers to fill in the blanks. Other times you can use your own judgment and initiative to provide an answer. Out of this process with its subsequent rewrites will come the requirements document.
Some organizations are very formal in their understanding of requirements. They will have company-standard formats which you must follow. But there is no magic format that will make for good requirements. It really all comes down to content.
Here’s an informal list of the requirements for the budget application, based on the conversation between Bob and Ellen.
Features:
Starts with a single lump sum of dollars.
How does this first sum get entered?
Each dollar amount is associated with an “account.”
Any account may be divided into two or more subaccounts.
The dollar amount associated with a subaccount is specified either in absolute dollars or as a percentage.
What if they don’t add up?
Can the user mix $ and %?
Can the user leave the last subaccount’s amount blank for “remaining dollars”?
Tracking of the dollars—not enough info, so not in first prototype.
Multiple users will have access to the data.
Concurrent use is allowed and supported.
Short development time, limited resources.
Has a graphical user interface; earliest versions may be command-line and terminal interaction.
Not all requirements will be easily forthcoming; not all can be traced back to an exact quote from the previous discussion. Other requirements will need to be inferred from the discussion or from department “culture,” or come from your own judgment:
Platform: “any” PC in Ellen’s department—but her developers are all using Linux platforms.
Future platforms: “any” PC in the company means any Windows, Linux, or Mac OS X.
Reliability: once entered, data is never lost.
Maintainability: the application must be easy to maintain.
Interoperability: there’s no requirement to interoperate with any other software but here’s an idea for a future version: export/import into CSV format for spreadsheets, and/or XML format for future expansion).
Response time: “reasonable” interactive speed; subsecond response when entering new accounts and values, so that the user can type quickly and continuously; waiting, if it occurs, should only be at button presses, not between data entries.
Stakeholder buy-in can be another important part of a software project. As we discussed in Section 11.5, stakeholders are any of those people who are touched in some way, direct or indirect, by this software project.
For this simple budgeting program, there will be few stakeholders—it will largely be Ellen and her direct reports. The system will not likely be a large drain on computing resources, so system admins don’t need to be brought in at this point. If and when the project expands to include other users across the network and across the enterprise, then the system administrators should definitely be included. There will be few reports from this first cut of the project, and what few there are will only be read by Ellen and her direct reports, so again, there are few others that need to be consulted as stakeholders.
The idea at this stage is to listen to other points of view—those of your stakeholders—to get a different perspective before charging headlong down one avenue of development.
It’s not that you will be able to satisfy all points of view—it can be a worthy goal, but it is often unattainable. Rather, you need to hear from all those involved since your software will affect all those people, and understanding something about how it will fit into their roles and daily tasks will help you make better tradeoffs and design better software. It will likely uncover previously unseen requirements. It also has the political benefit of those people knowing that you cared enough to listen to them before sending them a finished solution. It increases the likelihood that your software will be seen as a help, not hinderance.[4]
Prototyping can be an effective way to carry on the discussion of both requirements and user interface design. Given only a hypothetical or abstract description of some software, it can be very difficult for people to imagine what the implications of its use will be. A simple prototype can immediately bring the discussion down to the concrete; people can point at things and say “I like this” and “I don’t like that” and “How would I do so-and-so” and then see whether or not it would work. Sometimes, ideas that sound great on paper turn out to be pretty poor ideas when realized. Prototypes can help you discover that quickly and easily.
One very useful but inexpensive prototyping mechanism can be HTML—that is, creating Web pages. Simple static HTML can be fast and cheap to build, but can begin to approximate what the user interaction will look like—especially for, but not only for, Web-based solutions. It may not be an exact replica of the final product, but for a first step it can really get the discussion moving.
If the UI is too complex for a Web page mock-up, you can still use HTML for prototyping by getting images (screenshots) of what you want your final solution to look like and then making these images clickable on Web pages, to simulate some simple user interaction with hyperlinked image sequences.
The idea is to get something “tangible” in front of people as soon as possible, to further the discussion in a way that written descriptions never can. (“A picture is worth a thousand words.”)
Once you’ve built a prototype, shop it around. Hold informal meetings where you demonstrate the basic functions to stakeholders. We recommend, as much as possible, meeting with one group of stakeholders at a time. That way you can keep your conversations focused. If you have two different stakeholder groups represented and their expertise and interests are wildly different, you’ll be boring ½ the participants all the time. Even if their expertise is similar, you may have groups with competing or conflicting requirements. While you need to understand such conflicting requirements and eventually come to some detente, this meeting is not the best venue for settling those issues; it would more likely simply scuttle your meeting and void any value from it.
After each meeting, review your requirements and see what more you need to add. Likely at such meetings, you’ll begin to get requests for new features.
You have, in fact, begun the iterative process. Even the most bare-bones prototype that may only consist of a sequence of pictures is a first cut of your product. The sooner you can get to a running version, the sooner you will be able to respond to stakeholder suggestions by adding real features.
A good requirement is one that states a need but not a solution. Your first step is to uncover the needs, while listening to everyone’s solutions. These requirements will develop into feature descriptions. These should be documented and then prototyped. The prototype, which is in effect the first release of your product, can then be shown to various groups—stakeholders—as a way to elicit their feedback. This feedback should begin to factor in to what you will build, so now you need to move quickly on to building the real product; do not get stuck enhancing the prototype.
Writing good requirements is as much art as it is science, and it involves political science as well. This is not something easily taught in a book, but learned through hard experience.
One of the original purposes of the World Wide Web was to allow researchers to share their results. So, you should be able to search the Web for requirements documents from various projects for examples of requirements specification. As with any Web search, remember to consider your source. Just because someone has posted a requirements specification or a template doesn’t make it a good example.
Here are three examples that we found on a single simple Google search. They may still be there by now.
For those who are serious about their software development process, the Capability Maturity Model for Software from the Software Engineering Institute at Carnegie Mellon University is the standard. Visit their Web site at http://www.sei.cmu.edu/cmm/.
If you would like to know more about the spiral approach to software design, you might want to start with the seminal paper on the topic, “A Spiral Model of Software Development and Enhancement,” in Computer 21, no. 5 (May 1988), pages 61–72.
To see how the director of the Software Engineering Institute views the spiral approach, check out the short and readable introduction at http://www.dacs.dtic.mil/awareness/newsletteres/technews2-1/disciplined.html.
Another good look at the spiral, or iterative, approach can be found at http://www.stickyminds.com/se/S3420.asp which has a hyperlink for a PDF file of a paper by Philippe Kruchten of Rational Software. The paper covers some pitfalls common to the first uses of the iterative approach; worth the read.
A great survey of key papers on three major approaches—spiral and related topics (including newer work by Boehm), aspect-oriented programming (AOP), and the rational unified process—is at http://www.rspa.com/reflib/PrescriptiveModels.html.
[1] Did you notice that we tried to hint at that ever-enjoyable mid-project shifting of requirements as we went from a) to 2), changing our numbering scheme midway? Minimal humor, admittedly, but if you’ve lived it, you understand.
[2] We’re avoiding giving Bob a title because titles vary so much within our industry. Call someone an analyst and it may mean that they never code. Call someone a programmer and it may mean that they only code and never deal with requirements or even designs. Some use those terms interchangeably. We’ll just call him an IT guy.
[3] Remember, this is a fictional account. We are providing justification for why they can’t use the corporate application. Anyone’s use of such a tool can be less than optimal, reflecting more on themselves than on the value and usability of the tool.
[4] As engineering types it is difficult for us to understand and appreciate the importance of this, but in many ways these personal, political, and psychological factors are much more important to the success of a project than are technical choices. It has taken us years to appreciate that Dale Carnegie is as important to the software designer as Yourden or Booch. Your users need to be your friends if you want to succeed.