Chapter 12. Development Review

“May you have the hindsight to know where you have been, the foresight to know where you are going, and the insight to know when you have gone too far.”

Anonymous

The triad demonstrates to Sam the current state of the system. Debbie and Tom recount other stories that have been completed and tests that have been run.

The Rest of the Story

The triad meets with Sam. Cathy begins, “Sam, as sponsor, I’d like to update you on the current status of the system. I’ve shown you some of the user interfaces and tests as we’ve gone along. I’ve worked together with Debbie and Tom on the acceptance tests for Check-Out, Check-In, and Credit-Card Charging. I reviewed the other stories and acceptance tests they came up with. I’ll let Debbie tell you more about them.”

Debbie begins, “Because the system deals with customers and CDs, we needed a way to add, update, and delete customers and CDs. We had to add customers and CDs to test check-in and check-out. But we didn’t go into the stories themselves.”

“Adding, deleting, and updating data is so common that Tom and I know it needs to be done. We just create stories for those actions and let the customer schedule them. If the customer has the time, we work through the details with him. Otherwise, we just review the tests with him.”

“For adding a customer, we checked the business-related decisions with Cathy. For example, you want to ensure that a customer isn’t accidentally added twice. So we asked Cathy for a business rule to determine if you have two customers who are duplicates. She said the rule should look for the same credit-card number. This duplicate rule is also used when updating a customer.”

“Another action was to a delete a customer. Cathy said not to completely delete customers, but to deactivate them since they may still have outstanding rentals. She said she wanted to keep a record of previous customers so it would be possible to welcome them back or have on file that you don’t want them back.¹ We asked for similar rules for adding, editing, and deleting CDs.”

The acceptance tests Cathy approved were run through both the middle tier and the user interface. Talking about the user brings up the usability issues that Tom will talk about.”

Usability Testing

Tom starts, “Debbie described acceptance tests for functionality. There are also usability tests; quality attribute tests, such as security and performance; and exploratory tests [Chapter 3, “Testing Strategy”].”

“Debbie and I worked with Cary, Harry, and Mary on the usability of the check-out and check-in screens. Harry is color blind, so he couldn’t distinguish that messages displayed in red and green had different meanings. So we added textual indicators to the messages to clarify whether they meant “This is a problem” or “This is okay.” Mary doesn’t want to use her glasses to read the screen, so we increased the size of the font. These examples represent issues that we often find in usability testing.”

“We talked to a couple of your customers to see if the rental contract was readable. The wording that your lawyer approved seemed a bit obfuscated.” Sam interjects, “I think talking about lawyers and obfuscation is redundant.” Tom replies, “Agreed. And so did the customers. So we worked on a simple language contract, which is still undergoing review by the lawyer” [ABA01].

Separating State from Display

Tom continues, “We’ll talk more about the concept of separating display from state later [Chapter 14, “Separate View from Model”], but because we’re talking about what the user sees, I think it’s appropriate to introduce the idea now. A while back [Chapter 9, “Scenario Tests”], we presented to Cathy the idea of separating the form of input from the internal logic. For example, whether the input is from typing, the scan of a barcode, or the reading of an RFID tag, the middle-tier tests should not be affected. This separation makes for less test maintenance.”

“A similar issue applies to separating a state from the way it is displayed. As you can see for tests for check-out, we listed the error messages as CD_Rental_Limit_Exceeded. For example:”

Tom resumes, “Debbie coded this test with a reference to an identifier, such as CD_Rental _Limit_Exceeded. When Cathy decided what should appear on the screen, we put that into a separate table, as follows.”

Tom continues, “These two tables allow for separation of testing. One test verifies that the system produces the right state. The other test confirms that, given the state, the output is as is desired. It also allows testing the system even when the final wording has not been approved.”

“Because the contract wording has not yet been agreed upon, the test for the Rental Contract [Chapter 8, “Test Anatomy”] just verifies that the data on the contract is correct. Now we have a separate test for the printed contract:”

Debbie injects, “When the lawyer approves the wording, we need to change this test.”

Quality Attribute Tests

Tom resumes, “I often do extensive performance testing on stories. Just as with the other acceptance tests, I make up a table for the desired behavior. For example, if Sam is expecting to have many checkout people other than Cary, Harry, and Mary on the system simultaneously, I would make up a table such as what follows. As the stories are developed, they are checked to see if they meet the performance measures.”

Tom continues, “Security is an important area for testing. You need to ensure both physical security of the system and software security. You don’t want a customer to go behind the counter and check-in a CD that he really isn’t checking in. We have name/password security on the screens, but based on usage patterns, you may need to change the means and timing for that verification. You could have a logon at the beginning of the day or before every rental. You could have software verification, or you could have employee cards.”

“Because you are keeping credit-card information, Debbie discovered that you need to abide by the PCI Data Security Standard. So we’ll need tests to ensure that each of the requirements in that standard is met” [Security01].

“Security is such a broad issue that I can’t really get into much more detail in a limited amount of time. Suffice it to say that I can test a system to see if there are known security issues, but I can’t test to make sure that it is absolutely secure. Security is not about letting people do things; it’s about making sure they can’t do things. It’s easier to test the former than the latter.”

“The entire team can try exploratory testing [Chapter 3] on the system. Each member takes on the role of a different persona. Each performs the operations related to that persona and sees how the system feels. Issues may be discovered that do not come out in our predetermined tests. Because the system is in a runnable state and we’ll keep it that way as we add features, exploratory testing can continue throughout the project.”

Workflow Tests

Debbie starts, “Just because we have tests for each story does not mean that the system is fully tested. We need to have a test for an entire workflow. The workflow can correspond to a story map [see Chapter 11, “System Boundary”] or a set of story maps. The workflow test verifies that there are no issues between related stories and that the entire flow is usable. Here’s an example of a workflow test (see Figure 12.1). If the workflow was really complicated, we might have multiple workflow tests that go along alternative paths.”

Figure 12.1. Workflow Test

Debbie resumes, “Because this test involves many facets of the check-in process, it can break for a variety of reasons and may have to be maintained frequently. For example, if the wording of the rental contract changed, the rental rates changed, or other things changed, the test would have to be rewritten or it would fail. So we only want the essential workflow tests.”

Deployment Plans

Sam speaks up, “Cathy has been keeping me in the loop about your discussions. So let me ask the bottom-line question: How quickly can you get the system into operation? Other than the stuff you’ve already outlined, what else do you need to do?”

Debbie replies, “We already have a way to get customers and CDs into the system. We don’t have all the data. If we had a spreadsheet with all the data, we could input it into the system.”

Sam states, “I’ll have Cary, Harry, and Mary start inputting all that information. What else?”

Debbie answers, “You’ll need a transition plan to go from the old system to the new one. You could start with doing rentals both ways for a little while, or doing all new rentals in the new system, or doing rentals in the new system for just a few customers.”

Sam says, “Let’s work on that in a little while. Is that it?”

Debbie replies, “Anything else you can think of, Tom?” Tom responds, “I think that’s it.”

Sam states, “Then let’s go for it. The sooner this starts to happen, the quicker we can start renting more CDs with the same number of people and thus make the money to pay you for all your hard work.”

Cathy injects, “I think it’s time to look back at the objectives. The one we’re dealing with right now is “Within two months, CD check-outs and returns will be processed in 50% less time.” From what I’ve seen so far, it looks like that will be met. The manual system measurement was 1 minute, 40 seconds for a check-out and 55 seconds for a check-in. The preliminary measurements using the user interface were 46 seconds and 26 seconds, respectively. But as you say, the sooner we get it working, the sooner we’ll find out.”

From Charter to Deliverable

The triad (Cathy, Debbie, and Tom); Sam the sponsor; and the users Cary, Harry, and Mary have gone from the initial project charter through the first deliverable. Along the way, acceptance criteria have been created as objectives in the charter through the high-level features and stories. The criteria have been expressed as specific tests. Cathy now understands the importance of helping to create acceptance tests. Debbie and Tom have learned about the business domain. The more they comprehend the domain, the more effective the triad will be in producing quality software.

Summary

• It is insufficient to have just acceptance tests that revolve around the functionality of a system.

• Acceptance criteria need to be established for usability, security, performance, and other quality attributes.

• Workflow tests catch inter-story issues.

• Developing in deployable chunks allows for quicker cost recovery.