Footnotes
Chapter 01
1. If you can agree up front how much that extra 0.1 second is worth, developers can make quick decisions. See Don Reinertsen [Reinertsen01] for lean economic models.
Chapter 02
1. As suggested in the previous chapter, some agile teams work more like a soccer team or a basketball team. Players have particular strengths, but everyone plays both defense and offense. The testing unit consists of players currently acting one way—focused on testing—but the same players may be part of the developer unit that acts another way—focused on implementation.
2. There are other players, such as the stakeholder who owns the charter (see Chapter 5, “The Example Project”) and the users who use the system. Their roles will be introduced at appropriate times.
3. Particularly any team that does not use some form of test-driven development (TDD).
4. See [Shalloway01] and [Larman01] for another explanation of lean principles.
Chapter 03
1. Some people separate performance (and security for that matter) because it does not end in ility.
2. See [Crispin02] for a discussion of how to implement the other testing types.
3. Thanks to Scott Bain and Amir Kolsky for the discussion where this idea occurred.
Chapter 04
1. See Appendix D, “Tables Everywhere,” for an example of putting the rule into a table.
2. There could be even more interpretations of this business rule, as reviewers pointed out. For example, if Customer Rating is any other type than Good or Excellent, what should the discount be?
3. Fit is the Framework for Integrated Tests, developed by Ward Cunningham [Cunningham01], [Cunningham02]. Fit was incorporated into FitNesse by Bob Martin [Martin01]. Cucumber can be found in [Chelimsky01].
Chapter 05
1. Objectives are also related to management tests. See [III01].
2. This could be more detailed, giving the exact events that make up the process time, such as from a customer handing the clerk the CD to the clerk handing the customer the rental contract.
3. This was adapted from a diagram by Jim Shore.
4. See [Gottesdiener01], [Gottesdiener02], [Gottesdiener03], [Tabaka01] for more details.
5. Some teams do a high-level business value estimate to determine a high-level ROI for each feature. This ROI can help in planning.
6. Such as in Scrum [Pichler01].
7. Such as in Kanban [Anderson01].
8. If they wanted to, they could determine the business value of each feature as shown in Appendix C, “Test Framework Examples,” and use the rough return on investment as a consideration in determining the order of feature development.
Chapter 06
1. There are other roles to consider, such as system administration, operations, and help desk.
2. Some teams create stories for malicious roles. For example, “As a cheapskate renter, I want to check out a CD without paying for it.” The system needs to prevent this story from occurring.
3. There is a risk that each story is individually correct, but together, they do not fully deliver the feature. Once the acceptance criteria for individual stories are created, the acceptance criteria for the feature may be updated. Those criteria can then be turned into specific acceptance tests. These tests may be harder to specify, but there will be fewer of them.
4. Some teams credit business value to these smaller stories. It is either a separate estimate or a breakdown of the business value of the higher-level story.
Chapter 07
1. Another way is to use Event/Response, as shown in Chapter 15, “Events, Responses, and States.”
2. You can use video conferencing sites such as ooVoo.com or Skype.com.
Chapter 08
1. The structure is sometimes used for actions, such as recording that a CD is rented, particularly for lots of test cases for the same action. There is another structure used specifically for actions, which will be described shortly.
2. See [Koskela02] for other ways to run acceptance tests.
Chapter 12
1. A full deletion of a customer would require that all references to a customer (rentals, card charge history, and so on) be deleted to maintain what is called referential integrity [IBM01].
Chapter 13
1. Creating the tests for the Cumulative Rentals Level is left as an exercise for the reader.
2. If there was already a system in production with real data, Tom might try to find two customers: one who is allowed to reserve according to the rule, and one who is not. This could expose any data-dependent issues.
3. Alternatively, the system could increment the Late Rentals count when a CD was not returned by the Rental Due date. This would be a time-base event, as described in Chapter 15, “Events, Responses, and States.”
Chapter 16
1. Creating additional tests for phone numbers is left as an exercise for the reader.
2. A test process could include a step in which the customer approves the image that an application generates. That image could be used in a test to ensure that it did not change.
3. It could be generated by Hypertext Markup Language (HTML) attributes, Cascading Style Sheets (CSS) styles, or some other set of acronyms. However, a constraint might be that it uses the same framework as the rest of the system.
4. Creating requirements and tests for the details of sorting a table is left as an exercise for the reader.
Chapter 17
1. The methodology and symbols used for storing and retrieving values vary in each test framework.
2. As a resident of North Carolina, I highly prefer the two-character drop-down list. Can you figure out why?
3. Try to enter North Carolina in a drop-down that has full state names. How many keystrokes does it take?
4. For example, you might use Ajax [Riordan01].
Chapter 18
1. See [Wiki06] and [Ambler01] for details.
2. The first three tables might be part of a common setup, as shown in Chapter 31, “Test Setup.”
3. An exercise for the technical reader is to determine how the three tables can be viewed as the original single table.
4. In domain-driven design (DDD) terminology, CD is the root of an aggregate.
Chapter 19
1. There are some gray areas that might be considered either configurable or customizable.
2. Michael’s Feather’s definition of a legacy code is anything without tests [Feathers01].
Chapter 21
1. This is a DoFixture table from the Fit Library by Rick Mugridge [Cunningham01]. Slim also has a version like this [Martin03].
Chapter 22
1. These came from Amir Kolsky.
2. Any new test that passes the first time without a change in the implementation should be made to fail by briefly changing the implementation. This ensures that the test is actually testing something.
3. Depending on how the code is written, a single test might provide 100% code coverage. If you have that coverage, do you need more tests?
4. These tests may not be performed during the standard build-test cycle if they take excessive time.
5. In some cases, such as hardware chip design, it is acceptable for production code to have a built-in self-test to confirm that assumptions about the system’s environment still hold and all elements of the system are functioning correctly.
Chapter 23
1. In some complex situations, you can write an “unacceptance” test that passes with the bug. The “unacceptance” test should fail when the bug is fixed. If it does not, the changes did not totally fix the problem.
2. Other resources give detailed explanations of how to do this [Systems01] [Wiki07] [Wiki09].
3. If the “We had to get this done in four hours before release” occurs more than once, it is a common cause rather than a random event.
Chapter 24
1. It would be ideal if the terminology of both domains matched so that developers and testers did not have to switch meanings when working on the other domain. But the coordination effort required may not be worth it.
Chapter 25
1. See [Koskela02] and [Adzic01] for other change issues.
Chapter 28
1. See Chapter 15 in [Pugh01].
Chapter 30
1. The term derives from electrical engineers who used to design and build a circuit. If the circuit smoked when they turned it on, they knew immediately that something was wrong [Wiki04].
2. Tests of compilers can take a long time.
3. See [Myers01] or [Kaner01] for details on these techniques.
4. See [Adjic01] for more tester and developer issues.
Chapter 31
1. This is an example in the test world of the “Splitters versus Lumpers” prefactoring guideline for design [Pugh02].
2. The approach taken starts to define a test architecture [Rup01], which should be designed with the same care as the application.
3. This is an example of the DRY—Don’t Repeat Yourself—Principle from the Pragmatic Programmers [Hunt01]. It is also known as the Once and Only Once Principle. A corollary of this principle is Shalloway’s Law. If there are N places where a change has to be made, Shalloway will find N–1 of them.
4. A relative result is also referred to as a delta. A test that uses a relative result is called a delta test.
5. This is not the case for unit test frameworks.
6. A log of operations can include entries on both of these operations. So the side effect of logging is not cancelled out.
7. This is like what happened to George Bailey in It’s a Wonderful Life.
Chapter 32
1. If you are actually following this case study to perform your own validation, you may also want to eliminate characters that can inject SQL [Security01]. You would not allow a single quote or a forward slash.
Appendix A
1. This is similar to Alexander’s design by context [Alexander01].
2. The sequel to the TV series Lost may address this.
3. It may be tested during system operations. A classic way is through A/B (Champion/Challenger) testing [Decision01].
4. For example, FitNesse is a wiki version of Fit created by Robert Martin and Micah Martin [Martin01].
Appendix B
1. They could use story poker as described in [Shalloway01].
2. Also known as “Bang for the Buck.”
Appendix E
1. This table has a second row with values that do not appear in the original tests.