A1:

If Dave was a good programmer he would have questioned the “obvious” oversight and not just programmed his software to only work through 1999. Because he didn't, a software tester should have tested for and found the bug. The team could have then decided whether to fix it.

A2:

False. It's not important, but the term used often reflects the personality of the team and how they approach the finding, reporting, and fixing of the problems.

A3:

At most, that's only half the testing problem. Users don't always follow the rules, and testers need to prove out what happens when they don't. Also, if testers don't approach their testing with a gotta-break-it attitude, they will miss bugs.

A4:

From 10 to 100 times or even higher!

A5:

The goal of a software tester is to find bugs, find them as early as possible, and make sure they get fixed.

A6:

False. A good tester knows when perfection isn't attainable and when “good enough” is reached.

A7:

Often a spec isn't even written—remember, if you can't say it, you can't do it. Other reasons are that the spec exists but it isn't thorough, it's constantly changing, or it's not communicated to the rest of the development team.

A1:

The development team needs to understand the customer requirements and define the features in a product spec. A schedule should be created so team members know what work has been completed and what work remains to be done. The software should be architected and designed, and the test team should start planning their work.

A2:

There's no flexibility to adjust the software if the customer's needs change or the market moves in a different direction while the software is being developed.

A3:

It's simple. Period.

A4:

There's no real exit criteria for the code-and-fix process except for when someone, or the schedule, says that it's time to stop.

A5:

Just like with salmon, it's difficult to swim upstream. Each step is a discrete, standalone process that follows the one before it. If you get to the end and find that something should have happened further up, it's too late to go back.

A6:

They're involved very early in the development process and have the opportunity to find problems early, saving the project time and money.

A1:

There is no universal answer for when to stop testing. Each project is different. Examples of the information that might go into the decision would be: Are lots of bugs still being found? Is the team satisfied with the number and types of tests that have been run? Have the reported bugs been evaluated to decide which ones to fix and which ones not to fix? Has the product been validated against the user's requirements?

A2:

The answer you get is 0, not 4095 as you would expect. The reason is that the , (comma) was automatically converted to a . (decimal point). What you calculated was 5.000-5=0, not 5,000-5=4995. To determine if this is a bug, verify this operation against the product specification. It might state that commas are to be converted to decimal points. You should also validate this “feature” against the user requirements. Find out if most users want this to occur or if it would be confusing.

A3:

The purpose of a simulation game is to put the player into an artificial environment that mimics a real-world scenario. Flying a flight simulator should look and feel like flying a real plane. Running a city simulator should reflect what happens in a real city. What's most important is how accurately the simulator reflects reality. Does the plane fly like a Boeing 757 or Piper Cub? Does the skyline of the city look like the city it represents? Once you know the software is accurate, you can then test for precision. This is where you start caring about whether the windows in the buildings are in their precise locations and whether the plane movement perfectly maps to the joystick motion.

A4:

Yes, but it depends on the customer's expectations for quality. Many people buy high-performance sports cars that are considered to be high quality for their acceleration and speed, their style, and their fit and finish. Often these same cars are notoriously unreliable, frequently breaking down and being expensive to repair. The owners have their own perception of quality and don't see low-reliability as an important issue.

A5:

With any software other than the smallest and simplest program, there are too many inputs, too many outputs, and too many path combinations to fully test. Also, software specs can be subjective and be interpreted in different ways.

A6:

Two axioms come into play here. The first—that the number of bugs remaining is proportional to the number of bugs you've already found—means that you won't come in on Tuesday and miraculously find the software to be perfect. The pesticide paradox tells you that if you continue to run the same tests over and over that you eventually won't find new and different bugs until you add more tests. Given these two axioms, you will probably continue to find bugs at the same rate or slightly less.

A1:

Yes, white-box testing is simply using information about how something is designed to influence how the testing is done. The tester could attend the project's focus groups, usability studies, and marketing meetings to understand the underlying process being used to design the features and the overall product. With that information she could more effectively test the resulting specification. There is a risk, though, that this information could bias the tester into assuming that the spec is correct.

A2:

On the Mac, deleted files go to the Trash Can. In Windows, they go to the Recycle Bin.

In Windows, pressing F1 always displays Help for the software. It's Command-? on the Mac.

The File menu is always the far-left menu choice in Windows.

In Windows, selecting Help, About displays the software's copyright, licensing, and version information.

On a Mac, Command-X performs a cut, Command-C performs a copy, and Command-V performs a paste.

These are just a few examples of a wide variety of possible answers.

A3:

It uses the phrase as small as possible. This is impossible to test because it's vague and imprecise. The spec should state exactly what level of compression will take place.

The statement also isn't code-free. It explains how the feature will work on the algorithm level. This doesn't belong in a specification. The user doesn't care how the compaction occurs, just that it does.

A4:

Testability. It doesn't matter that typical usage is only 1 million connections. If the specification states that 100 million are possible, the 100 million must be tested. The tester needs to find a way to test something this large or get the spec writer to reduce the maximum possible number to something closer to what's typical.

A1:

True. The technique is called exploratory testing, and you essentially use the software as though it's the product spec. It's not an ideal process, but can work okay in a pinch. The largest risk is that you won't know if a feature is missing.

A2:

You could attempt to print with no paper and with jammed paper. You could take the printer offline. You could unplug its power. You could disconnect its cable. You could try printing with low ink or toner, maybe even with a missing cartridge. To identify all the possibilities, you could look in the printer's operator's manual and see what errors it supports and attempt to create all of them.

A3:

If you select the Pages option, the From and To fields are enabled. The obvious boundary conditions would be 0 and 99999, the smallest and largest values that the fields will hold. It would be wise to also test internal boundary conditions such as 254, 255, 256 and 1023, 1024, and 1025. There's also another internal boundary condition. Try asking it to print pages 1 though 8 of a 6-page document. Notice that in this case, the software has to stop printing at page 6 because it's out of data, not because it was told to stop at this page. It's a different, internal boundary. See if you can think of more.

A4:

You should have at least these four equivalence partitions, although you may think of more:

A5:

False. Think of visiting 50 different cities spread out across the entire United States You could plan a trip that would take you to each city, but it would be impossible for you to travel all the roads that connect to all the cities—that would be all the roads in the entire country.

A6:

A7:

The initial displayed value and the internal partial result are set to 0. The memory register (MC, MR, MS, and M+ buttons) is set to 0. The Clipboard contents (where data goes when you cut, copy, and paste) is left unchanged.

Another initial state variable is where the Calculator appears onscreen when it's started. Open up several copies of the Calculator and notice that the location isn't always the same (at least on Windows 95/98). As an exercise in exploratory testing, see if you can figure out what the rules are for where the Calculator opens.

A8:

You try doing multiple things at the same time. They could be related, like printing to two printers at the same time from the same application. Or, they could be unrelated, like pressing keys while a computation is taking place. What you want to do is force a situation where the software is competing (racing) with itself to perform some function.

A9:

False. No test is ever unfair. Your job is to find bugs. But, the realities of software testing may apply and any bugs you find in this situation may not be fixed.

A1:

Static white-box testing finds bugs early in the development cycle, making them less time-consuming and less costly to fix. The software testers can gain information about how the software works, what potential weaknesses and risky areas exist, and can build a better working relationship with the programmers. Project status can be communicated to all team members who participate in the testing.

A2:

True. Missing items are arguably more important than normal problems and can be found through static white-box testing. When the code is checked against published standards and guidelines and carefully analyzed in formal reviews, missing items become obvious.

A3:

Process. Having a process that's followed is what makes the difference between a formal review and two pal programmers glancing over each other's code.

A4:

The key difference is that with inspections, a person other than the original author of the code is the presenter. This obliges another person to fully understand the software being inspected. It's much more effective than having others simply review the software for bugs.

A5:

That would be a standard. If he was told that a name of more than eight characters was preferred, that would be a guideline.

A6:

No! It's provided as a generic example only. There are some good test cases in it that you should consider when you test your code, but you should research and read about other published standards before adopting your own.

A7:

Data Reference. They are caused by using variables, constants, arrays, strings, or records that aren't properly declared or initialized for the data that they can contain.

A1:

If you test only with a black-box view of the software, you won't know if your test cases adequately cover all the parts of the software nor if some of the cases are redundant. An experienced black-box tester can design a fairly efficient suite of tests for a program but he can't know how good that suite is without some white-box knowledge.

A2:

Both processes overlap, but the goal of dynamic white-box testing is to find bugs and the goal of debugging is to fix them. The overlap occurs in the area of isolating exactly where and why the bug occurs.

A3:

With the software delivered to you in one big piece, it's difficult, if not impossible, to figure out why a bug occurs—the needle-in-a-haystack problem. The second reason is that there are so many bugs, some hide others. You'll grab one and shout “gotcha” only to discover that the software is still failing.

Methodically integrating and testing the modules as they are built allows you to find and fix bugs before they start to become hidden or pile on top of each other.

A4:

True. You can, but you shouldn't. Doing so will cause you to miss bugs that could have been found earlier. Skipping or postponing testing generally makes your project take longer and cost more to complete.

A5:

A test stub is used in top-down testing. It stubs out, or substitutes itself, for a low-level module. It looks and acts to the higher-level code being tested just like the original low-level module.

A test driver is the opposite of a stub and is used in bottom-up testing. It's test code that replaces higher-level software to more efficiently exercise a low-level module.

A6:

True. Design your test cases based on what you believe the software is supposed to do. Then use white-box techniques to check them and make them most efficient.

A7:

Condition coverage is the best because it also incorporates branch coverage and statement coverage. It tests that the conditions within decision logic, such as if-then statements, are verified, as well as branches from those statements and the lines of code.

A8:

You can easily become biased. You might look at the code and say, “Oh, I see, I don't need to test that case, the code handles it properly.” In reality, you were blinded by the light and eliminated necessary test cases. Be careful!

A1:

Generally, a component is a hardware device internal to a PC. A peripheral is external to the PC. The lines can become blurry, though, depending on the type of hardware.

A2:

Rerun the exact same steps that revealed the bug on several different configurations. If the problem doesn't occur on those, it's very likely a configuration bug. If it occurs on the different configurations, it's likely a general problem. Be careful, though. It could be a configuration problem across an entire equivalence class. For example, it's possible that the bug shows up only on laser printers, but not inkjet printers.

A3:

This is sort of a trick question. You'd need to ship the hardware and software together as one package, the software would only work on that hardware, and the hardware would have to be completely sealed, not having a single interface to the outside world.

A4:

It depends. Typically, a cloned hardware device is mechanically and electrically identical to its brothers and sisters. Often they are 100 percent functionally equivalent, but sometimes the firmware or device drivers are different, allowing one device to support additional or different features than the others. In the case of this sound card, you would need to research how different or similar the sounds cards are that you plan to test before deciding on your equivalence classes.

A5:

Region or country is a possibility as some hardware devices such as DVD players only work with DVDs in their geographic region. Another might be consumer or business. Some hardware is specific to one, but not the other. Think of others that might apply to your software.

A6:

Yes, probably. You'll never be able to fix all of them. As in all testing, the process is risk based. You and your team will need to decide what you can fix and what you can't. Leaving in an obscure bug that only appears with a rare piece of hardware is an easy decision. Others won't be as easy.

A1:

False. There will be a few rare, standalone, proprietary first versions of software out there that don't interact with anything. For the other 99 percent of the world, though, some level of compatibility testing will be necessary.

A2:

True. The level of compatibility that your software has is based on your customers' needs. It may be perfectly fine for a word processor to not be compatible with a competitor's file format or for a new operating system to not support a certain class of gaming software. As a tester, you should provide input to these decisions by determining how much work would be involved in checking that compatibility.

A3:

Research whether your program follows existing standards for its files. If so, test that it meets those standards. Equivalence partition the possible programs that would read and write your program's files. Design test documents with representative samples of the types of data that your program can save and load. Test the transfer of these files between your program and the other programs.

A4:

Testing forward compatibility is tough—after all, how can you test against something that doesn't exist yet? The answer is to make sure that what you're testing is thoroughly and carefully defined to the point that it could be deemed a standard. That standard then becomes the means for assuring that what you're testing is forward compatible.

A1:

Translation is concerned only with the language aspects—translating the words. Localization takes into account the customs, conventions, and culture of the region or locale.

A2:

No, but someone on the test team needs to be fluent. You can test the non–language-specific portions of the software, but knowing a bit of the language will help you be more efficient.

A3:

Text expansion can occur when text in one language is translated into another language. The length of text strings can grow 100 percent or more. Text that used to fit onscreen, in dialog boxes, in buttons, and so on no longer does. It can be truncated or cause other text to roll off. It's even possible to have the software crash because the extra long text no longer fits in the memory set aside for the string and other memory is overwritten.

A4:

The order of sorted or alphabetized words and phrases, conversion between upper- and lowercase, and just general display and printing issues.

A5:

Localizing becomes much easier if the person doing the work has to modify only a text file rather than the programming code. It also makes the testing work easier because you'll know that the code didn't change on the localized version of the software.

A6:

Measurements such as pounds, inches, and gallons. Time in 24-hour or 12-hour format. Currency has recently become more important now that many European countries have converted to the Euro. There are many others.

A1:

True. Eventually, even the most deeply embedded software is exposed, in some way, to a user. Keep in mind that the UI may be as simple as a switch and a light bulb or as complex as a flight simulator. Even if the software is a single module in a code library, its interface, in the form of variables and parameters, is exposed to a programmer—who can be a user, too.

A2:

It's a little bit of both. Many user interface designs have been thoroughly tested in the labs, been through rigorous studies, only to be complete failures in the marketplace.

A3:

Software testers should check that it meets seven important criteria: That it follows standards and guidelines, that it's intuitive, consistent, flexible, comfortable, correct, and useful.

A4:

This answer will vary based on the products you use but think about these: Try setting the time on your car radio's clock—can you do it without using the manual?

Most Windows dialog boxes have the OK button on the left and the Cancel button on the right, whereas others have Cancel on the left and OK on the right. If you get used to one layout and click without looking, you could lose your work!

Did you ever accidentally hang up on someone when you clicked the receiver hook on your phone to use call waiting or conference calling?

And, the best one of all time…is up to you to find!

A5:

Visual, hearing, motion, and cognitive impairments.

A6:

Areas dealing with the keyboard, mouse, sound, and display. If the software was written to a popular platform that supports accessibility, the test effort will be a bit easier than if the accessibility features were programmed entirely from scratch.

A1:

Here are a few examples: There's rollover help—the little pop-up descriptions you get when you hold the cursor over a painting tool. Selecting About Paint from the Help menu displays a window with copyright and licensing information. Pressing F1 starts the online help system where you can read the manual, select from an index, or type in words to search for. There's also function help—for example, if you select Edit Colors from the Colors menu, click the ? in the title bar, and then click one of the colors, you'll get help about choosing and creating colors.

A2:

Every testing task is a risk-based problem. If you have time to test all the index entries, you might choose to do so. If you can't test them all, you'll have to create equivalence partitions of the ones you think are important to check. You could base your decision on information you get from the programmers on how the index system works. You might talk with the writer to find out how the entries were generated. You might try one of each starting letter, or the 1st, 2nd, 4th, 8th, 16th, … and last. You could even wait until you read Chapter 15, “Automated Testing and Test Tools.”

A3:

True. But, it's not just documentation testing. The content of the message needs to be tested as documentation, but forcing the message to appear and assuring that the correct message is displayed is testing the code.

A4:

Improved usability, improved reliability, and lower support costs.

A1:

Use/Leverage. Maybe a bit of Challenge/Prestige.

A2:

False. It is a formal process performed by the entire team.

A3:

Computation Errors—the value was only expected to be positive. When it went negative, it became a huge positive number. Data Reference Errors—because when the value became a huge positive number, the destination buffer was not limited to the size of the comment (65533 bytes).

A4:

Latent Data.

A5:

RAM Slack and Disk Slack.

A1:

The elements that are similar to what's in multimedia CD-ROM software—text, graphics, and hyperlinks.

A2:

Gray-box testing is when you can take a peek at the underlying code and use that information to help you test. You're not examining it to the same level of detail as you would with white-box testing. It's helping you test, but you're not basing all of your tests on it.

A3:

Because many websites are principally created with easily viewable HTML, a mark-up language, not an executable program. You can quickly and easily take a peek to see how the pages are built and design your tests accordingly.

A4:

Because a spell checker can only check ordinary text. It can't check graphical letters or dynamically generated text that changes on each viewing or over time.

A5:

The hardware platform, the operating system, the web browser, browser plug-ins, browser options and settings, video resolution and color depth, text size, and modem speeds.

A6:

Gratuitous use of bleeding-edge technology. Existing hardware and software is always susceptible to new technology being run on it for the first time. This was a bit of a trick question—it wasn't mentioned in the chapter, but hopefully you could arrive at the answer by applying what you've learned in Part III, “Applying Your Testing Skills,” of the book.

A1:

They can speed up the amount of time it takes to run your test cases. They can make you more efficient by giving you more time for test planning and test case development. They're precise and relentless.

A2:

Because software can change during the product's development, your test tools will need to change, too. You can fall into a trap of spending too much time designing tools and automation, neglecting actual testing. It's easy to rely on automation too much. There's no substitute for testing the software yourself.

A3:

A test tool will help you test, making it easier for you to perform a manual testing task. Automation is also a tool, but it will run without your intervention. Think power saw and hammer building a house while the carpenter sleeps.

A4:

Both types of tools hook into the software at points not normally accessible to the average user. Viewer tools are non-invasive as they allow you to see only what's happening. Injector tools are invasive—they allow you not only to see what's happening, but also to manipulate it. You can try test cases that might otherwise be difficult or impossible to perform at the normal user level.

A5:

False. An invasive tool may provide you with better information and control in some situations, but it has the possible downside of affecting the software and the test results. It's best to carefully evaluate each situation and select the tool that works best, with the fewest side effects.

A6:

Recording and replaying your test cases so that you only have to manually perform them once is very effective. It frees you from the monotonous repetition and can give you more time to hunt for those hard to find bugs.

A7:

Simple programming of steps rather than captured steps. The ability to pause or wait for the software to react to the actions. Some type of simple verification so that the macros know whether a bug has occurred.

A8:

They're almost self-aware. They know the software's state table so they know where they are and what they can do.

A1:

The pesticide paradox (described in Chapter 3, “The Realities of Software Testing”) is the situation that occurs if you continue to test software with the same tests, or the same people. Eventually, the software seems to build up an immunity to the tests because no new bugs are found. If you change the tests or bring in new testers, you'll find new bugs. The bugs were always there, it's just that the new approach made them visible.

A2:

It gets lots of additional people looking at the software. It's a good way to find configuration and compatibility problems.

A3:

A beta test is no substitute for an organized, planned, methodical test approach—it's not good at general bug finding. You should know who the beta testers are in regards to their experience level, equipment, and needs to ensure that you get what you expect out of the test.

A4:

The expense and overhead to stock and manage a configuration testing lab is very high and would likely be prohibitive for a small company or project.

A1:

To paraphrase the IEEE 829 definition, the purpose of a test plan is to define the scope, approach, resources, and schedule of the testing activities and to identify the items being tested, the features to be tested, the testing tasks to be performed, the personnel responsible for each task, and the risks associated with the plan. In short, to tell and get agreement from the rest of the project team exactly how the heck the test team intends to test the software.

A2:

Because all the issues and questions defined in a test plan either impact or are influenced by other project functional groups or team members. Getting everyone to understand and agree to the contents of the plan is what matters. Privately creating a paper document and putting it on a shelf is not just a waste of time, but also jeopardizes the project.

A3:

Because left to their own, everyone will have different ideas of what quality and reliability mean to them. Since they're all different, they all can't be achieved.

A4:

These requirements must be met to move from one testing phase to another. A phase can't be left until its exit criteria are met. A new phase can't be entered until its entrance criteria are met.

A5:

People, equipment, offices and labs, software, outsourcing companies, and miscellaneous supplies.

A6:

False. Because testing depends so much on other aspects of the project (for example, you can't test something until it's coded), a test schedule is best made relative to the delivery dates.

A1:

Organization, repeatability, tracking, and proof of testing.

A2:

Ad hoc testing is testing without a plan. It's easy and fun but it's not organized, it's not repeatable, it can't be tracked, and when it's over, there's no proof that it was ever done.

A3:

The purpose of the test design spec is to organize and describe the testing that needs to be performed on a specific feature. It outlines the features to be tested and the approach to be used. It identifies the test cases, but doesn't specify them, and what the pass/fail criteria is.

A4:

This document defines the actual input values used for testing and the expected outputs. It also lists any special environmental needs or procedure requirements and any inter-dependencies among test cases.

A5:

Tables, matrices, lists, graphical diagrams—whatever means most efficiently presents the test cases to you, other testers, and other members of your product team.

A6:

The purpose of the test procedure spec is to identify all the steps required to perform the test cases, including how to set up, start, run, and shut down the test. It also explains what to do in case the test doesn't go as planned.

A7:

That's a question without a specific answer. It greatly depends on who will be using the procedures. Too little detail makes the test procedures ambiguous and variable. Too much detail can bog down the test process. The level of detail should be set by the industry, the company, the project, and the test team.

A1:

There's not enough time in the schedule, it's not a bug, it's too risky, it's not worth it, and the bug wasn't reported properly.

A2:

Log them as soon as possible. Effectively describe the bug, making sure it's minimal, singular, obvious and general, and reproducible. Be nonjudgmental in your approach. Follow the report through its life cycle.

A3:

Record what you do and review it carefully. Use white-box test techniques to look for race conditions, boundary conditions, memory leaks, and other similar problems. See if the bug is state related, such as initial state or later state dependent. Consider resource dependencies and even hardware problems as the source of the bug.

A4:

Title: Adding two even numbers gives an answer that's one too much.

Description:

A5:

Probably Severity 3 (minor problem), Priority 2 (must fix before release).

A6:

Open, Resolved, and Closed are the basic states. Review and Deferred are two possible other states.

A7:

You can see at a glance what a bug's life cycle has been—even if it has been complex. The current status of a bug can be instantly known. Bugs can't be lost or neglected as easily. Project statistics can be quickly obtained.

A1:

It doesn't tell the entire story. You could be testing the most complex area of the software. Your area could have been written by the most experienced programmer. It could have been written by the least experienced programmer. The code you're testing may have already been tested or may be brand new.

A2:

Average number of bugs found per day. Total bugs found so far. Ratio of fixed bugs to all bugs found. Ratio of Severity 1 or Priority 1 bugs to all bugs found. Average time from the Resolved state to the Closed state.

A3:

Product EQUALS Calc-U-Lot AND

Version EQUALS 3.0 AND

Status EQUALS Resolved AND

Assign TO EQUALS Terry

A4:

It's possible that the software was released to testing in phases and not all the software was tested yet—it might only be leveling off for the current phase. The testers might be busy regressing and closing bugs and not looking for new ones. It could have been a very warm and sunny week or the testers might be out on vacation.

A1:

Because no matter how good the development process is, testing still needs to be performed one time to verify the product against the product specification and validate it against the user requirements. If no bugs are found, great, but all the costs associated with planning, developing, and executing the tests contribute to the costs of conformance.

A2:

False! Testing looks for bugs. Testers didn't put the bugs in the product and can't guarantee when they're done testing that no more bugs exist.

A3:

Because it implies that you are the one guaranteeing the product's quality. Are you ready for that responsibility?

A4:

If they report to the development manager or the project manager, there's a conflict of interest between finding bugs and the creation of the software or the meeting of the schedule.

A5:

They would probably be at CMM Level 3, possibly touching some of the Level 4 requirements. They aren't at Level 2 because Level 2 is just concerned with the project level. Level 3 deals with the entire organization or company. Level 4 is where statistical control starts to come into play.

A1:

Since the job names and descriptions for software testers are variable, you should try looking for software test, software testing, quality assurance, and QA.

A2:

Beta testing and usability testing are two good methods. A third way is testing open source software and using open source testing tools. You might even get paid to find bugs!

A3:

The goal of a software tester is to find bugs, find them as early as possible, and make sure they get fixed—tempered, of course, with all the realities of software testing that you learned in this book. Good luck and go find those bugs!