Chapter 10. Quality

As your CTO, I have several expectations about quality.

Continuous Improvement

I expect continuous improvement.

Human beings improve things with time. Human beings impose order upon chaos. Human beings make thing better.

Our computers are better than they used to be. Our cars are better than they used to be. Our airplanes, our roads, our telephones, our TV service, our communications services are all better than they used to be. Our medical technology is better than it used to be. Our space technology is better than it used to be. Our civilization is massively better than it used to be.

Why, then, does software degrade with time? I do not expect our software to degrade.

I expect that, as time goes by, the design and architecture of our systems will improve. I expect that the software will get cleaner and more flexible with every passing week. I expect that the cost of change will decrease as the software ages. I expect everything to get better with time.

What does it take to make software better with time? It takes will. It takes attitude. It takes a commitment to the disciplines that we know work.

I expect that every time any programmer checks in code, they check it in cleaner than they checked it out. I expect that every programmer improves the code that they touch, regardless of why they are touching it. If they fix a bug, they should also make the code better. If they are adding a feature, they should also make the code better. I expect every manipulation of the code to result in better code, better designs, and better architecture.

I expect continuous improvement.

Fearless Competence

Image

I expect fearless competence.

As the internal structure of a system degrades, the complexity of the system can rapidly become intractable. This causes the developers to naturally become more and more fearful of making changes. Even simple improvements become fraught with risk. Reluctance to make changes and improvements can drastically lower the programmers’ competence to manage and maintain the system.

This loss of competence is not intrinsic. The programmers have not become less competent. Rather, the growing intractable complexity of the system begins to exceed the natural competence of the programmers.

As the system becomes increasingly difficult for the programmers to handle, they begin to fear working on it. That fear exacerbates the problem because programmers who are afraid to change the system will only make changes that they feel are safest to make. Such changes are seldom those that improve the system. Indeed, often the so-called safest changes are those that degrade the system even more.

If this trepidation and timidity are allowed to continue, estimates will naturally grow, defect rates will increase, deadlines will become harder and harder to achieve, productivity will plumet, and morale will slide into the pit.

The solution is to eliminate the fear that accelerates the degradation. We eliminate the fear by employing the disciplines that create suites of tests that the programmers trust with their lives.

With such tests in place and with the skill to refactor and drive toward simple design, the programmers will not fear to clean a system that has degraded in one place or another. They will have the confidence and the competence to quickly repair that degradation and keep the software on an ever-improving trajectory.

I expect the team to always exhibit fearless competence.

Extreme Quality

I expect extreme quality.

When did we first begin to accept that bugs are just a natural part of software? When did it become acceptable to ship software with a certain level of defects? When did we decide that beta tests were appropriate for general distribution?

I do not accept that bugs are unavoidable. I do not accept the attitude that expects defects. I expect every programmer to deliver software that is defect free.

And I am not simply referring to behavior defects. I expect every programmer to deliver software that is free of defects in behavior and structure.

Is this an achievable goal? Can this expectation be met? Whether it can or can’t, I expect every programmer to accept it as the standard and continuously work toward achieving it.

I expect extreme quality coming out of the programming team.

We Will Not Dump on QA

Image

I expect that we will not dump on QA.

Why do QA departments exist? Why would companies invest in entirely separate groups of people to check the work of the programmers? The answer is obvious and depressing. Companies chose to create software QA departments because programmers were not doing their jobs.

When did we get the idea that QA belongs at the end of the process? In too many organizations, QA sits waiting for the programmers to release the software to them. Of course, the programmers do not release the software on schedule, so it is QA that is stuck trying to make the release date by cutting their testing short.

This puts the QA people under tremendous pressure. It is a high-stress, tedious job in which shortcuts are necessary if shipment dates are to be maintained. And that is clearly no way to assure quality.

The QA Disease

How do you know if QA is doing a good job? On what basis do you give them raises and promotions? Is it defect discovery? Are the best QA people the ones who find the most defects?

If so, then QA views defects as positive things. The more the better! And that is, of course, sick.

But QA may not be the only folks who view defects in a positive light. There’s an old saying1 in software: “I can meet any schedule you like, so long as the software doesn’t need to work.”

1 I first heard it from Kent Beck.

That may sound funny, but it is also a strategy that developers can use to meet individual deadlines. If it’s QA’s job to find the bugs, why not deliver on time and let them find some?

No word has to be spoken. No deal has to be made. No handshakes ever occur. And yet everyone knows that there is an economy of bugs flowing back and forth between the developers and QA. And that is a very deep sickness.

I expect that we will not dump on QA.

QA Will Find Nothing

I expect that if QA is at the end of the process, then QA will find nothing. It should be the goal of the development team that QA at the end never finds a bug. Anytime a bug is found by QA, the developers should determine to find out why, correct the process, and make sure it never happens again.

QA should wonder why they are at the end of the process, because they never find anything.

In fact, QA does not belong at the end of the process. QA belongs at the beginning of the process. The job of QA is not to find all the bugs; that’s the programmers’ job. The job of QA is to specify the system behavior in terms of tests with sufficient detail that defects are excluded from the final system. Those tests should be executed by the programmers, not QA.

I expect that QA will find nothing.

Test Automation

In most instances, manual testing is a huge waste of money and time. Almost any test that can be automated should be automated. This includes unit tests, acceptance tests, integration tests, and system tests.

Manual testing is expensive. It should be reserved for situations in which human judgment is necessary. This includes things like checking the aesthetics of a GUI, exploratory testing, and subjectively evaluating the ease of an interaction.

Exploratory testing deserves a special mention. This kind of testing depends entirely on human ingenuity, intuition, and insight. The goal is to empirically derive the behavior of the system through extensive observation of the way the system operates. Exploratory testers must infer corner cases and deduce appropriate operational pathways to exercise them. This is no mean feat and requires a significant amount of expertise.

Most tests, on the other hand, are eminently automatable. The vast majority are simple Arrange/Act/Assert constructs that can be executed by supplying canned inputs and examining expected outputs. Developers are responsible to provide a function-callable API that allows these tests to run quickly and without significant setup of an execution environment.

Developers should design the system to abstract out any slow or high-setup operations. If, for example, the system makes extensive use of a relational database management system (RDBMS), the developers should create an abstraction layer that encapsulates the business rules from it. This practice allows the automated tests to replace the RDBMS with canned input data, vastly increasing both the speed and reliability of the tests.

Slow and inconvenient peripherals, interfaces, and frameworks should also be abstracted so that individual tests can run in microseconds, can be run in isolation from any environment,2 and are not subject to any ambiguity regarding socket timing, database contents, or framework behavior.

2 For example, at 30,000 feet over the Atlantic on your laptop.

Automated Testing and User Interfaces

Automated tests should not test business rules through the user interface. User interfaces are subject to change for reasons that have more to do with fashion, facility, and general marketing chaos than with business rules. When automated tests are driven through the user interface, as shown Figure 10.1, those tests are subject to those changes. As a consequence, the tests become very fragile, which often results in the tests being discarded as too difficult to maintain.

Image

Figure 10.1 Tests driven through the user interface

To avoid this situation, developers should isolate the business rules from the user interface with a function call API, as shown in Figure 10.2. Tests that use this API are completely independent of the user interface and are not subject to interface changes.

Image

Figure 10.2 Tests through the API are independent of the user interface.

Testing the User Interface

If the business rules are automatically tested through a function call API, then the amount of testing required for the behavior of the user interface is vastly reduced. Care should be taken to maintain the isolation from the business rules by replacing the business rules with a stub that supplies canned values to the user interface, as shown in Figure 10.3.

Image

Figure 10.3 The stub supplies canned values to the user interface.

Doing so ensures that the tests of the user interface will be fast and unambiguous. If the user interface is significantly large and complex, then an automated user interface testing framework may be appropriate. The use of the business rule stub will make those tests much more reliable.

If the user interface is small and simple, it may be more expeditious to fall back on manual testing, especially if there are aesthetics to be evaluated. Again, the use of the business rule stub will make these manual tests much easier to conduct.

I expect that every test that can practicably be automated will be automated, that the tests will execute quickly, and that the tests will not be fragile

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.83.87.94