The first level of testing comes from you. When you implement a new feature, you make a small code change and then run the application to see whether the required functionality is ready. Compiling and running your code is a daily task that happens many times a day as you progress toward the required functionality.
Some features, such as “add a button here that sorts this table of the report,” are trivial enough that they can be implemented and tested in one run. But more-complex features, such as “we need to change the policy of approving/rejecting a loan,” will need several changes and runs of the application until the feature is marked as complete.
You can see this manual code-run-verify cycle in figure 1.2.
Manual testing is enough for small software projects. A quick prototype, a side project, or a weekend coding session can be tested manually by a single person. In order for the cycle to work effectively, a single loop must be quick enough for the developer to see the results of the code change. In an ideal case, a single code change should be verified in seconds. If running the whole application and reaching the point where the new feature is found requires several minutes, developer productivity suffers.
Writing software is a creative process that requires getting into the “zone.” Having constant interruptions with lengthy intervals between each code change is a guaranteed way to disrupt the developer’s thinking about the code structure (not to mention loss of time/money while waiting for the test to finish).
As the programming code grows past a certain point, this manual cycle gets lengthier, with more time spent running and testing the application than writing code. Soon the run-verify time dominates the “developing” time. Another problem is the time it takes to redeploy software with the new changes. Small software projects can be deployed in seconds, but larger code bases (think bank software) may need several minutes for a complete deployment, further slowing the manual testing cycle.
Spock is marketed as an enterprise-ready test framework, so it’s best to explain the need for automated testing in the context of enterprise software—software designed to solve the problems of a large business enterprise. Let’s look at an example that reveals why a test framework is essential for large enterprise applications.
Imagine you’ve been hired as a software developer for a multinational company that sells sports equipment in an online shop. Most processes of the company depend on a monolithic system that handles all daily operations.
You’re one of several developers responsible for this central application that has all the characteristics of typical enterprise in-house software:
The last point is the one that bothers you most. Several areas of the application have nonexistent documentation, and no one to ask for advice.
You’re told by your boss that because the snow season is approaching, all ski-related materials will get a 25% discount for a limited time period that must also be configurable. The time period might be a day, a week, or any other arbitrary time period.
Your approach is as follows:
You might be happy with your implementation and send the code change to the production environment, thinking you’ve covered all possible cases, as shown in figure 1.3.
The next morning, your boss frantically tells you to revert the change because the company is losing money! He explains that the e-shop has several VIP customers who always get a 10% percent discount on all products. This VIP discount should never be applied with other existing discounts. Because you didn’t know that, VIPs are now getting a total discount of 35%, far below the profit margin of the company. You revert the change and note that for any subsequent change, you have to remember to test for VIP customers as well.
This is a direct result of a large code base with several modules affecting more than one user-visible feature, or a single user-visible feature being affected by more than one code module. In a large enterprise project, some modules affect all user-visible features (typical examples are core modules for security and persistence). This asymmetric relationship is illustrated in figure 1.4.
With the change reverted, you learn more about the business requirements of discounts. The final discount of a product is affected by the following:
The next time you tamper with the discount code module, you’ll have to manually test more than 100 cases of all the possible combinations. Testing all of them manually would require at least four hours of boring, repetitive work, as shown in figure 1.5.
This enterprise example should make it clear that the complexity of software makes the manual testing cycle slow. Adding a new feature becomes a time-consuming process because each code change must be examined for side effects in all other cases.
Another issue similar to module interaction is the human factor: in a big application, communication between domain experts, developers, testers, system administrators, and so on isn’t always free of misunderstandings and conflicting requirements. Extensive documentation, clear communication channels, and an open policy regarding information availability can mitigate the problems but can’t completely eliminate them.
As an example, a sales manager in the e-shop decides that he wants to see all tables in the back-office application sorted by the value of the order, while at the same time an inventory manager wants to sort the same tables by order size. Two separate developers could be tasked with these cases without knowing that the requirements are conflicting, as shown in figure 1.6.
This enterprise example illustrates firsthand the problems of every large software code base:
Now you know the major problems faced by a software development team working on a big enterprise project. Next, let’s look at various approaches to tackling these problems.
All software companies suffer from these problems and deal with them in one of the following three ways or their variations (I’ve seen them all in real life):
Let’s look at each of these solutions in turn.
In the first case (which is possible with only small- to middle-sized software projects), developers aren’t entirely sure what’s broken after a code change. Therefore, they manually test all parts of the application after they implement a new feature or fix an issue. This approach wastes a lot of time/money because developers suffer from the repetitive nature of testing (which is a natural candidate for automation).
In addition, as the project grows, testing everything by hand becomes much more difficult. Either the development progress comes to a crawl, as most developers deal with testing instead of adding new features, or (the most common case) developers add features and test only parts of the application that they think might be affected. The result is that bugs enter production code and developers become firefighters; each passing day is a big crisis as the customer discovers missing functionality.
In the second case, the “solution” is to never perform big code changes at all. This paradigm is often embraced by large organizations with big chunks of legacy code (for example, banks). Management realizes that new code changes may introduce bugs that are unacceptable. On the other hand, manual testing of the code is next to impossible because of the depth and breadth of all user scenarios (for example, you can’t possibly test all systems of a bank in a logical time frame by hand).
The whole code base is declared sacred. Changing or rewriting code is strictly forbidden by upper management. Developers are allowed to add only small features to the existing infrastructure, without touching the existing code. Local gurus inspect each code change extensively before it enters production status. Code reuse isn’t possible. A lot of code duplication is present, because each new feature can’t modify existing code. Either you already have what you need to implement your feature, or you’re out of luck and need to implement it from scratch.
If you’re a developer working in situations that belong to these first two cases (manual testing and the big code base that nobody touches), I feel for you! I’ve been there myself.
There’s a third approach, and that’s the one you should strive for. In the third case, an automated test framework is in place that runs after every code change. The framework is tireless, meticulous, and precise. It runs in the background (or on demand) and checks several user features whenever a change takes place. In a well-managed software creation process, the testing framework runs automatically after every developer commit as part of a build pipeline (for example, with the Jenkins build server, available for free at http://jenkins-ci.org/). Results from this automatic run can influence further steps. A common policy is that code modules with failed test results should never be deployed to a production environment.
The test framework acts as an early warning system against unwanted code effects. To illustrate the previous example, if you had a test framework in place, you’d get an automated report after any change, as shown in figure 1.7.
A test framework has the following characteristics.
It reduces
It ensures
It provides
Code can be refactored, removed, and updated with ease, because the test framework continuously reports unwanted side effects. Developers are free to devote most of their time to coding new features and fixing existing (known) bugs. Features quickly come into production code, and the customer receives a software package known to be stable and solid for all scenarios supported by the test framework. An initial time investment is required for the testing framework, but after it’s in place, the gains outperform the time it takes to write the test scripts. Catching code regressions and severe bugs before they enter the production environment is much cheaper than allowing them to reach the final users.
A test framework also has other benefits not instantly visible with regard to code quality. The process of making programming code testable enforces several constraints on encapsulation and extensibility that can be easily neglected if the code isn’t created with tests in mind. Techniques for making your code testable are covered in chapter 8. But the most important benefit of a test framework is the high developer confidence when performing a deep code change.
Let’s dig into how Spock, as a testing framework specializing in enterprise applications, can help you refactor code with such confidence.
18.188.190.175