Testing, whether system testing or user acceptance testing, traditionally was done at the end of the waterfall life cycle, as illustrated in Figure 9.1.
Without further questioning the effectiveness of this model, let’s simply say that in an Agile or Scrum project, testing is no longer performed at the end of the life cycle but is “baked” throughout the different iterations or sprints, as shown in Figure 9.2.
Without doubt, testing is one of the things that make the most difference in Scrum development. Whether or not the team is capable of producing regular shippable increments will most likely be determined by how well testing is organized and run.
Even on a small Scrum project, it is difficult to conceive how teams can regularly deliver without some kind of automated testing mechanism already in place and fully functioning.
The more automated testing infrastructure is in place, the more the team’s velocity is likely to increase in the long run (Figure 9.3).
The Importance of the Definition of Done
Before we go further with our discussion of testing, let’s first talk about the definition of the word done. This is what will determine which types of testing should be done by the team. By testing types, we referred to something like useracceptance or technical testing.
Like most experts, including Henrik Kniberg, in his book, Scrum and XP from the Trenches, we prefer done to mean “ready to deploy to production.” But also like Kniberg, we sometimes have to accept that the definition of done can be somewhat different.
The definition of done often differs depending on the project situation. In the following sections, we will review two or three definitions of done as we have seen on real-life projects.
The first definition of done in Figure 9.4 shows that the team has decided to consider their work done by the end of coding and unit testing.
In the example illustrated in Figure 9.4, many teams are working in parallel on the same product line. The teams decided that it would be most beneficial for them to deliver their different work results after unit testing, so that integration and stabilization work can be done before they move into the next iteration.
This is a situation you are likely to encounter in real-life—that the done is declared at the unit testing level. This approach is fine as long as the team has also accounted for the additional integration and testing effort that will be needed before deployment.
Figure 9.5 shows a common scenario where the team considers that they are done only after all new stories have been integrated and tested before they do their sprint demo.
Figure 9.6 shows the best scenario, in which some business users are part of the Scrum team, where they are responsible for performing acceptance testing before done is certified. Whether or not a project is using Scrum, user acceptance test development has become more and more a widespread practice which should be incorporated into release iterations as early as possible in the process.
Because Scrum is mainly a project management framework, it is silent about the engineering practices related to coding and testing.
However, in our experience, the following testing should be in place in your organization to sustain your Scrum effort:
The reason we list automated testing first is because as soon as you start seeing your team churn out regular software increments, you will realize that you cannot really sustain their pace unless some kind of automation is in place.
Automated testing will save you time and will enable your team to cruise along quite smoothly.
Unlike manual testing where you are required to go through all the test cases yourself manually, automated testing can take hours, even days, depending on your code base. One of the main benefits of automated testing is that a software program will handle all the testing for you.
Of course, in order to do automated testing, you must do a little more work and have bought a tool and have got all the test cases created. But that is only a fraction of time compared to all the time it will take should you have to do everything manually.
Now that we have covered the different testing types and the ones we consider to be most fundamental, let’s turn our attention to organizing the testing.
If you are lucky and the company where you work is fully committed to Scrum, you would not need to do anything more than working and communicating closely with the quality assurance department. Otherwise, if you are like us, you may find yourself in companies that have not implemented Scrum throughout the enterprise and, because of that, little or no infrastructure has been set up in terms of automated and continuous integration testing for Scrum.
In case this happens, this chapter provides you with some idea of what you should see in terms of testing infrastructure, either for your own needs or for the company, should they ask you.
Figure 9.7 shows a very decent environment, in which the software is organized around three different environments: development, testing, and production.
There is a natural flow from development to testing and from testing to production.
Perhaps you are not so fortunate as to be in a company where everything is set up like this. We suggest that you get together with your technical team members at the very beginning of your project, and see how they can set up a makeshift testing environment within your project’s own development environment, as shown in Figure 9.8.
We know from experience that many teams do not start with this, but, it has been an essential part of all the successful Agile or Scrum projects we have worked on. This is why we include this information here to help save you time and headache. We think that you will thank us later.
Unless you have taken the time to organize your testing strategy and organization, you will not be able to deliver with Scrum if your testing does not follow or support you.
Testing no longer takes place at the end of the software life cycle. We have emphasized in this chapter just a few tests, which are critical to your Scrum project success. This is why you do not hear about test-driven development (TDD), despite its remarkable effectiveness for the average programmer team to churn out good code. Instead, this chapter focused more on automatic and continuous integration testing, the two most important types of testing for every Scrum project team to set up as soon as possible.
This chapter then focused on testing organization.
If you do not work in a company where a good testing infrastructure has been set up, we recommend that you work with technical team members at the beginning of your project to set up a makeshift testing environment—if necessary within your project’s development environment. This testing environment will provide an indispensible support for the development team as they turn out new increment after new increment, sprint after sprint.
Because the definition of done has a large impact on how much of a testing type you would need to do, this chapter also covered some different definitions of done in order to give you an idea of what you may need to do on your project.