5.3. Importance of Automated UI Testing

We feel that UI testing is an important part of a project. Though the tests are still automated, they have to be handled in a slightly different fashion and different considerations need to be taken into account when creating them.

With unit tests, the focus is on an isolated and targeted method with a clearly defined set of inputs and outputs, and as such, it is easier to understand the flow of the test and the interactions it is performing. With UI tests you lose this ability. This means that you need to combat this by reconsidering how you write and structure your tests to take this into account. As with most parts of software development, if you lose something — such as readability or automation — then you need to react by replacing or mitigating the impact. This is solved by paying more attention to another area that will help combat this. Because it is more difficult to understand the internals of the system when dealing with a black box, you need to make UI tests even more readable.

UI tests also have a different focus and role than unit tests. They both focus on catching regression bugs, but unit tests, and in particular TDD, focus on design and improving testability of the code base. Sadly there isn't this relationship to the UI. The UI tests can provide a form of documentation for your UI and the UI interface about how the software should work. If you combine this with your BDD or acceptance testing then you can create a very effective form of documentation for the customer.

The focus of the tests should be to ensure that the core functionality is working as expected. Every application has a core set of scenarios and features that must work for the application to be tested or demoed to a customer. This should be the initial focus for your automated tests because if this core functionality doesn't work as you expect you'll want to know as soon as possible. Common core-use cases could include being able to log in to an authenticate-only system. Of course, if you can't get past this initial screen then you can't do much else, so it's important to know if this breaks in an automated fashion as we discussed earlier. Just as with other core features of the application, if they fail you'll want to know very early on.

This is the most important point of UI tests. They provide you with early warning signs for when parts of your system have changed. With unit tests, you want high code coverage, while with UI tests you want high value systems that provide the most value for identifying breakages. If they can provide you with this level of confidence that they will catch the major showstoppers, then they're worth investing the time and effort you'll spend in creating and maintaining them.

Having this level of confidence is a great comfort, especially for testers. Although developers continue to provide code and implement new features, having the UI tests along with unit and integration tests means that the testers can remain focused on the same features as the developers are working on. Having developers and testers focus on the same feature or scenario at the same provides a huge productivity boost. Testers can start testing features as soon as the developer commits the changes, reporting bugs earlier in the lifecycle allowing them to lower the cost. Automation helps this. By having automation, testers are not as concerned with regression bugs on existing features as there will be some automation cover.

This is especially relevant in agile practices such as Scrum. One of the important parts of the scrum process is that all the team focuses on implementing a story and making sure it's finished. If the testers are focused on making sure that no regressions have occurred, they can't focus on moving the application forward.

After you have your UI testing automated, the next ideal stage is to automate the tests across multiple platforms and browsers to take full advantage of the tests you have invested in. With the right structure and UI automation, this is possible and can be a great time return on investment.

However, although UI tests are greatly important and can have great benefits, they are not without their problems.