9.1.1Omnipresent Connectivity

Although service-oriented architecture (SOA) is today less of a focal point, the rapid development of the Internet of Things (IoT)—more accurately, the Internet of Services (IoS)—coupled with the increasing spread of standardized web services in place of technical interfaces, of web interfaces in place of conventional user interfaces, and the development and restructuring of large applications using the principle of microservices pose great challenges for the implementation of sustainable test automation. Due to the multitude of different devices, display sizes, operating systems, and so on, the same applies to test automation for mobile applications too. The omnipresent interconnectivity of systems with all their (standard and proprietary) interface types means that test automation needs to be seriously well thought out. Desktop applications with a wide variety of GUI technologies and legacy systems (such as host applications) are still common in the current IT landscape and add considerable complexity to the automation process. This in turn means that most SUTs cannot rely exclusively on (fully or partially) standardized interfaces. In many cases, even the most powerful commercial tools cannot support all the interfaces necessary for sufficient test coverage without additional extension and expansion work.

9.1.2Test Automation in IT Security

The massive interconnection of software systems mentioned above generates new security requirements. Every vulnerability in a system means that hackers (often referred to as “black hats”) can find and exploit potential weaknesses. A notorious example of this kind of security vulnerability are the computer systems built into the Jeep Cherokee, which allowed hackers to access the vehicle’s driving control systems, manipulate the gearshift, and disable the brakes while the car was in motion [URL: Jeep Cherokee]. This is just one well-known example that illustrates the increasing importance of extensive IT security testing. In the long term, test automation needs to progress to the point at which largescale IT security testing can still be carried out economically.

9.1.3Test Automation in Autonomous Systems

The collection of huge amounts of data through smartphones has accelerated the development of autonomous systems more than just about any other field of engineering. Drones, cars, phones, recommendation and response systems, image categorization—these days, it seems that no app can get by without the buzzword “artificial intelligence (AI)”. The interesting thing about AI is that many of the methods and techniques it employs have been around for a long time, but it is only recently that growth in omnipresent interconnectivity and mobile data collection has provided sufficient amounts of diverse data to enable extensive use of the available AI systems, with all the pros and cons that autonomous systems entail.

The challenge in testing such systems is that it is often no longer possible to find plausible explanations for why the SUT behaves the way it does. Observed SUT behavior may initially appear plausible, while a second glance reveals that it is in fact more accurate than specified in the test.

AI and AI testing are currently enjoying huge research visibility, and test automation needs to rise to the challenge and respond with a suitable answer. However, we cannot right now say how this response will look. Perhaps AI systems can best be tested by AI test systems—in other words, we are looking at the automation of test automation (see also section 9.2.3).

9.2.1Agile Software Development Is Inconceivable without Test Automation

Another development is evident in connection with agile software development methods. Efficient use of test automation is an absolute necessity if an agile project (for example, in Scrum) is to succeed. Without it, it is impossible to ensure the functionality and reliability of each release at the end of a sprint. With each sprint cycle, the proportion of regression tests gets larger and, as a result, they have to be continuously adjusted. Moreover, the test cycles often provide little processing time for test execution [Baumgartner et al. 21].

9.2.2New Outsourcing Scenarios for Automation

For a while now, many managers answered the question of how to save money in software testing by outsourcing testing activities to places where human labor can still be bought cheaply. Whether offshore or nearshore, the decision follows the price. Outsourcing requires up-front investment in the development of preliminary products or work results. If test execution alone is outsourced, comprehensively and precisely formulated test cases have to be provided in advance. If test case creation and execution are outsourced, the functional specifications must be formulated so precisely that remote staff who are not familiar with the details of the domain can derive and describe sufficiently high-quality test cases from them. This often entails communication challenges due to language barriers.

However, this approach—i.e., a lot of up-front investment followed by the use of a lot of cheap labor to perform extensive manual system and regression testing—is already regarded as outdated. The profit margins in these types of scenarios are small, and a workforce that appears cheap today will probably appear too expensive in the space of just a few years. This means that the degree of test automation is destined to increase for all manual testing activities—initially for test execution and, in the future, in test design and test case specification too. Where this automation takes place, and who actually takes functional test cases and uses them to create and execute an automated test process, is defined within the corresponding testing or outsourcing strategy. These strategies are in turn influenced by the latest test automation innovations and their associated possibilities and practicability.

9.2.3Automating Automation

Don’t forget: test automation is a software project! Just as the use of AI is increasingly used to create a whole new dimension of software systems, AI can of course also be used for test automation and to mitigate the need for outsourcing described in section 9.2.2 above, whether in visual object recognition, smart generation of efficient and effective test suites, or self-healing approaches for existing, large automation solutions. For example, Facebook [URL: DEV INSIDER ] uses a custom tool that autonomously tests a code base and automatically corrects any defects it finds. With its Chaos Engineering [URL: Wikipedia: ChaosEngineering] approach, Netflix has created a whole new branch of engineering (and a name) for automated autonomous fault tolerance testing.

9.2.4Training and Standardization

Solid conceptual and practical training is required to teach software testers and developers the skills they need to become test automation experts. Many test-automation training courses are currently offered by tool manufacturers, who usually limit the curriculum to covering the concepts behind and operation of their own tools. Some service providers, too, offer occasional courses on the concepts underlying test automation, but with varying focus and depth. This makes it difficult to compare courses with one another.

With its Certified Tester Advanced Level - Test Automation Engineer syllabus, the ISTQB® is forging a path toward international standardization in training and parlance for test automation engineers. Building on the established Foundation Level and Advanced Level Certified Tester schemes, a syllabus has been created that teaches the concepts of test automation and provides a unified glossary to go with it. This is reason enough for us to structure the latest edition of this book around the ISTQB® curriculum. We hope that tool manufacturers will soon incorporate this standardized terminology into their tools, thus making them easier to learn and understand.