- UI and end-to-end tests
- When to write each type of test
- The business impact of end-to-end tests
- An overview of multiple UI-based end-to-end testing tools
Sublime pieces of sugary treats can take a bakery a long way. Excellent customer service, beautifully decorated tables, and breathtaking views take it even further.
For customers to be enchanted, their experience must be flawless from the moment they step in to the moment they decide to leave, hopefully, wanting to come back.
No matter how good a pastry chef’s desserts are, if their bakery looks dirty, no one will come in.
Building delightful software involves a similar amount of care and attention to detail. Elegantly designed APIs are worthless if the buttons in the client’s user interface don’t work or if the information it displays is unreadable.
In this chapter, you’ll learn about UI-based end-to-end tests, the conceptual differences between them and other kinds of tests, how they impact your business, and the different tools available for writing them.
I’ll begin the chapter by explaining UI-based end-to-end tests and how they differ from other kinds of tests. In this first section, I’ll emphasize where these tests fit in the testing pyramid, explain how much they cost, and the benefits they yield so that you can decide whether you should adopt them, and understand how to incorporate them into your development process.
Once I’ve explained the differences and similarities of these types of tests, I’ll teach you how to decide when to write each one of them. I’ll go through the pros and cons of each one and give you examples of how each type of test delivers value.
The third and final section contains an overview of end-to-end testing tools. In it, I will present and compare tools like Selenium, Puppeteer, and Cypress, which is the tool I’ll use throughout the next chapter. Through the use of diagrams, I’ll show you how these tools are architected and how they work. Additionally, I’ll explain the pros and cons of each so that you can pick the adequate technology for your project.
10.1 What are UI-based end-to-end tests?
Baking sublime desserts is different from establishing a successful bakery. No matter how delightful your cheesecakes taste, you won’t sell any in a scruffy bakery. Likewise, not even the best-looking bakery in town survives sour desserts.
To succeed, bakery owners have to make sure that the presentation of each product is flawless, that they taste delectable, and that both the customer service and the bakery’s decoration is held up to the highest standards.
Similarly, for software products to succeed, they must fulfill their user’s needs, behave correctly, and have responsive and understandable interfaces.
In this section, I’ll explain the difference between end-to-end tests, UI tests, and UI-based end-to-end tests.
End-to-end tests help you ensure that the entire application works as it should. As I’ve explained in chapter 2, the scope of these tests is large, and the reliability guarantees they generate are strong. These tests are at the very top of the testing pyramid.
To attach an “end-to-end” label to a test, it’s useful to look at what the software under test is.
As an example, think about the tests for the application used in chapters 4 and 5. When you consider the software under test to be the inventory management backend, the tests that validate this backend by sending HTTP requests are end-to-end tests.
On the other hand, if you consider the software under test to be the web client the bakery’s operators use, its end-to-end tests are the ones that click buttons, fill inputs, submit forms, and read the page’s contents.
Now, if you consider the software under test to be the entire inventory management application, including both its backend and frontend, the tests I’ve previously mentioned are not exactly “end-to-end” tests because they do not cover the entire software under test.
End-to-end tests for this entire application would use a browser to interact with the application through its web client, allow the HTTP requests to reach the backend, and expect the page to display the right contents once the server answers. Testing the inventory management application in this way covers all of the software involved in running it, as shown in figure 10.1.
Figure 10.1 An end-to-end test for the entire application covers all parts of its stack by interacting with it through a browser.
User-interface tests are different from end-to-end tests in the sense that they cover an application’s UI, not necessarily its entire range of features or its complete software stack.
A user-interface test for the frontend application from chapter 6, for example, could involve using a browser only to check if it displays the correct content.
Even the end-to-end test I’ve previously mentioned—the one that interacts with the entire application through its UI—is a UI test. I’d consider that test a UI test because it uses the client’s user interface as the entry point for its actions and as an assertion target.
Alternatively, a user-interface test could use a stub to replace the backend entirely and focus solely on the application’s interface elements. A test like this would then be exclusively a UI test.
Important User-interface tests and end-to-end tests are not mutually exclusive classifications. A test can be both a UI test and an end-to-end test, or only one of them.
I call UI-based end-to-end tests the tests that cover an entire application’s software stack by interacting with the application through its UI. You can see in figure 10.2 how these types of tests overlap.
Figure 10.2 End-to-end tests validate all of the application’s layers. UI tests validate an application through its UI. UI-based end-to-end tests are at the intersection of both kinds of tests because they validate all of the appli-cation’s layers by interacting with it.
10.2 When to write each type of test
When deciding whether and when to write pure end-to-end tests, pure UI tests, or UI-based end-to-end tests, I recommend readers follow the testing pyramid’s principles.
To follow those principles, you must be able to determine where each of these kinds of tests fit, which is what I’ll teach you in this subsection.
10.2.1 UI-based end-to-end tests
A UI-based end-to-end test involves the entire stack of software upon which your application depends. It goes at the very top of the testing pyramid because it has the broadest possible scope and generates the most reliable guarantees.
Even within the “end-to-end” test category, UI tests that cover your entire system go above end-to-end tests for RESTful APIs, for example, as shown in figure 10.3.
Figure 10.3 Within the end-to-end tests section of the testing pyramid, UI-based end-to-end tests go at the very top.
It interacts with your application exactly like users would: by interacting with your page’s elements through a real browser. Because these interactions depend on the client to send the adequate requests and the server to provide adequate responses, it covers both your frontend and backend.
For example, consider a UI-based end-to-end test for the inventory management application from chapter 6.
By filling the application’s inputs, submitting the form, and checking the item list, this test validates whether the frontend form sends the expected request, whether the server responds correctly, and whether the client adequately updates the stock once it receives the server’s response.
With a single test, you were able to cover a large part of your application reliably. Therefore, you need fewer UI-based end-to-end tests than other kinds of tests.
Because these tests tend to be more time-consuming to write and to execute, it’s wise to avoid having to update them multiple times as you develop new features. Therefore, I’d recommend readers to write these kinds of tests after implementing complete pieces of functionality.
NOTE In the next chapter, I’ll go into detail about why these tests can be time-consuming to write and maintain in the sections “Best practices for end-to-end tests” and “Dealing with flakiness.”
To decide in which circumstances you should write UI-based end-to-end tests, you must carefully consider how critical the feature under test is, how labor-intensive it is to validate it manually, and how much it will cost to automate its tests.
The more critical a feature is, and the more time you need to test it manually, the more crucial it is to write UI-based end-to-end tests.
If, for example, you had to test how the “undo” button behaves when users add items simultaneously, you’d have to have at least two clients open, insert items on both, and then try clicking the “undo” button in these different clients, in different orders. This kind of test is time-consuming, and, because it has many steps, it’s also error-prone.
Additionally, because the consequences can be incredibly dire if this button misbehaves, it’s essential to validate this functionality rigorously.
In this case, given how critical this feature is, you’d have to test it frequently, and, therefore, you’d invest a lot of time performing manual labor.
If you had UI-based end-to-end test instead, you could delegate the testing to a machine, which would do it much more quickly and would never forget any steps.
On the other hand, if you have a small, inessential feature, such as a reset button that clears the form’s content, you don’t necessarily need to invest time in writing a UI-based end-to-end test.
For these kinds of features, a test written using react-testing-library would offer guarantees that are almost as reliable and take much less time to implement and to execute.
Both developers and QA engineers can write UI-based end-to-end tests. In leaner teams, which take a more agile approach to software development, software engineers will write these tests themselves and factor into their estimations the time it takes to write those tests.
When a QA team is available, QA engineers can write UI-based end-to-end tests that run against an environment other than production. By automating repetitive tests, they can perform more proactive work and have more time for improving their own processes and performing exploratory testing.
10.2.2 Pure end-to-end tests
In the testing pyramid, pure end-to-end tests go a bit below UI-based end-to-end tests, as you can see in figure 10.4. They do not test your software precisely as users would, but they can be almost as reliable and are much quicker to write.
Figure 10.4 Pure end-to-end tests go below UI-based end-to-end tests in the testing pyramid.
Unlike UI-based end-to-end tests, pure end-to-end tests can be bundled into your development process and significantly shorten your feedback loop as you write code. These tests provide reliable quality guarantees and cover a large part of your code.
Additionally, when testing software with no graphical user interface, such as RESTful APIs or distributable software packages, it’s not possible to write UI tests. Therefore end-to-end tests offer the strongest reliability guarantees you can have.
You should write end-to-end tests as you develop code. These tests should validate your code from a consumer’s perspective. When you tested your backend in chapter 4, for example, your end tests validated your routes by sending them HTTP requests and checking the server’s response and the database’s content.
When writing end-to-end tests, developers should pay attention to how much their tests overlap and carefully adapt their test suite to reduce the burden of maintenance as their software grows, maintaining a balanced testing pyramid.
Because writing this kind of test requires direct access to the code, they must be written by developers, not QA engineers.
10.2.3 Pure UI tests
UI tests can come in two flavors: they can validate a UI either through a real browser or through a testing framework, such as Jest.
If you take the term “UI test” to the letter, you can consider the tests you’ve written using react-testing-library as UI tests. Thanks to JSDOM, they can interact with your components by dispatching browserlike events and validate your application by finding elements in the DOM.
Nonetheless, because those tests use JSDOM instead of a real browser run-time environment, they don’t exactly replicate what happens when your application runs within a browser.
In the testing pyramid, UI tests that run in a browser go above UI tests that run within testing frameworks. Tests that run within a real browser can replicate user interactions more accurately and involve fewer test doubles. However, they take more time to run and are more complex to write.
Considering that UI tests that run within testing frameworks are way quicker to write and that the guarantees they generate are almost as reliable, I’d recommend you choose them most of the time.
For example, think about what you’d have to do to run in a browser the tests you wrote in chapter 6. Instead of having a lightweight replica of a browser environment in which you could easily dispatch events, you’d have to interface with a real browser and deal with all of its complexities, like waiting for pages to load and interacting with its native APIs. Besides making the test much more complicated, it would take way more time to finish.
Because UI tests that run within test frameworks are quick to write and offer reliable guarantees, you can write them as you develop features. Given how small these tests tend to be, it’s also straightforward to include them in a test-driven workflow.
I’d recommend writing pure UI tests that run in a browser only when it’s critical to use browser-specific features that you can’t accurately simulate with JSDOM or when performing visual regression testing—a topic we’ll cover later in this chapter.
Most of the time, UI tests must be written by developers, not QA engineers, because they depend on having access to the code either to interface with the unit under test directly or to write test doubles for the applications with which your client interacts.
When interactions with other pieces of software are irrelevant to the test, QA engineers can then assume this responsibility if they find it suitable.
10.2.4 A note on acceptance testing and this chapter’s name
In the software industry, people often use terms like end-to-end tests, UI tests, and UI-based end-to-end tests imprecisely.
I’ve frequently seen, for example, people calling “end-to-end tests” any tests that interact with an application through a browser.
Even though that definition is correct, given that these tests’ scope is the entire application under test, I believe that we could adopt more precise terminology, like what I’ve used throughout this chapter.
Because of this inaccuracy, I had a difficult time picking a name for this chapter. Initially, I thought I’d name it “UI tests,” but that would mean its name would be reductive, given that the chapter isn’t exclusively about testing user interfaces.
I then pondered naming it “acceptance tests.” I considered this name because I was dealing with validating requisites mostly from a customer’s perspective and checking whether the application fulfils its customers’ needs.
The problem with naming it “acceptance tests” is that it may mislead readers into thinking that I’d ignore checking technical requirements at all. This name, too, could end up being reductive.
Naming this chapter “UI-based end-to-end tests” informs the reader that I will cover a broader scope of technologies and techniques within this chapter.
I believe this name is ideal because this chapter covers tests that interact with an entire application, from end to end, through its graphical user interface, most of which are acceptance tests.
The difference between this chapter and the previous ones that covered end-to-end tests is that this chapter focuses on testing your entire software stack, not just a single piece of software.
Because these tests are at the highest possible level of integration, when considering the testing pyramid, even within the “end-to-end” layer, they are at the very top.
10.3 An overview of end-to-end testing tools
As Louis’s business expands, the more challenging it becomes for him to oversee the bakery’s decor and supervise customer service while still baking delightful desserts. Because he is fully aware that his gifts are best used in the kitchen rather than in an office, Louis decided to hire a manager to oversee the whole business.
Even though the bakery’s new manager can’t bake a cheesecake that’s nearly as good as Louis’s, she is multidisciplinary enough to recognize an excellent dessert and guarantee that the business is running smoothly.
In this section, I will present tools that are to your software what Louis’s new manager is to the bakery.
Instead of specializing in interfacing with your code directly, the tools I’ll present in this section interact with your software through its user interface. They test your software as a whole and are capable of asserting on much broader aspects of how it works.
For example, instead of invoking a function and expecting its output to match a particular value, these tools can help you fill inputs, submit a form, and check whether the browser displays the correct result.
In this case, even though the tool you used doesn’t necessarily need to know about your server, it requires the server to provide the correct response for the client to update adequately. Even though this loosely coupled test knows less about each specific part of your software stack, it can evaluate all of them, just like Louis’s new manager.
I’ll start this section by talking about Selenium, one of the oldest and most widely known end-to-end testing tools available. I will talk about what Selenium is, how it works, how you can benefit from it, and its most critical and most frequent problems.
By exposing Selenium’s problems, it will be easier for you to understand how other testing tools try to solve them and what trade-offs they have to make.
After talking about Selenium, I’ll present Puppeteer and Cypress, which are two of the most popular tools in this niche at the moment.
As I teach about those tools, besides covering their strengths and weaknesses, I’ll explore the contrast between Selenium and them.
For reasons I will explain throughout this chapter and the next, Cypress is my personal favorite and, therefore, the tool I’ve chosen to use to write almost all of the upcoming examples and the tool on which I’ll focus in this section.
10.3.1 Selenium
Selenium is a browser automation framework frequently used for testing web applications through a real browser. It can open websites, click elements, and read a page’s contents so that you can interact with applications and execute assertions. Selenium is the precursor of the browser-based end-to-end testing tools you will see in this section.
NOTE You can find the complete documentation for Selenium at https://www.selenium.dev/documentation.
To understand why Selenium is useful, compare it to the tests you’ve written for the web application in chapter 6.
In those tests, you mounted your application to an alternative DOM implementation, JSDOM. Then, you used JSDOM’s pure JavaScript implementation of native APIs to dispatch events and inspect elements.
JSDOM is ideal for writing tests during your development process. Because it allows you to get rid of the need for a real browser instance, JSDOM simplifies setting up a testing environment and makes your tests quicker and lighter.
The problem with using JSDOM is that it may not always accurately reflect what real browsers do. JSDOM is an attempt to implement browser APIs as per specification.
Even though JSDOM does an excellent job in almost every case, it’s still an imperfect replica of a browser’s environment. Additionally, even browsers themselves do not always follow the API’s specifications adequately. Therefore, even if JSDOM implements those APIs correctly, browsers may not.
For example, imagine that you implemented a feature that relies on an API that behaves one way in Chrome and another in Firefox. In that case, if JSDOM’s implementation is correct, your tests would pass. Nonetheless, if neither Chrome nor Firefox implemented the specification correctly, your feature wouldn’t work on either browser.
Because Selenium runs its tests through a real browser instance, it’s the tool that more closely resembles how your users interact with your software. Therefore, it’s the tool that gives you the most reliable guarantees.
If the browser that Selenium is using to run your tests doesn’t implement a particular API or doesn’t adequately follow its specification, your tests will fail.
Besides being the most accurate way to replicate your user’s actions, Selenium provides you with the full range of a browser’s capabilities.
Instead of merely attaching nodes to a “document,” when using Selenium, you can freely navigate between pages, throttle the network’s speed, record videos, and take screenshots.
For example, if you have a test that guarantees that the images for all products are visible after one second, even on a patchy internet connection, you need to use a real browser. This technique will cause your tests to resemble your users’ environments as closely as possible.
Selenium interacts with a browser through programs called Webdrivers. These Webdrivers are responsible for receiving Selenium’s commands and performing the necessary actions within a real browser.
For example, when you tell Selenium to click an element, it will send a “click” command to the Webdriver you’ve chosen. Because this Webdriver is capable of controlling a real browser, it will make the browser click the selected element.
To communicate with the Webdriver, Selenium uses a protocol called JSON Wire. This protocol specifies a set of HTTP routes for handling different actions to be performed within a browser. When running the Webdriver, it will manage a server that implements such routes.
If, for example, you tell Selenium to click on an element, it will send a POST request to the Webdriver’s /session/:sessionId/element/:id/click route. To obtain an element’s text, it will send a GET request to /session/:sessionId/element/:id/text. This communication is illustrated in figure 10.5.
Figure 10.5 Selenium sends HTTP requests to the Webdrivers that control browsers. These requests adopt the JSON Wire protocol.
To communicate with the browser, each Webdriver uses the target browser’s remote-control APIs. Because different browsers have distinct remote-control APIs, each browser demands a specific driver. To drive Chrome, you will use the ChromeDriver. To drive Firefox, you’ll need the Geckodriver, as shown in figure 10.6.
Figure 10.6 Different web drivers control different browsers.
When you use Selenium’s JavaScript library, all it does is implement methods that will send to the Webdriver of your choice requests that follow the JSON Wire protocol.
Using a Webdriver’s interfaces without Selenium
As I’ve previously mentioned, even though Selenium is mostly used to test web applications, it is actually a browser automation library. Therefore, its JavaScript library, available on NPM under the name selenium-webdriver, does not include a test runner or an assertion library.
NOTE You can find the documentation for the selenium-webdriver package at https://www.selenium.dev/selenium/docs/api/javascript.
For you to write tests using Selenium, you will need to use separate testing frameworks, such as Jest, as figure 10.7 illustrates. Alternatively, you can use Mocha, which is exclusively a test runner, or Chai, which is exclusively an assertion library.
Figure 10.7 If you want to use Selenium for testing, you must pair it with a testing framework, such as Jest.
Because Selenium doesn’t ship with any testing tools, it can be cumbersome to set up the necessary environment to start using it to test your applications.
To avoid going through this setup process yourself, you can use libraries like Nightwatch.js, whose documentation is available at https://nightwatchjs.org, or WebdriverIO, about which you can find more at https://webdriver.io.
These tools, just like Selenium, can interface with multiple Webdrivers and, therefore, are capable of controlling real browsers. The main difference between these libraries and Selenium is that they ship with testing utilities.
Besides bundling testing utilities, these other libraries are also concerned with extensibility and offer different APIs that cater to a test-focused audience (figure 10.8).
Figure 10.8 Instead of having to set up your own testing infrastructure, you can use libraries like Nightwatch.js or WebdriverIO, which ship with all the tools necessary for writing tests.
The most notable advantage of tools like Selenium, Nightwatch.js, and WebdriverIO over other browser-testing frameworks and automation tools is its capability of controlling multiple kinds of browsers.
Due to its highly decoupled architecture, which supports interfacing with different kinds of drivers to control numerous distinct browsers, it supports all the major browsers available.
If your userbase’s browser choice is diverse, it will be highly beneficial for you to use Selenium or other libraries that take advantage of the Webdriver interfaces.
In general, I’d avoid Selenium itself if I’m using it exclusively to write tests. In that case, I’d generally go for Nightwatch.js or WebdriverIO. On the other hand, if you need to perform other browser-automation tasks, Selenium can be an excellent choice.
The most significant problem with these kinds of tools is that they make it too easy for you to write flaky tests, and, therefore, you’ll need to create robust testing mechanisms to have deterministic validations.
NOTE Flaky tests are nondeterministic tests. Given the same application under test, they may fail sometimes and succeed in others. In this chapter’s section called “Dealing with flakiness,” you’ll learn more about these tests and why you should try to eliminate them.
Additionally, because these kinds of tools control real browsers through HTTP requests, they tend to be slower than alternatives like Cypress, which run entirely within the browser.
Besides their possible slowness, configuring and debugging tests written with these tools can often be challenging. Without built-in tools to outline different test cases, run assertions, and monitor your test’s executions, they can take significantly more time to write.
10.3.2 Puppeteer
Like Selenium, Puppeteer is not exclusively a testing framework. Instead, it’s a browser-automation tool.
NOTE You can find Puppeteer’s documentation at https://pptr.dev. Within this website, you’ll also find links to Puppeteer’s complete API documentation.
Unlike Selenium, Puppeteer, shown in figure 10.9, can control only Chrome and Chromium. To do so, it uses the Chrome DevTools protocol, which allows other programs to interact with the browser’s capabilities.
NOTE At the time of this writing, support for Firefox is still in an experimental stage.
Because Puppeteer involves fewer pieces of software than Selenium and other Webdriver-based tools, it is leaner. When compared to those tools, Puppeteer is easier to set up and debug.
Nonetheless, because it still is exclusively a browser-automation tool, it doesn’t ship with testing frameworks or libraries to create test suites and perform assertions.
Figure 10.9 Puppeteer directly controls Chrome and Chromium through their DevTools Protocol.
If you want to use Puppeteer to run tests, you must use separate testing libraries, like Jest or Jest Puppeteer. The latter ships with all the necessary support for running tests using Puppeteer itself, including extra assertions.
A further advantage of Puppeteer over Selenium is its event-driven architecture, which eliminates the need for fixed-time delays or writing your own retry mechanisms. By default, tests written using Puppeteer tend to be much more robust.
Additionally, its debuggability is much better than Selenium’s. With Puppeteer, you can easily use Chrome’s developer tools to solve bugs and its “slow-motion” mode to replay the test’s steps in such a way that you can understand precisely what the browser is doing.
Given it’s much easier to write robust tests with Puppeteer, they take less time to write and, therefore, cost less. You also spend less time debugging them.
If you need to support only Chrome and Chromium, Puppeteer is a much better alternative than Selenium and other Webdriver-based tools due to its simplicity and debuggability.
Besides these advantages, because Puppeteer doesn’t focus on supporting numerous browsers, it can provide you with access to more features and offer more elaborate APIs.
The disadvantage of it focusing only on Chrome and Chromium is that you shouldn’t even consider it if you must support other browsers. Unless you can use different tools to automate the tests that will run in different browsers, Selenium or other Webdriver-based tools are a much better choice.
NOTE I’ll cover supporting multiple browsers in-depth in the penultimate section of this chapter, “Running tests on multiple browsers.”
Considering that Puppeteer doesn’t ship with testing-specific tools, if you’re not willing to set up a testing environment on your own or use packages like jest-puppeteer, it may also not be the best choice for your project.
10.3.3 Cypress
Cypress, shown in figure 10.10, is a testing tool that directly interfaces with a browser’s remote-control APIs to find elements and carry out actions.
NOTE Cypress’s full documentation is available at https://docs.cypress.io. It is exceptionally well written and includes many examples and long-form articles. If you’re thinking about adopting Cypress, or already did so, I’d highly recommend you to read their documentation thoroughly.
This direct communication makes tests quicker and reduces the complexity of setting up a testing environment because it reduces the amount of software needed to start writing tests.
Figure 10.10 Cypress spawns a Node.js process that communicates with the tests that run within browsers themselves.
Besides making it easier and quicker to write tests, this architecture allows you to leverage the Node.js process behind Cypress to perform tasks like managing files, sending requests, and accessing a database.
Furthermore, because Cypress is a tool created specifically for testing, it offers numerous advantages over tools like Selenium and Puppeteer, which focus only on browser automation.
One of these advantages is that Cypress includes many testing utilities right out of the box. Unlike Selenium and Puppeteer, you don’t have to set up an entire test environment yourself when using it.
When using Cypress, you don’t need to pick multiple packages for organizing your tests, running assertions, or creating test doubles. Instead, these tools are bundled into Cypress. Installing Cypress is all you need to do to start writing tests.
Even Cypress’s APIs are designed with testing in mind. Typical situations you’d encounter when writing tests are already built into Cypress’s methods. These APIs make tests simpler, concise, and more readable.
Imagine, for example, that you’d like to click a button that appears only a few seconds after accessing your application.
In that case, browser-automation tools would require you write a line of code that explicitly tells your test to wait for the button to be visible before trying to click it. When using these kinds of tools, your tests will immediately fail if they don’t find the element on which they must click.
Cypress, in contrast, doesn’t require you to write explicit code to wait for the button to appear. It will, by default, keep trying to find the button on which it wants to click until it reaches a timeout. Cypress will try to perform the click only once it finds the button.
Among Cypress’s marvelous testing features, another helpful one is the ability to “time-travel.” As Cypress runs tests, it takes snapshots of your application as it carries them out. After executing tests, you can revisit each of their steps and verify how your application looked at any point in time.
Being able to see how your application reacts to a test’s actions allows you to debug it more quickly and ensure that the test is doing what you intended it to do.
For example, if you have a test that fills out a form, submits it, and expects the page to update, you can use the test’s action log to see how your application looked at each of these steps.
By hovering over each action, you will be able to visualize your application’s state before the test fills any fields, after it enters data into each one of them, and after submitting the form.
Because you can run Cypress tests within real browsers, as you go through your application’s states, you can examine them in detail using your browser’s developer tools.
If your test couldn’t find an input to fill, for example, you can travel back in time to check whether it existed on the page and whether you’ve used the correct selector to find it.
In addition to inspecting elements, you can use the browser’s debugger to step through your application’s code and understand how it responds to the test’s actions.
When tests detect a bug in your application, you can add break points to your application’s code and step through its lines until you understand the bug’s root cause.
10.3.4 When to choose Cypress
Cypress’s main advantages over other tools stem from being it being a testing tool rather than a more general browser-automation software. If you’re looking for a tool exclusively to write tests, I’d almost always recommend Cypress.
Choosing Cypress will save you the time of setting up a testing environment. When using it, you won’t need to install and configure other packages for organizing tests, running assertions, and creating test doubles.
Because Cypress bundles all the essential tools for testing, you can start writing tests and getting value from them sooner. Unlike with Selenium and Puppeteer, you won’t have to set up multiple pieces of software or create your own testing infrastructure.
Besides being able to start writing tests earlier, you’ll also be able to write them more quickly, thanks to Cypress’s debugging features.
Particularly the ability to time-travel and inspect your application’s state at any point in time will make it much easier to detect what’s causing a test to fail.
Quickly detecting a failure’s root cause enables developers to implement a fix in less time and, thus, diminishes the costs of writing tests.
In addition to these debugging features, another factor that makes it quicker to write tests using Cypress are its simple and robust APIs, which have retriability built into them.
Instead of having to explicitly configure tests to wait for a particular element to appear or retry failing assertions, as you’d have to do with Puppeteer and Selenium, your tests will do that automatically and, therefore, will be much more robust, intelligible, and concise.
Finally, the other two characteristics to consider when choosing Cypress as your testing tool are its excellent documentation and comprehensible UI.
These characteristics improve developers’ experience, causing them to want to write better tests more often and, therefore, create more reliable guarantees.
The only scenarios in which I’d advocate against choosing Cypress is when you have to perform tasks other than exclusively running tests, or when you must support browsers other than Edge, Chrome, and Firefox—the only three web browsers Cypress supports.
I consider Cypress to be the most cost efficient of all the tools presented in this chapter, and I believe it will be suitable for the majority of the projects you will tackle. Therefore, it is the tool I’ve chosen to use for the examples in the next chapter.
Summary
-
UI-based end-to-end tests use your application’s interface as the entry point for their actions and cover your entire application’s software stack.
-
The more critical and challenging it is to test a feature, the more helpful it becomes to have UI-based end-to-end tests for it. When you have critical features that are difficult to test, UI-based end-to-end tests can accelerate the process of validating your application and make it much more reliable. These tests won’t forget any steps and will execute them much faster than a human could.
-
When UI-based end-to-end tests are too time-consuming to write, and the unit under test is not so critical, you should consider writing other kinds of tests that can deliver similarly reliable guarantees. If you’re testing a web server, for example, you can write end-to-end tests exclusively for its routes. If you’re testing a frontend application, you can use
dom-testing-libraryorreact-testinglibrary, if you have a React application. -
To write UI-based end-to-end tests, you can integrate browser-automation tools like Selenium or Puppeteer with your favorite testing libraries. Alternatively, you can opt for a solution like Cypress, Nightwatch.js, or WebdriverIO, which bundle testing utilities so that you don’t have to set up a testing infrastructure yourself.
-
Selenium interacts with browsers by communicating with a browser’s driver through a protocol called JSON Wire, which specifies a set of HTTP requests to send for the different actions the browser must execute.
-
Cypress and Puppeteer, unlike Selenium, can directly control a browser instance. This capability makes these tools more flexible in terms of testing and makes tests quicker to write, but it reduces the number of browsers with which these testing tools can interact.
-
The tool on which I’ll focus in this book is Cypress. I’ve chosen it because of its flexibility, how easy it is to set up, and its excellent debugging features, which include the ability to time travel, see your tests as they run, and record them. Most of the time, it is the tool I’d recommend if you don’t plan to support browsers such as Internet Explorer.