Possible Causes

• This is most likely an isolation issue. Since the test is interacting with other tests in the build, something in this interaction may not be working correctly; for example:The test is using a set of data that is being used by other tests.
• The test is trying to access elements or locators that are being used by other tests.

Possible Solutions

• Create a set of data for the flaky test, run it alone, and then run it as a part of the build, and see what happens.
• Identify other tests that are using or referencing locators and elements from your flaky tests. Try to find and fix any overlaps or conflicts.

Possible Causes

Assertions are a great way to verify that tests are working properly, but a poorly written assertion can cause more harm than good.

Possible Solutions

Verify all of the assertions in your test. The error might be as simple as a misspelling or a miscalculation.

Possible Causes

Sometimes, a flaky test is not really a flaky test, but its execution is unstable due to external services, including software (web services or internet service providers) or hardware (network infrastructure) services.

Possible Solutions

If this is the problem, there might not be much you can do, because the services might be out of your reach. What you can do is verify how your test interacts with the external services, as follows:

• Check how your test deals with information going to and from a service. Is it sending and receiving the right information? Is it handling the information in a proper manner?
• Is there a network setting that you have missed? This could be a firewall that is blocking connections, or a service that is listening at the wrong port.
• Are the external services unstable? Try to contact the team in charge. Maybe they are running maintenance or a clean-up routine.

Possible Causes

In previous chapters, we discussed how modern web applications work (pop-up windows and interacting with users in complex forms, for example) and how they can make writing tests a little more difficult.

Possible Solutions

Verify all of the locators in your test. It is always advisable to create unique and descriptive locators that are unlikely to change.

So, it is very important that the testing team understand how a web application works and establish the best way to interact with its elements, in order to avoid flaky tests.

Possible Causes

As we have previously mentioned in this book, elements on a web page can be visible at some point and disappear the next second. Or a button is no longer clickable after a specific interaction from the user. That is why implicit and explicit waits exist: to wait for an element to be visible or usable. But they could also cause issues in our test as most of the time, it is impossible to determine the exact amount of time we need our test to wait. It could happen, for example, that a 10-second wait for an element works most of the time, but under certain circumstances (network latency or processing times of certain tasks), those 10 seconds will not be enough and will cause our test to fail.

Possible Solutions

Do not use waits at all. This is the easiest way to avoid flaky tests caused by too-short or too long timeouts, but also an unrealistic one given the way web applications work.

Use wait times combined with user interface alerts. If you need to wait for a pop-up window to appear, you can set a waiting time on your test. But the waiting time you set maybe too long or too short. Create a UI alert as a fail-safe: even if the waiting time expires, your application will still continue working once it is notified that the window showed up.