Machine learning (eventually leading to artificial intelligence) is the current buzzword that's thrown around a lot in the automation sphere. It promises a lot, but what can it actually do for testing?  

Well, theoretically, it can take over the job of a tester by learning how a system works and then using its knowledge to search through the system for known bugs. It sounds amazing (or scary depending upon your point of view), and it is also a bit of a nebulous statement; there are hundreds of different things that testers look at. 

Now this all sounds like doom and gloom: the machine learning apocalypse is coming and all of us testers will soon be obsolete. Well, I don't think that is true. Machine learning systems need to be taught how to do things. If they are not properly trained, they will not work in the way they are expected to work. Also I've worked with lots of talented testers over the years with loads of experience and they have still missed problems in systems they have tested; I do it myself. I've never met a single person capable of finding every single bug in a system.  What makes us think that machine learning will be any different? At the end of the day, we are a product of our experiences and the way we test is shaped partly by the way we were trained and partly by our experiences of things that have gone wrong in systems we have tested in the past. 

Every single person will have slightly different training and experiences. This diversity is what allows us to find bugs that other people don't find. How would we provide artificial intelligence with all of this diversity and experience? Are we going to end up training multiple different artificial intelligences and then pointing a few of them at a system we want to test so that they can all find different things? If we have a couple of artificial intelligences looking at things, why not some humans as well to give us yet another point of view?

Then, we also have the task of validating that the machine learning algorithms used to create artificial intelligence work as expected. We will need to train these artificial intelligences and check that they have been trained correctly. How do you validate that you have trained somebody to find bugs when there are no concrete outcomes of the training?

Jobs in general always evolve as technology marches onward and I think that one of the ways that the job of a tester will change will be for us to spend more time teaching machines how to successfully test things. Some of this learning will be applicable across multiple different websites, some of it probably won't. I think the hardest thing for a machine to do is learn to check the look and feel of the system under test. At the end of the day look and feel are very emotive and what is right greatly depends upon a person's point of view. Secondly, depending upon your target audience, the look and feel of a website change greatly.

Let's explore this scenario a bit more. Imagine artificial intelligence that has been taught to look at sites that advertise soft drinks. Most of these sites are going to be colorful, cool, and aimed at the younger generation. They often also throw in eye-grabbing lifestyle statements that imply you will be the coolest extreme sports practitioner in the world if you drink that brand of soft drink.  

Let's take this artificial intelligence and point it at a funeral site. The sort of site that has neutral colors and doesn't have statements littered around it that try to make it sound "cool" to use their services. It wouldn’t know what to do, the entire style of the site would be completely alien to it. If you let this system loose to log all the look and feel errors it would have a field day, but most of what it found would be invalid.  Imagine if it was sufficiently advanced to start changing the site to automatically fix all of these errors it had found. Can you imagine it? A funeral website with loud eye-grabbing colors and statements such as "Only the coolest people let us bury their loved ones!" It would be a complete disaster.

You could, of course, train your AI to work on funeral sites, but what if you then pointed it back at your soft drink site? As ever, context is key, and one of the skills we have as testers is being able to switch context for different sorts of sites. Now, I'm sure you could eventually get to the point where you could train an AI to work out what sort of site it is and then utilize algorithms specific to that types of site. But that's a lot of work to be able to identify and switch between the different types of sites. Also companies that sell soft drink aren't usually involved in the funeral business, or vice versa. Why would they want to spend lots of time training an AI to context-switch when there is little value in them having that ability? I think it's far more likely that companies will create their own artificial intelligences that are designed specifically for the sort of content that they create.

I think the key thing to take away here is that no matter what people say, machine learning (and by extension artificial intelligence) is not going to be a silver bullet that fixes everything. As I've said previously; when it comes to automation, there are no silver bullets!

Now this is a book about Selenium, so obviously we are going to be interested in how machine learning can help us with our Selenium tests. There are three main areas of help that I can think of:

• Machine learning that helps identify problems by examining the results of your tests
• Machine learning that tries to automatically fix errors with your tests
• Machine learning that tries to write your tests for you

You will notice that the three points I have listed kind of depend upon each other; if a machine learning system cannot identify problems by looking at results it will not be able to automatically fix errors with your tests. If it can't automatically fix errors with existing tests, how is it going to start writing its own tests?

So, let's have a look at these three areas of machine learning and see what they have actually delivered so far. Well you may already be using the first area of machine learning in your Selenium tests today; it's called visual validation.