Tools, like friends, should be few and well-chosen.
An illustration depicts a clipart of a robot with a raised left hand.
This chapter introduces Boozang and how it addresses the pain points of a coded solution as described in the previous chapter. It also presents a possible selection approach for selecting an automation tool, particularly the one I followed before adopting Boozang.
Proof of Concept Phase
Every organization has its own rules, means, and degree of flexibility when adopting new tools, and the same applies when selecting a Test Automation tool.
In any case, a typical selection process starts with a more or less clear need, and a list of criteria that the tool must satisfy.
When investments and risks are high, RFI (Request for Information ) and RFP (Request for Proposal ) phases can be applied, where software editors formally provide information based on an input document.
Otherwise, for small investments, this information can be taken directly from the editors’ websites, where you usually find a list of features and supported technologies and integrations. But of course, from their own website, every tool is the best one :).
What I look for from the editors’ website is a clear list of features, transparency in the licensing scheme, and links to documentation to see more details.
Another common source of information for medium to large enterprises is the Gartner Magic Quadrant reports, which give you a good idea of the best players for a particular topic and the specific market dynamics.
These preliminary phases allow you to do a shortlist and eliminate some candidates. But until now, you haven’t tested them yet.
Here is where a Proof of Concept (PoC) phase typically comes into play. The PoC phase allows you to uncover hidden risks. In a PoC, you usually install the tool and implement real automated tests on a representative scenario. The scenario should be complex enough to challenge the tool, but not too much, so that you can implement it with a reasonable effort. In fact, within 1 or 2 days, you should already be able to make up your mind about the tool’s strengths and weaknesses.
In my case, around September 2019, I started looking for a tool to cover functional tests on End-to-End business processes and mainly web applications. I could afford to keep Mobile and Desktop technologies out of the scope.
I decided to evaluate the top five automation tools as rated by Gartner.
With two of them, I wanted to pursue the PoC phase.
The first one, at that time leader of the market, was soon ruled out: besides being expensive (even for a PoC phase), I found it too invasive and hard to fit into my organization in terms of skills, integrations, and change of processes. I like how we manage releases and manual testing with JIRA (I will cover these aspects in the section “Using JIRA with XRay”), and I don’t want to change our process because of a new automation tool.
The second one was a tool based on image recognition. I was initially skeptical. I thought the image recognition approach was not a good fit for our functional and business rules intensive tests, but I gave it a shot. In the first scenario, the validation on a text would fail when the window was smaller and the text broken into two lines. This was enough for me to rule it out.
Not convinced with the “big players,” I looked into smaller and newer solutions. Here Google searches, blogs, and word of mouth are the main channels.
I came across Boozang via the TestGuild podcast ( https://testguild.com/podcast/ ). The podcast caught my interest and convinced me to give Boozang a closer look. After following the Udemy courses,1 I played with it for a couple of hours and decided it was worth doing a PoC scenario.
Luckily, our Digital Services team (responsible for maintaining the eCommerce tcs.ch site and mobile app), just a few weeks before, had defined 10 critical Use Cases as scope for a potential automation effort. All these Use Cases had clearly defined steps and also screencasts to understand well the UI interactions.
I selected one of these Use Cases (number 6) as scope for the PoC. This case was a tough one, covering the e-Commerce site, a backend user activation system, and SalesForce (the CRM).
Use case description
UC06 - Acquisition PJ Privée IND by a member identified by inline eCare login
- Go to PJ Privée product page
Insured persons? Individual
Press continue → Modal “Are you a member already?” appears
Press yes → “enter membership information” appears
Enter identified eCare login + password
Press 'add to basket' → Shopping cart appears
- Press continue → Personal data page appears
Data is already pre-filled with identified account data
- Continue → Payment page appears
Select an activation date in the future
Select Mastercard as payment method
Check the legal checkboxes
Continue → Six payment modal appears
Finalize → Confirmation page appears → Confirmation email appears in the mailbox
In a Gherkin format (Gherkin syntax will be covered in Chapter 13, “Gherkin and Behavior Driven Development”) would be something like this:
Given an existing client with an active <existing_product>
And an active eCare account
When the client buys a <product> with option <option>
And the payment method is <payment_method> with <payment_option>
Then the confirmation page appears with <final_price>
A diagram depicts the relationship between a client and salesforce through the internet, with backend user management to manage the data.
System interactions in the PoC scenario
During the PoC, what impressed me about Boozang was the ease of use. The UI is self-explanatory, and recording the first scenarios was very easy.
Being able to perform a PoC at no cost, at my own peace, and without having to deal with salespeople was also a good point. Boozang has a free version unlimited in time that allows you to perform a PoC smoothly.
Pilot Phase
After the PoC phase that should last a few days, the next step is to perform a Pilot phase.
A Pilot phase is usually longer, it should involve more people, and it should be complex enough to stress the tool in all the needed functionalities that can be important in the long term.
In this case, you may need a certain budget and editor support.
In my case, I had planned around 20 man-days of effort.
During this time, I had also planned to involve the Boozang team to make sure I would start on the right track.
Having a Pilot phase was very important. It allowed us to anticipate many issues we would otherwise have experienced in a real project, where time constraints are different.
We found and promptly fixed some Boozang bugs, and we found ways to deal with application issues like SalesForce pages with slow loading iFrames and hard to automate popups from the backend application.
A Week with Mats
As part of the Pilot activity, I had the pleasure to work with Mats Ljunggren, Boozang co-founder and CEO , for a week.
We set the objective of automating the 10 e-Commerce Use Cases using the brand new Cucumber scenarios feature.
When we started, early on Monday morning, we could have jumped straight into the automation work, but instead, we decided to have a chat over breakfast where I presented to Mats our organization and short-term automation challenges.
After that, Mats figured out the basic building blocks to set up in Boozang. This approach gave us a good start with solid foundations.
In Boozang terminology this means a good separation of Modules and Sub Modules.
Each day we would start with some planning over coffee and pain au chocolat, and then hands-on work until the end of the day.
It became immediately clear that the critical aspect for us was the data management. When debugging a test, it must be clearly visible how the data flows from a Gherkin scenario down to tests. This part was definitely the number one improvement for Boozang.
At the end of the week, we had completed two Use Cases and most of the rest. Overall, I would say 80% of the automation code was done because a lot of tests were re-used among Gherkin scenarios.
We also managed to have our first Jenkins job running via the Docker container.
During Mats’ stay a couple of times, we made our priorities clear, and every time we were agreeing on the most important thing: robustness over features. As the principles in the Agile Manifesto, it does not mean that features are not important. But when compared to overall robustness, they have lower priority. There is nothing worse in Test Automation than mistrusts toward the tool; hence robustness should come before any new feature request.
What we did not manage to do during our week together, as initially planned, was a thorough time tracking per scenario. With each scenario saved in JIRA (I will cover our JIRA setup in detail in Chapter 14, “JIRA with Boozang via XRay”), we wanted to precisely track the implementation effort. But the fact is that the construction of the tests was not done on a per-scenarios basis, but rather on application modules. Therefore, we only tracked the time for translating the initial scenarios into proper Gherkin, which ended up being almost one and a half day.
A screenshot depicts a Boozang pilot page, which includes the automation rules in structure for JIRA with Level 1 and 2 features of e-commerce.
JIRA Structure page with features and scenarios of the Pilot phase
Pilot Results
During the Boozang pilot, I was suggested to evaluate another codeless tool, Testim,2 which seemed to have a similar approach. The Testim PoC was successful enough to deserve a Pilot phase against Boozang.
In the following figures, you can see the results of the two pilots. Since the pilot execution time many improvements were done on Boozang, I will report them below.
An illustration depicts Testim versus Boozang, which includes a color scheme of green, yellow, and red, along with the higher and better scores.
A table lists 3 columns: topic, Testim, and Boozang. It depicts different methods along with their corresponding scores.
A table has 3 columns, which depict different methods along with their corresponding score values of the automation tool.
Versioning: Boozang now provides a branching system to manage concurrent work. I will cover this in section “Branching and Merging.”
See upstream tests impacted: This functionality is now present in Boozang.
Cloud Grid: With the docker workers, it is now possible to set up your local or cloud-based grid for test execution.
Documentation: Boozang has a good documentation section and a forum for users to discuss new features.
Parallel execution: Support for parallel execution is now present in Boozang; see section “Tips to Keep Execution Time Low.”
Company size: The Boozang team is growing. Today it still cannot be compared to other top of the market tools, but I bet it will keep growing. Moreover, many times, having a small but highly skilled team has been quite an advantage in terms of agility and delivery time.
Why Boozang
In this section, I will summarize some key elements for choosing Boozang as your Test Automation tool. First of all, it has to be clear what Boozang is not for.
Boozang is not a Test Automation tool for desktop applications: in this case tools like Sikuli (free and open source, http://sikuli.org/ ) or commercial tools like Ranorex and Tosca are more suited.
Boozang is not a Test Automation tool for native mobile applications. In this case I would go with Appium ( http://appium.io/ ) for a coded, free, and open source approach, or other commercial tools.
Boozang is not a release and manual test management tool, where you can manage in an integrated way Requirements, Tests, and Bugs and have release readiness reports. In this case, tools like JIRA and its add-ons ecosystem are to be preferred.
Boozang is also not a load and performance testing tool. If you need to test the performance of your application and generate a load with thousands or more users, tools like JMeter ( https://jmeter.apache.org /) and NeoLoad ( www.neotys.com/ ) are more suited. Nonetheless, Boozang offers some interesting performance insights while running your tests (see Chapter 11, “Reporting and Troubleshooting”) that can spot basic performance issues.
If your focus is to automate functional tests and End-to-End scenarios on web applications that run within the browser, Boozang is definitely a good choice. Here are the main reasons, which we will cover in detail in the rest of the book.
Ease of Use
This is the first thing that you will probably notice while using Boozang. You can create a simple test without even looking at the training material.
The GUI is made in such a way that both technical and non-technical people will quickly find what they need. The SaaS approach means that setting up a project is a matter of a few clicks (see Chapter 6, “Starting Up with Boozang”).
Focus on Test Automation
Most organizations today have in place their own tool (or tools ecosystem) to manage requirements, manual tests, create and track Defects, and produce advanced reporting on release readiness .
Boozang’s smart approach is to focus on Test Automation and provide all the necessary solutions to integrate with your existing tools rather than duplicating these features.
Simple and Solid Language-Based Element Recognition
As we will see in Chapter 7, “Elements Location Approach,” Boozang does not rely on Selenium Web Driver to locate elements, removing a lot of complexity as we mentioned in Chapter 2, “Coded Approaches and Selenium Pain Points.” It instead uses an efficient language-based location strategy that allows you to build stable locators, and maintain them easily in case your application changes .
Data Management
Data management is probably the most important piece of functionality of every Test Automation tool. Boozang offers you the possibility to store data at three different levels: project, module, and test. Tests can also receive and pass parameters. Each of these options has its own peculiarity, as we will see in Chapter 10, “Data Management.”
Gherkin and BDD Support
More and more organizations apply Gherkin and sometimes pure Behavior Driven Development in their agile development. Boozang supports Gherkin and the BDD approach, making it a good fit for agile projects where people with different skills can participate in the automation effort.
We will cover BDD and Gherkin in Chapter 13, “Gherkin and Behavior Driven Development.”
Concurrent Development
Boozang can effectively manage concurrent automation implementation with solutions that have been used for decades in software developments: branching and merging. But you do not need to master GIT or SVN since the Boozang GUI hides a lot of the complexity that is behind VCS features.
Support and Vision
Last but not least, Boozang support is timely and eager to listen for improvement coming from users .
Their roadmap is well centered around common automation issues and keeps the focus where it is needed: easily build long-lasting automated scenarios.
Codeless Approach: How Important Is It?
I admit I was more than skeptical about this concept in Test Automation.
I had practiced tools based on the image recognition approach, then object recognition with HP Quick Test Pro, and then Selenium Web Driver. I appreciated a lot the flexibility that Selenium Web Driver offers, mainly due to the fact that within a development environment you can do almost anything, from GUI to API and Database access. In terms of modularity, you have no limits, and CI/CD integration is like for any other development project.
I assisted to lots of demos from tools claiming that they can be used by “Business Users,” without any need of a development background. But when you dig in a bit more, you realize that the typical Business User is not familiar with modularity concepts, refactoring, and reusability. Without considering the skills needed to accurately manage your HTML selectors. Even though they can easily record monolithic test cases, if you want your automation project to sustain the effects of time, avoiding good design and modularity is not an option.
If you ask companies who have applied such tools with success in most of the cases, you will find that they rely on a team of automation engineers certified on the tool.
Boozang is no exception, let’s be clear. Yes, it is indeed codeless in 80% of the cases, which is not bad at all, and it does not close the doors to programmatic solutions with JavaScript.
Good design and modularity are still of primary importance in Boozang. For this reason, I would suggest anybody to have at least one person in the team with programming or automation experience that can act as an architect and keep things in order.
A cool feature that Boozang provides to help in these housekeeping tasks is the quality checks, which I will cover in the section “Quality Control.”
Therefore, codeless is certainly important, but it is not at all the final goal. The goal is to easily build long-lasting automated scenarios. And I find that Boozang keeps this concept at its heart. It always strives to simplify your work, but it does not hide away from dealing with complex issues.
A similar reasoning can be applied to recording. The recording feature is very important, but you are not going to build a large and stable suite of automated tests just with recording. Recording is there to save you time while initially building your tests.