Chapter 10: Evaluating and Improving the Performance of a Website

Many things can make a website a success or a complete failure. In Chapter 9, Scraping tools, we talked about a real estate website that can't be launched without content. On many websites, content is the number-one feature. Amazon.com could be the best website in the world, but if it doesn't have the book you are looking for, you will go somewhere else.

For other websites, functionality is the number-one feature. A website such as Trello.com is a success because you can move cards from one list to another easily and intuitively. But functionality is not only about rich web pages. If we go back to the Amazon website, the website is pretty straightforward. It doesn't use any cool UI framework, but it has a great search and well-planned navigation.

The website design can also be considered a feature. While some websites such as www.google.com might look simple and focused on delivering content, you can see that other websites, such as www.apple.com, invest a lot in design. You can see that design is the number-one feature on www.apple.com.

But most websites will share the same feature: speed. Speed is a feature. When planning a website, the stakeholders might argue whether they want to invest in a rich client or not. They can discuss whether they should hire a designer or not. But if you ask about the speed, there is only one answer: "We need the website to be fast."

In this chapter, we will learn how to use performance metrics to solve several issues that can arise with websites. We will look at functionality, speed, and how we can measure these key performance points with Google Lighthouse.

We will cover the following topics in this chapter:

  • The issue of performance
  • Getting started with Google Lighthouse
  • Tracing pages
  • Analyzing code coverage

By the end of this chapter, you will be able to implement performance metrics on your website and help the development team improve the website's performance.

Let's get started.

Technical requirements

You will find all the code of this chapter on the GitHub repository (https://github.com/PacktPublishing/UI-Testing-with-Puppeteer) under the Chapter10 directory. Remember to run npm install on that directory, and then go to the Chapter10/vuejs-firebase-shopping-cart directory and run npm install again.

The Issue of Performance

As I mentioned in the introduction, speed is a feature. You might be asked to make a simple website, but no one, ever, will tell you to make a slow website. Websites need to be fast. Performance is the key to success. Performance increases user satisfaction; it results in high conversion rates and higher revenue. Let me share some facts with you:

As you can see, performance puts money in your pocket.

There is one more thing on which I'm sure you will agree with me: No matter the website, no matter what they sell or offer, performance is the number-one feature in the mobile experience. When you are on the street, you don't care about the style; you don't care about functionality. The first thing you need is the website to load, and load fast. You need to measure the performance on mobile.

The problem with this feature is that, in general, people don't know how to measure speed. If we go back to the other features, they are easy to measure. It's easy to discuss content. Content is easy to measure:

  • I want to ship the website with content.
  • How many items?
  • Over 1,000.

Functionality is, in general, something you can write down on a spec:

  • I want an e-commerce site with an outstanding search experience.
  • What does that mean?
  • It should support typos, and I should be able to search for part of a word.

Design is about whether you want to put effort into the look and feel or not:

  • We need a website with a great design.
  • OK, we need three designers.
  • Perfect.

But speed is hard to discuss:

  • The website needs to be fast.
  • How fast?
  • I don't know… fast?
  • But how fast do you think it should be?
  • I don't know… faster?

The second problem is that we tend to react to performance issues. We don't realize something needs to be fast until we realize it's slow.

Third, speed is a matter of expectation and comparison. The user would say that the website is slow. They would also say that they use Google Drive and that it's way faster. The developer would reply that the website seems fast in their opinion, and, of course, that they don't have Google's budget.

And the last problem is that we don't know how to test the website's performance. We would get a bug report from a user, saying that the website is slow, and the QA analyst would grab that bug report to validate it, but what's the tool that the analyst has to validate that issue? Going to the website and checking whether it "feels" slow to them.

That is the perfect cocktail for a disaster: No way to measure, no plan—it's all about feelings and different expectations.

We won't be able to solve all these problems in just one chapter in a web automation book. But, we will see some strategies and tools to help you and your team to measure and improve your website's performance. Let me share some tips with you to get started.

First, choose what you need to measure. If it's the entire website, that's OK. But I would start with the most popular pages first. Begin with the home page. Then, continue with the main workflow of the website. For instance, for an e-commerce website, you would want to test the product details and the checkout page. Ask the people in charge of analytics what pages bring more conversions, and focus on those pages.

Second, define the maximum amount of time a page can take to load. You could say that the home page should never take more than 30 seconds to load, under any circumstances. This is an excellent use for Checkly, the platform we saw in Chapter 6, Executing and Injecting JavaScript. You could code a test to check that the page doesn't take more than 30 seconds to load in production and keep that check running on Checkly. We will see how to implement this later in this chapter. Once you have set up that check, you and your team can set more strict goals. For instance, the search page should never take more than a second to load.

Third, measure performance degradation. Many times, setting a limit is hard, and it can become a guessing game. You can start by measuring how the performance evolves over time. Is the website becoming faster or slower? Are we getting better or worse? This is a great approach, but it requires a little bit more work. You need to start storing data over time and build something to visualize that information.

And lastly, use the tools you learned in this book. We talked about Checkly, but remember all the emulation techniques we learned back in Chapter 8, Environments emulation? You can set different goals for different network speeds.

This is all you can do to measure a website's performance. In this chapter, I want to show you which tools you have to implement these ideas. Let's start with Google Lighthouse, a tool we can use to measure several important metrics.

Getting started with Google Lighthouse

As we saw in the previous section, it's not easy to determine how fast "fast" is. Google came up with an idea. They built Lighthouse, "an open-source, automated tool for improving the quality of web pages. You can run it against any web page, public or requiring authentication. It has audits for performance, accessibility, progressive web apps, SEO and more" (https://www.hardkoded.com/ui-testing-with-puppeteer/lighthouse).

Lighthouse will grab the website you choose, apply a list of metrics and recommendations it finds important, and give you a score from 0 to 100. It will analyze the website under five categories:

  • Performance: The most popular category. Lighthouse will measure how optimized the website is, that is, how fast it gets ready for user interaction.
  • Accessibility: I would love to see developers paying more attention to this category. Here, Lighthouse will evaluate how accessible the website is.
  • Best practices: This is another popular category. Lighthouse will evaluate a few good practices to incorporate into the website.
  • SEO: This category is used a lot by people in charge of marketing. Some companies even have SEO experts looking at this. This category is about how optimized the website is for search engines. You might agree or not on how the other categories are measured, but here Google is telling you: "This is how we measure your website." You will want a score of 100 if you want to secure your spot on the first page of Google.
  • Progressive Web App: If the website is a progressive web app, this category will evaluate aspects of that progressive web app.

    Important Note

    Progressive web apps (PWAs) are websites prepared to be installed as native applications. Many PWAs have offline capabilities and try to get close to a native app experience.

In this chapter, we will focus only on the Performance category. But before getting into the details of the performance category, let's see how we can run this tool. Lighthouse comes in four flavors, which we will cover in the following sections.

As part of Chrome DevTools

If you open DevTools, you will find a Lighthouse tab. If you can't find it, you can add it by clicking on the three dots in the tool's right corner, then going to More tools, and then finding the Lighthouse options. You should see something like this:

Lighthouse inside DevTools

Lighthouse inside DevTools

You should now have the tab there with all the options to generate the report. The process will run Lighthouse locally, which is good, but that would mean that the Lighthouse results will be based on your hardware, CPU, available RAM, network speed, and so on.

Using PageSpeed Insights

Google saw that results might fluctuate based on your hardware, so they made a PageSpeed Insights (https://www.hardkoded.com/ui-testing-with-puppeteer/pagespeed) where you can run Lighthouse using Google's hardware. That would make it more stable, but you could get different results even using Google's hardware.

Using the command line

You can also use Lighthouse from the command line. I wasn't that excited about having Lighthouse in the command line first. But then I realized that it's way more productive to use it from the command line than opening a browser, going to DevTools, and so on.

You can install the Lighthouse CLI (command-line interface) by installing the Lighthouse module globally. This is the first time we do this in this book, but it's no different from how we installed the Puppeteer module. We just need to add the -g flag like this:

npm install -g lighthouse

NPM global modules

When you run npm install with the -g flag, the module will be installed in a shared directory rather than in the node_modules folder, and it will be accessible by any app. Additionally, if the module provides an executable command, it will be accessible from the command line like this Lighthouse module.

Once installed, you will be able to launch lighthouse from the command line, passing the URL and, additionally, extra command-line arguments such as --view, which will launch the report after evaluating the website.

With this line of code, you will be able to see the Lighthouse result for www.packtpub.com:

lighthouse https://www.packtpub.com/ --view

Wondering what the result is? We'll get there.

The last option available is one that we will use a lot, and it's using the node module as part of our code.

Using the node module

We will be able to use Lighthouse in our unit tests using the node module. Let's take a look at the example from the Lighthouse repository (https://www.hardkoded.com/ui-testing-with-puppeteer/lighthouse-programmatically):

const fs = require('fs');

const lighthouse = require('lighthouse');

const chromeLauncher = require('chrome-launcher');

(async () => {

  const chrome = await chromeLauncher.launch({chromeFlags: ['--headless']});

  const options = {logLevel: 'info', output: 'html', onlyCategories: ['performance'], port: chrome.port};

  const runnerResult = await lighthouse('https://example.com', options);

  const reportHtml = runnerResult.report;

  fs.writeFileSync('lhreport.html', reportHtml);

  console.log('Report is done for', runnerResult.lhr.finalUrl);

  console.log('Performance score was', runnerResult.lhr.categories.performance.score * 100);

  await chrome.kill();

})();

The code is not very complicated. We launch a Chrome browser using the chrome-launcher module. Then we launch lighthouse, passing a URL and a set of options. The lighthouse function will return an object, I called it runnerResult, which contains a report property with the report as HTML and also a property called lhr (Lighthouse result) with all the results as an object. We will use that property to assert the minimum values we want to get.

Now that we know how to launch Lighthouse, let's see how the report looks. In order to avoid hurting feelings, we will run Lighthouse against the very same Lighthouse website: https://www.hardkoded.com/ui-testing-with-puppeteer/lighthouse. Let's see whether it is as fast as they say. As I mentioned before, I felt really comfortable with the command-line tool, so I will run this command:

lighthouse https://developers.google.com/web/tools/lighthouse --view

After running that, I got a new tab in my browser with the following result:

Lighthouse result

Lighthouse result

To be honest, I picked a Google website on purpose. As you can see in the screenshot, the results can be rough. A score of 55 doesn't mean that your site is terrible. It just means that the website can be improved a lot.

You also have to keep in mind that a single company made this scoring system and, although many companies use it as a marketing number to show off how good the score is, this is not the final word. It's just one way to measure the performance of your website.

Another thing to keep in mind is that although it measures many things, its focus is on the time taken to load the page, and you should know that performance is more than that.

Let's dive into the performance category details.

The performance category

Each category in Lighthouse consists of three sections: metrics, opportunities, and diagnostics. Although only the metrics are used for the category score, it is by implementing the opportunities and looking at the diagnostics that you will improve the metrics.

Each category has its own set of metrics, opportunities, and diagnostics. In particular, the performance category has 6 metrics, 17 opportunities, and 13 diagnostics.

Let's take a look at the performance metrics.

Performance metrics

The first metric is First Contentful Paint. It has a weight of 15% on the overall performance score. According to Google, this metric "measures how long it takes the browser to render the first piece of DOM content after a user navigates to your page. Images, non-white <canvas> elements, and SVGs on your page are considered DOM content; anything inside an iframe isn't included." You will get a green score if this metric is under 2 seconds. You can read more about this metric and how to improve the score at https://web.dev/first-contentful-paint/.

The second metric is Speed Index. It has a weight of 15% on the overall performance score. According to Google, this metric "measures how quickly content is visually displayed during page load. Lighthouse first captures a video of the page loading in the browser and computes the visual progression between frames. Lighthouse then uses the Speedline Node.js module to generate the Speed Index score." You will get a green score if this metric is under 4.3 seconds. You can read more about this metric and how to improve the score at https://web.dev/speed-index/.

With a weight of 25%, the third metric is Largest Contentful Paint and is one of the most important metrics. According to Google, this metric "measures the render time of the largest image or text block visible within the viewport." You will get a green score if this metric is under 2.5 seconds. If you are interested in how they get to know what the "largest contentful element" is, check out their article at https://web.dev/lcp/.

The fourth metric is Time to Interactive. It has a weight of 15% on the overall performance score. According to Google, this metric "measures how long it takes a page to become fully interactive." You will get a green score if this metric is under 3.8 seconds. You can read more about this metric and how to improve the score at https://web.dev/interactive/.

The fifth metric is Total Blocking Time, which is the second metric with a weight of 25% on the overall performance score. According to Google, this metric "measures the total amount of time that a page is blocked from responding to user input, such as mouse clicks, screen taps, or keyboard presses." You will get a green score if this metric is under 300 milliseconds. You can read more about this metric and how to improve the score at https://web.dev/lighthouse-total-blocking-time/.

With a weight of just 5%, the last metric is Cumulative Layout Shift. According to Google, this metric "measures the sum total of all individual layout shift scores for every unexpected layout shift that occurs during the entire lifespan of the page." You will get a green mark if the score is under 0.1. You can read more about this metric and how to improve the score at https://web.dev/cls.

I came to a few conclusions after digging into all these metrics. First, it's clear that they were made by web performance professionals. It would have been impossible for me to build all these metrics in my daily job. The research behind these metrics is impressive.

On the other side, when you look at the weights and thresholds, while they look well thought out, they might sound quite arbitrary. Why is total blocking time more important than time to interactive? Or why do I get a green mark on a speed index under 4.3 seconds? Why not 4.2? Why not 4.4? But this is better than nothing.

You might also have had the feeling of this being too complicated or hard to understand. Some concepts, such as Largest Contentful Paint, sound like rocket science, at least to me. That's why you might find it easier to understand and follow the opportunities section of the report.

Performance opportunities

Opportunities are calls to action. These are not just simple recommendations. Here, Lighthouse will get to the point: "If you do this, you might get this bump in performance."

As I mentioned before, there are 17 opportunities under the performance category. We won't cover all of them. But I would like to go through a few of them so you get an idea of what this section is about.

Let's cover only the opportunities shown when we process the Lighthouse website. These are the opportunities I got in my report:

Performance opportunities for the Lighthouse website

Performance opportunities for the Lighthouse website

Here, we have five opportunities. Let's unpack them:

  • Properly size images: Lighthouse found images that are bigger than the size shown on the page.
  • Serve images in next-gen formats: Here, Lighthouse checks whether you are using JPEG or PNG files instead of "next-gen" formats such as WebP. I'm not particularly a fan of this opportunity. Although WebP is supported in most popular browsers these days, it is not a popular format in general yet.
  • Eliminate render-blocking resources: I think this is a critical opportunity. Here, Lighthouse found that many resources are blocking the first paint of the page. Paying attention to this opportunity would improve your metrics considerably.
  • Remove unused JavaScript: Lighthouse found JavaScript code that is not being used. Although this would be easy to detect by Lighthouse, this issue is not that easy to solve by developers. Developers these days use bundlers to pack all their JavaScript code, and shrinking the final code to only the code that the page needs can be challenging.
  • Remove unused CSS: This opportunity is similar to the previous one, but it's related to CSS styles.

I love this section because Lighthouse doesn't just tell you what the opportunities are; it also gives details showing you where the opportunity is and what would be the performance bump. Let's see, for instance, what we get when we click on the Properly size images row:

Properly size images section

Properly size images section

As you can see there, Lighthouse is showing us which images we can improve and what the potential savings we could get are. You will get the same kind of details on every opportunity.

The last section within a Lighthouse category is the diagnostics.

Performance Diagnostics

I see the diagnostics section as a list of things you should consider to improve your website. As I mentioned before, the performance category has 13 diagnostics, but you will see this number change over time.

This is how the diagnostics section looks:

Performance diagnostics for the Lighthouse website

Performance diagnostics for the Lighthouse website

As you can see, you will have the same level of detail as in the opportunities section, but these diagnostics sound more like recommendations to improve over time on your website. For instance, let's take a look at the Minimize main-thread work section:

Minimize main-thread work section

Minimize main-thread work section

As you can see, it seems to make sense what the diagnostics section reports. There is some script evaluation that is taking 1,490 ms. But that doesn't sound like a call to action. It's more something to consider if you need to improve the website's performance.

Now that we have learned about what Lighthouse is, let's see how we can test our website's performance by adding Lighthouse to our tests.

Using Lighthouse for testing

Let's be clear, Lighthouse is not a testing tool. It's a tool used more for developers to check their websites' performance. But, as we mentioned many times in this book, the role of QA is to honor the customer. It's to ensure that the customer gets the best product the team can deliver. We will use Lighthouse to ensure that the customer will get the fastest website we can deliver.

I can think of three ways we can test a Lighthouse report:

  • Ensure that a page has a minimum performance score.
  • Ensure that a metric is below a threshold.
  • Ensure that an opportunity is not found.

Let's begin by checking the performance score.

Ensure that a page has a minimum performance score

The first test we can make using Lighthouse is making sure that our page performance won't degrade over time. We will check that our page never falls below a specific score. How can we pick the minimum score? As we want to be sure that our website won't degrade over time, let's see the current performance score and enforce that. Let's go to the vuejs-firebase-shopping-cart directory, under Chapter10 of the repository, and we will run npm run serve and launch the web application:

npm run serve

That command should start the website. Once started, let's open another terminal and run Lighthouse on the home page:

lighthouse http://localhost:8080/--view

The result of that process was a score of 30 for performance. We can set our target score at 25. Time to write our test.

Important Note

As Lighthouse runs locally, you might get different results on different machines. You should consider that when picking your score goal.

We are going to add our test in the homepage.tests.js file. But before creating the test, we need to install the lighthouse module by running the following command:

npm install lighthouse

That will get us Lighthouse in our tests. The next step is importing the lighthouse module using the require function. Let's add this line at the top of the file:

const lighthouse = require('lighthouse');

This will make Lighthouse available in our tests. Now, let's see how our test would look:

it('Should have a good performance score', async() => {

  const result = await lighthouse(config.baseURL, {

    port: (new URL(browser.wsEndpoint())).port,

    onlyCategories: ['performance']

  });

  

  expect(result.lhr.categories.performance.score >= 0.25).to.be.true;

});

We solved the test using only two statements. We first call lighthouse, passing the URL we want to process, in this case, config.baseURL, and then we pass an options object. There we are passing the port it has to use to connect to the browser that Puppeteer is using. We get it by doing new URL(browser.wsEndpoint())).port, and then we tell Lighthouse we only want to process the performance category. We won't cover all the available options here. You can take a look at the full list of options at https://www.hardkoded.com/ui-testing-with-puppeteer/lighthouse-configuration.

In the next line, we just assert that the score of the performance category is greater or equal to 0.25. When you see the report, scores are in the range of 0 to 100. But in the JSON object, the range is from 0 to 1. That's why we need to use 0.25 instead of 25.

The next test is checking for specific metrics.

Ensure that a metric is below a threshold

We can also be more specific. We could say that, for instance, regardless of the performance score we want to check, First Contentful Paint should never take longer than 30 seconds. Our code will be similar to our previous test:

it('Should have a good first contentful paint metric', async() => {

  const result = await lighthouse(config.baseURL, {

    port: (new URL(browser.wsEndpoint())).port,

    onlyCategories: ['performance']

  });

  

  expect(result.lhr.audits['first-contentful-paint'].numericValue).lessThan(30000);

});

Here, we can see that the lhr object also contains an audits dictionary with all the metrics. We can grab the first-contentful-paint entry call and check that numericValue (in milliseconds) is under 30,000 (30 seconds expressed in milliseconds).

How can we know what the available metrics are? The easiest way is to add a breakpoint in your test and add a watch to see the value of result.lhr. You will see something like this:

result.lhr content

Result.lhr content

There you will be able to see not only the available entries but also numericUnit, among many other properties.

Based on this example, making sure that an opportunity is not found will be easy.

Ensure that an opportunity is not found

I think this is the most solid way to use Lighthouse. We introduced some arbitrary numbers in our previous examples, 30 for the score and 30 seconds for the metric. Now, let's say we don't want to get a certain opportunity; for instance, we don't want any images of the wrong size. We can look at the audits and try to find an entry with the name user-responsive-images. With that entry, we can write the following test:

it('Should have properly sized images', async() => {

  const result = await lighthouse(config.baseURL, {

    port: (new URL(browser.wsEndpoint())).port,

    onlyCategories: ['performance']

  });

  

  result.lhr.audits['uses-responsive-images'].numericValue.should.equal(0);

});

The code is the same as the previous example, but here, we assert that the metric value should be 0. That will mean that all the images are properly sized.

It's impressive all we can do with Lighthouse, but to be honest, you won't see many teams applying these ideas to their project. If you get to test your website's performance using Lighthouse, you will add a lot of value to your team.

Lighthouse is kind of a black box that you call, get values, and act in response. But what if you want to build your own metric? What if you want to analyze the performance of a page in a more granular way? Let's now explore all the tracing features Puppeteer offers.

Tracing Pages

In this section, we will cover how to get performance information using the tracing object you can find on the page.tracing property. I saw this question more than once on Stack Overflow: How can I get the Performance tab's information using Puppeteer? The answer is: You can get all that information from the tracing result. There is a high chance that you will get a reply like: "Yes, I say that, but the result is too complex." And yes, the tracing result is quite complicated. But we will try to see what we can get from that object in this section.

If you open DevTools, you should see a Performance tab like this one:

Performance tab

Performance tab

As you can see, the Performance tab is not processing information all the time because it's a costly process. You need to start "recording" the tracing, Chrome will begin collecting lots of data from the browser, and then you have to stop the tracing process.

If you click on the second button, which looks like a reload, it will automatically reload the page and start the tracing. If you click on that button and then stop the tracing when the page loads, you will get something like this:

Performance result

Performance result

The level of detail of that panel is impressive. You get to see every paint action, every HTML parsing, every JavaScript execution, everything the browser did to render the page there.

We can get the same using the tracing object. Let's create a test called Should have a good first contentful paint metric using tracing in our homepage.tests.js file, but we will add only the tracing calls for now:

it('Should have a good first contentful paint metric using tracing', async() => {

  await page.tracing.start({ screenshots: true, path: './homepagetracing.json' });

  await page.goto(config.baseURL);

  await page.tracing.stop();

});

The code is straightforward. We start the tracing, we go to the page, and we stop the tracing.

The start function expects an options object, which has three properties:

  • The screenshots property will determine whether we want Chromium to take screenshots during the tracing.
  • If you set the path property, the tracing result will be written in that JSON file.
  • Finally, you'll find a categories property, where you will be able to pass an array of properties you want to trace.

There is no fixed list of categories, but these are the categories I find more relevant to us:

  • Under the rail category, we will find many useful traces such as domInteractive, firstPaint, and firstContentfulPaint.
  • If you set screenshots to true, you will find all the screenshots under the disabled-by-default-devtools.screenshot category.
  • You will find that lots of entries will come under the devtools.timeline category. This category represents one of those items you see in the performance timeline.

When you call the stop function, you will get the result in the file you passed to the start function, and, whether you passed a path or not, the stop function will return the result as a Buffer.

The resulting JSON will be an object with two properties: A metadata object with information about the trace and the browser, and a traceEvents array, with all the trace information.

In my simple test example, traceEvents gave me 16,693. That was just for navigating to the page. I think now you get why this can be scary for some users.

The shape of each trace event might vary based on the category. But you will find these properties:

  • cat will tell you the categories for the event, separated by commas.
  • name will give you the name of the event, as you would see it in the Performance tab.
  • ts will give you the tracking clock, expressed in microseconds (1 microsecond is 0.000001 seconds). Most events are relative to the beginning of the trace.
  • pid is the process ID. I don't think you will care about that.
  • tid is the thread ID. You won't care about that either.
  • args will give you an object with specific information for that event type. For instance, you will get the URL and the HTTP method of a request. For a screenshot, you will get the image in Base64 format.

With all this information, let's code our first contentful paint test using tracing values. We are going to write a test that will start the tracing, navigate to a page, and then evaluate the results. It would be something like this:

it('Should have a good first contentful paint metric using tracing', async() => {

  await page.tracing.start({ screenshots: true, path: './homepagetracing.json' });

  await page.goto(config.baseURL);

  const trace = await page.tracing.stop();

  const result = JSON.parse(trace);

  const baseEvent = result.traceEvents.filter(i=> i.name == 'TracingStartedInBrowser')[0].ts;

  const firstContentfulPaint =result.traceEvents.filter(i=> i.name == 'firstContentfulPaint')[0].ts;

  expect((firstContentfulPaint - baseEvent) / 1000).lessThan(500);

});

We have some tricks to explain here. After stopping the trace, we get the result and parse it. That will give us the result with a traceEvents property. As ts is based on the beginning of the trace, we need to find the baseEvent, looking for an event with the name TracingStartedInBrowser. Then we look for the event with the name firstContentfulPaint, and finally, we calculate the difference. As it's in microseconds, we need to divide it by 1,000, so we can compare it with our target goal of 500 ms.

Notice that in this example, our goal is 500 ms versus the 30 seconds we used in the Lighthouse example. This is because, by default, Lighthouse performs several runs emulating different conditions.

Another thing we could do here is export the screenshots generated by the tracing tool for later analysis. We can add something like this at the end of the test:

const traceScreenshots = result.traceEvents.filter(x => (

    x.cat === 'disabled-by-default-devtools.screenshot' &&

    x.name === 'Screenshot' &&

    x.args  &&

    x.args.snapshot

));

traceScreenshots.forEach(function(snap) {

  fs.writeFile(`./hometrace-${snap.ts - baseEvent}.png`, snap.args.snapshot, 'base64', function(err) {});

});

There, we are filtering screenshots events with a valid screenshot, and then we just write all those Base64 snapshots to the filesystem. With that, you will see how the page was being rendered during the loading process. You would even code your own first contentful paint algorithm with those images.

Now you might be wondering whether you should use Lighthouse or Puppeteer's tracing. I think there are some pros and cons to every approach. Lighthouse is easy to use, and as we saw, it gives us metrics that would take us lots of effort to build by ourselves. With Lighthouse, you just call the lighthouse function and evaluate the results. But it can be slow, even if you select only one category.

On the other hand, Puppeteer's tracing can be hard to read and process, but if you know how to take the metric you need from the tracing result, it will be way faster than Lighthouse. Another important difference is that Lighthouse only evaluates the page load, whereas with Puppeteer's tracing, you could start the tracing at any moment. For instance, you could go to a page, start the tracing, click on a button, and then evaluate what the browser did to process that click. At the end of the day, it's about picking the right tool for your job.

Lighthouse also gives us two interesting metrics: Remove unused JavaScript and Remove unused CSS. Let's see how we can solve those metrics using Puppeteer.

Analyzing code coverage

In this last section, we will see how we can get code coverage using the Coverage class from Puppeteer. Code coverage is a metric that can be applied to any piece of code. To get the code coverage of a piece of code, you need some kind of tool to trace which lines of code are being executed, execute that code, and get the tracing result. It's like the performance tracing, but instead of measuring time, it measures executed lines of code.

You can see the code coverage on a page on Chrome using the Coverage tab. I didn't have that tab by default, so I needed to add it using the More tools option, as in the following screenshot:

Coverage tab

Coverage tab

The Coverage tab works like the Performance tab. You need to start the tracing, run the page, or perform an action, then stop the tracing to get the results.

The result will be something like what we see in the preceding screenshot: A list of resources with the total bytes of that resources and the unused bytes. At the bottom of the window, we can see that over 90% of the code was used (executed) during the tracing. That's pretty good. We could write a test to ensure that we will always have a code coverage of over 90%.

The JavaScript and the CSS code coverage have two sets of functions in Puppeteer. If you want to get the JavaScript code coverage, you need to run startJSCoverage to start the coverage and stopJSCoverage to stop it. startJSCoverage supports an options argument with two properties:

  • resetOnNavigation is a Boolean property we can use to tell Puppeteer to start over with the tracing if navigation was detected.
  • reportAnonymousScripts is a Boolean property we can use to tell Puppeteer to ignore, or not, dynamically generated JavaScript code.

If we want to get CSS coverage, we need to use the startCSSCoverage and stopCSSCoverage functions. startCSSCoverage also expects an options argument, but, in this case, it only has the resetOnNavigation property.

Once we run the coverage, both stopCSSCoverage and stopJSCoverage will return the same type of value. Both will return an array of objects with these properties:

  • url will give us the resource URL.
  • content will be the CSS or the script content.
  • ranges, which will contain an array of objects telling us which were the portion of code that has been executed. Each entry will contain two properties, start and end, telling us where that text range starts and ends.

Now we have all this information, we can write our code coverage test. Let's take a look at it:

it('It should have good coverage', async() => {

  await Promise.all([page.coverage.startJSCoverage(), page.coverage.startCSSCoverage()]);

  await page.goto(config.baseURL);

  const [jsCoverage, cssCoverage] = await Promise.all([

      page.coverage.stopJSCoverage(),

      page.coverage.stopCSSCoverage()

  ]);

  let totalBytes = 0;

  let usedBytes = 0;

  const coverageTotals = [...jsCoverage, ...cssCoverage];

  for (const entry of coverageTotals) {

      totalBytes += entry.text.length;

      for (const range of entry.ranges) usedBytes += range.end - range.start - 1;

  }

  const percentUnused = parseInt((usedBytes / totalBytes) * 100, 10);

  expect(percentUnused).greaterThan(90);

});

We start our test by starting both code coverages. We put startJSCoverage and startCSSCoverage inside Promise.all, so we wait for both coverages to be confirmed. Then we go to the page, and after that, we stop both coverages. That will give us two arrays that we can join (because they share the same shape) using [...jsCoverage, ...cssCoverage].

Once we have both coverages, we get the total size of the resource by using entry.text.length, and then we get the size of the coverage by adding the length of all the ranges.

The result will give us the total code coverage of our tracing, which we will check whether it's over 90%.

The pros and cons of this solution compared with Lighthouse are the same as we saw in the previous section. On one side we have Lighthouse, which gives us all the numbers already cooked. But here, we have more control over what we want to measure. This test was quite simple, but you could improve it by filtering out all the resources you don't want to measure. You can also download that result to a file and share it with your team if the test fails.

Now it's time to wrap up this chapter and this book.

Summary

If you get to apply performance tests in your team, you will be on a whole new level.

We started the chapter by talking about Lighthouse. We only covered the Performance category. But now that you know how it works, I encourage you to keep digging into the other categories and think about how to create tests for that. I would love to see more tests about accessibility.

We also learned how to use Lighthouse in our tests. That's not something you will see quite often. You will be able to test very complex metrics using two lines of code.

Most developers would run away from Puppeteer's tracing results. Although what you can get from there is way more than what we covered, we learned the foundations of such a powerful tool in this chapter.

The size of a page is critical for performance; that's why we learned about code coverage and how to measure it.

And this is also a wrap on this book. When I planned this book, my goal was to write a book that would cover the entire Puppeteer API, without being a reference book. And I think we accomplished that goal. We learned how to write high-quality end-to-end tests using Puppeteer and, at the same time, we covered most of the Puppeteer API.

With this goal in mind, we covered topics that were not strictly related to unit testing. We talked about PDF generation and Web Scraping. We also covered topics that many people would run away from, such as the tracing model.

If you read this book from cover to cover, I can assure you that you will know way more about Puppeteer than the average user of this library.

But I also hope you learned more than just a Node package. In this book, we also learned about the foundations of the web, and how to write good tests. We talked about the internet ecosystem, scraping ethics, and web performance. You have also empowered your role. QA is more than just about testing web pages. It's about honoring your users by delivering high-quality software they can enjoy using.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
34.201.16.34