Analytics for User Research

The price of light is less than the cost of darkness.
—Arthur C. Nielsen, Founder of ACNielsen

Knowing who your users are is crucial to any UX process. This is why user research plays such a vital role. User research consists of a whole range of different tools and techniques, but what underpins them all is gathering useful data.

Research is not about asking people what they like or what they hate, but establishing facts about your users. This ties in with analytics, where data is objective, providing facts rather than opinions.

This chapter will cover how your analytics data can support and inform your user research. The data you get from your analytics tool is no substitute for in-depth user research, but taking an analytics-first approach to research will help you build strong foundations.

Where Analytics Sits in the Research Process

There’s no one, fixed model for the user research process. Different people will approach user research in different ways, and the process may change depending on the project.

The following diagram shows some of the different forms of user research you might use as part of your process.

Where analytics fits in with some common user research methods (credit: Tim Minor)

Continuing the theme of an “analytics-first” approach, the diagram shows how analytics can be the starting point for other types of user research.

Your data can be used at the start of your process to get a broad idea of the types of users visiting your website. It can also be used to help you create detailed personas, and to analyze the behavior of different user types. Use your analytics data to support your other research methods to get the most out of the process.

Knowing Your Users

To understand why your users behave the way they do, you first need to get to know them. You may make assumptions about who those users are, but you should be constantly challenging those assumptions, or at least be backing them up with facts.

There’s a lot of data available in your analytics package that will help build up your knowledge of who’s visiting your website. The more you know about your users, the more informed your design decisions can be.

This data can form a useful starting point for many different types of research. One area where this data is particularly helpful is in recruiting people for usability tests. In usability testing, the better the participant matches the target persona, the better the test.

Usability testing should show how “real” users interact with your website, and where they may be experiencing issues. Knowing who your users are will improve the results of your usability testing, and will give you a better chance of uncovering the issues your “real” users are encountering.

The following section looks at the data in your analytics tool that will help build up your understanding of who your users are.

How Do Users Find Your Website?

Analyzing how users are finding your website can help you understand more about them, and about the context of their visit.

Different analytics tools will classify “traffic sources” or “channels” in different ways. The following are some typical sources for your traffic:

• Organic search. This typically identifies a user who has clicked an “unpaid” link from the search results.
• Paid search. Paid search users, sometimes known as “Pay Per Click” or “PPC”, will have arrived on your website via a paid advert on a search engine.
• Referral. Users from referral links will have followed a link from another website.
• Social. Social media is often shown as a separate channel from other referral links.
• Direct. This category includes users who type your domain into the address bar of their browser. It can also include users where the analytics package was unable to identify their traffic source.
• Email. Links in emails will need to be tagged, since by default, analytics tools are unable to identify users clicking links that don’t appear on web pages.

This analysis can give you a better idea about your users’ intentions. If you’re running a paid search campaign, for example, you’ll be able to see the keywords that were used to find your website (as long as you’ve linked up the Google Analytics and Adwords accounts, and enabled auto-tagging). If users are finding you based on “brand” search terms, you know they’re aware of your company and are searching for you specifically.

Analyzing the behavioral metrics for users, broken down by channel, can help content and marketing teams make decisions about the amount of effort, resources, and budget to dedicate to specific channels. From a UX perspective, it can be useful for identifying problem areas (see Chapter 4), but it also helps to give insight into the mindset of your users. Knowing where your users are coming from can help you to identify whether they’re already familiar with your website, and can start to give you clues about the likely purpose of their visit.

Where Do Your Users Come From?

To begin with, it’s a good idea to start by finding out where—geographically speaking—your users come from. This is a very broad level of analysis that will help you focus your research. Looking at the location of your users will show you the role international visitors play in the success of your website. Geodata will also give you insight into the behavior of users on a national and regional level.

Most analytics tools will give you location data for your users. In Google Analytics, this can be found in Audience > Geo > Location. This report will tell you where your users are coming from, and will also allow you to compare behavior metrics (and different dimensions) for users from different countries, regions or cities.

Location report in Google Analytics

Looking at the percentage of visitors from each country will help you understand the importance of international visitors to your website.

But you need to be careful with your analysis here. Just because no one visits your site from Canada, for example, doesn’t necessarily mean that audience is not important to you. You could be accidentally blocking them! Your marketing efforts may not be reaching where they should, or there may be a whole host of other reasons for the lack of visits. Once again, remember that your website analytics tell you what, but not why.

You may be assuming your website only attracts visitors from your own country, but this report may show that you should also consider the needs of international visitors. This could lead to practical considerations—such as the load speed of your website in other countries, international delivery rates for an ecommerce site, and possible cultural differences between other countries and your own.

Cultural differences based on the country of your visitors can be considerations for both your website’s design and functionality. These differences can be hard to cater for, as cultural differences may be subtle, and often won’t lead to clear ideas for design changes. Still, detailed research on the cultural needs of you users is definitely recommended.

On a more practical level, the way people use ecommerce websites varies dramatically depending on their country. According to data from Worldpay, only 12% of users in Germany make online purchases using cards.

This compares with 63% of UK users and 72% of users in the USA. If you notice that your ecommerce website is getting a lot of visits from Germany, you’ll want to look at offering alternative payment methods. The most popular type of online payment in Germany in real-time bank transfer.

Looking beyond the number of visits, you’ll be able to see behavioral metrics, such as time on site, bounce rate and conversion rate. Focusing on user behavior will enable you to pinpoint particular countries where there may be issues with your website, and thus opportunities to make improvements. If, for example, your ecommerce website is getting a lot of visits from a certain country, or countries, but the conversion rate is low, you may want to reassure those users that you deliver internationally.

There could be many reasons why your conversion rate is low in a particular country. You may not ship to that country, your site may be in the wrong language, the products may be cheaper in that country, or you may not offer the right payment methods (as in the example of Germany). This is where it’s important not to jump to conclusions. Remember, the data only tells you what is happening. It’s important that you set aside research time to find out why.

If you can see an opportunity to increase your conversion rate internationally, you might also want to consider personalizing your website in some way for those countries. This could be as simple as showing the flag of that country and being up front about the exact costs and delivery times for that location. This simple form of personalization will likely resonate with your international users, and will help you unlock potential additional revenue from international sales.

If your analytics shows you’re getting a lot of visits from other countries, you’re likely to be missing out if you don’t factor in cultural differences!

Geo reports can locate users down to state and city level. This means you may also want to assess your visitors by state, region and city to get a clearer picture of who your visitors are and how they behave. The importance of this level of detail will depend on the purpose of your website. For political websites, for example, localized data can be very important to see how a candidate is performing in a key state or region.

What Language Do Your Users Speak?

Knowing what language your users speak can give you additional insight into the content you should serve up. Language and location are sometimes confused when looking at analytics, but the two dimensions have no direct connection. A user can be located in Paris but speak Spanish. Location is ascertained by the IP address of your user, while their language can be derived from the language settings of their browser.

You may notice you’re getting a lot of visits from German-speaking users. If this is the case, you might want to look into creating a German version of your site if you don’t already have one.

As with location, language can also give an indication of the culture of your users. The language used can also have a big impact on your design. Arabic text is displayed from right to left, and the Chinese alphabet contains thousands of characters. Both of these will potentially have a big impact on your page layouts.

For example, websites targeted at Chinese audiences are likely to contain a lot of links, rather than offering users search options. This is due to Chinese keyboards being more difficult to use than western keyboards, because of the large number of characters in the Chinese alphabet.

You may be surprised to learn about a larger than expected audience you had no idea existed. It’s likely that users who don’t share your language will be “silent”, and, as a result, may be overlooked and unrepresented when it comes to your user research. By “silent”, I mean these users are less likely to contact you or offer feedback than users who share your language.

Finding out the languages used by your visitors is, of course, just a starting point. I wouldn’t recommend making any major changes based on this data alone. But it can play a very useful role in your user research process.

What Devices and Browsers Are They Using?

When you know which devices are being used to visit your site, you get a clearer picture of the context in which users are viewing your website. Analyzing browser usage can also help you better understand your users.

In user research, you should be careful not to rely on stereotypes or to make broad assumptions about your users. That said, knowing which browsers and devices they’re using can help you get a broad sense of their demographics.

These kinds of studies may help you hypothesize why your users are behaving differently, but I certainly wouldn’t use them on their own. Find out more about your users to test these hypotheses.

Device information is likely to tell you even more about your users. While you also need to be careful about assumptions, research shows, for example, that adults in higher-income households are more than three times as likely as those in lower-income households to own a tablet. There are also differences in the demographics of users of different device brands. A study has shown that iPhone users are better qualified and more affluent than Android users.

As with the previous example, there’s some value in these studies, but you need to be very careful how you apply it. Remember, the analytics data will tell you what is happening, but you’ll need to do further research to find out why.

A common mistake analysts make with device reports is to assume that mobile users are “on the move” (such as commuting), while “desktops” are always used at home. A study by Google showed over 60% of mobile usage is at home, while the “desktop” device category includes laptops, which people are likely to use while traveling.

You need to be careful how you use device and browser data. Don’t make assumptions about your users based on the data alone, but instead let it help shape your research.

Knowing the breakdown of browser and device types can also help identify who to recruit for user testing. If you know that 75% of your users are visiting your site on a mobile device, you’ll probably want to ensure the majority of your user testing takes place on mobile.

What are the Genders and Ages of Your Users?

Seeing how users from different demographic groups behave on a site can really help build up a picture of the different user types. In 2013, Google Analytics introduced demographic reports, which include information on users’ age, gender and interests. These reports are a potential gold mine for user research. It’s hard to tell exactly how accurate they really are, but the demographic data that Google Analytics reports is usually very similar to my own expectations for sites I know well. Google claims the reports 80–90% accurate, and a study by Humix concludes that the demographic data held by Google appears to be quite accurate.

When I get a new analytics client, I often ask them about what they expect the breakdown of the age and gender of their users to be. I then compare this to the reports in Google Analytics—and they’re often very similar. While I’d wouldn’t recommend treating this data as if it’s 100% accurate, based on my experience I’d say you can be fairly confident it gives a good representation of the age and gender of your users. If possible, I also recommend testing this yourself, by surveying a sample of site users and correlating the results against the same user segment in Google Analytics.

This information can be used to directly inform your personas, and to aid with your usability testing recruitment. You can also use this information to confirm, and fix, some concerns you may have about your audience.

I’m one of the organizers of UX Camp Brighton, an annual UX “unconference” that takes place in my home town of Brighton in the UK. Recently, along with my fellow organizers, I had concerns about the gender balance of our attendees. As with most tech events, we were generally getting more male than female attendees, and we were keen to redress this balance. Checking the demographics report for our website confirmed that we did in fact have a far larger percentage of male visitors. Armed with this knowledge, we partnered with Spring Forward, a series of events celebrating the role of women in digital culture. We both promoted each other’s events, and the result was a measurable increase in the percentage of women viewing our website.

Changes to demographics over time

We don’t collect gender data for ticket sales, so we can’t say for sure whether it changed the audience for the event, but it was certainly a step in the right direction, and is something we’ll look to do again next year.

As well as age and gender, the demographics reports in Google Analytics give data on the interests of your users. Personally, I find these reports less useful than the other demographic reports. I often see the same, very generic interest groups appearing. Some of the categories are very broad, such as “movie lovers” and “TV lovers”, and probably apply to 90% of people in every community!

How Frequently Are Your Users Visiting?

Your analytics tool will be able to help you gauge the loyalty of your users. Users who keep returning to your site are likely to behave differently from first-time visitors. Grouping your users in this way can be helpful when you’re creating user types or personas.

Below is a screenshot of a typical Frequency & Recency report from Google Analytics. This can be accessed by going to Audience > Behavior > Frequency & Recency

Frequency and Recency report

We can see the majority of visitors only visited once during the selected time period of 30 days. At first glance, this suggests you only need to focus on that group. But if you add the users who visited more than once together, you’ll see they make up nearly 40% of the total.

It can be helpful here to break these users down into groups that better meet your research requirements. You might want to group users as single visits, 2–10 visits, and 11+ visits. You can then label these groups as “one-off”, “repeat visitors” and “regular visitors”. This will give you a broad but useful overview of how often your users visit your site.

What Content Are They Interested In?

In the previous chapter, we covered how to identify problem pages or areas on your website. The Pages report can also be useful for finding out what type of content your users are interested in.

For example, if you’re looking at data for a real estate website, you may be able to group users into buyers, sellers, renters and letters, depending on the pages they’re viewing. This may not be an easy task, as users won’t always look just at pages targeted to them. In the real estate example, it’s likely people looking to sell their property will also look at properties already on sale to check prices. In this instance, you may need to choose pages further down the funnel to help group your users more accurately.

This type of information will be useful for various aspects of your research, including creating personas.

Using Data for Personas

Personas are used in the UX process to represent actual users of a website. They’re generally created to aid the decision making for the design process, and can be a powerful tool to help keep the focus on users at all times. The wealth of data available to you in your analytics package can be used as a starting point for creating new personas, and also used to analyze how those personas are using your website.

Using Data for Persona Creation

Personas aren’t real people, but they represent real people—the types of people likely to use your website. The best ones are not just “made up”, but instead formed from in-depth knowledge of the website users and their likely behavior. As we’ve covered, your analytics data gives you information on demographics that can be useful for creating personas based on details of real users.

There are several methods for getting the information needed to create realistic personas for your site. Some information may be gained by talking to stakeholders who may have first-hand experience of their users. User research of some kind is also vital, though, and could include surveys and in-person interviews with real users. Your analytics should also play a big part in persona creation.

Using the data from your Audience reports will help get you started in the persona creation process. In fact, almost any metric in Google Analytics can be used to help shape personas. Time on site, for example, may give an indication of whether your users are pushed for time or are likely to be leisurely browsers, while a high bounce rate may signal impatience and/or efficiency.

The key to creating truly accurate and useful personas is to draw from a range of sources to get a complete picture. I’m not recommending that you only use Google Analytics, but it certainly provides a lot of unbiased data, which can only help with the process. Don’t use analytics data in isolation. Use it alongside other research methods, rather than instead of them.

Persona creation is a bit of an art form, and is not the subject of this book! I certainly wouldn’t recommend only using data from your analytics tool to base your personas on. Instead, consider this data as a starting point for other research methods.

Creating Persona-based Segments

Once you’ve got a good idea who your users are, you can create segments to see how different groups of users are behaving on your site. A segment is a subset of your analytics data. For example, a segment might be made up of users from a particular country or city. Another segment might be mobile users, or users who visit a particular section of your website. Segments can be made up of a single dimension, or multiple dimensions, such as French-speaking women using tablets.

In Google Analytics, you can create custom segments by clicking the Segments bar at the top of your reports.

From here, you can select from a list of predefined “System Segments“, or create your own segments to use. To create your own segments, click New Segment and then select your chosen dimensions.

From the New Segments menu, you can create your segment based on any of the data stored by Google Analytics. In the example below, I’ve set up a segment for women visiting my site from the UK.

Setting up advanced segments

Under the Advanced options, you can select whether you want to base your segmented data on sessions or users, as well as creating segments based on users following specific pathways through your website.

As you can imagine, Google Analytics enables you to create very specific segments that can match your different user types or personas.

Using Persona-based Segments

Once you have your segments, you can apply them and view your navigational reports (either in the Site Content section or in the User Flow reports) to see how representatives of your personas are interacting with your site.

As well as looking at their paths through the site, you can begin to focus on the main exit pages for your persona groups, or to look at which user types are most likely to make a purchase. You may find one particular segment has a high exit rate at a particular stage of your checkout funnel, for example.

Because you can apply up to four segments at any one time, using persona-based segments will allow you to compare the behavior of your different user types. In the example below, we can see the differences in behavior of two different types of users—men and women.

Navigation by different genders

Whether you’re running side-by-side comparisons, or doing a deep dive into the details of one particular user type, persona-based segments are a vital part of your UX analytics analysis.

This whole process can be cyclical, too. You can use Google Analytics data to inform your personas, and then use those personas to interrogate your data further.

Benchmarking Against Competitors

In my training sessions, people often ask me things like, “My bounce rate is 47%; is this good or bad?” My answer is always the classic UX response: “It depends”.

I’ve touched on why a high bounce rate can be good or bad, and why it’s important to analyze your bounce rate on a page (rather than a site) level. Another factor to consider is how it compares to similar types of websites. Different sizes of websites, websites targeting different countries, and websites in different industries will have varying metrics. To find this data, look at the options available for benchmarking your website against your competitors.

You can learn a lot by analyzing how your users behave on your site. But to gain additional context, you should also consider how your competitors’ sites are performing.

Competitor research can form part of your user research process—and you’re able to do some basic competitor research, for comparison purposes, using Google Analytics.

Benchmarking in Google Analytics

As I mentioned in the first chapter of this book, Google Analytics is the most popular website analytics tool on the market. It’s used on millions of websites, and, as a result, Google has access to a lot of analytics data. Back in 2013, Google decided to make some of this data available within its benchmarking feature.

About the Benchmark Reports

Benchmark reports allow you to compare your website metrics with those of similar websites in your industry. These reports can be found in Audience > Benchmarking. There are three types of benchmarking reports indicating how your site is performing against competitors:

• Channels: lists the channels users have followed to arrive at your site.
• Location: shows the countries, or regions, from which users have arrived at your site.
• Devices: lists the devices used by site visitors.

Each report uses the same set of metrics:

• Sessions
• % New Sessions
• New Users
• Pages/Session
• Avg. Session Duration
• Bounce rate

Google Analytics offers three indicators of sites similar to your own for benchmarking purposes:

• Industry Vertical: for example, real estate > property management
• Geographical region: for example, United States > all regions
• Traffic size: for example, 500 to 999 daily sessions

Benchmark reports enable you to alter these indicators within each report. This means that you can change your site’s vertical and compare its performance to alternative business types, locations and sizes.

For the industry vertical, there are over 16,000 options, so you should be able to find a vertical that’s a good match for your site.

Using the Benchmark Reports

The first step in using the benchmarking reports is to check the indicators at the top of the report to ensure they’re appropriate for your website.

Benchmarking report indicators

You’ll need to select the most appropriate industry vertical for your site. With the geographical region, you’ll want to either select the country the majority of your users come from, or you may want to select “All” if you have an international website. You shouldn’t need to change the traffic size option, as Google Analytics works this out automatically based on your data.

Once you’re confident you have accurate indicators, you can start to look at how your website is performing against your competitor benchmark. Be aware that this isn’t necessarily going to include all of your direct competitors. This data will give you an indication of how you’re performing against websites in your industry of a similar size.

The first of the reports shows how your website is performing against its competitors across different channels. This type of report is generally of more interest to marketers, who’ll be interested in comparing the performance of traffic sources against the benchmark.

The second benchmarking report allows you to analyze data on how your website stacks up against its competitors in different countries. This is more useful to us, as it can provide extra context for the cultural differences I mentioned earlier in this chapter. It can help answer questions such as, “Is my bounce rate high in China due to an on-site issue, or is it due to bounce rates being higher in China in general?”, or, “Is my website failing to reach a specific state, or is that state just not interested in my type of product?” Answering these types of question can ensure you’re not trying to fix problems that don’t exist.

The final benchmarking report in Google Analytics focuses on device usage. For me, this is often the most useful of the benchmarking reports. Here you can see how your website is performing on mobile and tablet devices against your competitors.

There’s usually a big difference in the way mobile and desktop users interact with your site. But without context, it’s hard to gauge whether your mobile site is performing better or worse than you’d expect. If my bounce rate on desktop is 40%, is a 50% bounce rate on mobile to be expected, or is it a cause for concern? Device benchmarking reports will go some way toward answering that question. You’re able to see how the bounce rates of your competitors vary between device types. Keep in mind this is only one single indicator, and won’t answer all your questions. It’s also worth remembering that metrics like bounce rate and time on site are not the best indicators of the performance of your website. If your figures differ substantially from the benchmark, though, you may want to investigate further.

The ability to work out the breakdown of visits by device for your competitors is another really useful feature. One of my clients, who runs a global jobs board, believed they received fewer visits from mobile devices than their competitors. This was based on a hunch, but data from Google Analytics confirmed this was the case, and I was able to give him accurate figures on how their device breakdown compared to similar-sized jobs boards.

Benchmark comparison

This data is available in Google Analytics, but you’ll need to work out the percentage breakdown yourself. The data shown in Google Analytics is the number of sessions; I just copied it into Excel to work out the percentages and create the pie charts.

The benchmarking reports in Google Analytics are great for giving additional context to your data. They’re missing some key metrics, such as conversion rates, but the information that’s available can be really useful for getting insight into how your website is performing.

Other Benchmarking Techniques

The benchmarking reports in Google Analytics are useful, but they’re quite limited. I often use other sources to find competitor data for benchmarking purposes. As I touched on in the previous section, one key metric that’s missing from the Google Analytics benchmarking reports is conversion rates. The conversion rate is arguably the most important metric for many websites, particularly ecommerce sites. Without context, it’s difficult to know what a good or bad conversion rate is.

Desk research will help you to find industry statistics on conversion rates. Websites like smartinsights.com and monetate.com post quarterly statistics on conversion rates across different device types and different countries. This type of data will give you a ballpark figure to compare your own website to. There will, of course, be a lot of different factors that impact your conversion rate—but this kind of reliable data will at least give you an indication of typical conversion rates, providing more context when you’re evaluating the performance of your website.

An Analytics-first Approach to User Research

Hopefully this chapter has helped convince you that analytics has a big part to play in the user research process. I’m not proposing you only use analytics for research, as there’s no substitute for talking to real users. But you can get a lot of really useful information from your analytics package, and this can give a solid foundation for your user research process.

As UXers, we know that understanding who our users are is absolutely key to creating a good experience. Analyzing your website’s analytics data is a great way to improve that understanding.