Chapter 3

Lay the Foundations for Optimization Success

Before you begin to optimize and test your website, first you should realize that it's not going to be as easy you might expect. You need much more than just tools—to help improve your chances of optimization success you need to understand and leverage website optimization and testing fundamentals. In this chapter you will learn these, in particular how to build an optimization organization and some great testing best practices.

Chapter Contents

• Week 4: Learn Key Optimization Fundamentals to Help You Succeed
• Week 5: Understand and Create an Optimization Organization
• Week 6: Learn Testing Best Practices to Improve Your Success

Week 4: Learn Key Optimization Fundamentals to Help You Succeed

Now that you have set up analytics and testing tools, and checked success metrics and set targets, you may be eager to get started and apply the best practices you will find throughout the rest of this book. Unfortunately, when you begin to apply these, you are more than likely going to come up against some challenges and barriers within your company preventing you from quickly and easily running tests and making changes on your website to optimize it.

It’s essential that you break down these barriers and foster a website optimization culture; otherwise, your website optimization efforts and ability to test are going to be severely compromised. This week you will learn some key fundamentals to help you do this, including how to build a great testing culture and strategy. At the end of the week, you will learn how to evaluate whether you should outsource some or all of your optimization efforts to a third-party vendor.

Monday: Identify Your Company’s Current Website Optimization Performance

Before you go any further in this book, you first need to understand your company's current optimization and testing efforts, and then understand how they compare to companies using best practices. This will help you understand how much potential you have to improve your current testing and optimization efforts. Today you will evaluate this by using a website optimization checklist and a typical website optimization maturity lifecycle.

The Website Optimization Checklist

In order to start evaluating your current website optimization efforts, you first need to identify how your company currently performs. Begin by answering the checklist of questions in Table 3-1. The more questions you can answer “Yes” (and please be honest about your or your company’s efforts), the more advanced your website current optimization efforts are.

Table 3-1: Website optimization checklist

If you answered fewer than 5 questions with a yes, you have huge room for improvement in your optimization efforts. If you answered between 5 and 10 questions with a yes, you are well on your way to a good optimization start, but you still have plenty to learn. If you answered more than 15 questions with a yes, then your company is already in a great position to become a world-class website optimization organization. We will revisit this checklist in the last chapter to see how much your efforts have improved.

The Website Optimization Maturity Lifecycle

Now that you have used the checklist to understand your current optimization efforts, you need to evaluate how well your efforts compare to companies achieving an expert level of optimization.

To help you do this, Table 3-2 on the next page describes a typical website optimization maturity lifecycle that companies should evolve through to become expert at optimization. In this lifecycle, there are four main phases of optimization maturity listed as columns, with each phase consisting of nine rows of optimization elements and requirements (that you will learn all about in detail throughout the rest of this book). To reach the final expert optimization phase, you need to make sure you are fulfilling each requirement. This last stage is considered website optimization nirvana, because you are going to be armed with all the best practices, foundations, and tools necessary to continually optimize your website to its full potential.

Table 3-2: Website optimization maturity lifecycle

Unfortunately, it isn’t easy to get to the nirvana phase, and it can often take at least a year to arrive at this phase, depending on the scale and complexity of barriers and challenges within your online business. Indeed, with a lack of strategic help, many companies never even make it out of phase 2.

Many of you reading this book are likely to be in the beginning stages of website optimization, and therefore your company may only have elements of phases 1 and 2 at this point. Some companies may have many of the elements of phase 3, particularly ones relating to testing best practices, but many will be lacking the organizational and analytical elements, along with other essential tools to gain insight and improve conversion rates.

This lifecycle is also good to show your senior management and key stakeholders to make them understand what is need to be most successful when optimizing your website.

It’s also important to understand how you will likely go about doing this optimization. If you have a small company, with less than 10 people, it’s likely that you are going to be doing much of the optimization work yourself or hire outside help. But if you have a mid-size or large organization, you will need to help evolve it into and use an organization that lives and breathes website optimization, called the “optimization organization.” Unfortunately, doing this is a huge uphill battle for many businesses—in fact, this is often where most companies fail in their website optimization efforts. To help you avoid this common pitfall, Week 5 is devoted entirely to creating an optimization organization and the benefits of doing so.

Tuesday: Evangelize and Create an Analytics and Optimization Culture

In order to increase the chances of your website optimization efforts being effective in the long run, the next thing you need to do is start building an analytics and testing culture into your organization. Most importantly, this will help you gain understanding and buy-in from key stakeholders, break down internal optimization barriers, and obtain the internal resources necessary to conduct your optimization efforts.

Report Regularly on Key Success Metrics and Goals

The first thing you need to do is make sure your analytics and success metrics are commonly available across your company and updated in the form of a one page report. Most importantly this should contain the performance of your website success metrics that you set up in Chapter 2.

Ideally, this report should be distributed via email on a weekly automated basis to all the members of your online company, and then stored in a shared online document storage place for anyone to access.

The goal of such reports is to provide insight and recommendations based on current overall website performance and to ensure that all the members of your company know what your success metrics are and the performance of them.

You should also include some commentary, recommendations, and insights each week at the top of the report. This helps readers interpret what your metrics are revealing, without them having to study the report and risk their jumping to incorrect conclusions.

At least every month, you also need to do a metrics and optimization review presentation for your company (or at least with your key stakeholders if the company is a large one). At this presentation, you should discuss your website’s latest performance in comparison to the same period last year and last month, and then based on that, suggest recommendations and test ideas to help optimize it.

Run Regular Training Sessions

Web analytics and testing aren’t easily understood by everyone, especially in departmental areas where members are more focused on the creative or technical aspects of the website. Unfortunately, if your company doesn’t understand the importance of analytics and testing, it’s going to be difficult for you to get them to buy-in to website optimization.

To help address these issues, you should provide regular website optimization training for all the departments across your company. This training should include the following:

• Monthly internal training sessions on success metrics and how they relate to website optimization efforts, with key ways to try and influence them. Ideally, this should be mandatory for all new employees.
• External training from industry specialists and experts on website optimization topics and new tools, given to all analytics and optimization team members.
• Brown bag training sessions over lunch to discuss specific analytics and optimization topics. This would be for a department-wide audience and should be more informal and not mandatory.
• Website optimization and analytics conferences and online training; for example, the conferences called eMetrics (www.emetrics.org) and Conversion Conference (www.conversionconference.com), and the training from MarketMotive (www.MarketMotive.com). These should be mandatory for all analytics and optimization team members to attend and complete.

Give Regular Company-wide Presentations

Another great way to build a testing and optimization culture is to give presentations about this topic and to help evangelize the importance of it. These presentations should be made to all departments in your company, and ideally should be done every month (or at least quarterly).

In the initial stages of your optimization efforts, you will need to discuss the performance of your website’s current success metrics, your plans to test and optimize, and the potential impact on conversions and revenue. You should also present case studies from the industry that your website is in. These case studies should show the positive impact that website testing and optimization can have on your website.

Then as your optimization efforts grow, these meetings will need to focus on and communicate the results of recent tests and give updates on overall strategy and progress made.

Run Contests for Your Tests

Another great way to help build a testing and optimization culture and involve more people in testing is to run internal company-wide contests relating to your tests. Here are two ideas for contests:

• Ask people to submit their test ideas, and the person who submits a test that results in the highest improvement would win the contest.
• Show upcoming test versions and ask people to pick which version will win. The person who picks the most winning versions would win the contest.

You should even offer a small prize to the winners (such as an Amazon gift certificate), which can go a long way toward building a good testing culture.

Seek Out Company Influencers

Another great way to help evangelize and promote your website optimization efforts is to enlist influential members of your organization to help you do so. An ideal candidate for this would be someone fairly senior who is well-respected in your organization and has good communication skills.

After you have identified possible prospects, you should propose to meet each of them for lunch or after work so you can socialize and present your website optimization cause and plans to them. Offering free lunch or dinner to discuss this with them often works wonders.

Incentivize Your Team Based on Conversion Rate Improvements

Money is one of the biggest motivators to get someone to do anything. Therefore, to help drive the efforts of members of your optimization and testing team even further, you should consider changing your compensation model for these members so that they get bonuses based on reaching higher conversion levels. This could be in the form of tiered levels, meaning that they would get a higher bonus for each level of higher conversion rate reached—for example, a 10 percent bonus if they grow conversion by 10 percent.

Offering these monetary incentives is a great alternative way to drive your team even harder to improve your conversion rates. Therefore, if you manage an optimization and testing team you should try implementing this, or if you don’t, work with your boss and human resources department to get this done for your testers and analysts.

Wednesday: Prepare a Better Initial Website Optimization Strategy

The next thing you need to do is understand how to prepare an initial optimization strategy for your website. Having this strategy is going to be critical, not only to help determine what to test and optimize, but also to help build the optimization organization in the following week. If you don’t have a solid plan and strategy in place, it will be very hard to gain any buy-in from senior stakeholders and actually get much testing and optimization done.

Of course, you could do some ad hoc testing with no real strategy and get some limited results, but in order to really be effective at optimization in the long term, you need to prepare and stick to a long-term strategy.

Ultimately, to be more successful with your testing and optimization efforts you need to use a better initial optimization strategy. You will now learn several good approaches and best practices to help you form this.

Familiarize Yourself with the Test Idea and Test Results Tracking Documents

Rather than simply creating a long, unorganized list of test ideas for your website optimization strategy, you need to use a better approach. This involves using a test idea tracker document that allows you to state key details for each proposed test, including the hypothesis of your tests, a description, and location of the tests on your website. You also need to include ratings in terms of the difficulty of implementation and likely conversion lift value of each proposed test, using a rating system of 1–10 (with 1 being the lowest and 10 the highest). It is also best to organize your test ideas by area of your website—for example, tests for your top entry pages, product pages, and home page.

The test details and ratings in this tracker document will be essential for helping you and your senior executives and project management prioritize the order of your upcoming tests, and improve your optimization strategy.

This document is called the “Test Idea Tracker,” and you can find it in Appendix B as well as on this book’s companion website. I suggest that you download and print this out now so you can start filling it in as you go through the book. Here is the URL:

www.sybex.com/go/websiteoptimizationhour/test-idea-tracker.pdf

Next, you need to start tracking your test results by using another tracking document. In this document, for each test that you have run you need to include high-level test details, the impact on your relevant test success metric (percentage change) and what you have learned from running it, including insights gained from segmenting and targeting.

The details in this tracker will then help play a critical role in helping you iterate on your test results and improving your future test plan and optimization strategy.

This is called the “Test Results Tracker,” and you can find this in Appendix C and also on this book’s companion website. I suggest that you download and print this out now for when you start getting test results in. Here is the URL:

www.sybex.com/go/websiteoptimizationhour/test-results-tracker.pdf

Don’t worry about trying to complete both of these templates yet, as you need to read through the rest of the book first to review the ideas and best practices for optimizing your website. This day is more about helping you kickstart your test strategy and helping you understand how to document and track ideas for helping optimize your website.

Look for “Low-Hanging Fruit” to Test First

While you are reviewing the ideas for tests in this book, there is a simple strategy to use that will help you build some good test results quicker. This effective strategy is to note the test ideas that are likely to be easier and quicker to implement that also have good potential for increasing conversion rates and success metrics. This is often known as looking for “low-hanging fruit,” because you are in a sense expending the least amount of effort initially while still getting some good results. This is important because these quick testing wins will help you build a case for further budget and resources to improve your optimization plan.

Choosing these low-hanging fruit test ideas should require less technical, design, and marketing resources; therefore, they should be easier to implement. Some of the test ideas in this book, particularly simple text and call-to-action changes, may not even require any design help or resources at all, so they’re particularly good candidates.

Often you can even go “under the radar” and unofficially run these low-hanging fruit tests and implement these tests yourself, or with the unofficial assistance of an empathetic web developer that you might be able to find. This can be risky if things go wrong and your efforts get discovered, because it may minimize your chances of gaining trust and buy-in from stakeholders (a topic that will be covered in more detail in Week 5).

Consider Initially Outsourcing to Gain Quick Wins

Another approach to consider is to initially outsource some of your testing efforts. This is a particularly good option if you are having trouble obtaining internal resources and expertise to help you test (often this can take time and be expensive to hire all at once). There are many website optimization companies that will help you fill in these gaps, before you eventually obtain the resources internally to do this testing in-house. You’ll learn more about this outsourcing at the end of this week.

After Creating Your List of Test Ideas, Start Prioritizing the List and a Calendar of Tests

Earlier this week, you learned you need to create a list of ideas to test on your website. After you have gone through this book and created this list, you then need to return to this section and prioritize which tests you want to run first and start creating a test calendar. This will help form a big piece of your optimization strategy.

While you are prioritizing your test ideas, you should pay particular attention to the ratings you gave each test for difficulty of implementation and estimated conversion lift value. If possible, you should also consult with a project manager to get a more realistic idea of how difficult each test would be to implement. This is known as getting a level of effort (LOE), and your project manager can work with your IT department to gather this information for you.

Next, you need to come up with some potential launch dates for each of your proposed tests, and try to plan out at least three months’ worth of tests in advance. Depending on how fast you can launch tests, this should be at least three tests per month. Companies that use an optimization organization can easily run 10–20 tests per month.

After you have done this you need to create a detailed test plan for each of your proposed tests. You will learn more about this tomorrow because these are going to be critical for you to gain test buy-in and prioritization from key stakeholders.

Once your tests have been prioritized by project management for launch, you should create a testing calendar that stores details of launch dates, and what part of the website they are on. This is important to keep tabs on which tests are going live and where, and it is critical to share this with other departments so they are aware (particularly so that the IT department doesn’t make changes on a page when a test is on it and impact the running of it).

Thursday: Create Detailed Test Plans

To improve your website optimization efforts, it’s extremely important that you create detailed test plans. These enable you to flesh out important details for each test and are key to enable you and stakeholders to approve and prioritize test launches.

Putting this extra detail into your test plans will mean you will run tests more efficiently and be able to learn more from the results of them. You should create one test plan per test, and it should be in the form of a PowerPoint, Word, or PDF document. During the remainder of this day, you’ll learn what you need to put in an effective test plan.

Essential Elements for Better Test Plans

To make better test plans you first need to make sure your test plans contain essential elements, metrics, and details. Here are the most essential ones to include:

Use Wireframing Tools to Create Better Mockups in Your Test Plans

To help convey your test ideas better in your test plans, and increase the chances of approval, you should also try and create a visual mockup of your pages, elements, and variations being tested. A great free wireframing tool that you can use to help with this is called MockFlow (www.mockflow.com). This tool allows you to quickly and easily create wireframes using a variety of existing page elements. Figure 3-1 shows a wireframe example of the YouTube.com website created using this tool.

Figure 3-1: Example of a wireframe from the MockFlow tool

Friday: Consider Test Outsourcing—When and Why

The last important consideration you need to evaluate this week is whether you should outsource any or all of your testing and optimization efforts to a third-party vendor.

Because of the growing interest in website optimization, there are now several optimization companies that specialize in testing and optimizing websites. Many of these companies offer complete full-service solutions, including full-test creative services in addition to strategy and test implementation.

Although this is usually a more expensive way of doing testing, it can often bring some great expertise and fantastic results that you might not be able to achieve as easily by conducting in-house testing, particularly in the beginning when you may not have good testing resources or expertise available to your company yet. Indeed, many large companies completely outsource part or all off their website testing efforts, particularly companies that are savvy enough to dedicate considerable budget for testing and optimizing their website.

Major Reasons to Outsource

To help you determine whether to outsource some or all of your optimization efforts, here are the major reasons for doing so:

Pitfalls of Outsourcing

On the flip side, there are some distinct disadvantages of outsourcing your website optimization efforts. Here are three of the most common pitfalls:

Examples of Website Optimization Companies to Outsource To

While website optimization companies aren’t as common as web design or search engine optimization (SEO companies), they are now starting to gain in popularity. Over the last few years, several have appeared across the world, particularly in the U.S. market where optimization is more mature. These companies vary in their offerings, with some focusing on testing strategies and others offering full-service capabilities such as test creation and implementation.

Here are some website testing and optimization companies for you to look up and evaluate that can help with your optimization efforts:

• Adobe Digital (United States)
• SiteTuners (United States)
• Wider Funnel (North America)
• Elastic Path (North America)
• Ion Interactive (United States)
• Future Now (United States)
• Closed Loop Marketing (United States)
• EightByEight (United States)
• Conversion Voodoo (United States)
• Conversion Works (United Kingdom)
• Conversion Rate Experts (United Kingdom)

In addition to these, the more robust and expensive testing tools like Adobe Test&Target or Autonomy Optimost also have their own consultants that specialize in assisting with the creation and implementation of test strategies that you can outsource some or all of your testing efforts to.

Another option for you to consider is finding a web agency that has a good optimization and testing division. There are many of these agencies, and some of them have very competitive pricing models. A few good ones to consider are ZAAZ and Razorfish.

Questions to Ask Potential Optimization Vendors

To make sure that you will get the best results from outsourcing your website optimization efforts, you should ask potential vendors the following questions:

In general though, I strongly suggest you try running at least some initial testing efforts in-house first. This will at least give you some working knowledge and understanding of testing, and will allow for a more fruitful partnership with an optimization vendor if you later decide to go down that path.

Week 5: Understand and Create an Optimization Organization

In order to increase your chances of having an effective long-term website optimization program, much more is needed than just having a good website testing tool. Along with knowledge and usage of a good testing strategy and testing best practices, it is essential that you also help breed an optimization organization that increases your chance of success. This type of organization that lives and breathes testing and optimization and is often essential for the following three major reasons:

Sure, you can do some limited testing and optimization on your website without having a full optimization organization in place, and in fact, some small companies won’t even need all aspects of this. But if you work for a medium- to large-sized company, this week will be one of the most critical to read, understand, and implement. This is because the larger the company, the more potential there is for website optimization efforts to fail, due to factors such as problematic bureaucracy and slow website change process. Therefore, through the rest of this week, you will learn the key factors that make up an optimization organization and how to begin building one.

Monday: Find an Executive Sponsor for Your Optimization Efforts

One of the most important things to help you build an optimization organization is to find an executive sponsor to help with your efforts in the future.

This isn’t a full-time role; instead, it is more of a support function that someone in senior management takes on as part of their regular responsibilities. An executive sponsor doesn’t actually do any of the actual optimization work or strategizing, and although their role is fairly small, they can have a huge positive influence on the overall effectiveness of a website.

An optimization executive sponsor will have several key tasks to perform. You must make sure that potential candidates for this role understand and will be able to carry out the following responsibilities:

• Be able to influence senior stakeholders such as CEOs, CTOs, and CMOs and have strong relationships with them.
• Help prioritize testing efforts at the senior level.
• Champion the benefits of testing and optimization to senior stakeholders.
• Help influence and determine budget and resources for optimization.
• Help break down internal silos and barriers to improve optimization efforts.
• Help evangelize the benefits of optimization across the whole company.

An ideal executive sponsor would be a senior executive from the marketing or the IT department. This is particularly beneficial because of the large role these departments will play in your optimization efforts, which will make it easier to gain buy-in from the key stakeholders in these departments (which will be discussed in more detail tomorrow).

In order to find an executive sponsor, you should do some networking to determine which senior staff members are familiar with the basics of testing and optimization and the potential benefits of it. Once you have found a potential candidate, take them out for lunch and explain what you are trying to do in terms of optimizing, and how they can help.

Last, depending on your seniority at your company, you may even be able to take on this role yourself. This would give you even greater ability to promote your website optimization efforts; however, you need to make sure you have strong relationships with senior stakeholders in order to do this job effectively.

Tuesday: Gain Buy-in from Key Stakeholders

After you have found your potential executive sponsor (or sponsors), you need to gain buy-in from key stakeholders in your organization in order to increase the chances of optimization success. This is because they usually hold the keys to decision making and prioritization and often control the budget and resources you need to effectively test and optimize your website. Without buy-in from them, it will make it that much harder to begin adequate website optimization efforts and severely hamper your ability to optimize your website conversion rates and success metrics in the long run.

Today, you will learn some best practices to help you gain this all important buy-in from key stakeholders.

Forge Optimization Relationships and Break Down Barriers with Key Departmental Stakeholders

It’s important to realize that it’s very hard to run effective website tests without gaining buy-in and resources from many different departments. Gaining this will make it much easier to gain approval for your tests from senior management who unfortunately often have the authority to end your testing program, sometimes even before it begins.

It is particularly important to gain buy-in from the IT department to help you, because without approval from them, you can’t even implement and launch a website test or new testing tool, let alone be successful with it. Hopefully, by now you already have started to gain buy-in from the IT department using the methods outlined in Chapter 2.

You also need buy-in from the design and marketing departments to help you create the different versions of page elements in your tests. Often getting a “no” from just one of these departments can bring your testing efforts grinding to a halt.

The best way to forge these relationships is to reach out and explain your plans and goals to each of these departments and to make sure you involve them from the start. You should also always mention the potential high impact on revenue for the business. Once you have done that, you should look for testing advocates within each of these departments to help you gain buy-in from their corresponding department leaders.

If you work for a particularly large company, don’t forget about forging positive relationships with the legal team too, because they can often have the highest authority to say “no” when reviewing proposed changes to your website and can turn your best call-to-action test plans into boring stale copy that loses the interest of your visitors and greatly limits your potential to convert them.

You should also forge relationships with brand guardians, because they are also notoriously hard to work with at large companies and often don’t like to do any testing that involves changes to core brand messaging. An idea to help overcome resistance from them is to suggest running a test that relates to the brand, but only test a small percentage of the traffic. This way only relatively few people will see this test involving brand, and hopefully gain some good test results to prove brand guardians wrong.

If you continue to get little joy from the legal or branding team, you should communicate these issues higher up the command chain and get senior executives to try and influence them. This is why it is essential to have an executive sponsor help you when you run into issues like this that require senior management influence and power.

Always Mention the Potential Impact on Revenue and Profits from Testing

To help gain buy-in it’s important to remember to inform your boss and senior management about the impact that your tests and optimization will have on revenue and profit. For example, you always need to mention that “If I test and optimize X, it will gain Y additional revenue and profit.” Don’t just mention that you will improve conversion rates either, because they may not fully understand what the true impact of that is. Mentioning a potential increase in revenue and profit will be music to their ears and help open the door to an effective website optimization program for your website.

You should also convey the overall impact on ROI and revenue that having an optimization organization in place will have. Generally, because of the increased ability to test more frequently with better results on ROI each time, it’s definitely possible to achieve over 100 percent ROI in the long run, which if you invested $100,000 in optimization would equal an additional $100,000 ROI.

Table 3-3 highlights this by comparing results from a company that’s in Phase 1 of the website maturity lifecycle to the results of a company that’s in Phase 4 (the most advanced phase, as discussed in Week 4).

Table 3-3: Example of higher revenue

Another good way to convey potential profit from testing is to show particular visitor segments that are not converting as high as the visitor average for your website, and work out the revenue that would be made if these poorly performing segments were optimized and matched the average on your website.

Show Evidence and Case Studies That Your Competitors Are Effectively Optimizing Their Websites

To help increase the chances of gaining buy-in, you should find which of your competitors are already doing testing and optimization. This way you can prove that your company may be losing competitive advantage by not doing optimization (by saying something like, “Our biggest competitors are doing it, so it must be a good thing, and we should do it too”). As discussed in the last chapter, there are several good places to find case studies in order to prove how well they are doing with their testing efforts. Many optimization vendors also have great case studies that are quick and free to use to help you highlight this.

To get more competitor evidence, you could also try networking with competing companies to see if they are testing and what their results have been so far (although some might be reluctant to divulge this).

Make a Formal Presentation to Key Stakeholders Outlining Your Strategy

Back in Chapter 2, while trying to determine a budget for your testing tool, you made a presentation to senior stakeholders. Now is a great time to revisit and update that with your next steps and plans and have another follow-up presentation. You should reiterate some of the great candidates for testing, and as just discussed, don’t forget to mention the potential impact on revenue and profit.

You should also present the results from the 20-question optimization checklist from last week to make sure your senior stakeholders understand how the website is currently performing, in addition to showing them where your company currently stands in the website optimization lifecycle, and what needs to be in place to achieve expert website optimization (aka optimization nirvana).

Arm Yourself with Tips to Overcome Objections from Your HiPPOs

The HiPPOs (Highest Paid Person’s Opinion) in your organization will likely be senior management like CEOs, CTOs, or senior marketing directors, and often create major barriers to successful website optimization efforts. Unfortunately, problematic ones often demand things tested that they want, or worse still, sometimes don’t even allow testing on something if they have a strong opinion about it. Allowing HiPPOs to make website decisions regardless of any visitor insights or data that claim the opposite often has damaging implications to the conversion rates of your website.

To help overcome these issues and convince them to let you do more testing, you should always mention the following things to them:

• The potential impact of testing on revenue (as just discussed)
• That no-one is right 100% of the time, only your website visitors
• There isn't just one correct version of your website - it needs multiple variations to meet the varying needs of your visitors better.

You should also use web analytics data to support your reasoning for testing, and show them metrics highlighting problematic exit rates or poor conversion flows.

HiPPOs also often have website optimization objections based on the fear of negative potential impact of tests if they don’t go well. If this is the case, you should assure them that you will test on low traffic pages first or test low percentages of your traffic.

To help you gain some broader ideas to influence them, I strongly suggest you read these two excellent books:

What to Do if Your Senior Stakeholders Are Not Seeing Value

Lastly, it’s important to know what to do if your key stakeholders don’t see value in your website optimization efforts or plans. In order to counteract this, the first thing you need to do is identify the true source of who is raising the objection and their real motivation. Often this may not be immediately clear, or shrouded by some hidden agenda. Often one resistor among senior stakeholders is enough to cause unrest and problems when trying to do long-term testing, so it’s important to address their issues.

Once you have found the source and the reason, you can try and hone in on this. Set up a meeting with them (over lunch is a great way) and have a friendly discussion about why they aren’t seeing value and try and counter their reasons.

Often they may not just see the true value, so reiterate some of the benefits of optimization and the potential impact on revenue. If that doesn’t work, try and get them more involved with your testing so that they can take some of the credit when things go well.

If senior stakeholders are preventing you from running a first test, as a last resort you could go “under the radar” to run unofficially, using a developer’s help. You could even exclude internal office IP addresses from your test so that stakeholders don’t see it. Once you have got some good results, take them to stakeholders who aren’t seeing value. This approach is risky though, and you have to be very sure that the test is going to have a great impact on revenue.

Wednesday: Assemble a Dedicated Optimization Team

Unfortunately, as noted in the introduction to this book, website optimization is often neglected and misunderstood by companies. As a result, testing and optimization efforts are usually an afterthought and are assigned as part-time responsibilities to just one or two people, usually in the marketing or analytics team. Even if the company is progressive in terms of its website optimization efforts, there is often just one full-time dedicated optimization expert.

This unfortunately has the effect of immediately hampering your potential website optimization efforts, because there is a lack of the time and knowledge that are needed for a good optimization program. Soon the limited number of employees dedicated to optimization will become bogged down with office politics and a lack of resources available, forever chasing and hounding other employees just to gain a few hours of assistance.

Indeed, even with the greatest of test plans and full support from key stakeholders, your website testing and optimization efforts are going to be limited at best if you don’t have the correct team and skill set devoted to it. This lack of resources actually remains the biggest barrier to raising conversion rates, according to the “RedEye Conversion Rate Optimization Report 2011” by Econsultancy.

You also shouldn’t try to be a hero and do website optimization on your own and in a silo, because at best, you will have mediocre results, and at worst, you will damage your long-term hopes of website optimization. While this may give you some initial quick wins, starting off the wrong way and getting bad results can have a long-term negative effect on senior management’s opinion of website testing, damaging your chances of gaining optimization team members in the future.

Therefore, before you go any further with your testing and optimization efforts and strategy, you need to discuss building a great optimization and testing team with your boss and the necessary key stakeholders. Here are the ideal members of a successful website optimization team or department, ranked in order of importance:

There are actually a further two optional team members to help ensure even greater optimization and testing success:

Ideally these website optimization team members should all be contained within one independent group or form part of a larger analytics and optimization group. To help illustrate this, the following figures show a couple of examples of where these teams would sit in a hierarchy. Figure 3-2 shows a centralized team that sits under a director of analytics and optimization, which often works very well. Figure 3-3 shows a decentralized dedicated team with key members continuing to report to their corresponding departments.

While it may sound expensive and daunting to build such a large optimization team, there are some things you can do to help build this team more efficiently.

First, you should try finding people who intially can handle more than one role. For example, try finding a web analytics expert who also knows the basics of website testing. The website optimization best practices that you will learn in this book may give you enough knowledge to effectively take on two roles yourself and kickstart your optimization efforts while you’re looking for other dedicated website optimization team members.

Remember though, unless you have a very small company, in the long term you need at least one person in each role to ensure success. Don’t try and cut corners by permanently assigning one person multiple roles of the testing team, because this may short circuit your long-term testing plans and foster a “good-enough” mentality in your company.

Finally, you shouldn’t go in to meetings with your senior stakeholders with all guns firing and immediately request all these team members before you have even started your first website test or created a website strategy. You will have a far greater chance of succeeding if you wait to request these resources after you have gotten some good optimization results and built up a good case for them.

Figure 3-2: Centralized optimization team org structure

Figure 3-3: Decentralized optimization team org structure

Thursday: Integrate Testing and Optimization Web Processes into Your Company

One of the most important parts of trying to form a successful optimization organization is to integrate testing and optimization into your company’s everyday functions. If your company doesn’t have adequate processes or checklists to help embrace and push through website optimization efforts in a timely and efficient manner, your efforts and results are going to be limited at best, at worst negatively affecting your conversion rates.

Today, you will learn about this and begin doing it. While this may seem daunting, it will ensure that everyone in your company understands when website optimization strategy and efforts are most needed, and that your efforts will run smoothly.

Create an Optimization Checklist for New/Updated Website Content

All too often, an inconsistent method is used when a company launches new content or a new web design with a failure to adhere to a good process or checklist.

To help reduce the number of issues caused by this lack of process, here is an example of a website optimization and testing checklist you should run through when you are about to launch or update any content on your website:

Understand and Use a Better Optimization and Testing Process

Next, you need to audit and improve your company’s current process for website optimization. This is essential because an effective, continually used optimization and testing process is critical for long term optimization success. To get a better understanding of how this currently works at your company, I suggest you meet with all departments currently involved with your testing and optimization efforts and document what you find.

You will probably often find that the process is lacking in many areas, which causes lack of understanding and potential breakdown in communications and ultimately compromises your ability to effectively and efficiently run a testing and optimization program.

After you have determined your company’s current process, you need to learn what steps are essential to help improve it and make it more efficient.

Here is an example of a process that will help ensure that your website optimization efforts are most likely to succeed in your company:

These steps form part of a continual circular process, with Step 8 leading back around to Step 1. This helps you to iterate and improve on your tests to better optimize your website. It also essentially means that you should continually be optimizing and testing your website and that your website should never be considered finished or perfect—there is always room for improvement.

Figure 3-4 is a graphical illustration of this important flow.

Figure 3-4: Website optimization and testing process flow

Once you have started to implement this process, you need to get continual feedback from all involved departments so you can iron out any kinks and make improvements to it. The best way to get this feedback is to run regular optimization meetings, which you will learn about tomorrow.

Improve Your Project Management Process to Help Optimization Efforts

Many companies still struggle with efficiently implementing new website projects in a timely manner, and instead find themselves dealing with frequent delays and having to readjust their initial priorities. This can become a major barrier to optimization efforts, because the key to building an effective website optimization program is being able to make quick and frequent changes to the website while testing and launching winning versions.

If this issue occurs frequently within your company and projects often get derailed or take much longer than predicted, you are going to have to influence the decision making and prioritization process to make sure it doesn’t negatively impact your efforts.

One good way to try and do this is to get a project management expert within your company on your side and help them understand why you need to make quicker and more efficient changes to the website when running tests.

Ideally you should try getting the project manager to create a separate prioritization path that only contains projects and changes relating to optimization and tests. To increase the chances of this happening, you ideally need internal resources that are specific to just implementing these tests from a technical, marketing, and design aspect. This way, other major project plans that the company has won’t derail and slow down website optimization efforts.

You should also identify your company’s current project management process and work with a project manager to address any deficiencies. Ideally your company needs to be running what is known as an Agile (or scrum) process, which makes it easier to tackle much smaller and more frequently released projects as well as larger time-consuming projects that get released much less often. There are also tools that can help with running this, for example VersionOne (www.VersionOne.com).

NOTE: Agile is becoming a popular new project-management process. You can learn about it at http://en.wikipedia.org/wiki/Agile_software_development.

Friday: Communicate, Review, and Iterate

One of the most important steps to building an optimization organization is ensuring that your optimization efforts are communicated often, reviewed, and then iterated on. Without these last steps, you will also find it hard to make any headway on a long-term testing strategy and to build upon your previous testing results to maximize your optimization efforts.

Perform Weekly and Quarterly Optimization Meetings

One of the simplest yet most effective means of communicating is to hold weekly optimization review meetings. These should be run by the optimization team, and a consistent set of cross-functional team members should be invited each week. The major goal of these meetings is to ensure that current testing efforts are understood across the company and to communicate upcoming test plans. A very high-level communication of recent findings should also be discussed in the meetings. Another benefit of these meetings is that they really help to embed an optimization culture in your company.

To build on the foundation of these weekly meetings, any long-term successful optimization program needs more in-depth quarterly meeting reviews. These are essential to review recent testing efforts and results and to plan and prioritize tests for the next quarter. This is best done by setting up a quarterly optimization meeting that all testing key stakeholders should attend. Ideally these meetings should be at least three hours in order to devote enough time to adequately review and prioritize new efforts.

All recent test efforts should be reviewed at this meeting, including the lessons learned from the tests that worked well and those that didn’t work as well. This is vital to be able to iterate on your optimization efforts. In this quarterly meeting, it is also very important to review how your optimization process has been working and to identify and resolve any issues that still exist or have appeared.

Communicate Your Efforts via Regular Emails and Blogs

To ensure that testing efforts are consistently communicated and have high visibility across your company, another best practice is to set up a regular testing email communication schedule. These emails are sent every time a new optimization initiative or test is launched and should communicate details of page locations and timings of these tests or optimization initiatives, high-level results of previous tests and details on the winning versions that get pushed live on the website.

These email updates should be sent to all of the website optimization team members as well as senior management and key stakeholders. These emails should be short and to the point, and be formatted in bullet points to ensure that all recipients can read and digest the information quickly and easily.

Another great way to communicate your efforts is in an internal company blog that announces your latest optimization news and efforts. This blog can also serve as an education mechanism and include regular articles on optimization best practices and links to popular external blog articles.

Run Brainstorming Sessions

Another essential best practice to help communicate and iterate on your test results is to hold frequent brainstorming sessions. These should be run by the optimization team, and anyone from your company can come—the point of expanding the attendees of these is to promote that anyone from any department can get involved in testing and come up with test ideas.

These sessions should be held at least quarterly, and ideally a half day should be devoted to each session. The first half of the meeting should be an open forum where attendees put their ideas on sticky notes. The second half of the meeting is used to collate and prioritize these ideas based on their ease of implementation and likely impact on conversion.

To help improve this session and create better ideas for future tests, you should come prepared and bring a list of key pages to discuss that have high potential impact (top entry pages and ones relating to conversion key flows for example). Ideally you should also show visual click map data relating to these key pages, so attendees can understand what visitors are currently doing on them (you will learn more about this important visual analysis in Chapter 5).

Use Test Idea and Test Results Tracking Documents

As discussed earlier, it is essential to use tracker documents for storing your test ideas and results. These documents are essential to help communicate, review, and iterate on your efforts in your weekly and quarterly meetings. These templates are called “Test Idea Tracker” and “Test Results Tracker” and can be found in Appendix B and Appendix C, respectively, as well as on this book’s companion website.

Use a Shared Space for Your Optimization Documentation

Your optimization documentation should be easily shared and communicated within your company to ensure your efforts are transparent and easily accessible by anyone.

This is best done by making use of a shared platform on your company’s intranet to upload all of your optimization documentation. This should include uploading optimization meeting notes, test plans, test trackers, and they should be organized into categories that are easy to navigate and understand.

Ideally this shared space should be maintained by a single person in the optimization team to ensure the contents remain well organized and easy to use. If this doesn’t occur, these shared space areas can often just become a disorganized dumping ground and not much better than having no central storage space.

There are a number of fairly cheap solutions to help you store your documentation on a shared platform, such as Microsoft SharePoint and eRooms.

Week 6: Learn Testing Best Practices to Improve Your Success

Throughout this book, you will learn some great ways to test your website to improve your website conversions and success metrics. However, if you aren’t armed with website testing best practices, your testing may be inconclusive at best, and at worst, may actually hinder your conversion rates. To help you avoid this happening to your testing efforts, you need to understand and use a great testing approach, how to create good tests, and then learn some testing best practices. All of which you will learn about this long but very important week.

Monday: Use an Insightful, Hypothesis-Driven, and Iterative Testing Approach

It might seem easiest to just pick something to test on your website, come up with a few variations, and hope for good results. Although this may occasionally help improve elements of your website, you may often run tests that have very little impact on your website and your success metrics. Testing without using an adequate approach and process will also waste time on key resources and potentially damage the likelihood of being able to run more tests in the future.

Therefore, in order to gain the most success with your testing efforts, you need to first understand and follow an optimal approach and process to website testing. This five-step testing process flow is shown in Figure 3-5. As you can see, it starts with the creation of insights and a test hypothesis and ends with running test iterations to maximize the impact of your tests.

Figure 3-5: Best practice testing approach

The following subsections describe the important parts of this optimization process flow that you need to understand and apply.

1: Create Insights from Analytics and Feedback Tools

The first step is to always try and look for insights to help formulate test ideas. Rather than just guessing at what to test, or listening to HiPPOs all the time, you should be using the analytics tool you set up in Chapter 2 and feedback you generate from your visitors (which you will learn about in Chapter 4) to create new insights into your visitor behavior. These insights should form the pillars of formulating your website test ideas and ensure that you are testing for a valid reason with your visitors in mind, rather than just doing random testing.

Examples of an insight generated would be something like “Our visitor feedback has revealed that our home page visitors seem to be confused by the number of options they can click on” or “Our analytics data reveals that our shopping cart has an unusually high drop off rate on the shipping details page.”

Throughout the rest of the book, you will get some great ideas for tools and best practices to discover these critical visitor insights for your test ideas.

2: Create Your Test Hypothesis

You should always come up with a good reason to test something on your website; don’t just test something for the sake of it. This reason should be in the form of creating a good hypothesis for each proposed test—in other words, your hunch of what is likely going to happen. A few examples of a hypothesis would be “Removing the phone number form field is likely to increase form completion rates” and “Adding in reviews for all products is likely to increase visitor social proof and as a result increase sales.” The hypothesis you create should always relate to the initial insight you generated in Step 1 of the process flow.

Having a strong hypothesis will help ensure you don’t encounter the GIGO (garbage in, garbage out) effect with your test. The better your hypotheses, the better the tests are likely to be. Doing this also makes it easier to persuade senior management why you want to run a particular test, and help you can gain approval, prioritization and the necessary resources to run the test.

3: Create, Implement, and Run Test

The third step is to actually get the test up and running on your website. This includes getting the variations created that are being tested, creating the actual test in your testing tool, implementing the test code on your website (include performing QA on it) and then pushing it live. All of the aspects of this will be covered in more detail throughout the rest of this week.

4: Analyze Test Results and Launch Winners

This next step is often where sub-par test results get pushed live without proper analysis. This subject will be covered in much more detail at the end of this week. To summarize though, when analyzing your results you should make sure you gather enough data (at least 2 weeks of results, and at least 100 conversions for each version being tested) and also make sure the predicted winner has enough statistical confidence from the tool to declare it a winner (at least 80 percent confidence). If you don’t do this, you will risk pushing a version live that isn’t actually the best version, thus reducing your potential conversion uplift.

You should also not just roll out and push live one winner to all your visitors; instead, you should segment your test results to find higher conversion lifts for different visitor segments (for example your paid search visitors may convert higher to a particular test version) and push a winning version to each of these visitor segments.

5: Run Test Iterations on Tests to Maximize Impact

Sadly, this last step is often neglected by many online marketers who are testing and optimizing their websites. Rather than simply moving on and testing something different, you should do further iterations on your test by learning from the results (even if the test didn’t work well, you should be able to draw conclusions to learn from).

Doing this test iteration usually results in much higher conversion lifts than your initial tests generate. Many companies don’t do a good job of this, so it will be a great source of competitive advantage if you do it well. This important subject of test iteration will be covered in more detail at several points throughout the rest of this week.

To ensure maximum impact from your optimization efforts, this approach then should continuously loop back around after the last part and back to the start again. This follows the same logic as a popular business theory called Kaizen, where improvement is a continuous process, not just a one-off thing.

Tuesday: Learn Strategies for Creating Effective Tests

One of the most critical parts of any effective long term optimization plan is the understanding and usage of strategies for creating tests that result in higher conversion lifts.

Today you will learn several different strategies that you can try, but remember there is no one specific test strategy that works every time. Depending on your website, your type of visitors, and your value proposition, some of these will work better in some situations than others. The first few of the test strategies described here usually work well initially though.

Go for Bigger, Radical Page Changes First (Innovation Tests)

One good initial approach to your testing plan is to run a basic A/B test between one of your influential conversion pages and an entirely new design and style of it. This should involve simultaneously changing, adding, removing, or moving many major design elements. For an example, a radically different version could be created by moving the location of your signup form, removing the sidebar, and reducing the header area size. The key here is to not just test a few small things per page—instead, go for many more radical changes.

This is effective because going for this more-radical approach is more likely to significantly influence your conversion rates, hopefully with a major positive improvement. This form of testing is often referred to as an innovation test.

When you find a radical new page design that drives great conversion lifts, don’t just stop there and pat yourself on the back. It’s essential that you do a follow-up multivariate test; otherwise, you are likely to be missing out on additional potential conversion rate increases. This follow-up test is often known as an iteration test. (You’ll learn more about this type of test shortly.)

It’s important to bear in mind that this style of innovation testing also has the potential to lower conversion rates just as radically as it might increase them (and don’t be disheartened if not every innovation test that you run has good results). To limit any potential negative impact of this style of test, you should limit the percentage of your visitors who see this innovation test (although this can mean it takes much longer to get a significant test result if your page doesn’t have much traffic).

You could try for another radical design or simply begin doing multivariate tests on more precise things that you know you are more likely to have higher increases (like testing calls-to-action, headlines, and images as discussed throughout this book).

In order to get this alternative page created, you will need to involve your design and marketing team and collaborate to create it. Ideally this new page should be based on at least some previous findings or ideas you may have and not an entirely random design. If you can get the resources, you could also try creating a third page version to test against, which may increase your chances of finding a more significant conversion lift.

Use Multivariate Tests to Look for Conversion Influence on Page Elements (Conversion Influence MVTs)

A simple initial approach to check for what is influencing conversions on any of your web pages is to set up an MVT for your key page elements and create just one alternative version for each element (either a variation, or turning it off).

This is known a conversion influence MVT and enables you to see which page elements are worth focusing on and doing further testing on to find the best variations. It also works well because it means you don’t have to waste time creating and testing many different variations for a page element that might not even have much influence on your conversion rate.

To determine which versions have the biggest impact, you need to look for ones in your test results that have the greatest conversion influence. This can be done in most testing tools by examining conversion influencer reports—for example, the Element Contribution Report, as it’s called in Adobe Test&Target, or the Relevance Rating Report in Google Website Optimizer. As mentioned in Chapter 2, this will indicate which of the elements being tested have the highest impact on conversion, and also which variation of the element causes this impact.

Once you find which page elements contribute the most to conversion, you should iterate by doing follow up A/B tests to hone those elements and improve conversion lift.

NOTE: The optimal number of page elements to use for a conversion influence MVT is seven, which is known as a 7 × 2 multivariate test design (seven elements tested using two variations).

Test Moving Key Elements on Your Pages to Check for Conversion Influencers (Real Estate Tests)

You may have elements on your pages that are performing poorly for conversion solely because their location may not be optimal. For example, if you have a highly converting module like “Latest Deals” but you are placing it too far down your page, you would generate more conversions if you gave it more prominence by placing it above the page fold. And never just presume your page elements are already in their best locations.

This conversion influence of key areas on your web page should be checked and tested to maximize your conversion rates. This can be done by setting up a test to move around key elements of your pages and is often known as performing a real estate test.

A few examples of this would be to test moving your benefits module to higher up in your sidebar and by moving your testimonials from your sidebar to high up in your main content page area. Generally though, you will find that areas above your page fold will have more influence on your conversion rates, so you should take that into account when setting up these tests.

You will often have to work with your web designers on this type of test because your page still needs to look good and flow well after you have moved elements and this may have an effect on your conversion rates.

Ideally you should do this type of testing in association with what you learn from analytics click map tools, because this will identify your most and least clicked page elements that you should consider moving.

Once you have found out which page areas and positions have the greatest conversion influence, you should iterate by doing follow-up tests on the elements found there to gain even higher conversion rates.

Turn Off Page Elements to Check for Conversion Influencers (Inclusion/Exclusion Tests)

A simple and often effective strategy is to test turning off elements on your pages to find conversion influencers. This is known as running an inclusion/exclusion test. For example, on your home page you could test removing a promo module, or remove a set of subnavigation links. This type of test works particularly well on pages with many elements on them, such as homepages or category pages. Running these tests results in less cluttered and more focused pages, which often increases engagement rates and conversion rates further.

To run this type of test you need to determine a few things to test removing from your pages. Once you have set up a test and gained some results, you need to look at how much each element influenced conversion. If the element removed had a negative impact on conversion, you know that it should be kept because it causes higher conversion by showing it (and you should then do follow up A/B tests to determine winning variations of these).

The elements removed that resulted in no reduction in conversion rates should be considered for removal from your page or moved to a less important page or location.

However, please be aware of removing elements, because you may actually end up removing a key one without realizing it. This is because the one being removed may be poorly designed, hard to understand or unusable for visitors and could actually be influential on conversions if you tested improving it first.

Therefore, you should always think before you remove elements from your pages, particularly ones you think may relate to use cases or conversion goals. If you are wary about removing it, consider creating a better variation and then retest it to see if it still has no influence on conversion. You could even consider surveying your visitors to see if they find the element useful. If they do, instead of removing it, you could de-emphasize it by moving to a less-visible area of your page or website.

Segment and Target Your Tests to Maximize Your Success and Conversion Lifts

To increase your conversion rates further and squeeze even more juice out of your test results, you need push live higher converting test variations to different segments of your visitors. This is known as segmenting and targeting your tests. This works because some of your test variations will satisfy the needs better for particular segments of your visitors, and as a result of this, they will convert much higher.

This also works because it results in your having a much more dynamic website that better meets the needs of many different types of visitors, rather than just trying to satisfy everyone with just one version of your website.

To do this, first of all, you need to create key visitor segments that you want to analyze and target your content to (for example, repeat visitors, and these should match the ones that you created in Chapter 2). Then you need to filter your test results and look for segments that converted much higher than your average conversion lift for the test. You then push live the winning version to each of your visitor groups that performed higher.

This can be done in most advanced testing tools like Adobe Test&Target or Autonomy Optimost, but unfortunately Google Website Optimizer doesn’t currently offer this capability (and is a key reason to consider upgrading from it). These offer precreated visitor segments in addition to the ones you created in Chapter 2. To segment and target your tests, you need to filter your test results in your testing tool, review each of your segments and then push live any versions that you notice with particularly high conversion rate lifts. For an example of how to find these segments to review, in Figure 3-6 you can see how for Adobe Test&Target.

If you don’t do this and only look at your test results holistically, you may be missing out on some good potential lifts. For example, your paid search visitors may convert much higher to a particular test variation, and if you pushed the overall test winner to them instead (and not the version they converted higher for), you would be getting a lower conversion rate from those visitors, thus reducing your potential conversion rate.

Figure 3-6: Segment filter list in Adobe Test&Target

You also need to understand that when you combine all the higher lifts you gain from pushing winners live to many visitor segments, this can be significantly higher than the conversion rate lift gained pushing live just one winning version to your whole website.

NOTE: Ideally there should not be just one single test winner to push live on your website—there should be a winner to push live for every key visitor segment, and doing this will result in much higher conversion rates.

You will learn advanced segmenting and targeting best practices and ideas tomorrow to increase your conversion rates even further.

Use Competitive Intelligence to Come Up with Test Ideas

Another good testing strategy is to analyze your competitors’ websites to find great test ideas, just as you did to help improve your website’s unique value proposition. Reviewing each of your main competitor’s websites for their features, functionality, headlines, call-to-actions, and content offerings can be a goldmine for test ideas. Therefore, devote some time to check out what your competitors are doing and come up with some new test ideas. You should then repeat this every couple of months to see if they have launched anything new that you can create test ideas from. It’s best to take screen captures of any test ideas you find because this will help your marketing and design team create similar versions to test.

This competitive analysis also helps you gain test buy-in by showing your senior executives that other competitor websites are already offering what you are proposing to test.

But do realize that just because your competition might be doing it on their website doesn’t mean it will work on your website (or even work very well on their website).

Always Iterate Based on Your Test Results

This is one of the most important testing strategies that you should always use when testing and optimizing your website. You shouldn’t just do random or ad hoc testing whenever you get an idea or whenever your boss wants you to test an idea that they have. Instead, you should always base your tests on previous “learnings” whenever possible or test an additional element relating to the initial test. This is known as test iteration and is vital for a long-term successful optimization strategy.

To help you understand this testing iteration principle a little better, Table 3-4 lists some examples of initial tests and iterations on them.

Table 3-4: Sample initial and related iteration tests

Example of a High-Impact Initial Testing Strategy

Now that you have learned some great test strategies, let’s look at an example of an initial testing strategy that has high potential for success on a variety of website types. Here are the steps you would need to take:

Wednesday: Learn the Power of Targeting and Personalization for Improving Conversions

Today you are going to learn about an emerging advanced optimization strategy that can have an extremely powerful effect on your website’s conversion rates.

Historically, most websites offer the same version of their website to their visitors, no matter who they are or what their needs are. In a consumer-driven world that is increasingly customized, this “one size fits all” approach to websites is no longer good enough. At Starbucks you can have your drink made any way you like it, rather than them just having the same drink versions for everyone (such as a skinny soy mocha latte with light whip…), so why not have a website that can meet the needs of different types of visitors?

Smart website marketers are now beginning to realize the benefits of customizing their website to individual visitors based on their needs and previous behaviors—often with amazing impact on conversion rates. They are doing this by using tools that can target different segments of visitors with more relevant content that meets their needs better, and thus engage and convert them better. For example, they can detect whether the user is a new visitor and show them content designed to educate them about the benefits of the site or target offering special coupons for visitors who have purchased multiple times before.

Over the rest of the week, you will learn about tools and best practices for doing this to get even more out of your testing efforts.

Research Advanced Visitor Segmenting and Targeting Tools

This segment and targeting can be done fairly easily in most advanced testing tools like Adobe Test&Target and Autonomy Optimost. Some budget and free testing tools, such as Google Website Optimizer, don’t offer this segment targeting ability though. However, there are now some free tools that you can use to run a basic level of segmentation and targeting, the major one being BT Buckets (www.BTbuckets.com). This tool actually works very well with Google Analytics, and you can pull in the visitor segments that you have created from that tool and target content to them.

For an advanced, hands-free way of doing this, you can use a tool like Adobe Test&Target 1:1, which is an additional feature you can purchase and add on to Test&Target. This is automated behavioral targeting at its best and it actually learns from visitor behavior to automatically predict what content individual visitors (rather than groups of visitors) are likely to engage with the most. Another good behavioral targeting tool that you can research is called Cognitive Match (www.CognitiveMatch.com); specifically, this tool does a great job of behavioral targeting in ads (which will be covered more in Chapter 8).

Next, you’ll learn about several best practices for creating a more personalized and targeted website to help you push your conversion rates to the limits. Attempting some of these can often give your website optimization efforts a great boost.

Show New Visitors Content to Help Engage and Convert

As you will learn from several points throughout the rest of this book, visitors will often judge your website in a matter of seconds. Therefore it’s important to recognize your first-time visitors when they arrive on your website, and show them targeted content that helps explain the value proposition of your website better (which you will create and improve in the next chapter).

One of the best ways to do this is to create a new visitor segment in your behavioral or advanced testing tool, and when they arrive, show them a “First-Time Visitor” module on your home page and possibly in the right-hand rail of your other pages. This module should be titled something like “First Time Visitor? See Our Quick Start Guide” and should contain your value proposition, with a link to a page that explains this in more detail.

You could also test showing cheaper products, or showing coupons or trial offers specifically to your first-time visitor segment to see if that helps increase conversion rates. You could also show them whitepapers or guides for them to download that help them understand better what your website offers.

In later chapters, you’ll see examples of how to best engage your first-time visitors for different types of pages.

Target your repeat visitors with their most commonly browsed content or products

Wouldn’t it be great if you walked into your local book store and saw your favorite types of books right at the entrance for your convenience? Well that would be pretty hard to do in the real world, but you can do things like this on your website by using targeting.

This process of showing favorite content or products to repeat visitors to engage them better is known as content affinity targeting. It represents a great way to improve the relevance of your web pages to better meet the needs of your visitors, and usually is a great way to increase conversions as a result of higher visitor engagement levels.

To do this, set up a targeted test that tracks visitor’s favorite content (based on at least three views of a particular piece of content or product) that then shows them content relating to this in a module on one of your key pages. A pre-created option for doing this is available to you in advanced targeting tools like Adobe Test&Target. For example, you could use your hero or image slider on your homepage or category pages to show new or top links in that section that they browsed the most before. You shouldn’t just show them content based on a single page they saw before though, because visitors often browse websites and see pages, but that doesn’t mean that a visitor likes every page they see.

This is a particularly good targeting tactic to use on your home page, category pages, and top entry pages (as will be discussed in more detail in later chapters).

Create a Previous Purchaser Visitor Segment and Show These Visitors Recommendations and Coupons

It’s also very important to segment visitors who have purchased from you before and show them targeted content that will encourage them to buy again. This is because it’s often easier to generate orders from existing customers than from new visitors.

The best way to target these visitors is to create a segment that tracks visitors who have purchased before and include modules on your home page and your shopping cart page that show them product recommendations based on what they previously bought, and possibly offer them relevant coupons too. In later chapters, you’ll see relevant examples for different types of pages.

Segment and Target Your Visitors Regionally

Another powerful way to segment and target your visitors is to capitalize on which geographic region your visitors are coming from. For example, you could target your home page visitors by showing them their nearest local store or showing them international-related content if they are arriving from a different country.

This is also particularly useful for targeting and showing products or services to regions that are weather and seasonality sensitive. For example, test promoting fans to visitors from regions that get particularly humid or hot in the summer.

Figure 3-7 shows a great example of this type of targeting by Macys.com. The content displayed on their homepage is specifically targeted to visitors who arrive at the Macys.com website from outside of the United States and helps provide greater value to these types of visitors.

Figure 3-7: Example of international visitor targeting

Create an Abandoned Shopping Cart Segment for Retargeting

A visitor may abandon their shopping cart on your website before completing the purchase for any number of reasons, many of which are out of your control.

One great way of increasing the likelihood of a visitor returning to their cart and finishing purchasing is to create a segment of visitors who abandoned their shopping cart. When one of these visitors comes back to your website, you would target and show them a message that reminds them they still have items in their cart to try and influence them to complete the purchase. This could be in the form of offering a coupon for this purchase, mentioning that the price may go up in the future, or warning that an item may soon be unavailable.

You can also tie this segment of visitors into your email marketing efforts and target them by offering them a reason to finish their purchase (email marketing optimization like this is covered in more detail in Chapter 8).

Personalize Your Website Offerings Based on Peer Behaviors

Another great way to target your visitors and personalize what your visitors see is to use the social proof theory, which speculates that people are influenced by what their peers do. Using this theory that you will learn more about in Chapter 6, you would target visitors and show them content based on what purchasers with similar characteristics buy or look at.

Amazon.com has been doing this for years and has honed this to an exact science with some astounding results. The more you browse and buy on their website, the more intelligent it gets at predicting what you might like to see, as illustrated in Figure 3-8.

Figure 3-8: Example of personalized recommendations

Segment and Target Your Visitors Using Offline Data Sources

Wouldn’t it be great if you could show different content on your website to someone who had never even been to your website but had previously talked to someone in your call center or had been contacted by your sales team via email?

Previously, it was pretty much impossible to bridge the gaps between web analytics, website optimization, and third-party offline data sources. This meant it was usually hard to determine if a visitor had purchased before or if they had been sent marketing materials.

With recent improvements in web technologies, if you are using an advanced website testing tool, it is now possible to do just this. Now you can target visitors based on a multitude of offline activities and provide a more contextual experience for the visitor, often resulting in much higher engagement and conversion rates.

The Adobe Test&Target tool even gives you the option to use offline traditional marketing segments from Alliant. These segments enable you to target certain visitor affinity groups and demographics. This is very valuable offline data that can help increase your conversion rates even further.

Segment and Target Your Mobile Phone Based Visitors

One of the most important visitor segments to target is the growing number of visitors who arrive on your website via mobile phones and devices. This is because a mobile visitor’s needs are often a lot different due to the smaller screen size and slower connection speed, so they usually prefer seeing a different, easier-to-use version of your website. This will be covered in much more detail in Chapter 7.

Use Automated Predictive Learning to Improve Targeting

There are now advanced website testing tools that make these targeting best practices even simpler to use. These automate the process of continually showing the best content to individual and groups of visitors and often result in much higher conversions.

For example, Adobe Test&Target offers a great targeting add-on called Test&Target 1:1. This upgrade builds in-depth learnings from visitors’ behavioral patterns to predict and automatically show individual visitors more relevant content resulting in higher conversion rates. All you have to do is create and load into the tool several variations of content, provide some rules and logic that determine conversion success, and then give the tool a week or so to learn about your visitors patterns, and it will automatically start showing the best content to them.

Unfortunately, these advanced automated predictive learning tools are currently not very cheap and require you to gain an additional testing budget to make use of them.

Thursday: Avoid Common Mistakes When Creating and Running Your Tests

Today you are going to learn about important principles to follow while you are creating and running tests on your websites. These will reduce the chances of you making errors when creating and running them and result in greater efficiency and impact from them.

Don’t Go Crazy with the Number of Test Variations in Your Multivariate Tests

When setting up an MVT, don’t try testing too many elements and variations per page. Testing over five different page elements with more than three different variations for each element will take much longer to get significant test results. This is because the tool will have to get enough traffic for each combination being tested before it can declare a significant winner, and if the page you are testing has low traffic, it can take many months to do this.

Not only does it take longer to gain significant results when testing a large number of test variations like this, but it will also take longer for you to create the many different test versions needed (particularly if you need significant help to create these versions). This can dramatically reduce the number of tests you can run per month and slow down your optimization strategy progress.

To avoid this issue when testing many page elements at once, ideally you should try to limit your test to seven elements with just two variations of each (this is a conversion influence test that we discussed earlier).

And if you want to test more than seven, I suggest that you run an innovation A/B test instead, where you create an entirely different page and test it against the current page. This will take much less time to get a significant test winner, and you could then do further follow up testing on the elements on the winning page.

Make Sure Your Test Challengers Are Sufficiently Different

One of the more common reasons for poorly performing tests is because the challenger versions are too similar to the control version and visitors barely notice the difference, let alone influence conversion rates. This is particularly problematic when you are testing graphical elements like banners or images that designers have creative control over. To help alleviate these issues, you should meet with the design team and ensure they know the reasons behind the testing motives and direction to give them some examples of what constitutes a big difference.

Ideally, you need to create one of the challenger versions to be significantly different. For example, don’t just slightly change the color shade of a button; you should go for a bolder, more significant change, such as using very different imagery or wording. However, be aware that changing multiple things at once is not recommended without any iteration testing to find what elements of the change influenced conversion the greatest.

If your challenger versions require different text (for example, a new headline or call-to-action) you should always get assistance from your marketing team to help create your variations to increase the chances of it influencing your visitors better. Ideally you will have a marketing lead devoted to this function, and they would form part of the optimization team (as discussed earlier).

Don’t Spend Too Much Time Making Your Test Challengers Perfect

One common reason for a lack of testing efficiency can be spending too much time designing perfect test variations for your challengers. You shouldn’t spend hours on the design of them making them perfect, because most of the versions that are created won’t even end up winning and staying on the website. Instead, you need to invest just enough time to convey the main point of the variation—spend 80 percent of the normal amount of time that you would use on it, and don’t worry about spending the final harder, more time-consuming 20 percent on making it look perfect.

The key point to remember is that you can always tweak the winning version slightly if needed, such as exactly fulfilling brand requirements for example (as long as you don’t change anything major about the winner version, otherwise that would invalidate the test results).

You should definitely emphasize this point with your design department and make sure they aren’t spending too much time making each test design absolutely perfect. Doing this will mean fewer visual resources are needed to run tests, ultimately meaning you can run tests more often and more efficiently.

Plan Early for Your Test Resources

When you create tests, you will often need help from many different departments, including the IT department (to help implement your test), design department (to help design test variations), and marketing department (to help come up with marketing related improvements like calls-to-action or headlines).

Not planning early to obtain this help and resources will cause your tests to be created too slowly and inefficiently, ultimately reducing the frequency of tests (and you need to be able to test many things simultaneously for an effective long term testing program). The reason why this often happens is because last-minute test resource planning risks receiving pushback from other departments, and there are usually many last-minute questions that will need answering about the test, which will also delay the time of launch.

Therefore, you need to plan out and prioritize which resources you need as far in advance as possible and work closely with a project manager to increase your chances of success.

This planning is even more essential if you don’t have the luxury of having your own dedicated team of testing resources to help with the technical, design, and marketing aspects of your tests.

Limit the Potential Negative Impact

While it’s a good strategy to test and optimize pages that have the highest potential to increase conversion rates, you need to be cautious and make sure you limit any potential negative impact on your website if a test performs badly. Doing this will give key stakeholders less of a reason to question the value of optimization if your initial efforts go wrong.

There are two main ways that a test could have negative impact: first if you don’t perform a high-level QA on your test, it may break your website or slow it down; and second, if your test doesn’t do as well as anticipated, it actually lowers your conversion rate.

The best way to reduce this potential negative impact is to initially limit the number of visitors participating in your test to a small amount—for example, around 10 percent.

Then if the test shows no negative impact, gradually increase the amount of traffic seeing your test up to 100 percent. This option is available in most website testing tools and is fairly easy to set up (see your testing tool’s help section to find out exactly how to do this).

Be Wary of Changes in Major Sources of Your Traffic During Tests

Another thing to be aware of is the impact of your traffic source patterns that might change during the period you are running your tests. As mentioned earlier, conversion rates can differ tremendously based on traffic source and therefore can play havoc with your test results.

For example, if you are testing on your website and you send out a large newsletter campaign to it, this is likely to increase conversion rates for a few days and make it look like your test is performing better than it actually is for a brief period. This is because newsletter traffic often converts higher because it’s a higher quality source of traffic.

Another example would be if your marketing department started to run a paid search campaign to your website in the middle of one of your tests. This kind of traffic is also much more targeted and has a greater chance of converting, which will influence your results.

Most of the more advanced testing tools actually allow you to filter out segments of traffic to avoid situations like this. Therefore, before you launch a test, you should ask your marketing team whether they have any major traffic-driving campaigns planned during your test period that will likely change the quality of traffic on your website (thus affecting your test results), and then set up filters to remove these segments during the test.

Don’t Remove Poorly Performing Test Versions in the Beginning

Quite often people running tests are tempted to remove and exclude test versions that don’t seem to be working well, particularly if they or others are already convinced that these poorly performing versions aren’t as good as the other test versions.

This is a bad thing to do, because sometimes test versions that start off doing the worst can often do much better over time and even end up winning the test. Therefore, you should keep all your test variations running until the test has finished.

Friday: Learn How to Analyze Results and Determine a Test Winner

Now that you understand best practices for running your website tests, there are several other testing best practices that you need to understand to be able to interpret your test results and determine test winners. Today you will learn about these.

Gather a Significant Number of Conversions before Declaring a Winner

First, when analyzing your results, it’s important that you have enough conversions for each version being testing before deciding on a winning version. If you don’t have enough conversions, the test results won’t be statistically valid, and you will risk pushing live a version that actually isn’t the optimal version. In the worst case scenario it could mean you end up pushing a version live that actually ends up lowering conversions.

A quick rule of thumb is that you need to have at the very least 100 conversions per element or recipe you are testing. The more results you can get, the better, because this increases the chances of finding a valid winner.

Always Check Your Confidence Level before Declaring a Winner

Having enough conversions is important, but you also need to check whether your testing tool has gathered enough confidence in the winner it is predicting. This is particularly important to pay attention to if many of your versions are showing a similar uplift in conversion. Most testing tools give you an idea of this testing significance level by showing a confidence level in the test reports, as you learned in Chapter 2 when reviewing key reports in testing tools.

Ideally, you want as high a confidence level as possible for a test version before declaring it a winner, and ideally at least 90 percent confidence should be your goal (which in Adobe Test&Target is represented by three full green bars). However, sometimes it may take too long to get to this point, so you might want to consider using slightly less confidence, like 80 percent.

Learn How to Interpret Your Conversion Rate Lifts

While testing tools show you the conversion rate lift for each version you are testing, it’s important to understand what this is and what is considered a good conversion lift. This lift is calculated by comparing the conversion rate of each version being tested (test challengers) against the conversion rate for your test control (default) version.

Results for test challengers that show barely any conversion lifts like 1 or 2 percent, although positive, are not really considered to be very good and you might not want to consider pushing them live on your website.

Conversion rate lifts between 10 percent and 20 percent are considered relatively good and probably worth pushing live, while a good conversion rate lift is usually considered anything more than 25 percent.

If your test results manage to get above 50 percent lift, that is considered a very good conversion rate lift and not really that rare, particularly when you look for good results in your visitor segments (which will be discussed next). If you manage to reach over 100 percent conversion lift for any of your test results, that is really outstanding because you are essentially doubling the number of conversions on your website.

Anything below zero percent conversion lift obviously is going to be negative, but don’t just discard your tests if none of your test challengers get above zero percent. You should learn from results that don’t win, and use them to base further iteration tests on.

Always Segment Test Results to Find Higher Converting Versions

As discussed earlier, to achieve higher conversion rates you need to segment and target your visitors with more relevant content. Therefore, when you are analyzing your test results, you need to analyze each of your test visitor segments to look for higher conversion rates than the overall test result, and then push live those versions. For example, segments that quite often have a higher conversion rate lift are paid search visitors and newsletter visitors.

Don’t Rush into Making Quick Decisions from Initial Test Results

Rushing into making test result decisions may lead to suboptimal or negative conversion rate improvements without your realizing it. Even if you see great initial spikes in conversion rates, it’s essential that you wait for initial variances to level out; otherwise, you may pick a winning version that actually isn’t the best. You need to get at least seven days’ worth of nonfluctuating data before you can make a decision regarding the winners.

Many companies make this mistake of not waiting long enough and end up pushing live a test variation too early that isn’t actually the best one, potentially even lowering conversion rates on their site.

Depending on your traffic levels to the pages being tested, you usually need to wait a couple of weeks to gather enough significant data. If you have a particularly low-traffic page, you may need to wait up to a month to get results; if you have to wait any longer, you should question the value of optimizing that page because it gets such little traffic.

Know When to End a TestThat Doesn’t Beat the Original Version

Tests usually take at least a few weeks to provide significant data, particularly for a multivariate test or an A/B test that is being run on a page with lower traffic per day. But it’s important to know when to end a test if any of the challenger versions don’t gain conversion lift over the control version. You shouldn’t get bogged down by waiting months to get a result, because there will be times when you never actually get a winning version. If the results are still fluctuating quite a lot, then it may be worth waiting slightly longer than two weeks; but if the results have flat lined for over an extra week and still don’t indicate a good winner, then you can consider ending the test.

If you end up with a result that doesn’t beat the control version, as highlighted earlier, you should try iterating on your test and learn from possible reasons why it didn’t work.

For MVT Tests, Do Follow-up Tests to Increase the Likelihood of the Best Version Being Found

Given the nature of how Taguchi multivariate tests determine a sample of sets of experiences it needs to show, multivariate tests aren’t always perfect at predicting a winning test version. As a result, they can be occasionally prone to not revealing the best test variations, meaning that you might not always be pushing a test version live that has the highest lift on conversion rates.

The best way to make sure of the validity of your test results and that you are pushing live the best winning version possible is to do these follow-up tests: