A/B Split Testing - Instapage Blog https://instapage.com/category/ab-split-testing/ Tue, 14 Jan 2025 10:50:39 +0000 en-US hourly 1 Multivariate Testing: How to Run the Best Tests for the Best Results https://instapage.com/blog/what-is-multivariate-testing/ Wed, 29 May 2024 19:00:00 +0000 https://instapage.com/?p=49594
Experiments are crucial to all marketing campaigns—they help you understand what works and what needs work. While A/B tests work perfectly when comparing a few variations of your chosen landing page element, the tests are limited when you need to test more than a single page element. For these occasions, your best choice is multivariateRead More >]]>

Experiments are crucial to all marketing campaigns—they help you understand what works and what needs work.

While A/B tests work perfectly when comparing a few variations of your chosen landing page element, the tests are limited when you need to test more than a single page element.
For these occasions, your best choice is multivariate testing.

Multivariate testing, or MVT, allows marketers to compare multiple variables and determine the one that will maximize conversions.

A multivariate test is one of the most valuable optimization approaches, as it allows you to check radical web page modifications and validate multiple hypotheses simultaneously. You can use MVT to optimize your landing pages and ads.

First, let’s find out the specifics of multivariate testing, its goals, benefits and drawbacks, and how it works.

Multivariate testing: Definition, goals, and benefits

Multivariate testing tests multiple variables simultaneously and understands how they interact to influence user behavior. These variables include headlines, images, web forms, CTA buttons and links, and their location on the page. Instead of evaluating several versions of a single element, multivariate hypothesis testing involves systematically modifying the selected variables and analyzing how different components work together.

Elements to test in multi variate testing:

  • Copy
  • Headlines
  • Forms
  • CTA placement

The goal of multivariate tests is to evaluate user engagement and identify the winning variation, i.e., the most practical combination of elements to optimize landing pages accordingly. When implemented correctly, this improves conversion rate optimization.

Key benefits of multivariate tests

  1. Precise identification of conversion-driving page elements. The multivariate hypothesis suggests that in order to boost user engagement, you need to make significant changes to several different parts of the webpage. This provides a data-driven roadmap for future design and content strategy adjustments.
  2. Elimination of long A/B testing processes. Evaluating all potential variable combinations simultaneously allows you to get the desired results faster, eliminating the need to test every element separately.
  3. Better analysis of user interactions. An MVT test lets you get the best insights into how different page elements perform and resonate with your audience.

Along with these benefits, multivariate testing has one potential limitation—the amount of traffic required. The more variable combinations you want tested, the more traffic you need to conduct tests.

For example, to test six different page variations, you need to split your audience into six relevant groups, which requires a larger sample size (the number of visitors for each page version).

How multivariate testing is different from A/B testing

While both A/B testing and MVT methodologies aim to enhance performance and user experience, they are rooted in unique principles and offer distinct advantages. A/B testing, or split testing, is a technique where two test variations (A and B) of a web page or marketing element are compared to determine which performs better to achieve a specific goal.

This process involves dividing the audience into two groups and exposing each group to one of the variations. Marketers can pinpoint the variant that delivers superior results by measuring key metrics like click-through rates or engagement levels and make data-driven decisions accordingly.

This is what an A/B test looks like:

This image demonstrates a visual example of what an A/B test looks like

Let’s say you need to optimize a product page layout for an e-commerce website. For an A/B test, you might develop two versions of the product page: Variant A featuring a single, prominent call-to-action button and Variant B featuring multiple smaller buttons scattered throughout the page. By analyzing user interactions with each variation, you can identify which layout is more likely to increase conversions and implement the winning design.

On the other hand, multivariate testing represents an advanced level of experimentation that entails testing multiple elements and their interactions simultaneously. Unlike A/B testing, a multivariate test empowers marketers to assess the effect of different combinations of elements within a single test. This methodology proves beneficial when optimizing complex web pages or marketing campaigns with numerous interactive components.

Here’s what the difference between multivariate testing and A/B testing looks like:

This image demonstrates the difference between A/B testing and multivariate testing methods

Let’s say you’re looking to enhance the booking process on your travel agency website. Instead of just testing two versions of the booking form, an MVT test allows you to explore multiple possible combinations of form fields, button placements, color schemes, and messaging all at once. By evaluating how these combinations impact conversion rates and other crucial metrics, the website can uncover the optimal setup to enrich the booking journey for users.

And now, let’s look at another example showcasing the difference between multivariate testing and A/B testing. An e-commerce company is selecting the best-performing CTA button for its website. In an A/B test, they will compare options like “Buy now” and “Order now.” The wording change represents a single variable adjustment. For an MVT test, they will need to compare the buttons “Buy now” and “Order Now” in different colors (e.g., green, light blue, and navy blue) and multiple locations on the page, then analyze how these combinations perform.

This image shows the difference between A/B testing and multivariate testing when applied to CTA buttons

To sum up, there are several aspects that show the difference between A/B and multivariate testing methods:

  1. Number of variables. A/B testing focuses on comparing two page versions (A and B) with typically one element changed, while MVT compares multiple variables at the same time. This complexity in MVT influences the overall performance within a complex number of combinations.
  2. Scope of changes. A/B testing compares different pages with significant changes. In contrast, multivariate testing focuses on subtle variations across multiple page elements to gauge their impact.
  3. Get more insights.A/B testing aids in understanding the appeal of major page formatting, whereas multivariate testing offers insights into specific page elements’ engagement levels by exposing audiences to various unique combinations.
  4. Traffic and resource requirements.An MVT test needs enough traffic to guarantee the statistical validity of the experiment. Thus, it’s a more resource-intensive approach, calling for advanced analytical tools. On the contrary, A/B testing can be executed with lower traffic volume and demands fewer resources.
  5. Testing approach.A/B testing, being more straightforward and requiring less traffic, is ideal for swift data collection and insights. On the other hand, multivariate testing is a more advanced process that analyzes how multiple variables interact, better suited for extensive projects that demand in-depth analysis and experimentation.

The pros and cons of MVT vs. A/B testing

Comparing Multivariate Testing (MVT) and A/B Testing reveals unique advantages and limitations. Understanding these is crucial for digital marketers and web-optimizers. It helps them make informed decisions based on factors like page complexity and traffic volume. Let’s match the key aspects of these experimentation techniques and delve into their strengths and weaknesses.

Aspect Multivariate testing A/B testing
Experiment complexity High Low
Testing speed Slower Faster
Identifying interaction effects Yes No
Resource intensiveness Higher Lower
Precision of results Lower Higher
Sample size requirements Larger Smaller
Insight into user behavior Comprehensive Limited
Ease of implementation Lower Higher
Risk of interference Higher Lower
Costs Higher Lower

Multivariate testing formula and how it works

You can determine the precise number of web page versions needed for a specific MVT test by applying this formula:

This image is a visual representation of a multivariate testing formula

For example, you need to test three versions of headlines and two main images. In this case, the total number of combinations tested will be:

This image is a visual representation of a multivariate testing formula when testing different headline and image combinations

If you need to run an experiment for two different CTA button colors, two CTA wording options, and three headlines, the calculations are as follows:

This image is a visual representation of a multivariate testing formula when testing different CTA button and headline combinations

How to run multivariate tests

There are different approaches to conducting experiments in multivariate hypothesis tests, such as the full factorial, fractional, and Taguchi methods (though the last one is the least popular in online testing).

The full factorial test is considered the most accurate as it checks all variable combinations created based on the described formula. On the other hand, the fractional test checks only the most impactful variable combinations.

What can you test: Multivariate testing examples

Testing specific elements

With multivariate testing, you can test precise details like headlines, images, button colors, and call-to-action texts. Even variables such as font sizes, image captions, or the positioning of a subscription form can be scrutinized. The objective is to assess not only the performance of individual elements but also how the combinations of changes across elements work together to enhance the user experience and conversion rates.

AliveCor wanted to launch its new product, KardiaMobile Card, on their website without hurting the sales of their other products. They ran an A/B test using this hypothesis: website visitors tend to interact more frequently with highlighted elements and products. They added a “New” badge on the new product’s detail page and the product title from the listing page to check if they will see a conversion rate increase across desktop and mobile devices.

Control version:

This image represents the control version of KardiaMobile Card product page interface

Variation:

This image represents KardiaMobile Card testing variation of a product page interface with smaller badge

This image represents KardiaMobile Card testing variation of a product page interface with a bigger badge

The version of the product page featuring the “New” badge showed a significant increase in conversion rate (+25.17%) and revenue per user (+29.58%) across both desktop and mobile devices. AliveCor chose to keep the “New” badge on the website during the months that followed their new product launch.

Testing complete pages

Beyond individual elements, MVT can be applied to entire pages. This can involve testing different layouts, overall color schemes, or the arrangement of sections within a page. By analyzing how these more extensive changes impact user behavior, businesses can refine their website design with data-driven precision.

Groove, a customer support management platform, used testing to improve their landing pages conversion rate from 2.3% to 4.3% by completely revamping its layout to make it more “copy-first”. As a part of their experiment, they tested different versions of headlines and narratives, aiming to “lead with benefits, not features.”

This image shows the results of multivariate testing performed on Groove blog with focus on long-form content

Optimize your landing page performance with Instapage

Despite all its benefits, multivariate testing might not be the best option for marketers if the test group is too small to get reliable results, or if the variables they’re testing are too connected, which could mess up the findings.

If any of the above factors is the case, A/B testing will be more suitable for refining your existing pages.
Instapage’s A/B testing feature can help you determine if changing one or multiple elements on your landing page can increase conversions.

1. To create a new experiment, go to the Optimize tab and click on Experiment on the right sidebar and then on Create Experiment.

This image shows how to create an experiment using Instapage A/B testing functionality

2. Name your experiment and type in the hypothesis you’d like to test.

This image shows how to start an A/B test by creating an experiment in Instapage

3. Select the landing page experience that you will be testing, starting with the group the page is in, the name of the page and then the name of the experience, then click Create.

4. Your existing variations will automatically appear in the list. You can choose to add a new variation, edit the variations in the builder, and set the split.

This image shows how to name your experiment in Instapage

5. When all the changes are made, you can start the experiment.

Learn how to set up and run A/B testing within Instapage. By mastering A/B testing, you’ll gain valuable insights to enhance your landing page performance.

Instapage will empower you with the best optimization tools to help you elevate your landing pages. Sign up for a 14-day free trial with Instapage or request a demo today.

blog CTA enterprise demo illustration

See the Instapage Enterprise Plan in Action.

Demo includes AdMap™, Personalization, AMP,

Global Blocks, heatmaps & more.

]]>
Want to Build a Community? Start an Email Newsletter https://instapage.com/blog/email-newsletter/ Wed, 08 May 2024 13:39:53 +0000 https://instapage.com/?p=194194
The main goal of your landing page is to drive conversions—ideally, you are trying to convert prospects into customers. However, not everyone who visits your landing page or website is ready to convert. Many potential subscribers may not be ready to buy what you are selling, but want to stay engaged and connected with youRead More >]]>

The main goal of your landing page is to drive conversions—ideally, you are trying to convert prospects into customers. However, not everyone who visits your landing page or website is ready to convert.

Many potential subscribers may not be ready to buy what you are selling, but want to stay engaged and connected with you before they make the definitive decision of converting. They may be willing to give you their email address in exchange for time, thus qualifying them as leads.

A lead is just a future conversion that hasn’t happened yet. To ensure that the lead does eventually convert, you’ll need to nurture those leads with an email newsletter.

So, what does a successful newsletter look like?

In this article, we’ll discuss what an email newsletter is, how email marketing works, and best practices for creating effective email newsletters.

What is an email newsletter?

An email newsletter is a regularly scheduled email sent to newsletter subscribers who have opted in (via a signup form) to receive communication from your brand, including updates, news, stories, articles, offers, and more valuable information.

The purpose of email newsletters is to keep subscribers informed and engaged, with the general goal of building a relationship and cultivating a community with your target audience.

It’s not just an email. Newsletters are an essential part of email marketing strategies, as they provide a way to promote products or services, create brand awareness, and engage with your community beyond your website and social media platforms.

For example, The Washington Post uses its daily newsletter, The Post Most, to showcase trending website articles that other readers are loving.

By offering its email subscribers a sampling of the type of content they can expect to find on the publication’s website, they hope to entice readers to become paid subscribers.

This image shows a screenshot of The Washington Post newsletter, The Post Most, that showcases trending website articles that other readers are loving.

How do email marketing campaigns help nurture audiences?

Regularly sending communications to your email subscribers (without flooding their inboxes) helps keep your brand top of mind and gives you the chance to position yourself as a thought leader in your space.

Newsletter content is not solely about selling to your audience—it’s about helping your readers understand the voice of your brand. Newsletters (starting with your subject lines!) are a place to have discussions with subscribers, asking questions, telling stories, and helping them relate to your brand to build a loyal following that eventually leads to a conversion or sale.

For creators, newsletters are appealing because they typically deliver a high ROAS. They are cheap to make and send and tend to pay off, reportedly seeing an estimated ROI of $40 for every $1 spent.

Know what to say: Define your email newsletter goals and guidelines

As with any marketing strategy, email marketing requires thought behind what each email newsletter will entail. To maintain consistency with each email newsletter you’ll send, you must define your goals and guidelines.

Before you create an email, ask yourself questions like:

  • Where will my email signup forms live?
  • What do I want to accomplish with this newsletter? Will this newsletter be another avenue for monetization?
  • When do I want to send my newsletter?
  • What do I want my email frequency to be?
  • How can I use my subject lines to hook newsletter subscribers from the start?
  • What will the tone of my newsletter be?
  • Will my newsletter feature curated content or original content?
  • What is the unique value proposition I can offer in my newsletter versus others in my industry?
  • How will I segment audiences?
  • Do I want to use my newsletter to increase my website traffic?
  • What key performance indicators will I use to measure the success of my email marketing campaign?

The most successful email newsletters create a lasting impression with attention-grabbing, thought-provoking, and interesting (and quick) to-read content. They are not too sales-driven, impersonal, or corporate. And their subject lines succeed in setting expectations.

When you figure out what resonates with your leads and how you want to portray your brand through your email messaging, you can use that as a foundation or template to build your newsletter time after time.

Once you start consistently sending out your email newsletter, you can analyze your campaign metrics and understand where iterations can be made – perhaps your monthly cadence is not enough and a weekly newsletter will perform better. Maybe an earlier- or later-in-the-day send time will result in a better click through rate. Perhaps your subject lines are too long and are misleading your audience.

Let’s dive deeper into your email marketing strategy.

How to develop an email marketing strategy for a newsletter

After identifying your goals and your KPIs, here’s what you need to do to develop a comprehensive email newsletter strategy:

  • Choose an email management system. The right email marketing tools can make a big difference in the success of your campaigns. Email marketing software can run the gamut from informal to sophisticated, depending on your budget and your needs (if you’re just getting started, a free version of email marketing tools might be worth trying). An email management system will ideally enable you to plan and organize your content (similar to an editorial calendar), deploy your content to your distribution lists, and give you insights into the performance of your email campaign.

    Think you can get by without an email service provider? Before you decide that this is something you’d rather take on on your own, understand that email marketing software makes it more likely that your newsletter will actually end up in your subscribers’ inboxes instead of in spam and gives you a reliable way to be consistent with your sends. An email service provider can offer peace of mind, especially if you have a large number of subscribers on your lists.

  • Provide value every time. Think of how many emails we all receive on a daily basis. Our inboxes are saturated. If people are taking their time to complete signup forms, you want to ensure your newsletter content is worth their time and will help them somehow.

    Because you are ultimately trying to drive conversions, you should create an email that includes call-to-action buttons to nurture your lead further down your funnel. We recommend connecting all email offers with a dedicated landing page that has cohesive messaging and allows your readers to get what they need quickly and effectively.

This Salesforce Marketing newsletter aims to educate readers on how to grow customer journeys, with more information available in a downloadable guide.

This image shows a screenshot of Salesforce newsletter promoting the customer journey marketing guide

Clicking the email newsletter CTA for the guide brings the reader to a dedicated landing page where they see the same messaging about those customer relationships and can easily find the CTA button to download the guide. Plus, additional related CTAs give readers even more opportunities to convert.

This image demonstrates how clicking the Salesforce email newsletter CTA for the guide brings the reader to a dedicated landing page

  • Segment your audience. Email segmentation is important because you want your readers to see the right message at the right time. When you segment your audience, you can create an email that is more targeted and increase your potential ROAS.

    Various factors can affect how your email will perform and you can use these factors to build your distribution lists. Segmentation factors include:

    Demographics
    Geographic location
    – Consumer behavior
    Buyer personas
    Stage of the funnel/customer journey
    – B2B or B2C

  • Run A/B tests. Think of your newsletter content, especially in the beginning stages, as an experiment. Iterations are key. To figure out what is working and what can be improved, keep every detail in mind.

    Did you use the most engaging subject line? Are you showcasing your most dynamic content? Are your readers using both desktop and mobile devices? Trying out a different subject line, different layout, or bolder design, making sure to optimize for mobile devices, and testing out different high quality content are all ways to experiment and figure out what resonates most with your target audience.

    Using an email marketing tool that allows you to A/B test will give you the deep insights you need to maximize performance of your campaigns.

  • Track your metrics.Once you create a newsletter, it’s time to get down to business. Tracking your KPIs and making changes to your email newsletter content based on data will help you create highly effective campaigns.

    Specifically, metrics like open rate, click through rates, number of clicks, number of subscribers, completed sign up forms, and more will give you an idea of what your readers are finding most engaging.

10 email marketing newsletter examples from DTC and SaaS brands

Here are 10 email newsletters that provide valuable content to their subscriber list:

1. Welcome email newsletter

Food52 is a website that curates recipes, home goods, and crafts for anyone who likes to be in the kitchen. When new subscribers sign up, they are greeted with a welcome email that outlines what they can expect from Food52 and offers a discount code to use in their shop.

This email newsletter does a good job of immediately making the reader feel like they are part of the community and enticing them to convert into a paying customer.

This image shows a screenshot of Food 52 welcome email newsletter

2. Speaking to the right subscriber list

In this example of an email newsletter from Buffer, the copy is clearly written for a specific segment of their target market. Using slang like ‘stan’ and a pop culture reference to Zendaya make it clear that the audience for this newsletter skews younger and shows that Buffer creators have done their due diligence to create a newsletter that resonates.

We also like that they include a breakdown of what to expect in that day’s newsletter.

This image shows a screenshot of Buffer email newsletter targeted at a specific audience

3. Thought-leadership content

SEO experts at Moz understand that despite the fact that people are so busy, they need to stay in the know. Especially when it comes to something as ever-changing as optimal SEO practices.

They created their semi-monthly email newsletter called Moz Top 10 to share the 10 most valuable articles about SEO and online marketing that they source from around the web. By offering high-quality thought-leadership content, they hope to gain the trust of their readers and convert them into loyal customers.

This image is a screenshot of MOZ thought-leadership email newsletter

4. Mobile-friendly news

A lot of people wake up, grab their phone, and start reading emails before they’ve even had their morning cup of coffee.
Marketing Brew wanted to create an email newsletter that capitalizes on that by serving industry news and updates on a daily basis. The fact that their designs are easily viewable on mobile devices makes it highly likely that readers will commit to that email newsletter every day, regardless of where they are.

This image shows a screenshot of a mobile-friendly Marketing Brew email newsletter

5. Blog round-up

Want to get more eyes on that well-crafted blog post you wrote? A weekly newsletter rounding up recently published articles from your website is a good way to ensure that your readers don’t miss any valuable content.

Grammarly employs this tactic with their email newsletter, which also includes writing tips, grammar facts, and more. And, from their very first welcome email, they make sure to stick to their brand ethos mixing grammar lessons with a good dose of humor.

This is a screenshot of Grammarly email newsletter

6. Daily content

Not every email newsletter will benefit from having a daily cadence. For many companies, that volume of email sends can backfire, feeling too spammy for readers and getting too overwhelming to manage for creators.

The Skimm gets it right. They provide a highly popular daily newsletter that curates top news stories of interest to readers. It is an effective email campaign that educates people about what’s going on in the world while building a reputation as a credible, reliable source for daily news. Plus, it’s equally skimmable on desktop or mobile devices.

This image is a screenshot of theSkimm email newsletter with a top news roundup

7. Leading with visuals

The marketers over at The New York Times know what they’re doing. Whether it’s delivering news, introducing fun ways to exercise our brains, or helping us decide what to make for dinner, they know how to attract new subscribers with one successful email campaign after another.

With The New York Times Cooking newsletter, they let images do the talking. Large, vibrant photos of drool-worthy dishes are accompanied by recipes and anecdotes that leave readers eager for more.

This image demonstrates the use of attractive imagery in the New York Times cooking email newsletter

8. The reader chooses the cadence

People turn to TechCrunch as a dependable source for important tech news. And they have plenty of news to share. Knowing that, they offer 10 newsletters to opt into, some with a daily cadence, some with a weekly cadence, and all tailored to a specific topic (think fintech, crypto, or morning updates).

This puts readers in control of the content they will receive and makes the experience feel more personalized, which is a smart way for the publication to communicate their general approach to interacting with their subscribers.

This image is a screenshot of the Daily Crunch, an email newsletter by Tech Crunch

9. Mixing business with pleasure

Bloomscape is a direct-to-consumer (DTC) brand that uses their newsletter to support their ecommerce business. They send plants directly to their customers’ doorsteps and use their newsletter as an opportunity to educate them on how to become better plant owners while also introducing new and available products that are for sale.

Their email campaign is done in a visually appealing and easy-to-digest way, keeping potential and existing customers coming back for more.

This image is a screenshot of Bloomscape email newsletter

10. Bottom-of-the-funnel content

General Assembly is a company that offers next-level tech training for careers in software engineering, UX design, and more. Their email campaigns move engaged subscribers to the last stage of the funnel, where they can purchase a seat to an upcoming event.

In addition to offering industry expertise and educational content, they lay out titles, dates, and times for upcoming events and make it simple for readers to register for them. This strategy works well for subscribers who are consistently engaged and interact with your content regularly.

This image is a screenshot of a General Assembly email newsletter

Dedicated landing pages that maximize your newsletter campaigns

As you contemplate your own email newsletter, remember to take your time. There are a lot of factors that play into whether someone will fill out your email subscription form and take the first step to becoming a loyal customer. Whether they come from your social media accounts, website, or paid ads, your goal is to keep them in your ecosystem by serving valuable information.

You may aim to focus on a specific niche, get existing content out to more eyeballs, or increase your ecommerce business, and choosing the right tools can help you meet your goals.

It’s good to remember that a successful newsletter often has dedicated landing pages as part of the strategy, as landing pages reinforce your message and can be that extra push in driving a conversion.

Instapage is the most powerful landing page builder on the market, with high-valued features like a drag and drop editor, pre-existing templates, an analytics dashboard, an AI content generator, and in-app A/B testing.

Instapage users love the ability to conduct A/B testing with Instapage Experiments. It is an intuitive feature that allows users to test variations of their landing page elements to understand what resonates best with their visitors. With Instapage Experiments, users can:

  • Access heatmaps that provide a visual image of scroll depths, mouse movement, and on-page clicks. Using heatmaps, you can track your visitors’ behavior and improve element placement on your landing page accordingly
  • This image shows a screenshot of Instapage heat maps functionality

  • Rewrite headlines and page copy using an AI generator based on the existing text, target audience, product description, and recommendations from API
  • Compare elements like on-page videos, images, or text to see which is most engaging
  • View deep A/B analytics in an easy-to-use dashboard

See more successful campaigns when you pair your emails with Instapage landing pages. Start a 14-day free trial of Instapage today.

blog CTA enterprise demo illustration

See the Instapage Enterprise Plan in Action.

Demo includes AdMap™, Personalization, AMP,

Global Blocks, heatmaps & more.

]]>
How to Move Forward After Google Optimize Sunsets https://instapage.com/blog/google-optimize-sunsetting/ Tue, 06 Jun 2023 16:00:12 +0000 https://instapage.com/?p=186919
Recently, the digital landscape has been shaken by the news of Google Optimize’s departure in 2023, marking the end of an era for this popular A/B/n testing platform. As website owners and marketers, it’s natural to wonder about the implications and what lies ahead for your optimization journey. Fortunately, the departure of Google Optimize doesn’tRead More >]]>

Recently, the digital landscape has been shaken by the news of Google Optimize’s departure in 2023, marking the end of an era for this popular A/B/n testing platform. As website owners and marketers, it’s natural to wonder about the implications and what lies ahead for your optimization journey.

Fortunately, the departure of Google Optimize doesn’t mean the end of your ability to conduct effective A/B/n testing and drive conversions. In fact, it presents an opportunity to reassess your strategy, explore alternative solutions, and propel your optimization efforts to new heights. There are plenty of Google Optimize alternatives and strategies to test.

In this blog post, we’ll delve into why having an A/B/n testing strategy is essential for maximizing your website’s performance. We’ll also provide you with a step-by-step guide on how to set up your first A/B/n test with best practices.

What is Google Optimize?

What exactly is Google Optimize? Google Optimize is a web optimization tool offered by Google that allows users to test and optimize their websites for better performance and user experience. With Google Optimize, marketers can create different variations of their website’s content, such as headlines, images, buttons, or layouts, and test them against each other to determine which variation performs better in terms of conversions, engagement, or other defined metrics. This data-driven approach helps marketers make informed decisions about website design and content to improve user satisfaction and achieve their business goals.

But unfortunately, the tool will be sunsetting on September 30th, 2023.

So, if Google Optimize has been your main tool for experimentation, what can you do now?

Why you need an A/B/n testing strategy

With the sunsetting of Google Optimize, it becomes even more important to create and implement an A/B/n testing strategy for your website. A/B/n testing is a crucial component of any successful website optimization plan. It allows you to make data-driven decisions and maximize the impact of changes made. Here are a few key reasons on why you need an A/B/n testing strategy:

Reduce Guesswork: Without testing, your marketing decisions will rely on assumptions and guesswork. By testing different variations, you can gather concrete data on how each element impacts user behavior and preferences. This data-driven approach eliminates guesswork and provides valuable insights into what truly resonates with your audience.

Improve User Experience: A/B/n testing allows you to experiment with different design elements, layouts, and user flows to identify the combination that provides the best user experience. By testing and refining these elements, you can create a seamless and engaging experience for your visitors–leading to increased satisfaction, longer visit durations, and higher chances of conversion.

Establish a Cycle of Continuous Improvement: A/B/n testing is an ongoing process that allows you to continuously refine and optimize your website. It empowers you to make incremental changes based on real-time data, rather than relying on one-time redesigns. By embracing a culture of continuous improvement through testing, you can stay ahead of the competition and consistently enhance your website’s performance.

How to Set Up an A/B/n Test

Step 1: Define Your Objectives & Variables

Start by clearly defining the objective of your A/B/n test. What specific aspect of your website or conversion funnel do you want to improve? Establishing a clear goal will help you focus your efforts and measure the impact of the test accurately. Then, determine the specific elements you want to test in your A/B/n experiment. This could include headlines, call-to-action buttons, etc.

Step 2: Formulate Hypotheses 

Develop hypotheses about how each variation of your test variables might impact user behavior or performance. These hypotheses will serve as the basis for your predictions and help you interpret the results of your test.

Step 3: Create Test Variations & Allocate Traffic

Using your A/B/n testing tool or platform, create variations of your test elements. Ensure that each variation differs from others in a single, isolated way, allowing you to accurately attribute any changes in performance to the specific element being tested. Then, decide on the traffic allocation for each variation in your A/B/n test. The allocation can be evenly distributed among all variations.

Step 4: Launch and Monitor the Test

Launch your A/B/n test and monitor its progress. Regularly review the test results and check for any technical issues or anomalies that may affect the accuracy or integrity of the experiment. Allow sufficient time for the test to run to collect a meaningful amount of data.

Step 5: Analyze and Interpret Results

Once your test has reached statistical significance or has run for a predetermined duration, analyze the results. Compare the performance of each variation against the defined objective and evaluate the statistical significance of any observed differences. Interpret the results in light of your hypotheses and draw actionable insights.

Step 6: Implement Your Winning Variation

If one variation outperforms the others significantly, implement the winning variation on your website. Apply the insights gained from the test to make data-driven decisions and improvements. Tip: If no clear winner emerges, consider iterating on the test or conducting further experiments to gain additional insights.

A/B/n Testing Best Practices 

Now that we know how to set up an A/B/n test, let’s turn our attention to the CRO-focused tips and tricks that will take your A/B/n testing abilities to the next level. 

Think big, start small: Don’t try to fix everything all at once. Instead, make small changes that lead to learning, then iterate on those learnings. Building more tests based on your findings.

Ensure you have enough traffic: Use a large enough sample size to ensure statistical significance. Tests on low-traffic pages take longer to determine a winner, and they won’t provide accurate learnings or insight into the experience of the majority of your users. 

Create a prioritized roadmap for each page you plan to test: Determine the boldest and most impactful elements of the page to gain your initial learnings, then prioritize additional tests based on their ease of implementation and potential impact on your primary metric. Your testing roadmap should include the hypothesis, test idea, goals, desired learnings, and specific launch dates to keep your testing efforts on track.

Use heat maps and other user behavior analysis tools to understand how visitors are interacting with the landing page: Testing what you put above and below the fold can have a huge impact on conversion rates, especially with certain audiences and micro-audiences, so understanding what elements people are exploring and what they’re ignoring is key to creating a fully optimized landing page.

  • Here’s a hint, if you find people are consistently scrolling past the fold and not clicking above, it’s a major indicator that you’re not providing enough information above the fold.

Now you have a deeper understanding of the importance of A/B/n testing and how it can optimize your website’s performance, even in the wake of Google Optimize’s departure. Remember, while the end of an era may bring some uncertainty, it also brings the opportunity for growth and exploration! As you embark on your A/B/n testing journey, refer back to the insights we’ve shared about the structure of an A/B/n test, the value of testing strategies, setting up tests, and best practices.

Remember–optimization is a continuous process–and with the right mindset and tools at your disposal, you’ll have the ability to uncover remarkable insights and drive significant improvements. If you’re interested in trying out Instapage as your Google Optimize alternative, sign up for a 14-day trial.

Want to dive deeper with advanced analytics and heatmaps? Or are you interested in migrating your existing landing pages, and benefiting from a full conversion rate optimization analysis? Instapage Convert may be the right solution for you. Schedule a consultation today

blog CTA enterprise demo illustration

See the Instapage Enterprise Plan in Action.

Demo includes AdMap™, Personalization, AMP,

Global Blocks, heatmaps & more.

]]>
Optimize Your Campaigns With Instapage Experiments https://instapage.com/blog/google-optimize-alternative/ Tue, 11 Apr 2023 19:00:50 +0000 https://instapage.com/?p=184138
Google Optimize is shutting down It’s the end of an era. Google announced that Google Optimize will be shut down in September 2023. Google Optimize uses Google Analytics data to show you which version is performing best, so that you can optimize your site for conversions and increase traffic. Marketers have relied on this toolRead More >]]>

Google Optimize is shutting down

It’s the end of an era. Google announced that Google Optimize will be shut down in September 2023. Google Optimize uses Google Analytics data to show you which version is performing best, so that you can optimize your site for conversions and increase traffic. Marketers have relied on this tool to run experiments and improve the results of their Google Ads campaigns since it was launched in 2012. The good news for marketers who’ve relied on Google Optimize is that straightforward alternatives can help you accomplish the same goals.

For Instapage customers, the Google Optimize sunset is no big deal. Instapage offers many of the same features as Google Optimize (and some that Optimize lacks). Instapage can help marketers perform experiments like A/B/n tests and access in-depth analytics features like heatmaps for an all-in-one landing page experience.

What is landing page optimization?

Landing pages are the pages that you send your traffic to when they click on an ad. Since landing pages are so important, it’s worth taking the time to optimize them for maximum conversion rates.

When you optimize a landing page, you’re trying to improve its performance in terms of getting people who visit it (or click on it) interested enough to convert into leads or customers. This could mean generating more sales, qualifying leads better by having more people fill out forms on your website, and collecting information so you can reach out later with follow-up offers.

Optimization is a necessary part of any digital campaign. And Instapage is designed to help advertisers easily build and optimize pages for the best campaigns possible. Let’s explore how Instapage can replace or even exceed some standard Google Optimize features.

A/B testing

A/B testing is a way to compare two versions of a web page and determine which one performs better. A/B testing is a simple, fast and effective way to improve your website and increase conversions.

You can use Instapage to create multiple variations of the same page or experiment with different pages altogether. You can also run an A/B test on any element of your landing page (including images, text or videos).

And now, with Instapage AI Content Generation, you can build A/B tests faster and easier than ever before. Simply select the text element, like headline or button copy, that you’d like to test, and hit the “Rewrite with AI” prompt. You’ll receive several AI-generated variations on your copy that you can test against the original.

A/B testing is one of the most efficient ways to tailor a campaign’s landing page to a specific audience. But A/B tests focus on just one element at a time. The best optimization platforms let you take it a step further with A/B/n testing. This format allows marketers to create multiple variants and test them simultaneously. This form of split testing enables faster and deeper insights.

Instapage customers can easily build landing pages and conduct A/B/n tests all in one place—without the fuss of third-party tools.

Heatmaps

A/B and multivariate tests give you hard data on which version of a landing page performs best. But sometimes, if you want to understand what a visitor experiences on your landing page, you need to dig a little deeper.

That’s where heatmaps come in. Instapage Convert customers can access in-depth analytics of visitor behavior on their pages. Heatmaps provide a visualization of page activity, including:

  • Scroll depth
  • Page clicks
  • Mouse movement

Using heatmaps, you can identify the most important elements on your landing page, which can help you set up future tests for maximum impact. This feature is something Instapage has that Google Optimize doesn’t, making it your ideal alternative.

Conclusion

We know these tools will help you optimize your campaigns and ensure they reach their full potential. If you’re interested in trying Instapage as your Google Optimize alternative, sign up for a 14-page trial.

Want to dive deeper with advanced analytics and heatmaps? Interested in migrating your existing landing pages and benefiting from a full conversion rate optimization analysis? Instapage Convert may be the right solution for you. Schedule a consultation today.

blog CTA enterprise demo illustration

See the Instapage Enterprise Plan in Action.

Demo includes AdMap™, Personalization, AMP,

Global Blocks, heatmaps & more.

]]>
How to Maximize ROAS with Social Ad A/B Testing https://instapage.com/blog/how-to-maximize-roas-with-social-ad-a-b-testing/ Thu, 23 Feb 2023 17:00:24 +0000 https://instapage.com/?p=180562
It can be challenging to stand out and reach your target audience effectively. With ever-increasing competition, marketers are facing the task of using their limited budgets in a strategic manner to yield the best results. As we enter 2023, it’s crucial for marketers to ensure that their digital ad spend is optimized to drive strongRead More >]]>

It can be challenging to stand out and reach your target audience effectively. With ever-increasing competition, marketers are facing the task of using their limited budgets in a strategic manner to yield the best results. As we enter 2023, it’s crucial for marketers to ensure that their digital ad spend is optimized to drive strong campaigns and deliver maximum impact. One way to do this is with social ad A/B testing.

How to Benchmark your Campaign Success 

Maximizing your return on ad spend (ROAS) is a top priority for any marketer, and understanding how to accurately calculate the metric is crucial. Before diving into how to improve your ROAS, let’s first review how to measure ROAS and how it will be referenced throughout our discussion.

To calculate ROAS, take the total campaign revenue and divide it by the total campaign cost. 

 

For example, if you ran a digital ad campaign that you spent $5,000 on and you generated $15,000 in revenue into your conversion funnel, you would end up having a ROAS of 3 to 1.

As a note, in today’s blog post, you will see several B2C references and examples. But, everything we’re talking about today is relevant to B2B campaigns as well!

Optimizing your Ad-to-Page Journey 

Increasing your return on ad spend involves creating messaging that resonates with your target audience, so they engage with it and eventually convert. To ensure your messaging is relevant to your audience it’s important that the elements of your marketing funnel are optimized for each audience segment and that you perform A/B testing across the funnel.

Before we dive into optimization practices, let’s first understand the basic ad-to-page journey your potential customers will take. 

Featured below is an oversimplified, birds-eye-view of what the ad-to-customer journey looks like.

an image showcasing how social ad A/B testing effects your ad-to-customer journey

The conversion process begins when your audience first sees your ad. The exposure of your target audience to an ad, followed by a landing page. For a landing page to be high-converting all the elements need to highlight the offer–keeping in mind the audience’s needs. 

There should also be a perfect message match between your ad and your landing page and whatever else comes after it (welcome email sequence, nurture sequence, etc). 

Optimizing your ad-to-page experiences also involves A/B testing—including social ad tests and page tests to ensure you’re not paying high CPC (cost per click) on leads that aren’t converting on your page. 

Let’s analyze an ad-to-page example for message matching and see if there’s room for improvement.  

The Sasquatch Boots Social Ad A/B Testing

This Sasquatch Boots ad offers 10% off hiking boots. The ad copy includes social proof from world-class hiker Anish Anderson, and the creative features an image of the boots. 

an image showcasing how Sasquatch Boots uses social ad A/B testing

The landing page features the full quote from the hiker, ensuring the headline and sub-headline highlight the features of the boots from the point of view of the customer.

However, if you isolate the copy, it doesn’t really explain much about the product except for highlighting the comfort angle. There’s no mention of durability, how it performs against other natural elements, etc. 

This page also doesn’t feature the 10% off offer that brought prospects from the ad to the page. The missing discount offer also takes away any sense of urgency from the page. 

Let’s see what happens when you add “sale” messaging on the page. The page now also includes a sense of urgency with the addition of the CTA “Shop Sale Now.”

 

However, we can still tweak this landing page by showing prospects exactly how much they will save. It’s always better to save your potential customers the hassle of doing the math themselves. 

Here’s what the landing page would look like if you incorporate the 10% discount copy, while also highlighting the dollar value of the discount. 

As the real cherry on top, it also brings in the exact hiking boots from the ad. As a bonus: the landing page includes additional optimizations such as showing you the product’s star rating and adding in a more relevant CTA, “find my size.”

Tips on how to optimize your ad-to-page journey for your next campaign

Now, let’s dig into the tactics you can (and should) adopt to maximize your ad spend investment. Applying these five tips to your next digital campaign could help you improve your ROAS through social ad A/B testing. 

Be consistent with your messaging

Running A/B tests on Facebook Ads, for example, helps you analyze what creatives and copy are most relevant for the audience, making your messaging more impactful. 

Diagnose the right opportunity

In order to run a successful campaign, you as a marketer need to diagnose the right opportunity. How can you do that?

Just as cost per click or CPC is an input metric into return on ad spend, meaning that any changes to CPC will directly impact ROAS – CPC also has its own input metrics – clickthrough rate and conversion rate.

Click-through rate (CTR): The percentage of impressions resulting in a click. If CTR is lower than expected, there are usually two things that could be improved:

  • Quality of your ad creative. Does it capture someone’s attention enough to warrant a click?
  • Accuracy of your audience targeting. Are you targeting the right people who find the messaging relevant enough to click through? If not, rethink your audience. If you’re targeting groups based on interest or demographic categories, consider contextual targeting or retargeting groups who have interacted with your site as well.

Secondly, Conversion rate: The percentage of visitors who took a desired action (e.g., purchased a product, signed up for a newsletter, etc.) out of the total number of people who came to your website. Weakness in conversion rate diagnoses problems with

  • Does the layout and quality of your landing page match what the visitor was expecting from the ad that they clicked? Here is where consistency of messaging is very important
  • And again, the accuracy of audience targeting. Especially for B2B companies, this can mean that you’re targeting the right types of companies but perhaps not the right decision-makers.

Make adjustments to your audience targeting, ad creative design, messaging, and your landing page, and see how your cost per click, and by association your ROAS changes as a result. 

Provide a personalized ad-to-page journey

As we shared above, showing the same product in an ad as on the corresponding landing page is best practice. Dynamic ad and landing page templates can be further personalized to show the products that a particular visitor was looking at from a prior visit, as well as products that are related to the ones that your visitor was looking at. 

When you personalize the user experience to individual preferences through social ad A/B testing, you can create a more engaging and relevant journey that is more likely to result in conversions. 

By demonstrating that you understand and value your customers’ needs and preferences, you can foster a stronger connection and increase brand loyalty. And by offering a personalized ad-to-page journey, you can differentiate your brand from competitors and stand out in a crowded market.  

Increase your speed to conversion with social ad A/B testing

It’s all about increasing your speed to lead here. And making the most of the limited attention you will get from your page visitors. So employ tactics like:

  • Testing on-page promotions 
  • Heat map data
  • A/B Testing 
  • Focus on scarcity (ticking clock, deal end dates)
  • Offering upsells or cross-sells 
  • Highlight social proof

Optimizing your ad-to-page journey is critical to maximizing your return on ad spend. By focusing on the key components of an effective ad campaign, you can create an engaging experience for your audience and drive stronger results for your business. Remember, every touchpoint in the ad-to-page journey has the potential to impact your bottom line, so take the time to make sure each element is working together to deliver the best possible experience for your audience.

Now that you know what you need to optimize your digital campaigns, Instapage makes it easy to create personalized, relevant landing pages for every ad group and audience with hundreds of templates and easy in-app testing. Sign up for a 14-day trial and start improving your ROAS with social ad A/B testing today!

blog CTA enterprise demo illustration

See the Instapage Enterprise Plan in Action.

Demo includes AdMap™, Personalization, AMP,

Global Blocks, heatmaps & more.

]]>
Why Marketers Need Native A/B Testing https://instapage.com/blog/why-marketers-need-ab-testing/ Tue, 31 May 2022 16:00:18 +0000 https://instapage.com/?p=171538
For marketers, conversion rate is a crucial metric of success, but it sometimes requires extensive experimentation to optimize. In pursuit of conversion rate optimization, nearly two-thirds of companies employ A/B testing to figure out which messaging works best for their landing pages. Traditionally, you create two nearly identical versions of the campaign and send themRead More >]]>

For marketers, conversion rate is a crucial metric of success, but it sometimes requires extensive experimentation to optimize. In pursuit of conversion rate optimization, nearly two-thirds of companies employ A/B testing to figure out which messaging works best for their landing pages.

Traditionally, you create two nearly identical versions of the campaign and send them both out to a small sample of your audience. After the testing period, you look at the data to see which version performed better. It seems straightforward, right? Unfortunately, manual or external A/B testing can be far from simple—or accurate.

In one survey of marketers, 70% of respondents said their A/B testing was successful only half the time. Marketers need native A/B testing because it allows them to build, learn, and improve in a continuous cycle. With native A/B testing, they can rely on Instapage’s features that enable that cycle, which makes it easy to create test variants. Instapage’s tools are much more efficient than doing all this legwork in an internal CMS. Simplified A/B testing is even more valuable for businesses with multiple audiences. Native A/B testing is the best way to optimize each landing page quickly and efficiently in the same place you build.

Are you using the right A/B test strategy?

A/B testing is a crucial part of any effective digital marketing campaign, but comprehensive analytics tools can make or break a successful A/B test. These tools enable A/B testing that is more sophisticated than standard methods.

  • Multivariate testing: This type of A/B test is more complex and involves simultaneously testing multiple elements on a single page. This approach helps redesign pages to have significant CRO, but often requires large amounts of traffic to accurately test all variables.
  • Server-side testing: This more technical type of A/B test involves testing changes to the server-side code rather than the client-side code. This method allows companies to test changes that are more complex than the user interfaces without affecting performance.
  • Native A/B testing: This type of A/B test allows you to test changes to your landing page all in one place, connecting your CRO efforts to your landing page creation process.

So, why should marketers use native A/B testing when running an experiment? Building your landing pages can be difficult and time-consuming, especially if you don’t have the right tools. That’s where Instapage comes in. With our powerful landing page builder and built-in CRO tools, you can create high-converting pages without leaving our platform.

Consider these additional valuable reasons to try native A/B testing.

  • Efficiency: With native A/B testing, there’s no need to go through an external system or do it manually—you can test changes directly to your landing page, saving time and effort.
  • Accuracy: Native A/B testing provides more accurate results than other types of A/B testing, since a comprehensive dashboard can easily allow you to track your pages’ performance.
  • Flexibility: Native A/B testing allows you to test a wide range of changes, from minor tweaks to major redesigns.

Improve your testing with Instapage

A/B testing is a great way to optimize your conversion rate and increase ROI, but it can be tedious and challenging to manage if you are using an external system. At Instapage, we’ve seen what it takes to achieve success in this space, and we make it easy for customers to follow the path we’ve created.

Our tools let you connect CRO efforts to your landing page creation process and test changes to your landing page all in one place. Our years of expertise in results-focused advertising give us a firm grasp of our customers’ needs, and the tools and strategies we’ve developed give our clients an intuitive and flexible solution to build on for future success.

Instapage’s built-in features make native A/B testing easy and convenient by allowing you to make changes, see results and maximize conversions without compromising insights. The benefits of using Instapage for native A/B testing include:

  • Easy to use and manage
  • Simplifies the process of A/B testing
  • Provides insights that can help improve conversions

So why wait? Visualize, optimize, and deploy personalized campaigns from one easy-to-use platform, so you can achieve top-tier results and grow your business at any scale. Ready to get started? Begin your free trial today.

blog CTA enterprise demo illustration

See the Instapage Enterprise Plan in Action.

Demo includes AdMap™, Personalization, AMP,

Global Blocks, heatmaps & more.

]]>
Yes, Landing Pages Perform Better Than Your Website Homepage for Branded Search Traffic https://instapage.com/blog/branded-paid-search-ab-test/ Mon, 25 Nov 2019 09:30:00 +0000 https://instapage.com/?p=142876
At Instapage, we thrive on maximizing conversions with experimentation using continuous testing and learning to ensure business growth. We recently tested what impact a dedicated landing page connected to our brand name has on conversions in producing free trial signups. What did we test? For the experiment, we used two different ads and destination URLsRead More >]]>

At Instapage, we thrive on maximizing conversions with experimentation using continuous testing and learning to ensure business growth. We recently tested what impact a dedicated landing page connected to our brand name has on conversions in producing free trial signups.

What did we test?

For the experiment, we used two different ads and destination URLs (homepage versus landing page) to collect data on which experience generated the most conversions. The ads were prompted when prospects searched for “Instapage.”

We automatically distributed traffic through Google Ads with even distribution to ensure we produced more accurate results.

Here are the two ads that were tested; the only difference is the ad headline:

Instapage paid search ad homepage

Instapage paid search landing page

Let’s look at both the experiences separately.

Ad-to-website homepage

In the control, when prospects searched for “Instapage” they saw an ad that includes the phrase “Official Site” so they know that clicking the ad would take them to Instapage’s website homepage. The ad copy features Instapage’s UVP:

Instapage paid search ad homepage

Post-click, here’s the Instapage homepage where they land:

Similar to other homepages, the Instapage homepage features every major product offering with outbound links, including details on product features and what they will help digital advertisers accomplish. It also introduces the Advertising Conversion Cloud™ and landing page Automation.

Furthermore, the homepage also contains numerous navigation links in both the header and footer that take visitors to different pages on the website (Solutions, Resources, Support, etc.) to learn more about the platform and how it helps them solve their pain points. Most notably, it offers two different CTAs: Free Trial (“Get Started”) and Enterprise Demo. The homepage is meant to be a browsing experience for people to learn about the company.

Ad-to-dedicated landing page

The variation showed this ad with no mention of the official site or homepage:

Instapage paid search landing page

The headline features a statistic promising prospects up to 400% more conversions with the platform and the supporting copy describes what Instapage helps prospects achieve. Clicking this ad takes prospects to the following dedicated landing page:

Instapage paid search A/B test page

The page headline reiterates the “up to 400%” statistic mentioned in the ad, and the subheadline further elaborates that point by explaining that Instapage customers convert 4.2x higher than the industry average by automating 1:1 personalized landing pages.

Similar to the homepage, the page also showcases every major Instapage product and what they help customers achieve. The offer highlighted on the page is a demo request — the same as the homepage.

What sets the dedicated landing page apart from the homepage is that:

  • The landing page is dedicated to promoting the “demo request” offer without any distracting navigation links.
  • The ad uses a compelling statistic and the landing page headline immediately reinforces it.

So, which experience produced more free trials?

The test results

Instapage paid search A/B test results

  • The dedicated page had nearly 3x the conversion rate despite a lower CTR.
  • Total conversions for the 400% ad was more than double the website ad
  • The dedicated page had a lower CPC.
  • The total spend on the dedicated page is $800 less.
  • The dedicated page’s cost/conversion was about ⅓ less than the website ad.

Across the board, the dedicated landing page experience wins because it generated the most conversions while spending less than its counterpart.

Don’t waste ad clicks on your homepage

It’s common practice for advertisers to connect branded campaign search ads to their homepages because they believe a prospect searching for a brand name is still browsing for more information so they aren’t ready to convert. However, as this test indicates, dedicated landing pages perform better than a website homepage for brand campaigns.

Whether it’s a brand campaign search ad or any other, you must maximize ad clicks into conversions. Your best chance at converting ad clicks is to send users to dedicated landing pages that convey your UVP and promotes a specific offer without any distractions.

Always connect all your ads to personalized landing pages to lower your cost per customer acquisition. Start creating your dedicated landing pages by signing up for an Instapage Enterprise demo today.

blog CTA enterprise demo illustration

See the Instapage Enterprise Plan in Action.

Demo includes AdMap™, Personalization, AMP,

Global Blocks, heatmaps & more.

]]>
The Recommended A/B Testing Stack for Advanced Digital Marketers https://instapage.com/blog/ab-testing-stack/ Wed, 23 Oct 2019 08:30:00 +0000 https://instapage.com/?p=139768
The concept behind A/B testing is simple. You compare two versions of content against each other to determine which one performs better. As a result of running split tests, your business and website will benefit from more leads and higher conversion rates. My name is Lars Lofgren and before I started running a company atRead More >]]>

The concept behind A/B testing is simple. You compare two versions of content against each other to determine which one performs better.

As a result of running split tests, your business and website will benefit from more leads and higher conversion rates.

My name is Lars Lofgren and before I started running a company at Quick Sprout, I started my career in Growth. As the Director of Growth at KISSmetrics and I Will Teach You To Be Rich, I was personally responsible for hitting our lead goals and ran our A/B testing programs to improve conversions across each website.

I’ve conducted hundreds of A/B tests over my career.

At both companies, I doubled and tripled conversion rates. More specifically, I quadrupled the monthly lead flow at KISSmetrics and built lead-generation machine yielding 40,000 leads per month at I Will Teach You To Be Rich.

During my marketing career, I’ve tried just about every software related to testing available. Some were amazing, and some were not. If I were to ramp up an A/B testing program again, these are the 11 tools that I would trust.

Best tools for A/B testing

There’s quite a few different types of tools that you’ll need for A/B testing. It goes way beyond a single tool for running tests. Let’s go through all the applicable categories.

Transactional and custom emails

To properly test transactional emails, you’ll need complete control over your email creation and analytics. SendGrid offers this type of control. More popular email tools like MailChimp just aren’t robust enough, even many of the Marketing Automation tools aren’t flexible enough. It’s better to have an email tool that your developers can control completely.

Sendgrid does have some A/B testing built-in but it’s limited:

A/B testing stack SendGrid

The magic of Sendgrid doesn’t come from the built-in features. It comes from your developers being able to A/B test entire emails and funnels since they have complete control. When you have full control, you can run any tests that you want.

First, I personally never bother testing subject lines or click-through rates. They don’t have a material impact on the business. Second, testing content is okay, but I prefer to test completely different email funnels. That’s where I found my biggest wins. That’s why I ignore the built-in testing UI and use Sendgrid for entire emails.

Most of my A/B email tests are actually hold-out tests. Send your email to 50% of the test group, send nothing to the other 50%. You’ll be able to see the true impact of that email.

Surveys

SurveyMonkey is the industry standard for online surveys. Almost all of you reading this have taken a survey through their platform at one point or another.

I like SurveyMonkey because it’s very easy to use. Since nearly everyone is familiar with it, users are comfortable taking surveys powered by their software.

You may be asking, “Why do I consider surveys to be part of my A/B testing stack?”

Survey responses fuel the insights that lead to A/B testing wins. Before running any tests, I start with qualitative research. The better I understand the customers, the higher the odds I’ll find a winner with A/B tests.

To collect valuable qualitative insights, keep these tips in mind:

  • Don’t overcomplicate your surveys. Focus on asking 5 to 10 questions that you really need to know.
  • Focus the bulk of your questions on the problems that customers are having.
  • Start with open-ended questions until you have a really good intuition for the respondent.

For more guidance, there’s a bunch of great survey tips here.

User tests

A/B testing stack UserTesting

Another great way to find qualitative insights for A/B testing wins is with user tests. The best tool for these experiments is UserTesting.

The first thing you’ll need to do is to select your audience. You can use your own audience or connect with a specific target through UserTesting’s panel. The platform allows you to be precise with demographics and other factors to find the best users for your experiments.

Next, select 5 or 10 users to go through your onboarding process or other conventional user flow actions on your website. UserTesting will provide videos of these users interacting with your content. As a result, you can identify the major points of friction to give you wins for A/B testing.

This is a gold mine for A/B testing wins. I strongly recommend running a few of these on your signup or onboarding flows. I always find a major point of friction that leads to big wins later. At KISSmetrics, we learned how difficult it was for folks to map out all the data they needed to track. Then we revamped our onboarding, making it more structured and walking people through every piece of data step-by-step.

Business intelligence

A/B testing stack Tableau

For those of you who want a serious A/B testing program, you’ll need to bring all of your data and reporting in-house. To do this effectively, you’ll need to have a data warehouse of some kind. Once this is set up, then you can implement the best tool (in my opinion) for business intelligence reporting — Tableau.

I prefer Tableau because it gives me complete control and it will generate any report that you want. Since I know the exact queries, reports, and visualizations that I need, I can get everything built just the way I want it.

However, the unfortunate news is that someone must manage it and build the reports that you need, which is a full-time job. You’ll also need the expertise at the executive level to know what reporting is required. A traditional BI analyst can build the reports but won’t know which reports and metrics matter the most for the business. All this gets expensive fast.

This solution is not for everyone. Only go this route if you have the expertise and budget to make sure that it’s executed properly.

Statistical significance calculator

Depending on your skill level, two tools are recommended for statistical significance calculations.

Easy calculator
The A/B Significance Calculator from Neil Patel is your most straightforward option. All that’s required is to manually insert the number of visitors and conversions for each variation:

A/B testing stack easy calculator

The calculator then determines if the test was significant enough to draw conclusions.

Advanced calculator
For a more advanced option, the AB Testguide calculator is your best solution. This tool gives you more parameters to include in the calculation.

With this calculator, you can choose a hypothesis and select your confidence level. Even the results of the test are more advanced. Not only does the tool show you a graph, but other statistical data as well like the observed power, p-value, z-score, and standard error of the difference:

A/B testing stack advanced calculator

It’s a bunch of extra details that only make sense if you’ve done a lot of testing or have a background in statistics. If this is too complicated, opt for the simpler calculator.

Why do I use these?

Don’t most testing tools calculate statistical significance?

If there’s one lesson I’ve learned, it’s not to trust new tools blindly. Double-check statistical significance calculations until you’re confident an A/B testing tool can be trusted. There are lots of ways to calculate statistical significance and it’s also easy to mess this stuff up.

So whenever I’m working with new testing software, I always use one of the calculators above until I know for sure that their calculations are solid.

Web analytics

Look no further than Google Analytics, the king of traffic analysis.

To A/B test a website, you’ll need to understand the traffic dynamics, in particular:

  • What pages get the most traffic?
  • Which traffic sources are growing or declining?
  • Which pages bring in the most traffic?

Even if you only use Google Analytics to confirm you have enough traffic on the pages you intend to A/B test, it’s easily worth the effort to install. It’s also completely free and a world-class tool.

There’s a reason that it’s installed on nearly every site out there. The quality of data, the volume of reports and insights, and the fact that it’s completely free make Google Analytics an unbeatable deal.

Previously, I used to customize Google Analytics profiles heavily. Currently, though, I rarely touch the settings and rely on the built-in reports. So I suggest you do the same.

User analytics

Why get a second analytics tool? Isn’t Google Analytics enough?

Even after all these years, Google Analytics excels at traffic analysis. Yes, it has user tracking now, but it was never designed for that. You need to be a Google Analytics power user to use it for true user analysis.

For that, my favorite tool for user analytics and customer data is Amplitude because you can easily track user funnels:

A/B testing stack Amplitude

Most A/B testing programs typically focus on signup and onboarding funnels. Not only is there plenty of volume to test, but small improvements to these funnels can have a major impact on the business.

As you test your funnels, you’ll want to monitor them every month to make sure they’re steadily improving. A/B tests may report one result, but your monthly KPIs won’t match up perfectly.

If you’ve decided to go all-in with a data warehouse and business intelligence tool like Tableau, you can skip the user analytics tool entirely. I find that a user analytics tool is a good in-between step when a business isn’t ready to build out the entire infrastructure.

Post-click landing pages

For post-ad clicks, you’ll want a dedicated solution that specializes in landing page experiences — not an all-in-one software that happens to have “A/B testing” or a “builder.” This is especially true when each campaign deserves a unique post-click landing page and A/B testing each page in the process. Not only does Instapage help you scale landing page production for every campaign, it has A/B testing built right in. When you find wins, you’ll be able to launch them immediately:

A/B testing stack Instapage

My marketing programs always end up with tons of landing pages. If you’re not A/B testing the pages with the most traffic, you’re leaving revenue on the table.

Heatmaps

In customer research, there’s a common piece of advice: “don’t focus on what people say, focus on what they do.” That’s why heatmaps are another great data collection resource for qualitative insights. Heatmaps are perfect for getting a better idea of how people behave. Plus, they’re super easy to understand.

Knowing what people click on, how far they scroll, and where their mouse moves spur all sorts of ideas for optimization — it’s another gold mine of ideas for A/B testing.

Instapage has heatmaps built into their platform that can do all three for post-click landing pages (image below is only the click map):

A/B testing stack Instapage heatmap

You’ll also want to run heatmaps on your main site, too. In that case, I’d recommend Crazy Egg:

A/B testing stack Crazy Egg

Run heatmaps on your homepages, pricing pages, and onboarding flows. Look for things that get a lot of clicks that you didn’t expect, and items you thought would get more attention but don’t. Then come up with test ideas that highlight what people want while de-emphasizing the stuff they don’t care about.

CXL has a lot of great tips on how to use heatmaps here.

A/B testing

What about the A/B testing tool itself? What’s the best option for testing homepages, onboarding flows, and products? In my opinion, VWO:

A/B testing stack VWO

VWO has a solid A/B testing product that starts at $199/month. Compared to other tools that can easily be tens of thousands of dollars per year, VWO is much more accessible. The value per dollar is exceptionally high.

With VWO, you’ll be able to run multiple split tests simultaneously. There’s also a visual editor that allows you to run A/B tests without editing any code. Perfect for testing elements like headlines. You can also use their code editor to make more advanced changes to your tests. The code editor is a huge improvement.

WordPress A/B testing

WordPress is the most popular CMS in the world, but running A/B tests across an entire WordPress site can be a pain. The platform is definitely not designed for this.

That’s why you should use a plugin like Nelio for WordPress A/B testing:

A/B testing stack Nelio

Be warned; you want to get your site in order before doing this testing. If you don’t get everything cleaned up, you’ll run into countless plugin conflicts and your site won’t be able to serve testing variants fast enough for valid testing. Since WordPress was never designed for site-wide testing, it’s easy for A/B tests to conflict with plugins, themes, and poorly built sites.

First, make sure you get a good web host. The last thing you want is a slow host that will delay serving the variant, so the user notices the change. Then clean up your WordPress theme and remove any unnecessary plugins. This will reduce any plugin conflicts and serve your tests fast enough to avoid being noticed.

Once your WordPress site cleaned up and has a fast load time, install Nelio, and use that to run site-wide A/B tests. I used to run a lot of tests on our blog to increase conversions to email subscriptions.

Conclusion

A true A/B testing program needs a suite of tools to run effectively. After testing hundreds of tools myself across multiple companies and hundreds of tests, I prefer the stack above because they focus on different areas of digital marketing, and not one is “best” in all categories.

If there’s some hot, new tool that you’ve been considering, feel free to swap one of these out. My general rule is to experiment often with tools that are easy to start and stop like user recording or survey tools. But for infrastructure tools like analytics, I pick the one that’s reliable and that I’ll be able to depend on for years.

About the Author
Lars is currently the CEO of QuickSprout. Before that, he was the Director of Growth at I Will Teach You To Be Rich and KISSmetrics.

blog CTA enterprise demo illustration

See the Instapage Enterprise Plan in Action.

Demo includes AdMap™, Personalization, AMP,

Global Blocks, heatmaps & more.

]]>