AB Testing
Pritam Roy
You have changed the position of the call to action button on your landing page. However, the junior marketer suggests a different position for more conversions. To settle this, you create two variants and test them against each other using A/B testing.
How would you know which variant performed better?
By tracking A/B metrics, which in this case would be click-through rate or CTR. The variant with a better CTR is the winner!
But what are the common A/B metrics you should focus on? And more importantly how should you decide which metric to track and which to ignore? This article is there to help.
Below, we have listed the top 8 AB testing metrics to track and analyze the results of your conversion rate optimization efforts. You’ll also find a guide to help you determine the right metrics for your marketing efforts.
Overview
Here’s a brief overview of everything this article entails:
A/B Testing Metrics: Data points used to determine the effectiveness of your A/B tests, identify successful variants, understand the reasons for results, and make informed decisions.
Choosing the Right Metrics
Map the User Journey: Identify key interaction points (entry, engagement, decision, checkout) to understand user behavior.
Select Relevant Metrics: Choose metrics that reflect user actions and decisions at key points (e.g., engagement rate, conversion rate, sales revenue).
Key AB testing metrics:
Conversion Rate: Percentage of users completing a desired action (e.g., form submission, purchase).
Click-Through Rate (CTR): Percentage of users clicking on a link or element (e.g., CTA, ad).
Bounce Rate: Percentage of visitors who leave a page immediately without interacting.
Scroll Depth: How far down a page users scroll.
Average Order Value (AOV): Average amount spent per order.
Average Session Duration: Average time spent on a website during a single visit.
Abandonment Rate: Percentage of users who start a task but don't complete it (e.g., cart abandonment, form abandonment).
Retention Rate: Percentage of users who return to a page after a period of time.
How to improve the key AB testing metrics:
Conversion Rate: Define clear conversion goals, analyze user behavior, and improve page relevance and user experience.
CTR: Optimize CTA placement, use persuasive language, and improve visual elements.
Bounce Rate: Improve content quality, page loading speed, and user experience.
Scroll Depth: Use visuals, improve content readability, and optimize for user interaction.
AOV: Offer discounts, free shipping, and product recommendations.
Average Session Duration: Understand user behavior, improve content engagement, and address technical issues.
Abandonment Rate: Shorten forms, offer more payment options, and address user concerns.
Retention Rate: Offer personalized experiences, address technical issues, and use retargeting campaigns.
Top A/B Testing Tools
Fibr AI (unlimited experimentation forever free)
VWO
Optimizely
AB Tasty
Unbounce
Let’s start with the basics.
What are AB testing metrics?
A/B testing metrics are the data points that help you determine how effective your hypothesis was, which variant was most successful, what led to the results you achieved, and eventually make informed decisions.
Say your goal is to improve the conversion rate, i.e., increase the number of users who click the signup button.
Now, you create a hypothesis according to which changing the color of the CTA would help you increase the conversion rate.
You keep the color of the CTA button green for variant A and yellow for variant B.
Upon running the test and analyzing the results, you come to know that variant B performed better, i.e., drove more sign-ups.
Based on the results, variant B brought a higher conversion rate, which you decided to make live.
Because you tracked the required metrics, you could:
Determine how effective your hypothesis was: results showed the change brought a difference.
Which variant was most successful: variant B was the winner.
What led to the results you achieved: change in color of the CTA
Eventually, make informed decisions: make the variant B live
How to choose the right AB testing metrics?
One of the best ways to choose the right A/B testing metrics is by mapping your users’ journey right from when they interact with your website or page to the moment they leave.
Here’s an example:
Say you want to boost sales of your product, and you create 2 variations for A/B testing. Variation A has a traditional layout, while version B has a modern design with more product images and the “Buy Now” button positioned differently.
Now, to determine what metrics to track, you need to map the user journey:
Entry point: user clicks an ad or browses the website.
Engagement: user lands on the variant A or B.
Decision point: user either adds the product to the cart or leaves the site.
Checkout: user completes the purchase or abandons the cart.
Identify Key Interaction Points
Product page engagement: time spent on the page, interaction with the CTA or “Buy Now” button, and clicks on the description or product image.
Add to Cart: number of users adding the product to the cart.
Purchase completion: number of completed transactions
Selecting Metrics and KPIs
Engagement Rate: you can track the engagement rate or average time on the page to determine how long the users stay on the product page and interact with elements like buttons or images.
This will help you determine if the new UI holds the attention for longer.
Conversion Rate: track the percentage of users who add the product to the cart to determine which variation is more successful.
Sales Revenue: monitor the sales from each version to determine which UI variation drives more revenue.
By mapping the user journey, you can identify the points where users make decisions. This helps you pick the metrics that reflect those actions/decisions.
Say if the CTA placement in variation B increases your CTR but doesn’t improve checkout completions, you’d know to focus on the checkout process and the related metrics next.
8 key AB testing metrics you should track
The key A/B testing metrics you need to track include conversion rate, click-through rate, bounce rate, scroll depth, average order value, average session duration, and abandonment rate. Let’s discuss these metrics in detail:
Conversion rate
The conversion rate is pretty broad and the most obvious metric to track to determine the success of your conversion rate optimization efforts. The higher the conversion rate, the better it is.
However, you need to define what you consider as a conversion. For instance, it could be a form submission, a sign up, a demo request, or something else.
According to Ruleranalytics, here’s the average conversion rate by industry:
How to calculate conversion rate:
Conversion rate = (Number of conversions/ total number)*100
How to improve the conversion rate?
The answer to this largely depends on what you’re considering a conversion.
Say, for you, a customer making a purchase is a conversion. Now, you need to track the users’ journey to understand what is stopping them from taking action. And for that you need to track other secondary metrics, such as the bounce rate.
For instance, if the bounce rate is too high, you may want to work on it by making the content more engaging or relevant to the audience. When you improve the bounce rate, you’ll be able to keep the users on the website, which may eventually improve your conversion rate.
Click-through rate (CTR)
Click-through rate or CTR is another important AB testing metric that represents the percentage of people who clicked a link or a digital asset against the number of people who viewed it.
Let’s say your ad was viewed by 100 people, but only 10 clicked and landed on your landing page. Now, your CTR would be 10% (10/100*100). And out of those 10, only 2 clicked your landing page CTA, now, your click-through rate would be 20% (2/10*100).
A lower CTR indicates problems with your CTA or other visual elements like an image, video, or event text copy.
How to calculate Click through rate (CTR):
Click-through rate = (clicks/impressions)*100
How to improve the click-through rate?
Here’s how you can improve your CTR:
Optimize the visibility and placement of CTAs
Use persuasive language.
Make sure your digital assets or pages are relevant to your audience
Highlight the value clearly.
Use heat maps, scroll maps, and move maps to understand user behavior, identify the actual problem, and fix it.
Bounce rate
Bounce rate tells you the percentage of visitors who landed on your website but bounced right off without taking any action.
The lower the bounce rate, the better. According to HubSpot, the average bounce rate ranges from anywhere between 26%-70%. If your bounce rate is somewhere within this range, it’s great.
However, if the bounce rate of your landing page exceeds 70%, it indicates issues with your page quality, content, and UX. Common issues may include:
Visitors are not getting the information they clicked for
Visitors cannot relate to the content or aren’t engaged enough
Or maybe the page is not loading fast enough.
How to calculate bounce rate:
Bounce rate = (one-page visits / total visits)*100
How to improve the bounce rate?
To improve your bounce rate, you can:
Improve the page’s relevance to your target audience.
Improve the content quality and tweak elements like the heading, hero image, etc.
Ensure the page is optimized for mobile users and free from errors such as slow loading speed.
Scroll depth
Scroll depth tells you how far a user scrolls down a landing page or web page. Lower scroll depth indicates a lack of engagement across the page or after a certain point because of poor design and irrelevant content, among other issues.
By tracking scroll depth, you can also determine the average content length that is appropriate for CTA placement.
If most of your visitors don’t scroll past a point, even if there aren’t any issues, you can place the CTA above that point to ensure more clicks or conversions.
How to calculate scroll depth:
You can set a scroll depth trigger in Google Analytics for automated tracking. However, if you wish to calculate the value, you can use this formula:
Scroll depth = (furthest scroll point in pixels/total pixel page height)*100
How to improve the click-through rate?
According to Agency Analytics, good scroll depth ranges between 60-80%. However, a higher scroll depth may not necessarily be a good sign, especially when you compare it with other metrics.
For instance,
If your scroll depth is higher but the session duration is short, it could indicate users are viewing your page but not taking action.
And a low scroll depth with a high session duration could mean users find a certain section more engaging.
To improve your scroll depth:
Contextualize scroll depth by comparing it with other metrics.
Use visuals and multimedia.
Improve content readability and formatting.
Optimize for user interaction.
Average order value (AOV)
Average order value (AOV) tells you how much a customer spends on your website during a single purchase. This metric is particularly important for eCommerce businesses as it helps you evaluate if the changes you made persuaded the customers to spend more or not.
Another reason to track the average order value is to determine which version is actually good from a revenue perspective. How?
Imagine your variant A has an “add to cart” rate of 3X, while variant B has an “add to cart” rate of 5X. Without factoring in the AOV, you’ll declare variant B as the winner.
However, upon tracking AOV, you find out that the average order value of variant A is 4 times that of variant B. This means that variant A is driving more revenue and thus should be declared as the winner.
How to calculate average order value (AOV):
Average order value = (total revenue / total number of orders)
How to improve the average order value (AOV)?
Improving your average order value means convincing your customers to spend more per purchase on average. And to achieve this, you can:
Offer discounts when customers purchase multiple items.
Offer free shipping above a certain amount to convince customers to spend more.
Create urgency by offering limited-time discounts.
Recommend products that would go with the primary product the customer is buying.
Average session duration
Average session duration is the time a user spends on your website during a single visit. The session is measured right from the moment the users enter your site until they become inactive or leave your site.
If a user spends more time per session on your website, it indicates that they find your website valuable and they’re getting what they came in for. While you may not exactly know what is making them stay, you’d get an idea that you’re doing something right.
In a nutshell, by tracking average session duration, you can determine which page your audience is finding more engaging or interesting.
How to calculate average session duration:
You can find the average session duration from Google Analytics.
However, if you want a formula, here you go:
Average session duration: total session duration / total sessions
How to improve the average session duration?
To improve the average session duration, you need to understand where and why users are dropping off.
For that you can leverage Hotjar’s screen recording feature. This feature records the interaction of every user with your landing page element, allowing you to understand the user journey better. And when you can view/understand the user journey, you can make necessary improvements.
For instance, if users are not interacting with your CTAs, you can change its placement or make it more dominant. Or if the users are getting attracted to a particular section, you can place more valuable information there.
Abandonment rate
Cart abandonment rate gives you the percentage of users that added items to their carts but did not end up making a purchase against the total number of users that added items to their carts.
However, when you talk about the abandonment rate alone, it includes any action that was initiated but wasn’t completed. It could be filling out a form or a survey, etc.
A higher abandonment rate indicates a bad user experience, which could stem from lengthy forms, forms asking for personal information, unfavorable payment options, or a high delivery fee.
How to calculate abandonment rate:
Abandonment rate: (number of intended tasks completed / total number of tasks initiated) *100
How to improve the abandonment rate?
Here’s how you can improve abandonment rate:
Try shortening the forms and simplifying the checkout process.
Offer more payment options and lower or remove delivery fees.
Use popup surveys to ask users the exact reason why they’re leaving and work on the same
Retention rate
When it comes to A/B testing, retention rate is the percentage of users who return to a landing page after a period of time. This user behavior metric tells you how effective a landing page is. It also helps you determine which audience segment is more likely to convert.
How to calculate retention rate:
Retention rate: (number of returning visitors within a period of time / total number of visitors that landed on the website during the same period)*100
How to improve the retention rate?
Users may have left the page for several reasons, such as:
They had something urgent come up
They got distracted
Their need was not immediate
Technical glitches like slow loading speed
Your goal is to bring the users who’ve viewed your page back. Here’s how you can achieve this goal:
Offer value and a personalized user experience.
Ensure there aren’t any technical glitches at all.
Use retargeting campaigns to target users who have previously visited your site using promotional offers.
Now that you have a basic idea of AB testing metrics to track, you can start analyzing your A/B testing results. Afterall, that’s why you were tracking the metrics in the first place, right?
Things to keep in mind when analyzing A/B testing results with key metrics
By keeping these best practices in mind, you can analyze your A/B test results better and ensure you make the right decisions that positively impact your ROI. Here are the things you need to keep in mind:
Ensure statistical significance
It’s important to ensure all the findings of your A/B tests are statistically significant, i.e., your test results are accurate and reliable and are not a matter of chance or accident.
If your results are not statistically significant and you still optimize your landing page, you might ruin its performance altogether.
You can use tools like Fibr AI, which automatically ensures the results of your A/B tests are statistically significant and reliable.
Determine the influencing factors
You must identify the internal and/or external factors that might be playing with or influencing the results of your A/B tests.
Imagine you’re testing two subject lines to identify the one with the best open rate.
For variant A, you send the emails at 10 AM, and for variant B, you send the emails at 5 PM.
The audience (9-5 employees) is more likely to open the emails when they log in and not when they log out. But because you sent the emails for variant B at the time when employees log out, it would lead to a poor open rate, no matter how good the subject line may be.
Identifying the factors that might impact your results will help you draw better conclusions and make more informed optimization decisions
Optimize based on your analysis
No matter what results you achieve, it’s important to record the findings and take the next steps after the analysis.
If variant A performed way better than the other one, you can declare variant A as the winner and make it live.
If there wasn’t much difference between the two variants, you’d know that your hypothesis just did not work. You can create a more informed and data-driven hypothesis again, perhaps using MAX, Fibr AI’s AI-powered CRO experimentation agent.
Now that you know about the best practices for analyzing your A/B test results, start conducting your A/B tests right away using the below-mentioned tools.
Top 5 A/B testing tools for tracking A/B testing metrics
The best AB testing tools to conduct comprehensive AB tests and track the AB testing metrics include Fibr AI, VWO, Optimizely, AB Tasty, and Unbounce.
Fibr AI
Fibr AI is an AI-powered personalization platform that comes with a dedicated A/B testing tool. With this A/B testing tool, you can create, launch, and analyze as many A/B tests as you need for free.
Fibr AI’s A/B testing tool integrates seamlessly with Google Analytics 4, allowing you to track almost all AB testing metrics or KPIs mentioned in the above list.
Notable features
Statistical significance: Fibr AI’s A/B testing tool automatically ensures that your test results are statistically significant and aren’t based on random chance. This allows you to make data-driven optimizations that yield better results.
Max: AI-powered CRO agent: Max is an AI-powered A/B Testing agent from Fibr that continuously conducts experiments and makes optimizations on your website to help you identify high-performing variants.
This CRO agent helps you validate your hypothesis or create new ones powered by accurate data. You can also perform multivariate testing using this tool.
Create variations within minutes: Creating variations with Fibr AI becomes easy because of artificial intelligence. The AI engine gives you high conversion recommendations for your CTAs or headline text, allowing you to create high-performing variants.
No code editor: Using Fibr AI’s no code or visual editor, you can further simplify the process of creating variations. You can use the visual editor to create hundreds of variations within a few minutes, allowing you to run more tests more frequently.
Target different audience sets: Fibr AI allows you to experiment with specific audience segments using criteria like operating systems, custom events, browsers, visitor behavior, device type, and more.
Download reports: Upon test completion, you can download the test data for offline analysis. You can also share the reports with your colleagues for an in-depth analysis of your A/B test results.
Forever free: Fibr AI’s A/B testing tools help you test without limits, i.e., unlimited campaigns across any page, with no session limits as long as you’re running one campaign per URL.
Pricing: the A/B testing tool is absolutely free as long as you’re launching 1 campaign per URL.
P.S. If you want to automate the process of conducting and analyzing A/B tests on your website or landing page and need a tool that learns from the results and helps you implement best-performing strategies, try our Fibr AI.
VWO
VWO also comes with an A/B testing tool that allows you to conduct A/B testing, split URL testing, and multivariate testing. It uses the Bayesian-powered SmartStats engine to ensure you get reliable and real-time reports.
Using VWO, you can track the necessary KPIs or AB testing metrics and analyze the same to draw actionable insights.
Testing type: A/B testing, split URL testing, and multivariate testing
Notable features
Audience and behavioral targeting
Code Editor
It integrates with 40+ platforms, including GA4, Shopify, and more.
AI engine for optimization checks
Constant experiment health checks
Pricing: The starter plan is free for up to 50,000 monthly tracked users.
Optimizely
Optimizely is a digital experience platform that comes with content management, advanced personalization engines, and web experimentation. It allows you to perform omnichannel experimentation and track the right AB testing metrics.
Optimizely supports A/B testing, split URL testing, and multivariate testing and leverages the Bayesian and Frequentist statistics model.
Testing type: A/B testing, split URL testing, and multivariate testing.
Notable features:
AI-based text variation generator
CDN A/B testing
Multi-armed bandit testing
Audience targeting
Mutually exclusive campaigns
Pricing: Contact the sales team for pricing.
AB Tasty
AB Tasty is an experience optimization platform that helps you with web and feature experimentation and allows tracking of all the necessary AB testing metrics.
Testing type: A/B testing, split URL testing, and multivariate testing.
Notable features:
WYSIWYG editor
Multi-armed bandit testing
Audience and behavior-based targeting
Comprehensive widget options
Allows users to leverage gen AI for visitor segmentation, based on emotional states.
Pricing: Contact the sales team for pricing.
Unbounce
Unbounce is a landing page builder and a CRO platform that also supports A/B testing, real-time analytics, and AI-driven tools for copywriting and optimization.
Testing type: A/B testing and multivariate testing.
Notable features:
WYSIWYG editor
Integration with 3rd party platforms
Confidence intervals
Real-time reporting
No UX tradeoff.
Pricing: The plan that supports A/B testing starts at $112 a month
Now that you have a list of the best A/B testing tools, you can conduct A/B tests, compare your AB testing metrics, and make informed decisions.
If you wish to start right this instant, and that too for FREE, and leverage the power of AI to automate testing and optimization, try Fibr AI.
Conclusion
There you have it: the most common AB testing metrics to help you determine the success of your CRO efforts. However, make sure you map out the user journey to identify the right AB testing metrics to track. This is essential to derive meaningful insights and make well-informed decisions.
FAQs
Which tool is used for AB testing?
There are multiple tools in the market for A/B testing, as mentioned above. However, if you’re looking for a reliable yet free tool, you can go for Fibr AI’s A/B testing solution.
What is A/B monitoring or testing?
AB monitoring or testing is a method of putting your hypothesis to the test. Say you have two variants of the same landing page, A and B. You think variant A will perform better because of the bigger CTA, while someone on your team thinks variant B will perform better.
How’d you know for sure? A/B testing!
A/B testing allows you to compare both variants and identify which drives better results. This way, you’ll be able to find the winner backed by data.
