Fibr AI is an Agentic Web Experience Platform that transforms website URLs into intelligent, adaptive agents. Each page senses visitor intent, makes decisions, and reshapes itself in real time to deliver personalized web experiences.

Where is Fibr AI headquartered?

Fibr AI is headquartered in Delaware, USA.

Who is Fibr AI built for?

Fibr AI is built for enterprises looking to personalize at scale, growing businesses starting their web optimization journey, and agencies or marketing affiliates looking to optimize websites for their clients.

What problem does Fibr AI solve?

Fibr AI addresses the disconnect where ads, email, and search are hyper-targeted and AI-powered, but website visitors land on the same static page regardless of where they came from. Fibr makes the website itself as intelligent and context-aware as the marketing channels driving traffic to it.

How does Fibr AI personalize web experiences?

Fibr AI uses AI agents combined with human oversight to detect visitor signals, decode intent, and rewrite page experiences in real time. Personalization can be based on ads, location, device, browser, behavioral signals, visit frequency, LLM-sourced traffic, CDP data, CRM data, and custom audiences.

What results does Fibr AI claim to deliver?

Fibr AI claims results including +28% higher ROI from AI-driven personalization, +30% lower customer acquisition cost (CAC) from intent-based targeting, and 4X more leads from personalizing experiences at scale.

What are the pricing plans offered by Fibr AI?

Fibr AI offers three plans: a Starter Plan for growing businesses (up to 1,000 experiences), an Enterprise Plan for large organizations requiring unlimited visitor sessions and unlimited domains/URLs, and an Agency Plan for agencies and marketing affiliates covering 10,000 monthly visitor sessions and 5 unique URLs.

What features are included in the Enterprise plan?

The Enterprise plan includes Web-Journey Personalization, LLM-Traffic Personalization, AI Landing Page Creator, Customized Agentic Workflows, White-Glove Assistance, CDP/CRM and Analytics integration, On-Brand Agent Training, and 24/7 Dedicated Support with unlimited visitor sessions and unlimited domains and URLs.

What security and compliance certifications does Fibr AI have?

Fibr AI states alignment with SOC 2, ISO 27001, GDPR, and CCPA standards.

What integrations does Fibr AI support?

Fibr AI integrates with CDP (Customer Data Platform), CRM systems, and analytics platforms.

Does Fibr AI support A/B testing and experimentation?

Yes. Fibr AI includes an Experimentation Suite that provides AI-powered hypothesis creation, automated variant creation, audience-based experimentation, statistical significance monitoring, traffic allocation setup, and continuous learning and iteration.

How does Fibr AI handle AI ethics and human oversight?

Fibr AI states that its agents adapt experiences without manipulating them, and that it prioritizes transparency, security, and human oversight at every layer. The platform operates with a 'humans-in-the-loop' model where human allies guide strategy, brand alignment, and key decisions.

How do I get started with Fibr AI?

Fibr AI directs prospective customers to book a demo to get started.

What is the minimum sample size needed for A/B testing?

The minimum sample size depends on the desired statistical significance, baseline conversion rate, and expected improvement. Generally, aim for at least 1,000 users per variation for meaningful results. A small sample can lead to misleading results and undermine the test's validity.

What is a good sample size for A/B testing?

For most scenarios, at least 1,000 conversions per variation is recommended. However, this number varies depending on factors like traffic volume, conversion rate, and the smallest detectable effect. Using a reliable sample size calculator helps determine the ideal size for a specific test.

What inputs do I need to calculate an A/B testing sample size?

You need four key inputs: your baseline conversion rate, your minimum detectable effect (MDE), your desired confidence level, and your statistical power. For example, with a 5% baseline conversion rate, a 1% MDE, and 80% statistical power, the resulting sample size is approximately 3,900,000.

What confidence level should I use for A/B testing?

Most A/B tests use a 95% confidence level, meaning there is only a 5% chance of a false positive. A 90% confidence level produces faster results but carries a higher risk of incorrect conclusions, while a 99% level offers greater certainty but requires a significantly larger sample size.

What is statistical power in A/B testing and what level should I target?

Statistical power is the likelihood of detecting a true effect when one exists. A power level of 80% is standard for most A/B tests, meaning there is a 20% chance of a false negative. A power level of 90% reduces the risk of missing true effects but increases the required sample size.

What happens if I stop an A/B test too early?

Prematurely stopping an A/B test before reaching the required sample size can lead to false positives (incorrectly indicating a significant difference) or false negatives (missing a real difference). Decisions made from early-stopped tests are not backed by solid data and can result in suboptimal website changes.

What is the A/B testing time frame?

The A/B testing time frame is the duration required to gather enough data for statistically significant results. It is influenced by traffic volume, conversion rate, and sample size requirements.

A/B Testing Sample Size: A Definitive Guide for Beginners

Introduction

Without the right A/B testing sample size, your results could be misleading and cost you time, money, and opportunities. This guide walks you through the entire process of calculating your A/B testing sample size, covering what it is, why it matters, how to determine the right size, steps for calculating it, common challenges, and best practices.

What Is an A/B Testing Sample Size?

An A/B testing sample size is the number of users or sessions required to collect accurate data when testing website elements. It determines how many visitors need to participate in your test to ensure reliable results. Calculating the correct sample size is critical because a test with too few participants may yield inconclusive or misleading results, while an overly large sample can waste resources. To find the right sample size, you must consider factors like the expected conversion rate, the minimum detectable effect (the difference between variants), and the desired statistical significance level. A well-calculated A/B testing sample size ensures your conversion rate optimization efforts are based on valid insights, which improves user experience and boosts performance.

Why Is Sample Size Important in A/B Testing?

Sample size determines the reliability and accuracy of your A/B test results. It determines the amount of data needed to draw valid conclusions and minimizes the risk of making incorrect decisions. A sample that's too small increases the risk of false negatives (failing to detect a true difference) or false positives (detecting a difference when none exists). Conversely, using an unnecessarily large sample wastes time and resources.

1. Statistical Reliability

A sufficient sample size minimizes the influence of random chance on test results, allowing you to draw valid conclusions about which website element performs better. Tests with inadequate sample sizes risk producing skewed data, as minor variations in behavior may not represent your broader audience, which could lead to decisions that fail to address the actual preferences and behaviors of your users.

2. Contributes to a Higher Confidence Level

A well-calculated sample size is directly linked to achieving a higher confidence level in your A/B testing results. Confidence levels indicate the likelihood that the observed differences between variations are real and not due to random chance. Insufficient data undermines confidence, leaving you uncertain about whether the chosen variation will deliver consistent performance.

3. Encourages Accurate Decision Making

The sample size determines the quality of the insights you gather. Insufficient sample sizes can lead to misleading conclusions—for instance, implementing a design change based on results that do not reflect the behavior of your target audience. An adequate sample size ensures that decisions are informed, data-driven, and more likely to yield positive outcomes.

4. Reduces Sampling Error

Sampling error occurs when the sample does not accurately reflect the entire population. Larger sample sizes help reduce this error, ensuring that test results are more representative of the actual audience. With fewer sampling errors, your insights and subsequent decisions align better with user behavior across your website.

5. Resource Optimization

Running tests with an adequate sample size prevents unnecessary expenditure of time and resources on inconclusive or misleading experiments. While large sample sizes require more traffic or time, they guarantee that the resources invested yield actionable insights.

6. Sample Size Directly Affects A/B Test Power

Sample size influences A/B test power—the ability of your test to detect true differences between variations. Low-powered tests resulting from insufficient sample sizes might fail to identify meaningful differences between website elements, leading to missed optimization opportunities. A sufficiently large sample size increases test power so that your A/B test is sensitive enough to detect even small but significant differences in performance.

7. Improved Accuracy in Metrics

The accuracy of metrics like conversion rate, click-through rate, or bounce rate relies on an appropriate sample size. Larger sample sizes provide more precise measurements, giving you a clearer understanding of how each variation impacts user behavior. Small sample sizes can lead to variability and uncertainty, potentially leading to false positives or negatives.

8. Helps in Detecting Meaningful Differences

With an adequate sample size, even subtle changes in performance metrics become detectable. For example, a slight increase in the click-through rate on a call-to-action button might significantly impact overall conversions. Small sample sizes often fail to highlight such differences, leading to decisions that overlook valuable optimization opportunities.

9. Reducing Variability

User behavior naturally varies and is influenced by factors such as demographics, preferences, and external conditions. Larger sample sizes reduce the impact of this variability and provide a more stable and reliable dataset. Outliers have less influence on the overall results when the sample size is sufficient, allowing decisions to be based on consistent patterns rather than anomalies.

10. Achieving Statistical Significance

Statistical significance determines whether the observed differences in an A/B test are likely to be real and not due to random chance. Inadequate sample sizes often lead to inconclusive tests, leaving you uncertain about which variation to implement. A proper sample size enhances the likelihood of achieving statistical significance for actionable conclusions.

How to Determine the Right Sample Size for A/B Testing

Getting the sample size wrong can lead to unreliable results, wasted resources, or missed opportunities. The following tips help you determine the right number.

1. Understand Your Current Metrics

Before diving into calculations, analyze your existing website data and identify key performance indicators (KPIs) such as conversion rate (the percentage of visitors completing the desired action), click-through rate (the percentage of users clicking a particular element), and bounce rate (the percentage of visitors leaving the site without taking action). These metrics establish a baseline essential for estimating the expected performance of your test variants. For example, if your current conversion rate is 5%, you'll use this value to calculate the sample size required to detect changes.

2. Define Your Goals and Hypotheses

Clarity on your objectives is essential. Define what you are testing—whether increasing button clicks, reducing bounce rates, or improving sign-ups—to focus on metrics that matter. Additionally, set a minimum detectable effect (MDE), the smallest change in performance that you consider significant. For instance, if you expect a 5% increase in conversions, your MDE is 5%. Smaller MDEs require larger sample sizes to detect the difference.

3. Choose a Desired Confidence Level

Statistical confidence represents the probability that your results are not due to random chance. Most A/B tests use a 95% confidence level, meaning there's only a 5% chance of a false positive (Type I error). Increasing the confidence level to 99% reduces the chance of errors but also requires a larger sample size. Balance confidence levels and resource constraints for effective A/B testing.

4. Calculate the Statistical Power

Statistical power is the likelihood of detecting a true effect when it exists. A power level of 80% is commonly used, meaning there's a 20% chance of a false negative (Type II error). Higher power increases the reliability of your test results but requires more participants. When testing website elements like headlines, images, or CTAs, prioritize reaching sufficient power to ensure meaningful results.

5. Use Online Sample Size Calculators

Manually calculating sample size can be complex, as it involves statistical formulas for confidence levels, power, and MDE. Online calculators simplify the process. Input the baseline conversion rate (e.g., 5%), minimum detectable effect (e.g., 2%), desired confidence level (e.g., 95%), and statistical power (e.g., 80%), and the tool will provide the exact sample size needed for each variant.

6. Account for Variability

Real-world data varies due to random noise or external factors. Use software to randomly assign users to test groups to minimize biases, and run tests during periods that reflect typical user behavior, avoiding major events or holidays unless relevant. Accounting for variability reduces the risk of skewed results.

7. Adjust for Traffic Splits

Most A/B tests divide traffic equally between variations (50/50). However, some scenarios require unequal splits, such as allocating only 30% of traffic to the new variation for risk mitigation. In such cases, the smaller group needs a larger sample size to achieve statistical validity. Adjust your calculations accordingly.

8. Consider the Testing Duration

Sample size directly affects how long your test will run. A/B tests should capture enough data to account for natural fluctuations in traffic, including day-to-day variability (visitor behavior can differ between weekdays and weekends) and time-on-site patterns (certain elements, such as forms, may perform differently at various times of day). A good rule of thumb is to run tests for at least two full business cycles (e.g., 2 weeks) to ensure comprehensive data.

9. Monitor External Influences

External factors can distort your test results. Launching a promotion or paid ad campaign during your test can artificially inflate traffic and conversions. Seasonal trends such as Black Friday or holiday shopping can temporarily change user behavior. Plan your test timing carefully to avoid confounding variables.

Steps for Calculating A/B Testing Sample Size

Step 1: Define Your Baseline Conversion Rate

The baseline conversion rate represents the current performance of your website element and serves as the starting point for calculating sample size. It acts as a benchmark to evaluate whether your variation achieves significant improvement over the control. Use analytics tools like Google Analytics, Mixpanel, or internal tracking systems to assess performance under normal conditions. For example, if your website gets 10,000 visitors per month and 500 complete a purchase, the baseline conversion rate is 5%. Tests with lower conversion rates typically need a larger sample size because detecting subtle changes is statistically more challenging.

Step 2: Set Your Minimum Detectable Effect (MDE)

The MDE is the smallest performance improvement you deem meaningful. Tie it to business goals—for instance, if increasing conversions by 1% significantly boosts revenue, set your MDE to 1%. Balance precision and practicality, as smaller MDEs require larger sample sizes and potentially longer test durations. Example: if your baseline conversion rate is 5% and you aim to detect an increase to 6%, your MDE is 1%.

Step 3: Select Your Confidence Level

The confidence level measures the probability that your test results are not due to random chance. Common options are 90% (faster results but higher risk of incorrect conclusions), 95% (a balanced approach, suitable for most tests), and 99% (greater certainty but requiring a significantly larger sample size). For website elements with high business impact, such as pricing pages or checkout processes, prioritize higher confidence levels to reduce risks.

Step 4: Determine Statistical Power

Statistical power measures the likelihood of detecting a true effect if one exists. A power level of 80% is standard for most A/B tests, offering a good balance of reliability and feasibility. A power level of 90% reduces the risk of missing true effects but increases the required sample size. Incorporating test power into your calculations ensures your test is well-equipped to detect meaningful changes.

Step 5: Use a Sample Size Calculator

Use an online sample size calculator such as Optimizely to simplify the process. Key inputs are your baseline conversion rate, minimum detectable effect, and statistical significance level. Example: with a baseline conversion rate of 5%, an MDE of 1%, and statistical power of 80%, the resulting A/B testing sample size is 3,900,000.

Step 6: Adjust for Traffic Allocation

While most A/B tests split traffic evenly (50/50), some scenarios require uneven splits such as 70% control and 30% variation. Use tools to allocate traffic proportions automatically and ensure the smaller group has enough participants to maintain statistical validity. Traffic allocation impacts your test's duration and reliability.

Step 7: Account for Variability

User behavior is rarely consistent, and variability in traffic sources, devices, or external factors can affect test outcomes. High variability demands a larger sample size. Segment your audience to ensure test participants are representative of your target audience, and avoid seasonal bias by running tests during typical traffic periods.

Step 8: Validate Your Assumptions

Before launching your A/B test, validate all assumptions to ensure the calculated sample size is accurate and the test design is feasible. Verify that your baseline conversion rate reflects current performance, confirm your test will reach the required sample size within a reasonable timeframe, and assess potential external influences such as ad campaigns that could distort results.

Step 9: Monitor the Test

Even after starting the test, ongoing monitoring is crucial. Regularly check traffic distribution and conversion metrics to ensure the test progresses as planned. Avoid stopping the test prematurely, as doing so can lead to misleading results.

Common Challenges in A/B Testing Sample Size

1. Miscalculating the Required Sample Size

If the sample size is too small, results might not be reliable and any detected differences may be due to chance. If the sample size is too large, it can lead to unnecessary resource allocation, making the test more time-consuming and expensive. Accurate calculation requires considering the baseline conversion rate, minimum detectable effect, and power of the test (typically set at 80% or higher).

2. Balancing Test Duration and Traffic Availability

Websites with high traffic can reach the required sample size quickly, enabling shorter testing periods. Websites with lower traffic may need extended testing periods, delaying actionable insights. Rushing a test by shortening its duration before reaching the required sample size compromises statistical validity.

3. Accounting for Variability in User Behavior

User behavior is often variable and can be influenced by location, device type, time of day, or marketing channel. Mobile users might behave differently from desktop users, or users from different regions may interact with the website in distinct ways. Without accounting for this variability, results may not reflect the broader audience's behavior, leading to skewed conclusions.

4. Overlooking Statistical Significance and Test Power

Focusing solely on sample size without considering statistical significance and test power is a common mistake. If test power is too low, even a large sample size may fail to identify meaningful differences between variations. A balance between sample size, statistical significance, and test power is necessary for accurate results.

5. Dealing with High Drop-Off Rates

Tests involving complex website elements such as multi-step forms or lengthy user journeys often face high drop-off rates. When users abandon the test midway, it reduces the effective sample size and impacts the reliability of results. Adjust for drop-offs by recalculating the sample size or redesigning the test to account for these losses.

6. Handling Multiple Variations

When testing multiple variations (such as an A/B/n test), the required sample size increases because traffic must be evenly distributed across all variations. For example, testing three variations of a landing page requires more traffic than testing two. Failure to account for this can lead to an underpowered test, making it harder to detect significant differences.

7. Adjusting for External Influences

External factors like seasonal trends, ongoing marketing campaigns, or algorithm updates can impact A/B test results. A sudden increase in traffic due to a viral campaign might skew results and lead to inaccurate conclusions. To mitigate these factors, adjust the sample size or extend the test duration.

8. Ensuring Balanced Traffic Distribution

An imbalanced distribution of traffic between variations can distort test results, biasing outcomes toward the variation with more data. Proper randomization and tracking mechanisms are necessary to ensure traffic is evenly distributed across all variations.

9. Avoiding Early Stopping

Prematurely stopping an A/B test before reaching the required sample size can lead to false positives (incorrectly indicating a significant difference when there isn't one) or false negatives (missing a real difference). Early stopping, often driven by impatience or resource constraints, can lead to hasty decisions not backed by solid data.

10. Reconciling Business Goals with Statistical Rigor

Business priorities often demand quick results, which can conflict with the time needed to obtain statistically significant results. Balancing business goals with the need for statistical rigor requires careful planning, setting clear expectations, and establishing realistic timelines that allow enough time to reach the required sample size while meeting business objectives.

Best Practices for Managing Sample Size in A/B Testing

Calculate the ideal A/B testing sample size: Consider factors like baseline conversion rates, minimum detectable effect, and test power. Use sample size calculators to determine the exact amount of data needed to achieve statistically significant results without wasting resources or time.
Consider test duration and traffic availability: Ensure sufficient traffic is available over an appropriate duration. If traffic is low, extend the test duration to balance the need for reliable data while preventing rushed decisions that can skew test power and results.
Adjust for variability in user behavior: Account for differences in user behavior such as device type or geographic location. A higher sample size may be needed to compensate for behavior discrepancies and ensure test power is maintained.
Focus on statistical significance and test power: Ensure that both statistical significance and test power are prioritized when determining sample size. A sample size that is too small can lead to inconclusive results, while a large enough sample ensures that even minor changes are detected.
Monitor drop-off rates and adjust accordingly: High drop-off rates in tests involving multiple steps can reduce the effective sample size. Adjust for these losses by increasing the total sample size or redesigning the test to preserve test power and data accuracy.