A/B Test Calculator
Variant A
Variant B
Confidence level
No statistically significant winner
How to use this calculator?
1. Enter the number of visitors and conversions for each variant
2. Statistical significance is calculated automatically
3. A result above 95% is considered statistically significant
Quick answer
How do you know if an A/B test is statistically significant?
Compare the conversion rates of both variants with a Z-test: the gap between the rates is divided by the combined standard error, then turned into a confidence level. Above 95%, the result is considered significant. Example: A converts 100 of 1,000 visitors (10%), B converts 130 of 1,000 (13%): confidence reaches about 98.2%, so B wins.
| Variant A | Variant B | Confidence | Significant at 95%? |
|---|---|---|---|
| 100 / 1,000 (10%) | 110 / 1,000 (11%) | 76.7% | No |
| 100 / 1,000 (10%) | 120 / 1,000 (12%) | 92.4% | No |
| 100 / 1,000 (10%) | 130 / 1,000 (13%) | 98.2% | Yes |
| 100 / 1,000 (10%) | 140 / 1,000 (14%) | 99.7% | Yes |
| 500 / 5,000 (10%) | 550 / 5,000 (11%) | 94.9% | No |
| 40 / 2,000 (2%) | 60 / 2,000 (3%) | 97.9% | Yes |
About the A/B Test
What is it?
The A/B Test calculator is a statistical tool that determines whether differences observed between two variants are statistically significant, or simply due to chance.
Why use it?
Without statistical significance, you risk making wrong decisions based on random results. This tool gives you mathematical certainty before declaring a winner.
How does it work?
Enter the number of visitors and conversions for each variant. The tool automatically calculates the conversion rate, relative improvement and statistical confidence level.
Usage tips
- Aim for at least 95% confidence level before drawing conclusions
- Wait for a minimum of 100 conversions per variant
- Test only one variable at a time for reliable results
- Avoid checking results too early - let the test run its course
- Document each test with hypothesis, duration and result
Embed this tool on your websiteFreeCopy this code into your website or blog: the tool shows up right on your page, free and without sign-up.
All embeddable toolsRelated tools
- Margin CalculatorProfit margin, markup, gross margin and price multiplier
- Bold Text GeneratorWrite in bold, italic or strikethrough to copy and paste on Instagram, LinkedIn or X.
- Resize ImageResize your images in pixels, by percentage or to social media sizes
- Favicon GeneratorCreate your website favicon from a letter or an image, as ICO and PNG
