Significance

By October 9, 2020 No Comments

Glossary

A knowledge hub for digital marketers.

Significance

For personalized A/B testing, usually a hypothesis is made and the statistical significance of the variations will define if the tests actually worked or not. A campaign or personalization is said to have high significance (or statistically significant ) if that particular personalization has shown visible effectiveness over its control group.  This effectiveness is usually measured in terms of a positive change in the impressions or an increase in uplift of conversions or average order values

Why Should You Track Significance (AOVU)?

While A/B testing, both the A version and B version will have different results and you would want to know if a particular personalization has actually worked. You would also want to compare two different personalizations and their effectiveness against the control group. Significance gives you the percentage of effectiveness, so you can pick the best performing version for further campaigns. A low significance indicates that your personalization is not working well and needs additional changes or a change in strategy.