Significance

Short Description: Expressed as a percentage, significance shows whether a campaign or personalization is reliable to take an action depending on the impressions or uplift.

Long Description:  For personalized A/B testing, usually a hypothesis is made and the statistical significance of the variations will define if the tests actually worked or not. A campaign or personalization is said to have high significance (or statistically significant ) if that particular personalization has shown visible effectiveness over its control group.  This effectiveness is usually measured in terms of a positive change in the impressions or an increase in uplift of conversions or average order values. 

Why Should You Track Significance (AOVU)?

While A/B testing, both the A version and B version will have different results and you would want to know if a particular personalization has actually worked. You would also want to compare two different personalizations and their effectiveness against the control group. Significance gives you the percentage of effectiveness, so you can pick the best performing version for further campaigns. A low significance indicates that your personalization is not working well and needs additional changes or a change in strategy.