Power

The Power of a test refers to how efficiently it can detect true uplift. We would like this to be as high as possible, but the trade-off is that high-powered tests require more data. For a standard test, we have set our parameters so that a variant that has an actual 5% uplift has an 80% chance of winning the test. We say a test has reached power if the power calculator deems it to have gone on long enough to have the 80% power we aim for.