Statistical Significance Calculator

This runs a two-proportion z test on the results of an A/B test or any other pair of yes-or-no counts. Enter how many people saw each version and how many converted, pick a confidence level and a one or two sided test, and it returns each rate, the z score, the p value, a plain-language verdict and a confidence interval on the difference between the two rates. A second action goes the other way and works out how many visitors each version needs before the experiment can detect the improvement you care about, given the confidence and power you want. The interval uses the unpooled standard error while the p value uses the pooled one, which is why the two can disagree on a borderline result. It assumes each visit is an independent trial and that you fixed the stopping point before you started looking.