I’ve listed some further reading at the bottom of each section if you’re interested in learning more.

You’re probably familiar with the Frequentist approach to testing.

So what the hell does Bayesian statistics mean for a/b testing?

As Leonid Pekelis wrote in an Optimizely article, “Frequentist arguments are more counter-factual in nature, and resemble the type of logic that lawyers use in court.

Most of us learn frequentist statistics in entry-level statistics courses.

An important aspect of this prior belief is your degree of confidence in it.” Matt Gershoff, CEO of Conductrics, explained the difference between the two as such: Matt Gershoff: “The difference is that, in the Bayesian approach, the parameters that we are trying to estimate, are treated as random variables. Random variables are governed by their parameters (mean, variance, etc), and distributions (Gaussian, Poisson, binomial, etc).

The prior is just the prior belief about these parameters.

So what the hell does Bayesian statistics mean for a/b testing?

As Leonid Pekelis wrote in an Optimizely article, “Frequentist arguments are more counter-factual in nature, and resemble the type of logic that lawyers use in court.

Most of us learn frequentist statistics in entry-level statistics courses.

An important aspect of this prior belief is your degree of confidence in it.” Matt Gershoff, CEO of Conductrics, explained the difference between the two as such: Matt Gershoff: “The difference is that, in the Bayesian approach, the parameters that we are trying to estimate, are treated as random variables. Random variables are governed by their parameters (mean, variance, etc), and distributions (Gaussian, Poisson, binomial, etc).

The prior is just the prior belief about these parameters.

In this way, we can think of the Bayesian approach as treating probabilities as degrees of belief, rather than as frequencies generated by some unknown process” In summary, the difference is that in the Bayesian view, a probability is assigned to a hypothesis.