could you answer for my questions? 1. Firts questi...
# experimentation
r
could you answer for my questions? 1. Firts question is about equations (5) and (6) form the DOC. I understand, the average and the variance of posterior equal to formulas on the picture, if the likelihood is the normal distribution, but why we can do this assumption? 2. And as I understood, the stats engine uses only normal distribution for all type of metrics? But in pdf from 2021 the beta distribution was mentioned too.
h
but why we can do this assumption?
Do you mean the assumption that the data are distributed normally? That follows from the CLT since we are working with differences in sample averages.
And as I understood, the stats engine uses only normal distribution for all type of metrics?
Yes, and the distribution is over the experiment effect (
\Delta
) not over the metric itself! So we do not model the distribution of the sample mean itself but rather the change. This allows us to use one simple model where the priors are defined over effect sizes rather than sample means.