rapid-petabyte-36078
11/07/2024, 2:51 PMhelpful-application-7107
11/08/2024, 4:52 PMbut why we can do this assumption?Do you mean the assumption that the data are distributed normally? That follows from the CLT since we are working with differences in sample averages.
And as I understood, the stats engine uses only normal distribution for all type of metrics?Yes, and the distribution is over the experiment effect (
\Delta
) not over the metric itself! So we do not model the distribution of the sample mean itself but rather the change. This allows us to use one simple model where the priors are defined over effect sizes rather than sample means.