Hi! I’ve been wondering why in some cases the credible interval has a visible skew like in this case with my revenue per user metric. I kinda get the general idea and intuitively it’s somewhat understandable, but maybe someone can give me more detail? I’m well versed in confidence intervals in Frequentist approach but they d”n’t have this concept of “thickness”

@helpful-application-7107 ^

In the Bayesian engine, we show a posterior distribution, so these literally represent some density curve about what your posterior beliefs about uplift should be. The posterior probability of uplift being < 0% will be (roughly, due to some vis constraints) the pct of the area of this graph that is to the left of 0. So that's nice! So why is it skewed? • A reasonable experimenter might assume it's because we have some informative prior that skews results towards zero; but it isn't this reason. We use non-informative priors, so we shouldn't see any weight from the prior dragging results towards zero. • The real reason, is that we are showing you the posterior distribution of

uplift = (treatment_mean - control_mean) / control_mean

using a lognormal distribution. We approximate

uplift

using

log(treatment_mean / control_mean)

which simplifies computation of the posterior. Using this log-transformation results in the skew that you see.

Thanks for the explanation. I’m not very familiar with this approach and it was only yesterday that I read about log transformation but in the context of making the distribution less “skewed”. I checked the whitepaper but didn’t find anything about this approach in Growthbook.