Hello, I have a question on the calculation of the Relative Uplift, shown as a violin plot.
If I understand correctly, the Violin plot represent the distribition of relative uplift, is that correct ? What we are trying to understand is what this distrubution represent exactly ? For example, is it based on the relative uplift observed for each day of the test, or for different users subsets/segments,... ?
The graphs are very clear but as dummy in statistics I currently not being able to explain clearly what this distribution represent.
I already had a look at the ressources below but couldn't find (or understand) the answer:
• Growthbook Statistics: https://docs.growthbook.io/statistics
• Bayesian Statistics Engine
Also interested to be pointed to an other thread if this have already been asked.
Thanks a lot
11/10/2022, 2:29 PM
It's the probability distribution. Kind of like a confidence interval. The thicker parts of the graph are more likely to be the true uplift. In your example, there's a tiny chance that the uplift is actually negative (the red area), but it's most likely closer to a 10-20% uplift
11/10/2022, 2:51 PM
Thanks Jérémy, I'll stick to that forulation to respond to stakeholders and try to find a good stats book to better understand the underlying calculation ^^