https://www.growthbook.io/ logo
#ask-questions
Title
# ask-questions
b

billowy-window-97200

01/24/2024, 11:00 AM
Chance to Win
Hey folks, where can I see the Chance to Win calculation in more detail? Are there differences between Bayesian and frequentist engines? For Bayesian, is it considering loss function and risk? I was reading this page here, but there is not much explanation in detail for the Chance to Win calculation, and I can't see the expected loss on the new UI aparrently.
h

helpful-application-7107

01/24/2024, 6:19 PM
Hey. Chance to Win is a Bayesian calculation, the analog (not 1:1) is the p-value in the frequentist engine and is described lower in those docs. You can read more about how it is calculated in the White Paper linked at the top of those docs.
For Bayesian, is it considering loss function and risk?
Expected loss, or risk, is separate from chance to win. They often go hand-in-hand, but not always.
I can't see the expected loss on the new UI aparrently.
It can be found in the metric tooltip, and we use tooltips on Chance to Win to alert when it is high.
b

billowy-window-97200

01/25/2024, 10:39 AM
Thanks @helpful-application-7107, I will take a look into the white paper. About the metric tooltip you mentioned, I can't still find it. Is it the config part inside the metric where we see the risk parameters?
@helpful-application-7107 one more question here: I understand that Chance to Win can be affected by sample size. Does Growthbook has any tool or metric to assist the teams to estimate the sample size needed for the test? Or is there any online calculator for the bayesian version of the experimentation?
h

helpful-application-7107

01/26/2024, 6:42 PM
The tooltip in the results, although I think we only surface it when it is higher than either of your threhsolds.
Does Growthbook has any tool or metric to assist the teams to estimate the sample size needed for the test? Or is there any online calculator for the bayesian version of the experimentation?
Frequentist calculators can be useful for getting a rough ballpark estimate, but as I mentioned in the other thread, we plan on building a dedicated growthbook tool in H1 this year.
🙌 1
4 Views