Hi ๐
I'm new to GrowthBook and I wasn't able to find in the documentation how does it handle multiple primary metrics for the Bayesian Approach? Is there something similar for the Bonferroni correction for Bayesian Metrics?
Thanks in advance! ๐
f
fresh-football-47124
05/16/2023, 3:35 PM
Hi Pedro, we donโt do multiple testing corrections for Bayesian currently
p
plain-thailand-11534
05/16/2023, 5:02 PM
Thanks for the answer. Out of curiosity, do you have plans to implement corrections on multiple metrics/hypotheses for Bayesian or is there a good reason to not adjust them?
f
fresh-football-47124
05/16/2023, 5:07 PM
@helpful-application-7107 ^?
h
helpful-application-7107
05/16/2023, 5:11 PM
There aren't immediate plans because there isn't a direct analog in the Bayesian framework. The Bayesian systems that traditionally help deal with comparing many things all at once require additional scaffolding about the relationship between tests and assuming priors for those relationships, which is somewhat more involved.
Some people would argue that multiple comparisons aren't a problem for Bayesian analysis, but I would say those arguments are mostly theoretical, and practically one should indeed be concerned with this. Unfortunately building a general system to handle this is harder for Bayesian analysis than for frequentist analysis.