https://www.growthbook.io/ logo
s

some-planet-44104

05/11/2023, 7:57 AM
Hi everyone. I am happy to see that you implemented bonferroni-holms corrections in your stats engine. However, there’s one question I have. Is there some confidence interval adjustment going on as well? Since we adjust p-values to account for multiple testing, there should also be some adjustment for confidence intervals as well, but I’m not sure I’m seeing one
h

helpful-application-7107

05/11/2023, 11:52 AM
Hi Yevhen. As the tooltip on the confidence interval column suggests, there is no confidence intervals that match the Holm-Bonferroni correction. Unlike Bonferroni corrections, where you can simply adjust the CIs, this procedure does not have comparable CIs. This is a small cost to pay for the increase in power from the Holm-Bonferroni correction.
s

some-planet-44104

05/11/2023, 12:51 PM
Got it, so I just have to be slightly skeptic about those
h

helpful-application-7107

05/11/2023, 1:35 PM
Yeah it's unfortunate that they're out of alignment, but the alternative is p-values that are unnecessarily high with just the bonferroni correction.
2 Views