Hi everyone. I am happy to see that you implemented bonferroni-holms corrections in your stats engine. However, there’s one question I have. Is there some confidence interval adjustment going on as well? Since we adjust p-values to account for multiple testing, there should also be some adjustment for confidence intervals as well, but I’m not sure I’m seeing one

Hi Yevhen. As the tooltip on the confidence interval column suggests, there is no confidence intervals that match the Holm-Bonferroni correction. Unlike Bonferroni corrections, where you can simply adjust the CIs, this procedure does not have comparable CIs. This is a small cost to pay for the increase in power from the Holm-Bonferroni correction.

Got it, so I just have to be slightly skeptic about those

Yeah it's unfortunate that they're out of alignment, but the alternative is p-values that are unnecessarily high with just the bonferroni correction.