Hi, we run Build: f58a91e (2023-03-31). When using the frequentist statistics, we still get the Bayesian behaviour for the metrics (see screenshot). This means we can’t change the critical p-value for our results to be considered significant, as one could in previous versions. Is anyone else observing this behaviour?

Update: I can still set the critical p-vaue on the global level in the gerneal settings, but not for the individual metrics.

Yes, that's a limitation of our settings. However, the analog to the critical p-value threshold in the bayesian engine is also only configurable at the global level (Confidence Level). The Sample Size and Max/Min Percent Change Thresholds apply to both engines, while the Risk thresholds only apply to the Bayesian engine

Ahoi @helpful-application-7107. thanks for the answer! THisBased on your input, there is a difference between the expected behaviour and how the current GrowthBook version behaves. The risk thresholds for metrics are still configurable even when using the frequentist engine. I created a bug ticket for this, since I didn’t address it explicitly my original message.

I'll respond in the ticket. I agree it may not be the best user experience, but I'm not sure I would categorize this as a bug.

OK, I’ll follow the comments on the ticket.