Hi. Are there any plans to allow running one-sided tests with the frequentist engine?

Hi Yevhen, I'll ask our Data Scientist about this once he's online today

Hey Yevhen. We actually have it in the stats engine code but not hooked up in the app. So no immediate plans but it's obviously on our radar. Can you say a bit more about your use case for me?

Hi Luke! We routinely run one-sided experiments (as a matter of fact, more often than two-sided ones). The thing is, most of the time we are not interested if a certain treatment is worse, all we care is if it’s better and if it is not - we get rid of it. We even launch left-sided tests to see if the treatment is actually worse than the control and only reject it if it is (this type of tests is just like a precaution measure for us to make sure there is no significant drop). We’ve been able to use this approach with the Bayesian engine, in a way, thanks to this: https://en.wikipedia.org/wiki/Bernstein%E2%80%93von_Mises_theorem But honestly, I’d be much more comfortable going frequentist all the way, with confidence intervals and all that.