Hi! Reading through the docs, I see the frequentist engine uses 2 tailed tests. Is there an option to run a 1 tail test or is there a reason this would not be best practice?

I've asked this question before here in Slack. Didn't get an answer, but you'll have to settle with 2 tailed unfortunately. Depending on your hypothesis (IMO) 1 tailed is most suited for a strong majority of use cases since you are usually interested in improving metrics and would not ship if they did not do so. For Guardrail metrics however I think 2 tailed is suitable. I believe that GrowthBook has a page where you can do feature requests, but I don't remember where. I didn't do it though, but if you manage to find out where i'll upvote it!

thanks for the reply and context! I created a ticket here, feel free to add/react to it! I think I might just run my own 1 tail test in bigquery when I really want to for now, but repeating this for every metric/test is obviously not ideal. hopefully it's already on their radar!

Thanks for submitting a request! I've reacted to it now. Me personally would not go through the hassle with you method hehe, but that's of course up to you to decide. Lets cross our fingers 🤞🏿

Hey, Lead Data Scientist at GrowthBook here. Definitely a reasonable suggestion, and thanks for opening an issue. You can even see in our stats engine we have one-tailed tests (https://github.com/growthbook/growthbook/blob/main/packages/stats/gbstats/frequentist/tests.py#L211-L230), but we haven't hooked it up to the app and settled on a UI for it.