re: bayesian - is there a way for us to tell the engine what are our informed priors? or does the engine just assume an alpha=1,beta=1 prior? ideally we'd like to use the previous x months of conversion data as the prior going into the evaluation (but it would be good if this could also be configured). one of the reasons GB is particularly attractive is its use of bayesian probabilities - but in order for us to maximize the value there we would need to supply the calculation process with our prior distribution data.

Hi Peter. @helpful-application-7107 is our expert on such things.

We recently added support in our Python stats engine for custom priors, but we haven't built out the SQL or front-end to fully support this yet.

thanks for the reply. any idea when this might be released as feature to the front-end? not looking to chase / hassle - there's a trade-off for us whether we change the source to support such a feature, that really depends on whether it's likely to arrive in the release version ~2 months or ~12. If we did implement such a feature, are you interested in receiving pull requests, or it would have to be on a case by case basis, i.e. specific permissions given to folks to implement certain issues, rather than accepting any old PR?

There's the really basic version of this feature where we just give you a text field on the experiment page to manually enter the priors. Then, there's a more advanced version where we query previous data in your warehouse and determine the priors automatically. That first simple manual version is really quick to implement (a few days), but we're not sure how useful it will actually be to users. Have to make sure the pros outweigh the cons before we add this option to the UI. Manually entering priors is very error prone.