Hi Luke. Yet, it was lower than that. However, the tag disappeared but I’m still seing this “Chance to beat control” that is suspiciously stuck at 50%,
some-planet-44104
04/14/2023, 8:40 AM
it haven’t moved at all since the beginning of the experiment
h
helpful-application-7107
04/14/2023, 1:47 PM
This is probably the result of our particular bayesian approach causing problems when a group mean could potentially being negative (large variance and enough of the posterior is negative) and the 50% is a default return value when we can't compute uncertainty accurately. I can confirm this if you share the users, sums and sums of squares from the raw query.
We're aware just returning 50% is a bad user experience and improving it is our intention.