Hi team, I have noticed that for experiments where the conversion rate is very small, plotting metrics by date cohorts can result in the following graph, where for some dates the chance to win seems to be miscalculated to 50% . After investigating your codebase, I have realised that this is the expected behaviour due to this. I was wondering if there is any strong motivation behind returning the default output rather than performing the full calculation and showing a warning message through the UI?

Hey Manuel. You're right that this is unsatisfactory. We're investigating moving our entire bayesian testing apparatus for mean metrics over to a different model that won't suffer this same problem.

Great thanks @helpful-application-7107