Visualisation of “gained confidence” I think to mitigate the early peeking problem, but at the same time allow for early stopping of tests it would be great to have a visualisation of the uplift over time as more data comes in. This would allow to see when test results stabilises, but at the same time also highlight (likely) false-positives due to fluctuations in the data

Hi Lukas - we'll take a look - we've had requests for this in the past. Our concern was users might look for trends and call tests, but your ui makes that more clear

It is definitely a tradeoff 👍 Additionally there could be a warning if the sample size was not yet reached or maybe an overlay in the visualisation for the days where that is the case 🤔