This message was deleted.

If a test is flat, chance to beat control will hover around 50% forever, but risk will approach zero the more data you collect. So for inconclusive tests, you can wait until the risk is low enough and then just make a decision one way or the other and move on. There are usually lots of external factors in making a decision so there's no hard and fast rule for this.

I see, that’s useful to know that risk will lower over time in that case. I’m sure we’ll build a better intuition of these metrics over time. One thing that would be incredibly useful to see, and that we would be happy to contribute to, is a set of post-mortems / examples of experiments people have run, how they’ve designed them, how they chose conversion delay / window, how they analyzed the results, etc. I felt like I had built up that intuition deeply in the frequentist world, but since I’m new to the Bayesian metrics, I’m having trouble reasoning through all the nuances. Are there resources you’d recommend for “real world Bayesian A/B testing” with stories like this from other companies?