Hello all, I am curious as to why the displaying of results is tied to the number of conversions, rather than the number of total samples (exposures) in the experiment? And I wonder if it's possible to modify this somehow, so that results are not shown below some specified threshold of exposures irregardless of the number of conversions?

This is by design and intended to prevent premature conclusions when there is insufficient data, as statistical results can be highly unstable with very few conversion events (even if the total number of users is high). I'll check with our Data Scientists to see if they have any other input!

This is reasonable; however, one would at times also may like to hide the results if the total sample size is small.

Isn't this happening already by the fact that a small sample size will also equal few conversions? Another aspect of this that you might be interested in is the Experiment Decision Framework. Here, it's not about hiding results, but being able to provide totally customizable criteria for interpretation. You can set up your own rules that will guide interpretation recommendations.

It's not necessarily happening already. Example: You have a baseline rate of some metric at 5%. You launch an experiment with the default MDE of a 10% relative uplift. To achieve the standard power of 80% to detect a different between control/intervention, one would need roughly 31 000 users in each group. However, with the default "display results" threshold at 150, you would begin to see results already at 150/0.05 = 3000 users in each group, i.e. one tenth of the necessary population. Consequently, results that are underpowered are being displayed which means that they are susceptible to type 2, type S and type M errors.

Ah, gotcha. That makes sense