Hi - for one of our tests I get the following repr...
# announcementsg
gorgeous-whale-20294
06/29/2022, 12:46 PMHi - for one of our tests I get the following representation of data, it looks a bit off (risk of choosing = 0% and the graph is not shown) - what could be the issue for that?
gorgeous-whale-20294
06/29/2022, 12:46 PMalso chance of beating is 50%, which does not make sense if the decrease is 29.6%
f
future-teacher-7046
06/29/2022, 2:23 PM
Looks like the number of conversions is really low. 8 vs 13, so it's way too early to draw any conclusions from the data.
future-teacher-7046
06/29/2022, 2:24 PM
The stats kind of break down when you don't have enough data, which is why we set a minimum sample size of 150 by default. In certain circumstances, it's fine to lower it a little bit, but not that much. You can control the minimum sample size in the metric behavior settings.
g
gorgeous-whale-20294
06/29/2022, 4:05 PMmhh… but for all our other tests this did not look like this even though traffic was similarly low…
f
future-teacher-7046
06/29/2022, 4:29 PM
if the number of conversions is very low, the results will be unreliable. So sometimes it may look fine, sometimes it may look like this. We recommend setting a higher threshold for minimum sample size so that the results are more reliable
g
gorgeous-whale-20294
06/29/2022, 4:36 PMOkay, thanks 👍
gorgeous-whale-20294
06/29/2022, 4:37 PMI mean we will wait anyways for at least one week that the data becomes more reliable but it just looked like this for the first time - so just wanted to be sure if this could also be a bug or so - thanks for your help 👍
27 Views