I'm running my first experiment using GB. Last week, the experiment showed some statistically significant results, and then after the audience increased, the results became not statistically significant any more...is it because of the i*ncreased noise?* As more users use our product, there may be increased variability in user behavior, which can lead to decreased statistical power and reduced detectability of significant effects? Thank you
f
fresh-football-47124
07/17/2024, 9:16 PM
There are a number of effects that can cause this - could just be random variations, could be that the sample size originally was too low- but could also be something like a novelty or primacy effect.
f
fresh-piano-28194
07/17/2024, 9:19 PM
Thanks @fresh-football-47124 I increased the sample size...will see how it goes.