We have a fresh GB setup using JavaScript SDK and ...
# experimentation
w
We have a fresh GB setup using JavaScript SDK and GA4/BQ as the data source. Just finished an A/A test, and it shows a significant lift for all 3 key metrics for V1. No SRM or other anomalies. Looking into the segments, the only thing I noticed was that while V1 has noticeably more iOS users, Control has more Windows users. Any ideas what could be the problem here? Definitely running a new A/A test, but just thought maybe someone has been here before.
Screenshot 2025-06-03 at 10.11.48.png
h
The main thing is that these metrics are strongly correlated so it is really more like one false positive.
I would re-run it, and even consider running a second in parallel.