Hey all, We just started our first real experimen...
# experimentation
m
Hey all, We just started our first real experiment last night after a successful A/A test. The A/B experiment targets desktop only, whereas the A/A test was all users. This is the only difference we can see between them, yet the A/B test is receiving no data at all. I've tried making changes to the A/A test so as not to make things worse with the A/B test, and it just got stranger: • Added attribute-targeting for desktop only in a new phase - the experiment went unhealthy with a 77/23% split, and if I changed the dimension to device I see data for phone and tablet • Changed the attribute-targeting to chrome only in a new phase - the experiment health changed to a 98/2% split! Viewing with the dimension to browser showed data for other browsers • Removed the attribute-targeting completely - the experiment was still unhealthy with a 54/46% split • Created a new phase with no changes - healthy! Each time I created a new phase I re-randomised traffic. We are at a loss for what is causing this, as nothing seems to be a constant
s
Yeah, that's weird. Are you able to share some more details? What kind of experiment is it? How are you doing tracking? How are you doing attributes?