https://www.growthbook.io/ logo
q

quaint-window-98285

05/04/2023, 4:11 PM
hi all! my company has just started using growthbook and started with a set of 10 A/A tests. we noticed right away that the group assignment does not seem to be independent across the tests. The tracking keys are very similar -
aatest_1
,
aatest_2
, and so forth. is this expected behavior?
here’s a count of users by test group combinations (looking at the first five tests only, across 12 days of testing). 1 indicates experiment group, 0 indicates baseline
f

future-teacher-7046

05/04/2023, 5:36 PM
Hi Chloe. How did you run the A/A tests? Was it using feature flags or inline experiments in the SDK?
q

quaint-window-98285

05/05/2023, 7:42 AM
hi Jeremy. we did it using feature flags
hey @future-teacher-7046, any ideas on what could be going on here? it seems like there are different hashing functions which would avoid this issue - wondering what it would take to switch
@fresh-football-47124 do you have any ideas? sorry to message multiple people, I’m on a bit of a time crunch here 😅
f

future-teacher-7046

05/09/2023, 1:43 PM
Yes, we're in the process of switching feature flags over to the new hashing function. Should be finished this month. I can see if there's a workaround for you in the mean time. Which SDKs are you using?
q

quaint-window-98285

05/09/2023, 2:19 PM
great to hear! we’re using the Go SDK
I think we’ve figured out a workaround for now.
2 Views