hi all! my company has just started using growthbook and started with a set of 10 A/A tests. we noticed right away that the group assignment does not seem to be independent across the tests. The tracking keys are very similar -
, and so forth. is this expected behavior?
here’s a count of users by test group combinations (looking at the first five tests only, across 12 days of testing). 1 indicates experiment group, 0 indicates baseline
Hi Chloe. How did you run the A/A tests? Was it using feature flags or inline experiments in the SDK?
05/05/2023, 7:42 AM
hi Jeremy. we did it using feature flags
hey @future-teacher-7046, any ideas on what could be going on here? it seems like there are different hashing functions which would avoid this issue - wondering what it would take to switch
@fresh-football-47124 do you have any ideas? sorry to message multiple people, I’m on a bit of a time crunch here 😅
05/09/2023, 1:43 PM
Yes, we're in the process of switching feature flags over to the new hashing function. Should be finished this month. I can see if there's a workaround for you in the mean time. Which SDKs are you using?