https://www.growthbook.io/
Join Slack
hi all! my company has just started using growthbook and started with a set of 10 A/A tests. we noti...
# announcements
q
hi all! my company has just started using growthbook and started with a set of 10 A/A tests. we noticed right away that the group assignment does not seem to be independent across the tests. The tracking keys are very similar - 
aatest_1
, 
aatest_2
, and so forth. is this expected behavior?
here’s a count of users by test group combinations (looking at the first five tests only, across 12 days of testing). 1 indicates experiment group, 0 indicates baseline
AA-Test_Cross-Test_Segment_Sizes,_First_5_2023_05_04.csv
f
Hi Chloe. How did you run the A/A tests? Was it using feature flags or inline experiments in the SDK?
q
hi Jeremy. we did it using feature flags
hey @future-teacher-7046, any ideas on what could be going on here? it seems like there are different hashing functions which would avoid this issue - wondering what it would take to switch
@fresh-football-47124 do you have any ideas? sorry to message multiple people, I’m on a bit of a time crunch here 😅
f
Yes, we're in the process of switching feature flags over to the new hashing function. Should be finished this month. I can see if there's a workaround for you in the mean time. Which SDKs are you using?
q
great to hear! we’re using the Go SDK
I think we’ve figured out a workaround for now.
68 Views
Open in Slack
PreviousNext
Powered by

GrowthBook Users

Open source platform for stress free deployments, measured impact, and smarter decisions.

Powered by