Hi, I'm confused about the randomisation in my tes...
# ask-questionsr
rough-twilight-98762
03/17/2025, 8:27 AMHi, I'm confused about the randomisation in my test setup. So for example, I started 3 similar A/A tests within a few minutes, and now for all my tests I get more or less identical results, which makes me believe all users are assigned to the same variant in all three tests. Why is that?
I'm using pseudo id as identifier in the experiment assignment query
f
fresh-football-47124
03/17/2025, 10:01 AM
are you using the same experiment tracking key?
r
rough-twilight-98762
03/17/2025, 10:50 AMhey, no. If that is the same as "Experiment key"
they are similar, but differ like this:
• aa-13mars
• aa-13mars-2
• aa-13mars-3
f
fresh-football-47124
03/17/2025, 12:41 PM
are you using v1 assigning hash?
r
rough-twilight-98762
03/17/2025, 12:46 PMehm, no idea. Where can I find the answer to that question?
t
thankful-postman-39410
03/17/2025, 12:49 PM@fresh-football-47124 we are running the html script tag
r
rough-twilight-98762
03/17/2025, 1:06 PMit seems like growthbook is using some kind of fallback universal id, instead of pseudo id (which is in the ga cookie)
could this be the issue?
f
fresh-football-47124
03/17/2025, 1:50 PM
how many users?
fresh-football-47124
03/17/2025, 1:50 PM
the ga cookie is often too slow on initial load
fresh-football-47124
03/17/2025, 1:52 PM
the script tag served from our cdn will be the latest- and using v2 hashing
fresh-football-47124
03/17/2025, 1:52 PM
can you share the code you're injecting?
t
thankful-postman-39410
03/17/2025, 3:01 PMUsers: approx 55 000 in each AA-test
Script:
<script async
data-client-key="xxxxxxxxxxxxxx"
src="https://cdn.jsdelivr.net/npm/@growthbook/growthbook/dist/bundles/auto.min.js"
>/script
thankful-postman-39410
03/17/2025, 3:01 PMWe are running AA-tests so no code in the experiments.
r
rough-twilight-98762
03/18/2025, 12:06 PM@fresh-football-47124 hi, please follow up on this
f
fresh-football-47124
03/18/2025, 4:32 PM
hi - the experiment tracking key is hashed with the user id (in this case a random uuid will be generated as you havent' specified one) - and that gets a value from 0 to 1 that is used for assignment. What we would want to check is if the assignment in fact identical. Are you able to confirm they assignments are the same for each of the experiments?
r
rough-twilight-98762
03/19/2025, 8:27 AMyes, they are identical
rough-twilight-98762
03/19/2025, 12:47 PMwe changed the experiment keys, separating them in a clearer way between experiments, and it seems to be working better now.
But one thing I dont really understand: if I use user_pseudo_id for analyzing experiment data with BigQuery, do I also need to use user_pseudo_id for hashing in GrowthBook to keep the assignment consistent? or can I expect it to work using the standard html script tag without any attribution changes?
rough-twilight-98762
03/19/2025, 12:47 PM@fresh-football-47124
15 Views