Hello everyone, I recently started a 50/50 experim...
# experimentation
h
Hello everyone, I recently started a 50/50 experiment on my e-commerce site. I can't figure out why I'm not getting a 50/50 split in my experiment. What are the usual reasons for this kind of behavior ? Thx for you help !
s
There could be a few different reasons. First, how long has the test been running? Do you get any warnings on the Health tab? Finally, what kind of test is it (feature flag/visual/redirect)?
h
Since 3 weeks It's a feature flag
The idea is to test 2 conversion funnels the "old" one and the "new" one
s
This looks like there's something up with the implementation (feature flag in code) that's causing the experiment exposure to be logged incorrectly. Can you share your flag setup?
h
You mean the way we call growthbook or the setup of my flag inside growthbook ?
Are you available for a call today? It would be really helpful for our understanding of the product.
s
I mean how you set it up in your codebase. Are you able to share your code snippets where you set up GrowthBook and make your experiment exposures?
h
yes let me know if this is enought for you
s
There are two things that could be causing issues: 1. The "anonymous" user id. Whenever a user doesn't have a google analytics id, then they'll use anonymous as their user id, meaning that they will all get the same variation. If a large portion of your users don't have this cookie, then that'll definitely be the cause of your issue. To remedy, you should assign them a random ID. We also have a tracking plugin you can use with the SDK that may help. 2. The
loadFeatures()
method is deprecated in favor of
await growthbook.init()
. I'm not sure this is contributing to the issue, but I think there could be some timing issues with your setup. We do have a guide on Vue here: https://docs.growthbook.io/lib/vue
h
Thx for you answer ! Given the difference in volume between 5,000 and 3,000, this would mean that around 20% of my traffic does not have a Google Analytics ID. Does this ratio seem consistent to you? How can I assign a random ID? Do you recommend a particular library? How can this random ID be persistent? Via a Google Analytics fallback cookie, I imagine?
s
What percentage is blocking is hard to say, as it varies widely by audience and other factors. However, it's definitely where I would start debugging. For the persistent ID, you can use our tracking plugin, which will set up a cookie for you. You can also do something yourself using a cookie and
crypto. randomUUID()
.
h
Thx you so much for yout time Ryan ! I let you know if it fix my issue !
s
Please follow up if you're still having trouble.
h
@damp-evening-86702