https://www.growthbook.io/ logo
#ask-questions
Title
# ask-questions
p

purple-winter-48671

12/15/2023, 12:17 AM
Hello! I've found that quite a few customers in an experiment have been assigned multiple variants (which was helpfully flagged on the UI thankyou ), but in digging into it I've identified that customers are getting multiple variants if they are associated with multiple anonymous_ids. The experiment and feature are set to assign based on customer_id, so I'm quite confused as to why this is happening. Screenshots and an example in the thread.
Screenshot 2023-12-15 at 1.17.47 PM.png,Screenshot 2023-12-15 at 1.17.30 PM.png,Screenshot 2023-12-15 at 1.05.30 PM.png
f

fresh-football-47124

12/15/2023, 12:18 AM
where is the anonymous_id coming from?
p

purple-winter-48671

12/15/2023, 12:21 AM
From Rudderstack
It's an external ID that we use to stitch customers outside the product to authenticated users
f

fresh-football-47124

12/15/2023, 12:21 AM
and the customer id?
p

purple-winter-48671

12/15/2023, 12:22 AM
That's an internal ID that we generate (or call) when a user authenticates
f

fresh-football-47124

12/15/2023, 12:22 AM
I wonder if an adblock is causing the rudderstack cookie to not be as sticky as your internal cookies
what percentage of your users had this?
p

purple-winter-48671

12/15/2023, 12:24 AM
About 5% currently, but it is increasing overtime (likely due to more opportunities to be assigned another variant)
f

fresh-football-47124

12/15/2023, 12:24 AM
one thing you can check is the cookie length
I wonder if rudderstacks cookie is more short lived
p

purple-winter-48671

12/15/2023, 12:25 AM
But shouldn't the Growthbook assignment not even know/care about the anonymous_id when assigning, given we've said to assign based on customer_id?
f

fresh-football-47124

12/15/2023, 12:26 AM
the fact that you're getting this multiple assignment error means that you're not doing that
are you sure that youre not assigning by anon id and doing the report by customer id?
p

purple-winter-48671

12/15/2023, 12:27 AM
Given the feature in product says split users by customer_id, I assumed that would mean assigning by customer_id - is there another place we should check?
Screenshot 2023-12-15 at 1.27.55 PM.png
f

fresh-football-47124

12/15/2023, 12:28 AM
on the top of the experiment reports
you should see a bar, that has some info about the experiment data
to the right of "Analysis settings"
p

purple-winter-48671

12/15/2023, 12:30 AM
That also shows logged in visitors (meaning with customer_id)
I wonder if perhaps the Growthbook assignment is being called in a race condition at login, and perhaps the customer_id isn't always available when it's called - would it default to the alternative assignment ID if the customer_id is null?
f

fresh-football-47124

12/15/2023, 12:32 AM
if you call our sdk to get a value of a feature flag that includes an experiment on an attribute which is not defined, it will return the default value, but the trackingCallback is not going to be fired
p

purple-winter-48671

12/15/2023, 12:33 AM
hm strange so shouldn't be assigning based on the other attribute even in absence of customer_id
f

fresh-football-47124

12/15/2023, 12:34 AM
sometimes users can get this with sdk setups like id = some_id || another _id
that can cause this
p

purple-winter-48671

12/15/2023, 12:54 AM
I will confirm the implementation and keep you posted!
So it does not appear that we're ever sending the anonymous_id in place of or as a default or alternative for the customer_id... any other suggestions of where we might look to see why it seems to be using that alternative id?
Is it possible that because we've set both as identifiers in Growthbook (broadly, not for this experiment) it is concatenating them somewhere along the way? and then assigning based on both?
f

fresh-football-47124

12/15/2023, 9:17 AM
it might be, if you added a metric that is marked for anon_id, and the experiment is customer id, it will use the join query
3 Views