https://www.growthbook.io/
Join Slack
We're at a loss facing multiple exposure warning (no SRM) across a couple of sites. It's from 2%-25%...
# ask-questions
a
We're at a loss facing multiple exposure warning (no SRM) across a couple of sites. It's from 2%-25%, seemingly getting bigger over the life-time of a test . It shows in a segment overlap in GA4 explorations already before the BigQuery integration sends data to GrowthBook and gives us the additional warning. We've checked: • No difference between new and returning users • Adding the gbuiid value (hashed id + variant id) to a custom dimension - still showing multiple exposure • All browsers and device categories seem similarly affected • GTM forwards data exactly per documentation with consent mode active. • Can't provoke wrong dataLayer-values, e.g. changing variant_ids when I clear cache/reload etc. • Sticky bucketing is not activated (saw in a couple of posts that a wrong config here might cause the issue). We're using the HTML SDK, allocating traffic via the default id. We haven't configured the SDK, so it's defaulting to setting it's own id as far as the documentation goes. Both sites we have with GrowthBook are Single Page Applications. Ping me in a DM if domains are needed to support on this! But alas, still multiple exposure. What do we do as a last hurrah and continue the trouble shoot? The only mitigating factor that doesn't blow this right up is that the multiple exposure is seemingly totally random with no SRM in the groups.
We track the value of the 
gbuuid
as a event parameter in the 
experiment_viewed
GA4 event and use the following BigQuery query to analyze the number of users that are exposed to mutiple variations.
Copy code
WITH events AS (
  SELECT     
    user_pseudo_id,
    (select value.string_value from unnest(event_params) where key = 'gbuuid') as gbuuid,
    (select value.string_value from unnest(event_params) where key = 'experiment_id') as experiment_id,
    (select value.string_value from unnest(event_params) where key = 'variation_id') as variation_id,
  FROM `analytics_12345.events_202411*`
  WHERE event_name = "experiment_viewed"
  AND privacy_info.analytics_storage = "Yes"  
),

variation_count AS (
  SELECT    
    user_pseudo_id,
    gbuuid,
    experiment_id,
    COUNT(DISTINCT variation_id) variations
  FROM events
  GROUP BY 1,2,3
)

SELECT
  experiment_id,
  variations,
  COUNT(DISTINCT user_pseudo_id) ga4_users,
  COUNT(DISTINCT gbuuid) gb_users
FROM variation_count
GROUP BY 1,2
f
Hi Klaus
checking with the team
Multiple exposure warnnigs will happen if the GA4 attribute does not match the gbuiid (or gbuuid)
sounds like you're splitting based on the gbuuid
but doing analysis on the GA4 client ID
if the cookies (GA4's and GrowthBooks) don't live for the same amount of time, this can happen
so it could mean that a single GA4 user can have multiple GrowthBook IDs
perhaps the cookie consent is deleting cookies - like it's assigning to a variant, but somehow deleting the cookies after/during consent, and then generating a new UUID for them in GB. But GA4 tracks them as the same person, and hence the multiple exposures
that's our theory anyway
a
@hundreds-easter-23383
h
@fresh-football-47124 Hey, thanks for your thoughts on this issue. yes, we split on the 
gbuuid
and then we track the value of the 
gbuuid
cookie as an event parameter in the 
experiement_viewed
GA4 event, to see how well it matches the GA4 cookie id. Consent should not be an issue. On this client, the Growthbook SDK is loaded from GTM when analytics consent is given and we only query consented data from GA4 in BigQuery. This is the result of the above query, which shows the number of users (based on the GA4 and the 
gbuuid
ID) exposed to 2 variants. The share of users is almost identical regardless of which user ID we look at.
w
@worried-monitor-77515 misschien kunnen we hier nog iets mee? Hebben nooit reactie meer gehad op onze multiple exposure warning.
h
@fresh-football-47124 Hey, please take a look at this export for one specific user. It is the same 
user_pseudo_id
but 
gbuuid
changes when the user return on the next day. The users gets a new variation on the 
find-store-add-information-click-collect
experiment. The result is that the GA4 user has 2 variations while the GB user only has 1 variationer per user. In reality it is the same user, which means the same user has seen 2 variations even though that is not clear when only looking at the 
gbuuid
cookie. So either the 
gbuuid
cookie has been deleted or GB has generated a new ID for this user. Please let me know what you think?
@fresh-football-47124 This is the total number of GA4 users with multiple 
gbuuid
This suggests that it is not an isolated case, but something that happens for 1,6% of users.
a
Through the percentage tends to increase through the life-time of the test
f
Hi Jacob
that last table, what is that gbuuid_count?
h
The number of DISTINCT gbuuid's per 
user_pseudo_id
Copy code
WITH events AS (
  SELECT     
    user_pseudo_id,    
    (select value.string_value from unnest(event_params) where key = 'gbuuid') as gbuuid,
    (select value.string_value from unnest(event_params) where key = 'experiment_id') as experiment_id,
    (select value.string_value from unnest(event_params) where key = 'variation_id') as variation_id,
  FROM `analytics_253505787.events_*`
  WHERE event_name = "experiment_viewed"  
  AND privacy_info.analytics_storage = "Yes"
  AND _table_suffix >= '20241206'
),

gbuuid_count AS (
  SELECT    
    user_pseudo_id,    
    COUNT(DISTINCT gbuuid) gbuuids,    
  FROM events
  GROUP BY 1
)

SELECT
  gbuuids AS gbuuid_count,
  COUNT(DISTINCT user_pseudo_id) ga4_users
FROM gbuuid_count
GROUP BY 1
ORDER BY 1 ASC
a
we take the value from gbuiid-cookie and send it as an additional parameter on the 'experiment_viewed'-event in GTM
f
okay
h
Here is our GTM setup.
f
can you share the implementation code you're using ?
h
Sure, I'll send the website in a DM.
✅ 1
SDK is hardcoded and placed before GTM in the code.
67 Views
Open in Slack
PreviousNext
Powered by

GrowthBook Users

Open source platform for stress free deployments, measured impact, and smarter decisions.

Powered by