Question for the GrowthBook team: When a page loads on our site and the DOM is ready/fires, GrowthBook gets initialized fairly early in the processes that succeed that event. However, GTag loads asynchronously based on how the browser is loading the various elements we have set to populate per page. GB recommends loading/initializing GB after GTag loads, but we do not have GrowthBook set up in this way currently. The reason for that is our Paywalls and various pop-up modals are also early JS browser loads, and these are where our A/B testing focus currently is as a business. So, when we run an experiment on these JS elements, there’s currently a very slight time delay between the browser serving the base Control to the client, and then the GB trackingCallback determining if a Variant should be served to the client instead, which is unnoticeable to the user since this is currently a fraction of a second. However, if we loaded GrowthBook after the Gtag initializes (which could also be after some of these JS expressions), the Variant replacement time duration could extend and become noticeable to the user depending on how quickly the browser is initializing Gtags and other site elements. Our question is: do we need to wait for Gtag to initialize every time? Is there a different way to implement this so that the experiment-viewed cookie is applied to the user before or independently of Gtag loading? An example for an ongoing experiment: we’re running a paywall test on the site currently where the visual editor is adding an additional button/option to the paywall. We do not specifically have GrowthBook initializing after the Gtag initialization. We noticed in GA that some users were seeing the existing paywall (effectively the Control) but did not have the experiment-viewed cookie assigned in GA and therefore did not get served either variation of the experiment from the perspective of GrowthBook. We believe this is because a race condition is happening here, where most of the time Gtag is loading first and then GrowthBook is assigning the experiment-viewed cookie, serving a variation, and is then tracking results, but some of the time Gtag doesn’t load ahead of GrowthBook and so GrowthBook does not assign the experiment-viewed cookie or serve a variation at all. Please let us know your thoughts on how we can adjust for these excluded users @fresh-football-47124 @helpful-application-7107 cc @refined-addition-9216 @gorgeous-electrician-7828

Hi Matt - The reason we suggest waiting is so you can use the Google Client ID.

Thanks @fresh-football-47124. Relatedly, we're noticing that because of this issue, the trackingCallback wasn't always sending the

experiment-viewed

event to GA since in some instances the

gtag()

wasn't loaded yet, which we're adjusting. However, we can still see in the GB dash that there are more reported users in the experiment population cohorts and higher reported Metrics than we're able to see in GA. Is GB logging this anywhere we can access? And, if so, would we be able to retroactively fill GA for the users GB has accounted for but are missing in GA as a result of the

gtag()

not loading fast enough for these missing users?

the data is coming from BigQuery

In a perfect world these numbers would match. But I guess the question is as it relates to a specific example: GrowthBook dash reports 73.9k users were exposed to our Variation for an experiment. As an example, we have a trial_starts Metric attached to this experiment. GB dash reports 163 trial_starts for this Variant. When gathering data to compare from GA, the number of users exposed to this Variation is 69.9k, and trial_starts are 93. If all data is stored in Big Query, then GB and GA should have equal reporting. But, GB is serving the experiment to all users, and this is independent of the trackingCallback missing the

experiment-viewed

event send to GA from the

window.gtag

load, so what causes that large of a discrepancy? Does GrowthBook store experiment data in Google Big Query in more ways than via the

window.gtag

?

no, GrowthBook is modular, and the experiment reporting is entirely based off your data (BigQuery)

What's the mechanism by which BigQuery collects information like

variation_id

from GrowthBook?

from the trackingCallback method

Have you seen other customer instances of GB reporting higher numbers than a GA query for certain metrics?

its usually lower, as we do some deduping and data quality checks

That's what I would expect as well, having it report near double a GA query is puzzling

we give the raw query, you could run it yourself and debug it if you like

We also have our Metrics queries running a date range as

BETWEEN '{{startDateISO}}' AND '{{endDateISO}}'

In an instance where we set up a Phase 2 for an existing experiment, I assume the results of our Metrics in the dash automatically adjust for the date range we have set for the Phase selected?

If I select Phase 2 in the results tab and Update Data, the results for the Metrics tracked should be from Dec 15th-today, right?

Yes. I suggest you look at the queries we produce, by clicking the three dot menu on the results tab and clicking View Queries. There you'll see some of our date filtering logic and how our queries pull data from BQ to produce the numbers in the dash. I'm happy to help you understand them if you have questions.

Perfect, thanks! Very helpful, exactly what I was looking for