Hello! I am facing some issue with growth book configuration with Ga4/BigQuery: 1. I have done integration as mentioned in documents and also was able to identify metrics from GA4 tables, but i am not able to see any data regarding the experiments we are running right now in dashboard. 2. We are using Feature Flag testing Experimentations and Visual Editor Experiments but no data is visible in Dashboard for these two.

Hi, Harshul - Are you seeing any errors when trying to run the experiment analysis? One common issue is the

Experiment Assignment

queries need some customization - we make an attempt to build the default queries, but they don't work all of the time. Specifically, with BigQuery and GA4, the issue is with the

FROM

clause in the Experiment Assignment queries - you need to update the dataset in the from query (by default it's

my_dataset

. To do this, you can go to the data source and edit the Experiment Assignment Queries - when editing, there is a test functionality to know if the updates work as expected.

Hi Michael, I tried two approaches: 1. Integrating BigQuery directly and using a custom data set. a. In this approach GrowthBook wasn’t creating and populating any table on its own. 2. Integrating Via GA4. i.e using tables populated by GA4. a. In this approach I am not seeing any experiment details on experiment dashboard. b. But when i try to find the count a metrics is fired in GA4. That count fetch query is working. We can connect and try to resolve this and see if there is something missing in configuration done on our end.

Hey, Harshul - thanks for following up. When you integrated the SDK code into your codebase, did you customize the

trackingCallback

at all? It sounds like whenever a feature flag/experiment is evaluated, nothing is actually writing to BigQuery. For example, instantiating the GrowthBook object, you'll need to do something like:

trackingCallback: function(experiment, result) {
          // track with GA4, for example:
          gtag("event", "experiment_viewed", {
            event_category: "experiment",
            experiment_id: experiment.key,
            variation_id: result.variationId,
            gb_user_id: gbuuid,
          });
        }

Here is a full how-to on integrating GrowthBook with GA4. We have a section on how to integration the SDK here.

Hi Michael, Yes i am using it the same way you just mentioned. here attaching code snippet for reference.

function startGrowthbook() {
      
      if (!window.growthbook) return;
      console.log(window.growthbook,"window.growthbook")

      var exp = dataLayer[0].all_experiment_objects[0]


      window.gbuuid = window.readCookie('_ga');
      window.gb = new growthbook.GrowthBook({
        apiHost: "<https://cdn.growthbook.io>",
        clientKey: "sdk-XXXXXXXXXXXX",
        // TODO: Add decryptionKey if using encryption
        attributes: {
          id: gbuuid,
          expValue:dataLayer[0].optimize_all_experiment_experiment_id
          //add any other attributes here
        },
        debug:true,
                enableDebugMode:true,

        enableDevMode: true,
        subscribeToChanges: true,
        trackingCallback: function(experiment, result) {
          // track with GA4, for example:
            console.log("experiment-result",experiment,result)
          dataLayer.push( {
                        event:'experiment_viewed',
                        type:'experiment',
                        experiment_id: experiment.key,
                        variation_id: result.variationId,
                      gb_user_id: window.gbuuid,
                    } )
          
         
        }
      });
      window.gb.loadFeatures();


      // TODO: Instrument DOM with AB test logic
      //window.gb.loadFeatures().then(function() {
        // if you want to do anything after GB loads
        Object.keys(window.gb.getFeatures()).map(function(ele){
          console.log(ele+" variation assigend -> "+window.gb.getFeatureValue(ele, "fallback"))
        })
      //});
    }

Thank you! I see the clientKey is

sdk-XXXXXXX

- I take it that was just for copy and pasting the code snippet? If this is installed on a public site, you're welcome to send me the link and I can look at it further. You're welcome to DM it to me if you feel more comfortable.

Sure.

Thanks so much for sending that over. I see two things that could be causing issues. 1. When I go to the link you shared, I do see an error in the console that

window.readCookie

is not a function. So I'm not sure

window.gbuuid

is being set correctly. 2. It looks like you have two accounts. and a few different data sources in each account, but for each one, the

userIdTypes

I see are

userId

and

anonymous_id

, but you're not passing in a

user_id

attribute, just an

id

attribute. So when we query the BigQuery

events_*

table(s) we're looking for the

user_id

column that doesn't exist. A few things to do, can you confirm if a value is being set for

window.gbuuid

? You will either want to update the

trackingCallback

to pass in

user_id

instead of

gb_user_id

or update the

Experiment Assignment Queries

so that instead of looking for

user_id

it looks for

gb_user_id

. And as a health check, if you have access to BigQuery, just checkout some recent

events

tables, and see if there are any events fired for 'experiment_viewed'. I suspect there are events, so we just need to polish the queries GB is using to make sure we're looking for the correct columns.

Hi Michael, 1. Yes the snippet i send you is not complete and just growth book SDK part so remaining part of code is there when it runs on webpage. 2. Yes, i am not passing and

user_id

and

anonymous_id

. 3. But what should be the value of these two attributes is it different from id that i am currently sending as attributes. 4. Also i tried and didn't found any event with experiment_viewed in big query 5.

1. Gotcha - the error I saw (window.readCookie not a function) was on the live site, though. 2. Gotcha. 3. What I would do to simplify things, is update the trackingCallback snippet to something like:

trackingCallback: function(experiment, result)
   {
      // TODO: track experiment impression
      dataLayer.push({
         'event': 'experiment_viewed',
         'user_id': gbuuid,
         'experiment_id': experiment.key,
         'variation_id': result.variationId,
      });
   }

With regards to the attributes, when you're passing in attributes, those are attributes that you can use to define experiment rules: e.g. if county is 'US' show experiment. Then, inside the trackingCallback, GB defaults to using

user_id

. This trackingCallback call essentially adds a row to BigQuery that says "user123, saw variation 1, of this experiment, at this date/time". GB is able to use that info to build the experiment analysis. Currently, we're looking for the

user_id

column in BigQuery, and we don't see it. So, while you can use different attributes for setting attributes and in the trackingCallback, it's best practice to use the same one (its just easier). If you use the trackingCallback that I pasted above, you won't have to update any queries in GrowthBook. • 4. Hmm... that's odd. Sometimes GA4 takes 24 hours to sync data to BigQuery, but I'm guessing that's not the issue here. Can you confirm the

Service Account

has the following permissions? • BigQuery Data Viewer • BigQuery Metadata Viewer • BigQuery Job User

Yes, i will try this configuration also. But is it also possible to use Bigquery as data warehouse .i.e GB will write experiment view and event fire data on it own in custom table and use that data to show traffic and experiment report. Yes i have added these permission to service account.

Not currently - GrowthBook is warehouse native, so all of the data lives on your database(s) and requires you to define the

trackingCallback

accordingly. We don't offer a hosted data solution where we handle writing the data to our own databases. One of our core principles is not building any vendor lock-in into our product, and this warehouse native approach reduces the chances of your user's PII data ever leaving your control.

Let me try and get back to you. Also is it possible for us to do allocation on our end and force a specific with an attribute with specific value to fall under certain experiment behaviour and GrowthBook will show data accordingly. As of now GB is the one who is doing the allocation and users are getting fall under any variant. Do you think its possible where we can control this?

Hey Harshul - to confirm, you're asking if you cant force a specific experiment variation for certain users based on their attributes? Ex: Force all users with the attribute

country === "CA"

to get variation 1? That is not possible with an experiment as the bucketing of users wouldn't be random, and therefore the analysis wouldn't be accurate. When creating an experiment, you can target by attribute, so you can say, only add users to the experiment if their country === 'CA' - and then of those users, GrowthBook will assign them a variation. If you do need to force a certain variation for a user, you can use feature flags with force rules. This allows you to say "show variation 1 to users if their country is 'CA'. This is different from an experiment where GrowthBook will randomly bucket the users. But in that case, since GrowthBook isn't doing the targeting, the trackingCallback isn't fired.

Hi Michael, I checked and data has started flowing in GB dashboard. Its just i verified the data being showed in GB dashboard with GA4/Bigquery. I am seeing some difference. Like for example in GB dashboard i am seeing total users for all my variants as 777 but in bigQuery its 821. There is a difference of 5.3% This is the query i am using.

SELECT count(distinct(user_id)) FROM `trending-pages-edi.analytics_343679482.events_*`,UNNEST(event_params) AS event_param where event_name = 'experiment_viewed' and 
 event_param.key = 'experiment_id' AND REGEXP_CONTAINS(event_param.value.string_value, r'shoppageexp2')

Hey, Harshul - awesome - I'm so glad to hear that data is flowing into BigQuery. To confirm, you're running the above query via GrowthBook and then directly within BigQuery and you're seeing the 5.3% difference?

Yes i am running above query in bigquery console and comparing it with numbers visible in GB dashboard

Gotcha. It's likely that the query we are running for the experiment is doing some additional filtering (by date, segment, or something like that). If you want to send me the experiment id, I can take a deeper look and see if I can find anything. To find it, you can view the experiment in GrowthBook, and grab the ID at the end of the URL. It'll start with

exp_

Sure, here is the id

exp_19g61olsbcryvz

Hmm... I'm not seeing an experiment with that id. Are you self-hosting GrowthBook, or using the cloud version?

I am using cloud version only.

Can you confirm the id again? The id you sent has 18 characters, and usually they contain 21.

This is the only id thats available to me

Thank you - I was able to find it. Without looking at the data, it's hard for me to tell where the 5.3% discrepancy is coming from. There are a few things that it might be. 1. The experiment assignment query does filter by date, to only select rows that were added after the experiment started, and before it ended 2. The experiment assignment specifically eliminates

null

user_id values 3. The experiment assignment query doesn't contain the

regex

matching clause like the query you shared 4. The experiment analysis query (which is a larger query that contains the experiment assignment subquery) runs by default every 6 hours (though, you can always manually refresh it via the GrowthBook UI), so its possible that the 6 hour delay could be responsible for the difference. Again, without having access to the data, it's hard for me to pinpoint what could be causing the difference.

Hi Michael, Just wanted to raise one more issue i am facing. I am running one experiment with visual editor changes. But as i am seeing data in other experiments in this particular one i am not seeing any data. https://app.growthbook.io/experiment/exp_19g61olsbgd8un can you please look into it and let us know what could be the reason for this?

Hey Harshul - of course. It looks like when you created your SDK Connection, the "Include Visual Experiments" toggle wasn't enabled. You'll want to enable that, and then when calling GrowthBook, we'll return visual experiments in the endpoint's response.

Thanks Michael. I can see experiment_viewed event for Visual Editor Experiment. It must update and refresh data in GB dashboard by tomorrow.

Awesome! And yes, now that you can see it, users should start seeing the experiment, and data should start flowing into BigQuery. And once you see data in BigQuery, you can refresh the experiment results in GrowthBook. And, as a reminder, GrowthBook automatically tries to refresh experiment results every 6 hours by default.

Okay will check and let you know in case of any issue.

Hey @mysterious-hospital-86275 @billions-xylophone-11752. Thanks for raising the question nr. 3 "We need some mterics which are used in GA4 like session, bounce rate, session duration etc.". We are facing the same challenge. Did you already found a solution how to implement it? Otherwise i would appreciate an answer of this question aswell :)