https://www.growthbook.io/ logo
#ask-questions
Title
# ask-questions
m

mysterious-hospital-86275

02/09/2024, 4:52 AM
Hello! I am facing some issue with growth book configuration with Ga4/BigQuery: 1. I have done integration as mentioned in documents and also was able to identify metrics from GA4 tables, but i am not able to see any data regarding the experiments we are running right now in dashboard. 2. We are using Feature Flag testing Experimentations and Visual Editor Experiments but no data is visible in Dashboard for these two.
b

billions-xylophone-11752

02/09/2024, 1:32 PM
Hi, Harshul - Are you seeing any errors when trying to run the experiment analysis? One common issue is the
Experiment Assignment
queries need some customization - we make an attempt to build the default queries, but they don't work all of the time. Specifically, with BigQuery and GA4, the issue is with the
FROM
clause in the Experiment Assignment queries - you need to update the dataset in the from query (by default it's
my_dataset
. To do this, you can go to the data source and edit the Experiment Assignment Queries - when editing, there is a test functionality to know if the updates work as expected.
m

mysterious-hospital-86275

02/12/2024, 3:41 AM
Hi Michael, I tried two approaches: 1. Integrating BigQuery directly and using a custom data set. a. In this approach GrowthBook wasn’t creating and populating any table on its own. 2. Integrating Via GA4. i.e using tables populated by GA4. a. In this approach I am not seeing any experiment details on experiment dashboard. b. But when i try to find the count a metrics is fired in GA4. That count fetch query is working. We can connect and try to resolve this and see if there is something missing in configuration done on our end.
b

billions-xylophone-11752

02/12/2024, 11:57 AM
Hey, Harshul - thanks for following up. When you integrated the SDK code into your codebase, did you customize the
trackingCallback
at all? It sounds like whenever a feature flag/experiment is evaluated, nothing is actually writing to BigQuery. For example, instantiating the GrowthBook object, you'll need to do something like:
Copy code
trackingCallback: function(experiment, result) {
          // track with GA4, for example:
          gtag("event", "experiment_viewed", {
            event_category: "experiment",
            experiment_id: experiment.key,
            variation_id: result.variationId,
            gb_user_id: gbuuid,
          });
        }
Here is a full how-to on integrating GrowthBook with GA4. We have a section on how to integration the SDK here.
m

mysterious-hospital-86275

02/12/2024, 12:02 PM
Hi Michael, Yes i am using it the same way you just mentioned. here attaching code snippet for reference.
Copy code
function startGrowthbook() {
      
      if (!window.growthbook) return;
      console.log(window.growthbook,"window.growthbook")

      var exp = dataLayer[0].all_experiment_objects[0]


      window.gbuuid = window.readCookie('_ga');
      window.gb = new growthbook.GrowthBook({
        apiHost: "<https://cdn.growthbook.io>",
        clientKey: "sdk-XXXXXXXXXXXX",
        // TODO: Add decryptionKey if using encryption
        attributes: {
          id: gbuuid,
          expValue:dataLayer[0].optimize_all_experiment_experiment_id
          //add any other attributes here
        },
        debug:true,
                enableDebugMode:true,

        enableDevMode: true,
        subscribeToChanges: true,
        trackingCallback: function(experiment, result) {
          // track with GA4, for example:
            console.log("experiment-result",experiment,result)
          dataLayer.push( {
                        event:'experiment_viewed',
                        type:'experiment',
                        experiment_id: experiment.key,
                        variation_id: result.variationId,
                      gb_user_id: window.gbuuid,
                    } )
          
         
        }
      });
      window.gb.loadFeatures();


      // TODO: Instrument DOM with AB test logic
      //window.gb.loadFeatures().then(function() {
        // if you want to do anything after GB loads
        Object.keys(window.gb.getFeatures()).map(function(ele){
          console.log(ele+" variation assigend -> "+window.gb.getFeatureValue(ele, "fallback"))
        })
      //});
    }
b

billions-xylophone-11752

02/12/2024, 12:05 PM
Thank you! I see the clientKey is
sdk-XXXXXXX
- I take it that was just for copy and pasting the code snippet? If this is installed on a public site, you're welcome to send me the link and I can look at it further. You're welcome to DM it to me if you feel more comfortable.
m

mysterious-hospital-86275

02/12/2024, 12:06 PM
Sure.
Yes its for copy purposes. dev1.editorialisttest.com/shop/gucci/
👍 1
I integrated sdk using gtm tag
b

billions-xylophone-11752

02/12/2024, 12:35 PM
Thanks so much for sending that over. I see two things that could be causing issues. 1. When I go to the link you shared, I do see an error in the console that
window.readCookie
is not a function. So I'm not sure
window.gbuuid
is being set correctly. 2. It looks like you have two accounts. and a few different data sources in each account, but for each one, the
userIdTypes
I see are
userId
and
anonymous_id
, but you're not passing in a
user_id
attribute, just an
id
attribute. So when we query the BigQuery
events_*
table(s) we're looking for the
user_id
column that doesn't exist. A few things to do, can you confirm if a value is being set for
window.gbuuid
? You will either want to update the
trackingCallback
to pass in
user_id
instead of
gb_user_id
or update the
Experiment Assignment Queries
so that instead of looking for
user_id
it looks for
gb_user_id
. And as a health check, if you have access to BigQuery, just checkout some recent
events
tables, and see if there are any events fired for 'experiment_viewed'. I suspect there are events, so we just need to polish the queries GB is using to make sure we're looking for the correct columns.
I've got to step away for a bit, but I should be back in ~2 hours.
m

mysterious-hospital-86275

02/12/2024, 2:48 PM
Hi Michael, 1. Yes the snippet i send you is not complete and just growth book SDK part so remaining part of code is there when it runs on webpage. 2. Yes, i am not passing and
user_id
and
anonymous_id
. 3. But what should be the value of these two attributes is it different from id that i am currently sending as attributes. 4. Also i tried and didn't found any event with experiment_viewed in big query 5.
b

billions-xylophone-11752

02/12/2024, 3:20 PM
1. Gotcha - the error I saw (window.readCookie not a function) was on the live site, though. 2. Gotcha. 3. What I would do to simplify things, is update the trackingCallback snippet to something like:
Copy code
trackingCallback: function(experiment, result)
   {
      // TODO: track experiment impression
      dataLayer.push({
         'event': 'experiment_viewed',
         'user_id': gbuuid,
         'experiment_id': experiment.key,
         'variation_id': result.variationId,
      });
   }
With regards to the attributes, when you're passing in attributes, those are attributes that you can use to define experiment rules: e.g. if county is 'US' show experiment. Then, inside the trackingCallback, GB defaults to using
user_id
. This trackingCallback call essentially adds a row to BigQuery that says "user123, saw variation 1, of this experiment, at this date/time". GB is able to use that info to build the experiment analysis. Currently, we're looking for the
user_id
column in BigQuery, and we don't see it. So, while you can use different attributes for setting attributes and in the trackingCallback, it's best practice to use the same one (its just easier). If you use the trackingCallback that I pasted above, you won't have to update any queries in GrowthBook. • 4. Hmm... that's odd. Sometimes GA4 takes 24 hours to sync data to BigQuery, but I'm guessing that's not the issue here. Can you confirm the
Service Account
has the following permissions? • BigQuery Data Viewer • BigQuery Metadata Viewer • BigQuery Job User
m

mysterious-hospital-86275

02/12/2024, 3:33 PM
Yes, i will try this configuration also. But is it also possible to use Bigquery as data warehouse .i.e GB will write experiment view and event fire data on it own in custom table and use that data to show traffic and experiment report. Yes i have added these permission to service account.
b

billions-xylophone-11752

02/12/2024, 3:39 PM
Not currently - GrowthBook is warehouse native, so all of the data lives on your database(s) and requires you to define the
trackingCallback
accordingly. We don't offer a hosted data solution where we handle writing the data to our own databases. One of our core principles is not building any vendor lock-in into our product, and this warehouse native approach reduces the chances of your user's PII data ever leaving your control.
m

mysterious-hospital-86275

02/13/2024, 6:47 AM
Let me try and get back to you. Also is it possible for us to do allocation on our end and force a specific with an attribute with specific value to fall under certain experiment behaviour and GrowthBook will show data accordingly. As of now GB is the one who is doing the allocation and users are getting fall under any variant. Do you think its possible where we can control this?
b

billions-xylophone-11752

02/13/2024, 12:03 PM
Hey Harshul - to confirm, you're asking if you cant force a specific experiment variation for certain users based on their attributes? Ex: Force all users with the attribute
country === "CA"
to get variation 1? That is not possible with an experiment as the bucketing of users wouldn't be random, and therefore the analysis wouldn't be accurate. When creating an experiment, you can target by attribute, so you can say, only add users to the experiment if their country === 'CA' - and then of those users, GrowthBook will assign them a variation. If you do need to force a certain variation for a user, you can use feature flags with force rules. This allows you to say "show variation 1 to users if their country is 'CA'. This is different from an experiment where GrowthBook will randomly bucket the users. But in that case, since GrowthBook isn't doing the targeting, the trackingCallback isn't fired.
m

mysterious-hospital-86275

02/14/2024, 6:48 AM
Hi Michael, I checked and data has started flowing in GB dashboard. Its just i verified the data being showed in GB dashboard with GA4/Bigquery. I am seeing some difference. Like for example in GB dashboard i am seeing total users for all my variants as 777 but in bigQuery its 821. There is a difference of 5.3% This is the query i am using.
Copy code
SELECT count(distinct(user_id)) FROM `trending-pages-edi.analytics_343679482.events_*`,UNNEST(event_params) AS event_param where event_name = 'experiment_viewed' and 
 event_param.key = 'experiment_id' AND REGEXP_CONTAINS(event_param.value.string_value, r'shoppageexp2')
b

billions-xylophone-11752

02/14/2024, 2:22 PM
Hey, Harshul - awesome - I'm so glad to hear that data is flowing into BigQuery. To confirm, you're running the above query via GrowthBook and then directly within BigQuery and you're seeing the 5.3% difference?
m

mysterious-hospital-86275

02/14/2024, 2:24 PM
Yes i am running above query in bigquery console and comparing it with numbers visible in GB dashboard
b

billions-xylophone-11752

02/14/2024, 3:13 PM
Gotcha. It's likely that the query we are running for the experiment is doing some additional filtering (by date, segment, or something like that). If you want to send me the experiment id, I can take a deeper look and see if I can find anything. To find it, you can view the experiment in GrowthBook, and grab the ID at the end of the URL. It'll start with
exp_
m

mysterious-hospital-86275

02/14/2024, 3:17 PM
Sure, here is the id
exp_19g61olsbcryvz
b

billions-xylophone-11752

02/14/2024, 3:47 PM
Hmm... I'm not seeing an experiment with that id. Are you self-hosting GrowthBook, or using the cloud version?
m

mysterious-hospital-86275

02/14/2024, 3:49 PM
I am using cloud version only.
b

billions-xylophone-11752

02/14/2024, 4:04 PM
Can you confirm the id again? The id you sent has 18 characters, and usually they contain 21.
m

mysterious-hospital-86275

02/14/2024, 4:23 PM
This is the only id thats available to me
b

billions-xylophone-11752

02/14/2024, 6:25 PM
Thank you - I was able to find it. Without looking at the data, it's hard for me to tell where the 5.3% discrepancy is coming from. There are a few things that it might be. 1. The experiment assignment query does filter by date, to only select rows that were added after the experiment started, and before it ended 2. The experiment assignment specifically eliminates
null
user_id values 3. The experiment assignment query doesn't contain the
regex
matching clause like the query you shared 4. The experiment analysis query (which is a larger query that contains the experiment assignment subquery) runs by default every 6 hours (though, you can always manually refresh it via the GrowthBook UI), so its possible that the 6 hour delay could be responsible for the difference. Again, without having access to the data, it's hard for me to pinpoint what could be causing the difference.
m

mysterious-hospital-86275

02/15/2024, 3:38 AM
Hi Michael, Just wanted to raise one more issue i am facing. I am running one experiment with visual editor changes. But as i am seeing data in other experiments in this particular one i am not seeing any data. https://app.growthbook.io/experiment/exp_19g61olsbgd8un can you please look into it and let us know what could be the reason for this?
b

billions-xylophone-11752

02/15/2024, 4:10 PM
Hey Harshul - of course. It looks like when you created your SDK Connection, the "Include Visual Experiments" toggle wasn't enabled. You'll want to enable that, and then when calling GrowthBook, we'll return visual experiments in the endpoint's response.
m

mysterious-hospital-86275

02/15/2024, 4:26 PM
Thanks Michael. I can see experiment_viewed event for Visual Editor Experiment. It must update and refresh data in GB dashboard by tomorrow.
b

billions-xylophone-11752

02/15/2024, 4:31 PM
Awesome! And yes, now that you can see it, users should start seeing the experiment, and data should start flowing into BigQuery. And once you see data in BigQuery, you can refresh the experiment results in GrowthBook. And, as a reminder, GrowthBook automatically tries to refresh experiment results every 6 hours by default.
m

mysterious-hospital-86275

02/15/2024, 5:06 PM
Okay will check and let you know in case of any issue.
4 Views