https://www.growthbook.io/
Join Slack
Hi I am trying to use my Google Analytics 4 as data source and run a string A/B experiment for users...
# ask-questions
w
Hi I am trying to use my Google Analytics 4 as data source and run a string A/B experiment for users completing a reservation form on my web platform. I have already connected BigQuery and I can see all my analytics events data under Growthbook > “Data Sources” and I can see the historical event data over time as well. So far so good. But when I connect this metric as a Goal Metric on the experiment I get the notification: “No data yet. Make sure your experiment is tracking properly.” On my server I init the growthbook properly and run an inline experiment %100 of users and I can see the experiment result on my client console. I get the anonymous user id from google analytics visitor cookie so each visit is properly triggering the experiment. And even though many visitors complete the reservation everyday form I still don’t see any experiment result on the Growthbook experiment results. I have followed every youtube videos I could find and followed documentation to the detail over and tried many things - fact tables, custom SQL queries etc for the last few days but the results is the same - no data yet. At this point I would really appreciate any helpful tips. I am going to post some screenshots under this post thread hoping to give a better idea about my config.
r
@wonderful-easter-10646 has opened an issue Close Issue button
w
String feature we use to A/B test user survey form images and theme.
The experiment config
Reservation Complete event metrics from Google Analytics 4
Query Result
Analysis Settings for the experiment
no data yet
console.log on my web platform showing the experiment result for each user
my google tag script for growthbook API
Copy code
<script>
  (function() {
    // Wait for the SDK to load before starting GrowthBook
    if (window.growthbook) {
      startGrowthbook();
    } else {
      document.querySelector("#growthbook-sdk").addEventListener("load", startGrowthbook);
    }

    function startGrowthbook() {
      if (!window.growthbook) return;
      var gb = new growthbook.GrowthBook({
        apiHost: "<https://cdn.growthbook.io>",
        clientKey: "sdk-ZmmBkra16DhXLDp",
        // TODO: Add decryptionKey if using encryption
        attributes: {
          id: document.cookie.match(/_ga=(.+?);/)[1].split('.').slice(-2).join(".")
        },
        trackingCallback: function(experiment, result) {
          console.log('trackingCallback experiment + result::::')
          console.log(experiment)
          console.log(result)
          // TODO: track experiment impression
        }
      });
      gb.run({
          key: '231024_colortheme',
          variations: ['231024_a', '231024_b'],
          weights: [0.5, 0.5],
          coverage: 1.0,
      })

      // TODO: Instrument DOM with AB test logic
      /*
      gb.loadFeatures().then(function() {
        // NOTE: We may wish to remove `gb.loadFeatures()` and instead manually implement experiment logic
      });
      */
    }
  })();
</script>
r
Hello, Mehmet, thanks for reaching out! I will look into this and try to get you some updates before the end of the day today, but it's possible you may not hear from us until Monday. (Official Support hours are 6:30am - 5:00pm Pacific)
f
Can you double check the assignment query?
its under metrics & data -> data sources
w
image.png
image.png
image.png
So user_pseudo_id is cookie based identifier for Google Analytics 4. The following query works without any issues which has pretty much the the same query:
Copy code
SELECT
  user_pseudo_id as anonymous_id,
  TIMESTAMP_MICROS(event_timestamp) as timestamp
FROM
  `regina-lpo.analytics_247374183.events_*`
WHERE
  (
    (
      _TABLE_SUFFIX BETWEEN '{{date startDateISO "yyyyMMdd"}}' AND '{{date endDateISO "yyyyMMdd"}}'
    )
    OR (
      _TABLE_SUFFIX BETWEEN 'intraday_{{date startDateISO "yyyyMMdd"}}' AND 'intraday_{{date endDateISO "yyyyMMdd"}}'
    )
  )
  AND event_name = 'reservation_complete'
I am kinda stuck at this point. When I query the API returns the following result.. I see that total users are “0”. I understand that no user seems to have joined the experiment:
Copy code
{
  "result": {
    "id": "snp_19g61mlonb29yu",
    "dateUpdated": "2023-11-06T19:38:46.134Z",
    "experimentId": "exp_19g61tlo5xl26u",
    "phase": "5",
    "dimension": {
      "type": "none"
    },
    "dateStart": "2023-10-01T20:02:00.000Z",
    "dateEnd": "2023-11-06T19:38:46.760Z",
    "settings": {
      "datasourceId": "ds_19g61mlo7fihod",
      "assignmentQueryId": "anonymous_id",
      "experimentId": "231024_colortheme",
      "segmentId": "",
      "queryFilter": "",
      "inProgressConversions": "include",
      "attributionModel": "firstExposure",
      "statsEngine": "bayesian",
      "goals": [
        {
          "metricId": "met_19g61ilobmzacg",
          "overrides": {}
        }
      ],
      "guardrails": [],
      "activationMetric": {
        "metricId": "met_19g61olo7fzgn9",
        "overrides": {}
      }
    },
    "queryIds": [
      "qry_19g61mlonb2aer"
    ],
    "results": [
      {
        "dimension": "All",
        "totalUsers": 0,
        "checks": {
          "srm": 1
        },
        "metrics": [
          {
            "metricId": "met_19g61ilobmzacg",
            "variations": []
          },
          {
            "metricId": "met_19g61olo7fzgn9",
            "variations": []
          }
        ]
      }
    ]
  }
}
I would appreaciate any ideas
r
Hi Mehmet, sure thing, I have a meeting with one of our engineers this afternoon and we'll discuss this
w
Hi August, that is great to hear. While waiting your return I have created some additional fact tables based on other GA4 events to test with. Still the same, SQL tests shows results, but connected experiment says “No data yet…”
Ok I think I might have found the issue with my case. When I look into assignment query I see that it is looking for an “experiment_viewed” with “experiment_id” and “variation_id” event at the analytics data SQL. So as I understand I must send that event to my analytics warehouse with each and every experiment session.
image.png
If that is the case this section in the docs needs an urgent update:
image.png
b
Hi Mehmet, my colleague and I had a long chat about this yesterday. The first thing we want to try is this: In your 
<script>
tag you have this bit of code:
Copy code
trackingCallback: function(experiment, result) {
          console.log('trackingCallback experiment + result::::')
          console.log(experiment)
          console.log(result)
          // TODO: track experiment impression
        }
You will need to fill in the code in your 
// TODO
to track the experiment impression. This could be the source of the issues.
Right now, the code is only logging to the console and isn't actually tracking or sending event data
Could you let me know if you fixed this already?
w
Hi August, I have updated clientside code the other day and currently I am sending event_data to analytics via gtag:
Copy code
trackingCallback: function (experiment, result) {
dataLayer.push({
'event': 'experiment_viewed',
'event_category': 'experiment',
'experiment_id': experiment.key,
'variation_id': result.variationId
})
})
after I realized that I had to send experiment_viewed event to GA4 analytics for data source query to match I have started collecting the event.
For testing I have removed ‘experiment_reviewed’ lookup it as not empty anymore but now I have another issue
image.png
Multiple Exposures Warning. I have checked whether pseudo_user_id is unique and for our metrics event anonymous_id is looking fine
image.png
Ok I have figured it out. I have modified the experiment_id query a bit for the experiment_id 
(SELECT value.string_value FROM UNNEST(event_params) WHERE key = 'experiment_id') AS experiment_id,
So 
experiment_viewed
event must be located in bigquery collection. I have lost quite some time trying to figure this out. Please update your docs and tell that this is required to send the event to warehouse in order for AB testing results to work with BQ/GA4.
image.png
f
Thanks for this feedback. Sorry about the confusion. I updated the language around that doc
it’s going live now- let me know if this is an improvement
w
Yeah much better now, thanks.
121 Views
Open in Slack
PreviousNext
Powered by

GrowthBook Users

Open source platform for stress free deployments, measured impact, and smarter decisions.

Powered by