Hi all. We're just getting started with A/B testin...
# ask-questions
r
Hi all. We're just getting started with A/B testing using the javascript SDK. The split test is working with changing the feature but no results data is displaying in the admin UI. We're getting data in big query but if we run the experiment assignment query from the data source page it returns "no rows returned, could not verify result". When we run the same query in big query it returns the data. Any ideas on what we may be missing? Happy to add more context if needed.
r
Hi Nick, thanks for reaching out. Do you have Google Analytics (GA4) sitting in between GrowthBook and BigQuery? If so, is the data showing correctly in GA4? Are you using ​`intraday`​ tables in BigQuery? The settings in BigQuery can affect how quickly data is available. For example, you could include the intraday table in assignment queries to ensure timely data availability.
If you need further assistance, could you please send me a screenshot or a code snippet of the Experiment Assignment Query you're using, along with a screenshot of the columns/table in BigQuery to match?
e
Hi@brief-honey-45610, We integrated our GA4 with BigQuery. Yes, we have
intraday
in our BigQuery but we are not using it in our assignment query.
Here is the one of the assignment query that we are using:
Copy code
WITH
  __rawExperiment AS (
    SELECT
      user_pseudo_id as anonymous_id,
      TIMESTAMP_MICROS(event_timestamp) as timestamp,
      experiment_id_param.value.string_value AS experiment_id,
      variation_id_param.value.int_value AS variation_id,
      geo.country as country,
      traffic_source.source as source,
      traffic_source.medium as medium,
      device.category as device,
      device.web_info.browser as browser,
      device.operating_system as os
    FROM
      `weezietowelsdaton`.`analytics_279996039`.`events_*`,
      UNNEST (event_params) AS experiment_id_param,
      UNNEST (event_params) AS variation_id_param
    WHERE
      (
        (_TABLE_SUFFIX BETWEEN '20231130' AND '20240118')
        OR (
          _TABLE_SUFFIX BETWEEN 'intraday_20231130' AND 'intraday_20240118'
        )
      )
      AND event_name = 'experiment_viewed'
      AND experiment_id_param.key = 'experiment_id'
      AND variation_id_param.key = 'variation_id'
      AND user_pseudo_id is not null
  ),
  __experimentExposures AS (
    -- Viewed Experiment
    SELECT
      e.anonymous_id as anonymous_id,
      cast(e.variation_id as string) as variation,
      CAST(e.timestamp as DATETIME) as timestamp
    FROM
      __rawExperiment e
    WHERE
      e.experiment_id = 'ab-quiz-test'
      AND e.timestamp >= '2023-11-30 15:57:00'
      AND e.timestamp <= '2024-01-18 19:02:14'
  ),
  __experimentUnits AS (
    -- One row per user
    SELECT
      e.anonymous_id AS anonymous_id,
      (
        CASE
          WHEN count(distinct e.variation) > 1 THEN '__multiple__'
          ELSE max(e.variation)
        END
      ) AS variation,
      MIN(e.timestamp) AS first_exposure_timestamp
    FROM
      __experimentExposures e
    GROUP BY
      e.anonymous_id
  ),
  __distinctUsers AS (
    SELECT
      anonymous_id,
      cast('All' as string) AS dimension,
      variation,
      first_exposure_timestamp AS timestamp,
      date_trunc(first_exposure_timestamp, DAY) AS first_exposure_date
    FROM
      __experimentUnits
  ),
  __metric as ( -- Metric ([Octane AI] Quiz ROUND ONE - Question Wh)
    SELECT
      anonymous_id as anonymous_id,
      1 as value,
      CAST(m.timestamp as DATETIME) as timestamp
    FROM
      (
        SELECT
          user_pseudo_id as anonymous_id,
          TIMESTAMP_MICROS(event_timestamp) as timestamp
        FROM
          `weezietowelsdaton.analytics_279996039.events_*`
        WHERE
          (
            (_TABLE_SUFFIX BETWEEN '20231130' AND '20240121')
            OR (
              _TABLE_SUFFIX BETWEEN 'intraday_20231130' AND 'intraday_20240121'
            )
          )
          AND event_name = '[Octane AI] Quiz ROUND ONE - Question Wh'
      ) m
    WHERE
      m.timestamp >= '2023-11-30 15:57:00'
      AND m.timestamp <= '2024-01-21 19:02:14'
  ),
  __userMetricJoin as (
    SELECT
      d.variation AS variation,
      d.dimension AS dimension,
      d.anonymous_id AS anonymous_id,
      (
        CASE
          WHEN m.timestamp >= d.timestamp
          AND m.timestamp <= DATETIME_ADD(d.timestamp, INTERVAL 72 HOUR) THEN m.value
          ELSE NULL
        END
      ) as value
    FROM
      __distinctUsers d
      LEFT JOIN __metric m ON (m.anonymous_id = d.anonymous_id)
  ),
  __userMetricAgg as (
    -- Add in the aggregate metric value for each user
    SELECT
      variation,
      dimension,
      anonymous_id,
      MAX(COALESCE(value, 0)) as value
    FROM
      __userMetricJoin
    GROUP BY
      variation,
      dimension,
      anonymous_id
  )
  -- One row per variation/dimension with aggregations
SELECT
  m.variation AS variation,
  m.dimension AS dimension,
  COUNT(*) AS users,
  SUM(COALESCE(m.value, 0)) AS main_sum,
  SUM(POWER(COALESCE(m.value, 0), 2)) AS main_sum_squares
FROM
  __userMetricAgg m
GROUP BY
  m.variation,
  m.dimension
@brief-honey-45610 In the initial Common Table Expression (CTE) of this assignment query, an examination of the WHERE clause reveals the inclusion of filtering conditions for event_name, experiment_id_param.key, and variation_id_param.key:
Copy code
event_name = 'experiment_viewed'
AND experiment_id_param.key = 'experiment_id'
AND variation_id_param.key = 'variation_id'
It should be noted that there is no occurrence of an event_name equal to 'experimat_viewed'. Is the query intended to be executed in its current state?
b
Hi Aditya, I believe GrowthBook is expecting there to be an
event_name
column in your data source and for the value therein to be "experiment_viewed". What do you have configured instead?
e
@brief-honey-45610 There is a
event_name
column in the data source, but there is no value that matches "experiment_viewed."
These are all the events name captured in the data source: event_name add_payment_info add_to_cart Wunderkind Submission MiniCartUpsellClick event scroll user_engagement first_visit begin_checkout social_click HolidayDrawerClicks footer_tracking view_item error_tracking Wunderkind Impression SearchIconClick [Octane AI] Quiz ROUND ONE - Question Wh page_view session_start purchase Quiz ROUND ONE Started Quiz ROUND ONE Completed view_item_list button_click select_item newsletter_click remove_from_cart Wunderkind Click [Octane AI] Quiz ROUND ONE Viewed [Octane AI] Quiz Enjoy the Stay - Questi link_click add_shipping_info view_cart view_search_results page_variant view
b
Hi Aditya, seems like the
experiment_viewed
events are not being tracked or sent to the data source. To resolve this issue, ensure that the
experiment_viewed
event is being sent with the correct event name. 1. Verify that your application code is correctly configured to send the
experiment_viewed
event. This event should be sent whenever a user is exposed to an experiment. 2. If the event is not being sent, update the tracking code to include the
experiment_viewed
event. Here's an example of how you might send this event using a trackingCallback:
Copy code
trackingCallback: function (experiment, result) {
 dataLayer.push({
  'event': 'experiment_viewed',
  'event_category': 'experiment',
  'experiment_id': experiment.key,
  'variation_id': result.variationId
 });
}
r
Hi @brief-honey-45610 Thank you for this info. For the experiment id and variation id, do I plug in the experiment id that's in the growthbook admin url for the experiment? Also I'm not sure what should be set for the variation_id.
Hi @brief-honey-45610 just following up here in case you missed my message.
@brief-honey-45610 just following up again. I'd really like to know how to proceed 🙂
Hi @brief-honey-45610 I'm confused because we're on shopify and I don't see any info in these docs https://docs.growthbook.io/integrations/shopify about a datalayer push.