For experiments, can we use a GA4-created funnel as metrics?

Hey, I am still hoping against hope to find out why my ‘count’ experiment doesn’t show results? The metric is defined with this SQL:

SELECT

user_id,

user_pseudo_id as anonymous_id,

TIMESTAMP_MICROS(event_timestamp) as timestamp,

1 as value

FROM

``analytics_356435236.events_*``

WHERE

event_name = 'Click counter button'

AND _TABLE_SUFFIX BETWEEN '20230328' AND '20231231'

And it shows a graph that looks as I’d expect from my mocked-up BigQuery data, but the experiment is showing a result of ‘1’ instead of the expected 5 to 10 clicks per user. This is the experiment query:

-- Counter button clicks, no intraday in query (count)

WITH

__rawExperiment as (

SELECT

user_id as user_id,

TIMESTAMP_MICROS(event_timestamp) as timestamp,

experiment_id_param.value.string_value AS experiment_id,

<http://variation_id_param.value.int|variation_id_param.value.int>_value AS variation_id,

geo.country as country,

traffic_source.source as source,

traffic_source.medium as medium,

device.category as device,

device.web_info.browser as browser,

device.operating_system as os

FROM

(

SELECT

*

FROM

``analytics_356435236.events_*``

WHERE

_TABLE_SUFFIX BETWEEN '20230330' AND '20230512'

UNION ALL

SELECT

*

FROM

``analytics_356435236.events_intraday_*``

WHERE

_TABLE_SUFFIX BETWEEN '20230330' AND '20230512'

),

UNNEST (event_params) AS experiment_id_param,

UNNEST (event_params) AS variation_id_param

WHERE

event_name = 'experiment_viewed'

AND experiment_id_param.key = 'experiment_id'

AND variation_id_param.key = 'variation_id'

AND user_id is not null

),

__experiment as ( -- Viewed Experiment

SELECT

e.user_id as user_id,

cast(e.variation_id as string) as variation,

CAST(e.timestamp as DATETIME) as timestamp,

CAST(e.timestamp as DATETIME) as conversion_start,

DATETIME_ADD(CAST(e.timestamp as DATETIME), INTERVAL 72 HOUR) as conversion_end

FROM

__rawExperiment e

WHERE

e.experiment_id = 'new-secondary-panel-title'

AND CAST(e.timestamp as DATETIME) >= DATETIME("2023-03-30 17:01:00")

),

__metric as ( -- Metric (Counter button clicks, no intraday in query)

SELECT

user_id as user_id,

m.value as value,

CAST(m.timestamp as DATETIME) as timestamp,

CAST(m.timestamp as DATETIME) as conversion_start,

CAST(m.timestamp as DATETIME) as conversion_end

FROM

(

SELECT

user_id,

user_pseudo_id as anonymous_id,

TIMESTAMP_MICROS(event_timestamp) as timestamp,

1 as value

FROM

``analytics_356435236.events_*``

WHERE

event_name = 'Click counter button'

AND _TABLE_SUFFIX BETWEEN '20230328' AND '20231231'

) m

WHERE

CAST(m.timestamp as DATETIME) >= DATETIME("2023-03-30 17:01:00")

),

__distinctUsers as (

-- One row per user

SELECT

e.user_id as user_id,

cast('All' as string) as dimension,

(

CASE

WHEN count(distinct e.variation) > 1 THEN '__multiple__'

ELSE max(e.variation)

END

) as variation,

MIN(e.conversion_start) as conversion_start,

MIN(e.conversion_end) as conversion_end

FROM

__experiment e

GROUP BY

e.user_id

),

__userMetric as (

-- Add in the aggregate metric value for each user

SELECT

d.variation,

d.dimension,

d.user_id,

SUM(m.value) as value

FROM

__distinctUsers d

JOIN __metric m ON (m.user_id = d.user_id)

WHERE

m.timestamp >= d.conversion_start

AND m.timestamp <= d.conversion_end

GROUP BY

d.variation,

d.dimension,

d.user_id

),

__stats AS (

-- One row per variation/dimension with aggregations

SELECT

m.variation,

m.dimension,

COUNT(*) AS count,

SUM(COALESCE(m.value, 0)) AS main_sum,

SUM(POWER(COALESCE(m.value, 0), 2)) AS main_sum_squares

FROM

__userMetric m

GROUP BY

m.variation,

m.dimension

),

__overallUsers as (

-- Number of users in each variation/dimension

SELECT

variation,

dimension,

COUNT(*) as users

FROM

__distinctUsers

GROUP BY

variation,

dimension

)

SELECT

u.variation,

u.dimension,

u.users AS users,

'mean' as statistic_type,

'count' as main_metric_type,

COALESCE(s.main_sum, 0) AS main_sum,

COALESCE(s.main_sum_squares, 0) AS main_sum_squares

FROM

__overallUsers u

LEFT JOIN __stats s ON (

u.variation = s.variation

AND u.dimension = s.dimension

)

I’ve essentially been stuck on this for weeks, I can’t see how to make a ‘count’ experiment work. I can download some sample data as CSV if that will help? Please, I am sure there must be something I’m doing wrong, but I cannot see it.

Hi, what are steps by steps to integrating growth book and a node express js app?

Hi! I'm having trouble adding a ClickHouse datasource, I'm getting

URI malformed

error, I already check that the credentials, ports and database are correct, is there any documentation of what GrowthBook expects me to do to integrate it with ClickHouse?

Hi Folks! I had a couple of quick questions regarding the GA4 / BigQuery integration. Someone set up the tracking callback in the implementation to use something slightly different to what is in the documentation. Instead of having the experiment_viewed event with the event_category, experiment_id and variation_id properties set on it like shown in the first picture below they have an experiment_viewed event with the following event_category is correct but the data that would normally be in experiment_id is instead inside of event_action and variation_id is labeled event_label as shown in pictures 2 and 3. So I wanted to know: 1. Is that going to work out of the box (I assume not) 2. Can I configure GrowthBook to look for these other event parameters instead or should I update the code to use the correct parameter names?

We have Growthbook set up in our React app already, but are looking to set it up for our landing pages, which are currently built on Webflow. Do you have any recommendations on the best setup practices for a website built in an external Website builder (noting that we can add in customer code, including HTML / script tags). We'd prefer not to use Google Tag Manager.

Hi Team, After creating growth book instance, i want to get all the assigned features of the user in the NODE JS code. could you let me know what approach i have to follow. If gb is my growthbook instance, gb.experiments is giving undefined value.

Hey! I need to change the marked row in the picture (the query run for a test-experiment) to this:

AND regexp_extract(_TABLE_SUFFIX, '[0-9]+') BETWEEN '20230515' AND '20230518'

This is changed in the query for the specific metric. Further down the automatic script for the experiment, this is changed as well. But not for this first part of the experiment script. Where does it come from and how can I edit this? Thanks in advance 🙏

I am looking to create some time bound metrics - let’s say Day 7 conversion for argument’s sake. Is there a way to exclude users from a calculation (both numerator and denominator) if the user, for example, registered fewer than 7 days ago? My first thought was to create a calculation that could be used as a denominator, which returned the

user_id

and the registration timestamp, with a where clause that required the date diff between registration timestamp and current timestamp to be greater than 7 days. I need to return the registration timestamp so that the conversion window for the conversion metric works properly, but by definition the registration timestamp is before the user is bucketed in the experiment, so is not counted in the experiment calculation. Would very much appreciate help here - there are lots of metrics that we need to write in this way to align with our other reporting.

Hi 👋 I'm new to GrowthBook and I wasn't able to find in the documentation how does it handle multiple primary metrics for the Bayesian Approach? Is there something similar for the Bonferroni correction for Bayesian Metrics? Thanks in advance! 🙏

Hi, we are wondering if Growthbook should be mentioned in our website's privacy policy (Europe) in case we will connect it to BigQuery? Can Growthbook access user level data in this case? Does Growthbook copy our BQ dataset for any length of time or not? Thanks in advance!

Hey All, I was kind of curious how stable and trustworthy the auto-generated SQL queries were for BigQuery / GA4 because I tried to add some basic metrics like add_to_cart and purchase for example and when I try to add them I get initial data indicating all is good like shown in the second picture. But when I try to actually use them I instead get errors like this and I have no real context for debugging it unfortunately.

Hiya, this is probably a dumb question, but having set up a visual experiment, I can’t see any way to view / change the configuration? Is there any way to reaccess the visual editor? “Edit variations” only seems to edit variation names.

Hi, We are using BigQuery with a custom event source and are trying to set up the Experiment Assignment Queries for the data source. Have some questions about that process: 1. Confused on how to get the

viewed_experiment

table in BigQuery. Are the records in this table supposed to be events triggered by the SDK's

trackingCallback

? 2. We want to add Dimensions to this table (like the GrowthBook UI suggests). If Dimensions = Targeting Attributes, how do we get those into the same event that populates the

viewed_experiment

table? It doesn't seem like the

trackingCallback

returns with the Targeting Attributes, what's the best practice here?

How can I listen to Growthbook-proxy using Swift? I couldn't find any samples on the documentation, or any public APIs that look like what I need.

So I am trying to run growthbook self hosted on my server but I run into the following issue:

Error connecting to the GrowthBook API at http://<my-server-ip>:3100.

I have added these variables in my docker-compose.yml:

environment:
      - APP_ORIGIN=http://<my-server-ip>:3000
      - API_HOST=http://<my-server-ip>:3100

removed my ip from the example above but it does not seem to be working, I still get the issue. Does someone know why?

Hi, we are testing GrowthBook and setting up datasources to analyse experiment. we are setting them up in bigquery. Although I get that you need to have experiment assignment queries and metrics living in the same datasource, we are still struggling to set up a datasource with multiple datasets in it. The UI asks for a default dataset, but then there is only that one, even if the project (on bigquery) contains multiple datasets. In our case, I common to have multiple datasets coming from different business units (e.g. to track offline conversions) and then datasets that track online behaviors (e.g. Google Analytics). can someone help me out to understand if we are missing something?

Is there a way to reference the timestamp that the user gets bucketed into the experiment in a metric’s SQL? I’ve seen one can include the start/end date of the experiment with SQL template variables, though this is obviously not the same thing…

Hey all, I realize that the localStorage is not updating after expiring staleTTL. Any thoughts? It's requesting after this expiring directly from API on every reload, not updating the localStorage.

If a Feature has 2 Override Rules (Force Rule, followed by Experiment Rule), and a user falls under the Force Rule, doe it make sense that the

trackingCallback

does NOT fire for that user for that feature? Is that expected behavior

Hi, I want to add a a/b test, do I have to first bind a data source to collect data from a/b test?

What would be the best way to support this type of experiment in GrowthBook? Let's say that only 70% of my users have language set to English and I want to test some UX changes in English only. I'm using the javascript SDK's trackingCallback() to track when users are put into an experiment in my analytics system. If our UX logic happens to call gb.isOn("myEnglishFlag") for a non-English user, would that cause them to be added to an experiment? Let's assume that we could have the user.language along with user.id available as attributes when initializing GrowthBook. I'm guessing we should either (a) use a "targeting condition" when setting the feature flag / experiment in GB dashboard or (b) add some logic to the trackingCallback() in our client to avoid sending experiment_viewed events for non-English users, otherwise we may be falsely reporting non-english users as part of the experiment_viewed. Thanks for any advice on this.

With growthbook is there a way to only submit attributes on the server side (nodejs), but still do feature flagging on the client side? The goal is to avoid exposing the attributes to the client side.

Does growth book require a separate instance of the SDK for each user? IE: If I have a node.js server, and I have 10 users, I have 10 instances of the growthbook SDK initialized in my app, each bound to a particular user via attributes?

When setting up the Experiment Assignment Queries, what is

experiment_id, experiment_name, variation_name

? This is the SDK output that our Engineers are sending to me:

"experiment": {
      "force": null,
      "variations": [
        {
          "id_2": 2,
          "id_3": 3,
          "id_1": 1
        },
        {
          "id_1": 4,
          "id_2": 5,
          "id_3": 6
        },
        {
          "id_1": 7,
          "id_2": 8,
          "id_3": 9
        }
      ],
      "namespace": null,
      "hashAttribute": "id",
      "key": "test-json",
      "weights": [
        0.3334,
        0.3333,
        0.3333
      ],
      "active": true,
      "coverage": 1,
      "condition": null
    },
    "experimentResult": {
      "value": {
        "id_1": 7,
        "id_2": 8,
        "id_3": 9
      },
      "hashUsed": true,
      "hashAttribute": "id",
      "hashValue": "ec62aa1bc0b142348b3ea6ea0382a722",
      "inExperiment": true,
      "variationId": 2
    }

• Feature Key = Experiment ID? like

test-json

in the example above • What is an example of "Experiment Name"? Where in the GrowthBook UI is that input? • How do we get "Experiment Name" and "Variation Name" to be passed back by the SDK?

We're running our first A/B-test, it's only been running for two days. Getting this "suspicious result" in one of the variants, (for one of the metrics). Should we be worried or is it normal in the beginning of the test period? I'm guessing there's nothing to worry about but just want to double check. Thanks!

Hey, Is it possible to have an assignment condition that is not verified for users that are already assigned to the experiment? We have an experiment that depends on the user not having a user id. The thing is that once they log in, they are unassigned from the experiment. I would like to be able to change our condition so that those who have a user id but were assigned before (traffic is split based on another id), keep their assignment

Hi there! I appreciate any help given here. I just installed it on my k8s cluster but I've got some strange error message. It's running

v2.1.1

{"level":50,"time":1685991856283,"pid":500,"hostname":"growthbook-56b644d9b8-f89xj","err":{"type":"MongoParseError","message":"Load balancer mode requires driver version 4+","stack":"MongoParseError: Load balancer mode requires driver version 4+\n    at QueryReqWrap.callback (/usr/local/src/app/node_modules/mongodb/lib/core/uri_parser.js:111:27)\n    at QueryReqWrap.onresolve [as oncomplete] (node:dns:281:10)","name":"MongoParseError"},"msg":"Failed to connect to MongoDB"}
 | {"level":50,"time":1685991856285,"pid":500,"hostname":"growthbook-56b644d9b8-f89xj","err":{"type":"Error","message":"MongoDB connection error.","stack":"Error: MongoDB connection error.\n    at /usr/local/src/app/packages/back-end/dist/init/mongo.js:35:15\n    at Generator.throw (<anonymous>)\n    at rejected (/usr/local/src/app/packages/back-end/dist/init/mongo.js:6:65)"},"msg":"Failed to initialize application"}

Hey all, I’m investigating how to work with GB, and our situation is that most of our users are conflicted, because typically use multiple devices, and our period of conversion is long enough., and we test usually features before login, so we can’t match them by user_id. What the solutions could be implemented with GB to deal with it?

Hi, I'm trying to get Growthbook running on a local machine so we can test it. I had it working a couple weeks ago, but it no longer works. The error that I'm getting is on

growthbook-growthbook-1

with the error:

Failed to connect to MongoDB. Retrying with field remapping for mongodb v3 to v4

Is this a known issue?