https://www.growthbook.io/
Join Slack
Hi everyone, I launched a/a in order to check the system, but i encountered with the following issue...
# ask-questions
l
Hi everyone, I launched a/a in order to check the system, but i encountered with the following issues: 1. I checked the number of participated users in 14th June in health tab - 983 for both variations, next i checked the query in bigquery in 14th June - 2223 for both var (and it's equal to the numbers if users who triggered 'experiment_viewed' event in ga4). I also checked the same numbers for period 14-16 June, and the difference is less, but still significant 2. And one more strange thing i noticed is that total numbers of users who visited the page i implement new variation on is equal to 5362, while 'experiment_viewed' event was triggered by 4236 users (in Growthbook's health tab it's 3300 users). But variation was implemented directly in a page header So can anybody help with this 2 issues please P.S The SDK was implemented via HTML generic directly (without GTM)
f
for 1, we unique the users, so I would expect it to be less than the total count of events
also the experiment conversion window can lower the event totals too
l
but i also presented the numbers of users everywhere (not the events), since ga4 also count unique users And moreover i didn't set any conversion window, since i'm only trying to match initial amount of users, not conversions they did What other hypothetical reasons could be?
@fresh-football-47124 help please)
f
Hi Pavel. sorry for the delay
you can click on the three dots to the right of the results and choose 'view queries'
I Would start there and debugging the assignment part, see if there is a where clause that is causing the traffic numbers to be reduced
l
@fresh-football-47124 good day! I checked the first issue - now data matches each other, that's good But i'm still struggling with the second issue: from 14th to 17th the number of page visitors was 7855 while the number of the same page visitors who triggered "experiment_viewed" event only 6400 (according to ga4/bigquery/growthbook, so they are equal) The variation is placed in a page header, so there is no chance to miss it out What is hypothetical problem can be?
f
is this from the same data source?
l
yeah, i have the only one
@fresh-football-47124 i tagged you just in case if you lose the message 🙂
f
Its hard for me to debug from here
its possible that the user could have triggered a page view event but somehow left before the AB test event loaded
(though we usually find that growthbook loads first)
l
I checked it also, it's impossible to avoid exposure
@fresh-football-47124 can i ask also to check the code for New users segment?
Copy code
SELECT
  user_pseudo_id as anonymous_id,
  FORMAT_TIMESTAMP('%Y-%m-%d', TIMESTAMP_MICROS(event_timestamp)) AS date
FROM
  `anyrun`.`analytics_273112850`.`events_*`
WHERE
  (SELECT value.int_value FROM UNNEST(event_params) WHERE key = 'ga_session_number') = 1
  AND event_name = 'session_start'
  AND ((_TABLE_SUFFIX BETWEEN '{{date startDateISO "yyyyMMdd"}}' AND '{{date endDateISO "yyyyMMdd"}}') OR
       (_TABLE_SUFFIX BETWEEN 'intraday_{{date startDateISO "yyyyMMdd"}}' AND 'intraday_{{date endDateISO "yyyyMMdd"}}'))
Because i'm not sure how data filter need to look like. If the code is correct, can i use some alternative handlebars in order to reduce bigquery select cost?
f
is ga_session_number how ga tracks new users? I'm not super familiar with their structure
l
yeah yeah, it's just the number of session when session_start event is occurred
45 Views
Open in Slack
PreviousNext
Powered by

GrowthBook Users

Open source platform for stress free deployments, measured impact, and smarter decisions.

Powered by