https://www.growthbook.io/ logo
f

fast-telephone-84882

01/20/2022, 9:47 AM
Hi team, Today we are using sitespect for our AB Testing. I am trying to understand here how growthbook help to supplement AB testing or to replace sitespect? I understand the side of analysis & reporting create your own
f

fresh-football-47124

01/20/2022, 10:01 AM
Depends a bit on your tech stack and data stack - do you have a data warehouse currently?
The biggest difference is that GrowthBook connects to your data warehouse to pull in data for analysis.
f

fast-telephone-84882

01/20/2022, 10:05 AM
@fresh-football-47124 Yes entirely on Snowflake & BQ
So running experiment & collecting can't be done using growthbook?
I am reading this article https://medium.com/growth-book/where-to-a-b-test-front-end-vs-back-end-vs-cdn-vs-api-29201bd491a9 so trying to understand So growthbook comes under frontend & backend?
f

future-teacher-7046

01/20/2022, 11:43 AM
GrowthBook also has lightweight SDKs you can use to run experiments. https://docs.growthbook.io/lib
šŸ‘ 1
f

fast-telephone-84882

01/20/2022, 5:53 PM
and do we collect data of all those experiments?
logs? then transfer it to DW?
Ah i see now
rest API is also there
f

fresh-football-47124

01/20/2022, 6:00 PM
the SDKs help with the variation assignment side, but leave hooks for you to call the event tracking of your choice- those events are sent to your data warehouse for analysis with the GrowthBook platform.
f

fast-telephone-84882

01/24/2022, 12:17 PM
@fresh-football-47124 @future-teacher-7046 If I got it right, I can use one of these https://docs.growthbook.io/lib to run the experiment, collect the experiment events/data by any other framework like snowplow collector service which will in turn dump data into BigQuery then connect GrowthBook to run hypothesis. Is my understanding right?
f

future-teacher-7046

01/24/2022, 1:02 PM
Yes, that's correct
f

fast-telephone-84882

01/24/2022, 2:04 PM
Thank you @future-teacher-7046 for confirming it. How do we perform MAB in the GB? and when & how do I use API & web hook of GB?
f

future-teacher-7046

01/24/2022, 2:05 PM
GrowthBook does not support multi-armed or contextual bandits at the moment.
f

fast-telephone-84882

01/24/2022, 2:06 PM
Is there any plan for GB to include it in future?
f

future-teacher-7046

01/24/2022, 2:11 PM
Possibly. We wrote an article on bandits and why they are usually not the right tool for the job. https://medium.com/growth-book/guide-to-multi-arm-bandits-what-is-it-and-why-you-probably-shouldnt-use-it-ecc9bb2e5a84 That being said, they definitely have their use for specific optimization problems.
As for API and Webhooks, they are 2 different ways to sync experiment configuration to your database or cache layer. With the API, you would pull configuration from GrowthBook on a schedule (say every 5 minutes). With Webhooks, GrowthBook would push configuration to your servers whenever something changes.
f

fast-telephone-84882

01/24/2022, 3:14 PM
Is it to address the use case of feature flag management?
f

future-teacher-7046

01/24/2022, 3:21 PM
It's only required if you want to be able to control experiments from within the GrowthBook UI, for example starting/stopping an experiment or changing the traffic split between variations. If you don't care about that you can just use the SDKs without integrating with the API or webhooks. Instead, you would just need to modify the code and do a new deploy if you want to start/stop or change an experiment.
f

fast-telephone-84882

01/24/2022, 3:25 PM
Ideally, we want everything at one stop or one platform which can provide us everything. Generally, I observed to achieve one stop platform, folks end up building in house solution we are trying to avoid to build one. I am trying to find is GB our answer? We want our UI Team, Backend team & Data science team use one platform using multiple
f

future-teacher-7046

01/24/2022, 3:35 PM
It might be easier to set up a quick meeting over Zoom to go over your requirements and see if GrowthBook is a good fit for what you're looking for. https://calendly.com/growthbook/demo
f

fast-telephone-84882

01/24/2022, 4:24 PM
Sure thank you @future-teacher-7046, I will block something on the calendar. Currently we use SiteSpect to run our AB Testing experiments.
@future-teacher-7046 We will reach out to you soon. We are trying to learn more about existing implementation using SiteSpect. Meanwhile I will try to spin up GrowthBook on GCP
Are there any documentation available to install it on GCP as self-hosting?
f

future-teacher-7046

02/09/2022, 3:54 AM
There is not anything specific to GCP. For all cloud deployments, we recommend using MongoDB Atlas. For GrowthBook itself, it just requires a single Docker container, which can be run directly on a VM or through something like Kubernetes
f

fast-telephone-84882

04/27/2022, 1:01 PM
Hi Team, requirement i am trying to address is, we are collecting PDP (Product Detail Page) click events using Tealium via pub-sub connector to BQ Tables (It has got user_id, profile_id etc etc). We do have recommendation engine data pipeline on which we want to run experiments on the user we are collecting from Tealium & user base dataset we collect from Snowflake. We don't go to user interface. We want to bucket/group/split the users, stick users to the various experiments running in backend (pipeline running on GCP using Kubeflow & Airflow) Is it possible to address this use case using Growthbook? Post running the experiment we want to analyze the data in growthbook. ā€¢ First, need to way to bucket user & whenever we run experiments, it should stick users for certain amount of time. ā€¢ Collect the experiment data & need a UI to analyze the data.
f

future-teacher-7046

04/27/2022, 1:59 PM
I don't 100% understand your full use case. Our SDKs let you bucket users using deterministic hashing. So the same user id will always be assigned the same variation for an experiment. And the GrowthBook UI can be used to analyze any experiment results as long as you get all the data into either BigQuery or Snowflake.
f

fast-telephone-84882

04/27/2022, 2:45 PM
Thank you @future-teacher-7046 I would need info on that SDK or API
Our SDKs let you bucket users using deterministic hashing
Where can I find it. How do I use it? In our use case, UI won't be involved in it. User click events (PDP) & user info will be collected via Tealium connectors into our BQ Table. Problem statement: I've needed to generate random but consistent values in a BigQuery data pipeline. This is a pipeline that runs many times per day and constantly has new data feeding in through upstream pipes. As an example, let's say that we have 10m users and need to randomly place them into groups depending on the week of the month. Additionally, we only want to do this once; once a user gets placed into a group, they stay in that group. This way, downstream (experiment pipeline) work can rely on how each user is assigned without having to worry about checking for changes .
f

fresh-football-47124

04/28/2022, 2:40 AM
you can read about our hashing algorithm we use here: https://docs.growthbook.io/lib/build-your-own#hashstr-string-float
f

fast-telephone-84882

04/29/2022, 2:59 AM
Thank you @fresh-football-47124 So the string of that hash function can be combination of user_id & name of the test as salt.
f

fresh-football-47124

04/29/2022, 3:00 AM
Yes
f

fast-telephone-84882

05/15/2022, 10:54 AM
I saw this on the release note of GB 1.3
Previously, GrowthBook had two hard-coded user identifier types (or randomization units) ā€” *user_id* and *anonymous_id*. This worked for some websites, but was not very flexible. For example, B2B products might want to split users by their company instead of an individual user_id. Or an app might only have logged-in users and have no use for an anonymous_id.
How do I achieve the same using Python SDK?
@fresh-football-47124 if I use BQ to track my experiments how do I use this block like so that I can run analysis using GB UI?
Copy code
def on_experiment_viewed(experiment, result):
  # Use whatever event tracking system you want
  analytics.track(attributes["id"], "Experiment Viewed", {
    'experimentId': experiment.key,
    'variationId': result.variationId
  })
f

future-teacher-7046

05/15/2022, 1:37 PM
When adding an experiment rule to a feature, you can choose which user attribute to split by.
And in your tracking callback, people usually use some sort of event tracking library to queue up events so they can be bulk inserted into BQ. Some popular options are Segment, Jitsu, Rudderstack, and Snowplow.
f

fast-telephone-84882

05/16/2022, 1:40 PM
Oh I got your point here i will respective SDK here from Segment or Snowplow etc etc. Once I use this callback emit the structure of the event to pub-sub then push to BQ. Then GB UI can work out of the box from the structure or do I need to do intermediate transformation to get picked up by GB Dashboard or Metric Dashboard from BQ
f

future-teacher-7046

05/16/2022, 2:46 PM
We have out-of-the-box support for a few standard data schemas - Segment, Snowplow, GA4, and Rudderstack. For everything else, you will need to write some SQL to map your data structure to what GrowthBook expects
3 Views