Hi team, Today we are using sitespect for our AB Testing. I am trying to understand here how growthbook help to supplement AB testing or to replace sitespect? I understand the side of analysis & reporting create your own

Depends a bit on your tech stack and data stack - do you have a data warehouse currently?

@fresh-football-47124 Yes entirely on Snowflake & BQ

GrowthBook also has lightweight SDKs you can use to run experiments. https://docs.growthbook.io/lib

and do we collect data of all those experiments?

the SDKs help with the variation assignment side, but leave hooks for you to call the event tracking of your choice- those events are sent to your data warehouse for analysis with the GrowthBook platform.

@fresh-football-47124 @future-teacher-7046 If I got it right, I can use one of these https://docs.growthbook.io/lib to run the experiment, collect the experiment events/data by any other framework like snowplow collector service which will in turn dump data into BigQuery then connect GrowthBook to run hypothesis. Is my understanding right?

Yes, that's correct

Thank you @future-teacher-7046 for confirming it. How do we perform MAB in the GB? and when & how do I use API & web hook of GB?

GrowthBook does not support multi-armed or contextual bandits at the moment.

Is there any plan for GB to include it in future?

Possibly. We wrote an article on bandits and why they are usually not the right tool for the job. https://medium.com/growth-book/guide-to-multi-arm-bandits-what-is-it-and-why-you-probably-shouldnt-use-it-ecc9bb2e5a84 That being said, they definitely have their use for specific optimization problems.

Is it to address the use case of feature flag management?

It's only required if you want to be able to control experiments from within the GrowthBook UI, for example starting/stopping an experiment or changing the traffic split between variations. If you don't care about that you can just use the SDKs without integrating with the API or webhooks. Instead, you would just need to modify the code and do a new deploy if you want to start/stop or change an experiment.

Ideally, we want everything at one stop or one platform which can provide us everything. Generally, I observed to achieve one stop platform, folks end up building in house solution we are trying to avoid to build one. I am trying to find is GB our answer? We want our UI Team, Backend team & Data science team use one platform using multiple

It might be easier to set up a quick meeting over Zoom to go over your requirements and see if GrowthBook is a good fit for what you're looking for. https://calendly.com/growthbook/demo

Sure thank you @future-teacher-7046, I will block something on the calendar. Currently we use SiteSpect to run our AB Testing experiments.

There is not anything specific to GCP. For all cloud deployments, we recommend using MongoDB Atlas. For GrowthBook itself, it just requires a single Docker container, which can be run directly on a VM or through something like Kubernetes

Hi Team, requirement i am trying to address is, we are collecting PDP (Product Detail Page) click events using Tealium via pub-sub connector to BQ Tables (It has got user_id, profile_id etc etc). We do have recommendation engine data pipeline on which we want to run experiments on the user we are collecting from Tealium & user base dataset we collect from Snowflake. We don't go to user interface. We want to bucket/group/split the users, stick users to the various experiments running in backend (pipeline running on GCP using Kubeflow & Airflow) Is it possible to address this use case using Growthbook? Post running the experiment we want to analyze the data in growthbook. • First, need to way to bucket user & whenever we run experiments, it should stick users for certain amount of time. • Collect the experiment data & need a UI to analyze the data.

I don't 100% understand your full use case. Our SDKs let you bucket users using deterministic hashing. So the same user id will always be assigned the same variation for an experiment. And the GrowthBook UI can be used to analyze any experiment results as long as you get all the data into either BigQuery or Snowflake.

Thank you @future-teacher-7046 I would need info on that SDK or API

Our SDKs let you bucket users using deterministic hashing

Where can I find it. How do I use it? In our use case, UI won't be involved in it. User click events (PDP) & user info will be collected via Tealium connectors into our BQ Table. Problem statement: I've needed to generate random but consistent values in a BigQuery data pipeline. This is a pipeline that runs many times per day and constantly has new data feeding in through upstream pipes. As an example, let's say that we have 10m users and need to randomly place them into groups depending on the week of the month. Additionally, we only want to do this once; once a user gets placed into a group, they stay in that group. This way, downstream (experiment pipeline) work can rely on how each user is assigned without having to worry about checking for changes .

you can read about our hashing algorithm we use here: https://docs.growthbook.io/lib/build-your-own#hashstr-string-float

Thank you @fresh-football-47124 So the string of that hash function can be combination of user_id & name of the test as salt.

Yes

I saw this on the release note of GB 1.3

Previously, GrowthBook had two hard-coded user identifier types (or randomization units) — *user_id* and *anonymous_id*. This worked for some websites, but was not very flexible. For example, B2B products might want to split users by their company instead of an individual user_id. Or an app might only have logged-in users and have no use for an anonymous_id.

How do I achieve the same using Python SDK?

When adding an experiment rule to a feature, you can choose which user attribute to split by.

Oh I got your point here i will respective SDK here from Segment or Snowplow etc etc. Once I use this callback emit the structure of the event to pub-sub then push to BQ. Then GB UI can work out of the box from the structure or do I need to do intermediate transformation to get picked up by GB Dashboard or Metric Dashboard from BQ

We have out-of-the-box support for a few standard data schemas - Segment, Snowplow, GA4, and Rudderstack. For everything else, you will need to write some SQL to map your data structure to what GrowthBook expects