Hey team! I have a question regarding the new metric denominator feature. In the image below the first metric (row) is used as a denominator for the second metric (row). I would expect the denominator in the second metric to be exactly the same as the number of converted people in metric one. But this is not the case as you can see it differs slightly. Is there anything you know that could cause this, or should this not be able to happen?

Does the low base rate problem apply with Bayesian A/B testing? We have always done the “lazy assignment” suggested by this related article and assigned users to the experiment as late as possible in the funnel. Now that we’re using Growthbook, I’m curious if that applies. Total number of conversions would still be the same regardless of the assignment time, so I’m just reasoning through whether the difference in rates will matter

Hi guys 🙂 I read this article from Google last year, Overlapping Experiment Infrastructure: More, Better, Faster Experimentation, I wanted to know how do GrowthBook deals avoiding assigning the same user to experiments that might affect each other significantly, that can introduce complex confounders, how to manage domains etc, there is a easy way to use this framework on GB? or is there another strategy in the platform?

I thought I knew exactly how the ab test start callback behave but now that we have a different experiment here I am not very sure anymore. We used one event that is the one that happens the least compared to others as metric and the others as guardrails. So these numbers for control and variation 1 are from the number of people that reached that metric with the respective variation, right? So even though we have more people in the control or the variation group they only count as users once they trigger this metric?

How does one handle having different data sources for different environments (we have different BigQuery data sets for Segment staging/dev and production data)? Do we create one feature flag and 1 experiment per environment and assign appropriate data source? How does that play with GrowthBook environments?

Another thing, somehow me. and my coworker set up account for the same company and now I see two accounts for the same name. Can we somehow remove the other one?

Last thing (I promise 😂) based on the advice yesterday about inline experiment and UI prompting you to create an experiment when it detects new inline one. We don't see a way to do that in UI:

Hey guys - does anyone know what is mean by "Local processing means no flickering or blocking." - its on the website. Does this refer to SSR or something else?

hi folks, happy to join this community. Yaniv here from Honest, we are a fintech startup planning on going live soon, and use growthbook as our experimentation platform few quick questions I have which I wasn't able to find in the documentation: 1. does growthbook support killswitch/auto-stops of experiments? so if we have a really bad feature going live, once that's significant enough it should point 100% traffic to A 2. is there a place to configure some metadata info like- team running the exp, tags of experiments etc 3. is there a place that stored the historical experiments and their results that can pulled to our DWH (we use bigquery)?

putting another question here 🙂 I am not seeing where allocation is being determined, is this something we would need to handle internally? maybe I missed that part if documentation, apologies if so

another question 🙂 we are thinking how are we going to be able to track the users that only "see" an experiment meaning, a user lands on a page, gets allocated to 'B' but doesn't do anything else, how will growthbook know this user was part of the experiment? cc: @hallowed-portugal-31753

Hi <!channel>, I'm having difficulties integrating with the iOS SDK hosted in this repo https://github.com/growthbook/growthbook-swift

hello folks, do you have an estimate to average cost of hosting GrowthBook? we are not expecting loads of traffic (few months down the road, this can be few 10s of thousands of users each month)

Hi, in the link during set up, it has a key, what permissions does this key have? Is it only to list running experiments? Or it also has write access? If it does have write access, how do we expose experiment features to our android applications? (I'm guessing it's experiment list readonly access) 🙂

hi 👋 is it possible to limit who can publish changes? so something like

engineer

role but would require a mandatory approval? and maybe more general question are you thinking about adding access control list support in the future?

Does anyone know if there are any CMS plugins around for growthbook? How do people implement changes? Completely manually?

Hi! I wanted to add Growthbook via unpkg script in a SPA (Nextjs). I know I can just npm install the package but this is the requirement I currently need to work with. Trying to understand what best practice here would be 1. User navigates to a page with growthbook. Growthbook gets loaded client-side. 2. User then navigates to a page without growthbook. Since its a SPA, Growthbook variables will still be in memory. The more general question is: are there any good ways to cleanly load and unload client-side scripts for single page apps?

quick question: if we want to see in an A/B test whether Google or FB traffic performed better, we can create a data-source for this and a dimension, right? BUT: is it also possible to then filter for this for past A/B tests as well or only for those that we start after adding this?

hi folks, when an experiment gets stopped, is there any way to restart it? and if so does it mean that users get reallocated? thanks!

Hi is it possible to update feature definition without using webhooks?

Hi, I have one more question, How do we use experiments? Say, I go to ideas page, create an idea, turn the idea into an experiment, and start an experiment, There's no experiments on my https://cdn.growthbook.io/api/features/<key> api, IF we're using features to do experiment, then what's the effect of clicking on the stop button? also how do I turn on the experiment only on a specific environment? Are these all supposed to be from features? Is there a documentation on how we're supposed to use the "experiments" tab under analysis?

Is there a way to create experiments and attribute targets in growthbook via an API? https://docs.growthbook.io/app/features#targeting-attributes (I'm a total noob doing some feasibility research)

Posthog has a feature comparison page for 5 feature flag solutions incl. GrowthBook, do you think it’s a fair and up to date comparison? https://posthog.com/product/feature-flags

Is there a plugin API or similar for extending growthbook?

Hi there, I would like to draw the attention of the team to the issue described in https://growthbookusers.slack.com/archives/C01T6Q1SVFV/p1641828963004700 & https://github.com/growthbook/growthbook/issues/405. Our use case of Growthbook implies that a user can see an experimental version of the feed multiple times (each time with new content) & convert at any time with attribution not only to the first exposure but to the experimental version of the feed in general. There are two possible ways to address this problem: either set a nearly-infinite conversion window, or enable attribution to cohorts for further exposure cases (not just the first one for a specific experiment). However, if we use the former, conversions happening after the experiment is stopped still contribute to the statistics. It would be really nice to have the latter mechanism as an option. What is the opinion of the team? Do you plan to implement that?

hello folks, when running experiments in GrowthBook, how are outliers treated? so for example something like spend (similar to order value) that is extremely high and happens only on A, is that something that is excluded when calculating exp results?

Hello guys, is it possible to get tag's value in the code?

Hey, we are filling in some risk assessment for working with growthbook, can you please explain how the connectivity works between growthbook and our Bigquery?

Hello! Is manually toggling the feature flags the only options? Or is there a way to schedule a change on the flag?

Hi there, previously when we created feature flags they were disabled by default. Now we are seeing them enabled by default. Is that a setting we can change on our side?