Great job for building this product @GrowthBook - We're using currently "Cloudbees feature management" (A way to long product name) former rollout.io. After bought from Cloudbees they lost unfortunatly the most of their traction. So I'm happy to see that a new competitor is here, that has the potential to take the innovation over (And hopefully keeps it on a high level 😉 )

Nice product guys! When we evaluating this, I have a little suggestion about adding override rules. I found when I add new override rule, it seems like being taken effect immediately. better to default

disabled

or add an option when added new override rule to control. So that we can enable the change after we adjusted the rule order and being confident before it take effect.

Thanks for such a nice and easy to use product! I have a couple comments from myself and my team: • Is there a reason overrides cannot work unless features are "on"? I originally set a feature flag to "off" and an override of

if (admin) return true

and it does not work. So I ended up turning on the feature flag but having it return

false

and then the override works. It would be more intuitive to me to have the override always work. • Is there plans for allowing additional metadata alongside the feature flag boolean? • I saw in some GH issues that multiple environments is coming; is there an ETA on this? It would be nice to have multiple environments beyond just dev and prod. • Is there any plans for a feature flag streaming service such that feature flag states are updated to all connected clients without a refresh?

I've found a bug in Metrics: If I try to edit the query in Query Settings, which is made using the query builder, and hit "Save", my query is replaced by the auto-generated SQL query. I then have to edit the metric again and specifically select the query builder. Also, the "User Type Supported" seems to reset when this happens. I would assume the SQL query should be identical to the query I built with the query builder? See the attached the recording for an example.

in the experiments query in data sources, if you end the query with a comment e.g.

...
FROM schema.table --this is our new experiment table

update data will fail because it becomes

...
FROM schema.table --this is our new experiment table,

and that comma is important… easy to fix, but might want to join the strings differently to protect against that

Also running into install issues w gbstats. Looks like I need to use poetry to manage my env if I want to use this package (I know that poetry exists but have never worked w it. I see a poetry.lock file in the gbstats folder though) Pip isn't aware of any dependencies at all

One small thing I noticed which can be improve is about changing the order of the metric in experiment. We usually want to order the metric in a logical way. so currently change the metric order is not convenient. better to have a UI drag and drop feature to let us adjusted the metric order🙌

This use case might not be common. I have set up 2 different Data Sources, but only one of the Data Sources contains the variation assignment data for all experiments. I have multiple metrics set up - some for each Data Source. When creating an experiment I select the Data Source to import from (the one that contains the variation assignment data for all experiments). But say I want to run the experiment against some metrics that are only present in the other Data Source - do you intend to support that, or do you have any suggestions for workarounds? Thanks - I like GB so far ⭐

Hey! Would just be awesome to have a link from the metric directly to the definition under Metrics 😃 With increasing number of metrics it takes a while to find it in the Metrics list. Also to be able to sort team members would be super! 😄

It looks like the "SQL" and "Query builder" tabs in Metric > Query settings both combine together, as opposed to just using the contents of one or the other. Is that expected behavior? (It's not what I would've expected? But maybe that unlocks add'l functionality, not sure)

It would be great to have a “copy sql” button/link next to the queries when I am in a view query window. When I am looking to QA a metric etc I grab those queries and then run them elsewhere so that I can look at an individual user or parts of the query in detail. For me at least the most common thing to do in a view query window is expand the sql highlight all of it copy it and paste it into another tool for further exploration.

In Data Source settings, there's a "User Id Join Table (optional)" field. We don't link anon ids to user ids (as a policy). Sooo I left it blank, but it looks like that breaks any experiment that includes both anons and users. Is that field meant to be optional? If so, under what conditions?

Hey! Thanks again for a great product! 🙌 I have another feedback/feature request 😊 Would it be possible to add the possibility to use the

'{{ startDate }}' AND '{{ endDate }}'

parameters to Segments (and Dimensions..?) too? Context: We have users going in and out of certain habit groups, and I’d like to make sure to use the group they are in at the point of the experiment, and not before. I get errors trying to use these parameters now.. 🤔

Cute "time ago" grammar when an experiment starts in the future

Could there be a way to one-click start an experiment for any feature? I know there is a hint to start an experiment for a recently created feature on the Analysis -> Experiments page which pre-fills information from the feature definition, but I haven't found one for previously created features

Hello! We have been using Growthbook for at least a month now and its been a great experience 🙂. We ran into one situation that is a nice-to-have feature request and wanted to bring it up to the Growthbook team. We currently do some forced feature flag states via attributes by user IDs however we also wanted to be able to define "groups" of IDs that we can update. This can be done on our end and then we can pass a boolean attribute as well however that requires a larger degree of thought of where we store those groupings. An example would be creating "roll out" groups where we have

engineering

,

product

, and then

all-team

.

all-team

could be an

employee

attribute and a custom flag can be made for

engineering

and

product

but it would be nice to have an option of just creating custom groupings of attributes and setting a rule of

If "id" is in "engineering"

.

Hi, in Experiment Results with Date dimension, if there are no conversions per variation at a particular date, the query fails to count the number of users in the variation. I see this behavior in a low-traffic and low-conversion-% test - obviously this wouldn't be an issue for regular tests yielding conversions daily. See below the graph shows no users in a variation that has no conversions for that day, causing a seemingly large discrepancy in the split between the variations. I believe this could be fixed with a left join from __overallUsers to __stats In total aggregate results the numbers seem to be ok so only a minor issue any case

Hey Growthbook team! I need to include a

table( flatten( input => parse_json() ) )

sub-query as part of one of my metrics. The Metrics SQL editor is failing on the '=>' though. There's lots of additional context I can share, but wanted to keep this concise. Can I jump on a huddle with someone to troubleshoot? Thanks!

Hi team. Further to @numerous-hospital-35835’s issue above, I'm running into this warning on the Results tab of a new binomial experiment:

Warning: Missing data from the following variation ids: 0

I'm using a MySQL data source and the final query is not returning a row for variation 0 as it has no recorded metrics. I believe changing the final query to

SELECT
  u.variation,
  u.dimension,
  COALESCE (s.count, 0) as count,
  COALESCE (s.mean, 0) as mean,
  COALESCE (s.stddev, 0) as stddev,
  u.users
FROM
  __stats s
  RIGHT JOIN __overallUsers u ON (
    s.variation = u.variation
    AND s.dimension = u.dimension
  )

would produce the correct result, but understand that you'll want to test to see what else is affected 😁

Would it be nice to have setting to enable/disable A/B testing at least from UI? I’m asking this is while we are thinking to adopt AB testing in a long run but we would probably start with feature flags first. We are a small team. Dont want them to be overwhelmed suddenly by flooding questions about AB testing

I like the new feature flag versioning. Would like to not have to do it for every time I change a flag targeted towards my dev api token if possible.

We're changing the tag filtering UI. Let us know if you have any thoughts.

Hi! Are you familiar with this OpenFeature standard? Any thoughts on this? • https://blog.thepete.net/blog/2022/05/19/openfeature---a-standard-for-feature-flagging/ • https://openfeature.dev/

On MP data sources, there is a page view event setting, but it doesn't seem to be configurable

One thing I'd lightly flag - the Go lib feels very "non-Go-like" - there's no safe way to detect that feature/featuremap parsing has failed, as it always returns non-nil values from functions like

BuildFeatureMap

,

ParseFeature

, and

BuildFeatureRule

- even if the input is totally invalid. Offering more of a typical Go interface would require breaking changes though. Would you be open to a PR around this?

flagging that from what I can tell from the React sdk code path the tracking callback for experiment assignment is called during render rather than as a side effect (useFeature doesn't have a

useEffect

anywhere)

Hi there. Very new here so if this is posted in the wrong place, appologies. I was wondering if growthbook has or is looking at a import / export feature. Dealing with multiple teams working on different features, I was hoping to use a file as part of the code repositories that we can use to import on our CI CD pipelines. That way when a new flag needs to be added to the platform ther is no blocker between the dev team and or the admins working with growthbook. Basically manage our growthbook FF as part of our CICD pipelines...

Hey 👋 Is it possible to add a button on the

Features

page that exports a json file with the features viewable on the dashboard?

Hi! is there a way to have Growthbook store the latest features configuration to a file on the system? This would be like using a webhook but for storing the data to file(s) instead of sending it over to some other service. If i understand correctly right now the features data is fetched on every request, is that right?

Hello GrowthBook! And thank you again for a great tool! 🎉 We have a couple of use cases for metrics that are a bit tricky to handle right now, and were wondering about your thoughts on these. 1. Firstly, sometimes we want to analyze on a lower level than users. Examples could be sessions, pages or specific event types. For the latter, we have for example the type “search event” where the search team wants to minimize the number of typed letters before reaching the desired search result. We could technically do this using your brilliant new mapping tables of users to event_ids, but we would easily violate statistical assumptions, like very well written here. The article suggests a couple of methods to correctly estimate the variance in such cases (like the delta method and bootstrapping). Is this something you have thought about before, or that others need also? Anyone else here that have similar needs, and found a good work-around? 2. Secondly, sometimes we want to analyze and optimize on shares of something. An example: We sell groceries and recipes along with them. The products in an order can be added from our recipes or just from the users other needs. We ran an experiment on tagging the recipes that were climate friendly. We wanted to improve the share of recipes that were climate friendly, not just the # of such recipes sold. I.e. wanted the denominator to be total recipes sold, not users. Optimizely does something like this for changing denominators. Any thoughts on this? (btw, we tried to use recipes as activation metric, but we quickly ran into SRM as our treatment increased also the # of any recipe sold). 3. (also just a sidenote - as our currency is not dollars, it would be a nice to have to not have the $ sign on the revenue metric as it confuses some users 😊 ) Thanks again! 😊