set the channel topic: Provide feedback or bug reports here. Use #request-features to request new features. All other questions should go in #ask-questions or an SDK-specific channel.

set the channel description: Provide feedback or bug reports here. Use #request-features to request new features. All other questions should go in #ask-questions or an SDK-specific channel.

[2.4.0 - self-hosted] When editing a metric to change the cap user values to go from an absolute cap to a percentile cap the edit doesn't save and no error is shown. (attempted

0.99

and

.99

etc as values in case there was some validation happening that it was between 0 and 1)

Hi! About ad-hoc reports - love then! 🎉 A couple of improvement suggestions. 1. It would be great if one could add an end date in retrospect. Right now I cant find a way of adding it again if it has been removed (picture 2) 2. Would be great to be able to copy an ad hoc report, or create another one from the existing report (picture 1)

Hi! I’ve just realized that the p-value that can be found in the frequentist report is based on two-sided hypothesis test. Is there a way to choose what type of hypothesis test I need (two-tailed or one-tailed) so I get the correct p-value? I actually found one-sided calculations in the source code but can’t find the appropriate setting in the interface.

Hi, it seems image upload for variants has stopped working on GB Cloud. I used it before, but now when I upload images (PNG), they don't show anymore.

Enjoying the project, thank you 🙏 It'd be great to open source the docs (if not already) to allow the community to contribute With Google Optimize retirement you'll no doubt have an influx of people coming across from that product (myself included) The journey of implementing growthbook via GTM was not the smoothest During installation I found this page misleading for 2 clear reasons: https://docs.growthbook.io/guide/google-tag-manager-and-growthbook 1. It's not clear in this configuration that you need to pass

url

as a user attribute a. Most other web based A/B testing packages detect URL via their own script and don't rely on that being passed through b. When you first create an account, the URL attribute is set up automatically c. The visual editor prompts you to do URL targeting d. URL targeted experiments wont work unless url is passed as a user attribute, I had to scrutinise a lot of pages to detect this issue. 2. It'd be great to provide an example in javascript of generating a UUID for anonymous users, a reliable example was hard to come by. but additionally when using GTM it needs to be an instance which does not use ES6

In SDK integration for GA4, it is recommended to get the user Id and/or Client ID attribute, and adjust the trackingCallback to send the experiment exposure. It is not obvious what adjustments the doc is referring to since passing the ID to the GA event should not be necessary.

I understand it’s a very long shot, but are there any plans to implement some functionality that would allow us to detect primacy and novelty effects? The literature is very scarce on the methodology but I’ve been trying to decypher this paper: https://arxiv.org/pdf/2102.12893.pdf but been having trouble

Was wondering what the plans are for slack. Would love to see things like: 1. Alert for experiment reaching significance 2. Alert for when a team stops and experiment

So when we implemented the self hosted version of Growthbook there was no information about implementing our own database solution and we just now sadly lost the data in our mongo db instance after our server ran out of RAM and we restarted it. Are there some docs regarding how to connect it to a separate database or any guidelines regarding how it should be implemented? 🙂

Hey Team, My colleagues and myself are interested in a session where different stakeholders across our org would like to ask questions and possibly a quick demo to outline features, roadmap, etc. How can I initiate this ?

Would be awesome to be able to see histograms of the metric data, and also for there to be normality- and heteroscedasticity test integrated. Do you have some thoughts on that? 😊

It would be really helpful if it was possible to duplicate tests between projects without loosing the targeting setup and feature configuration

Ahoi @fresh-football-47124 and @future-teacher-7046, the GrowthBook documentation page is down with 402 payment required. Can you bring it up again, please? Alternatively, is there a locally renderable version available as well? FYI @adventurous-dream-15065

Hi, there's an annoying lil bug with eventName on metrics. It doesn't seem to save the first time I create a new metric. After saving I have to edit and enter the eventName in again for it to save properly. Note: I've only tried this via duplicate metric.

Still having big problems with old data sources/metrics, that have been severely exacerbated by the new experiments system Context: we originally used Growthbook directly with Mixpanel (and our very oldest archived experiments are set to it), but moved to BigQuery We archived all the Mixpanel metrics (and seemingly these are now entirely invisible in the metrics list?), but it won't let us delete the Mixpanel data source, and there's no concept of archiving It was super annoying but not that big a deal - we just renamed it to

zzz do not use: Mixpanel

, though sadly it still appears first and is the default But with the new experiments system, every single experiment gets assigned to it automatically. Every single one requires our staff to go in and realize it's wrong and to update it, and if they don't, the results will be wrong/confusing Please could this be fixed somehow? Archiving data sources, or allowing us to delete the metrics, or setting a default source or something

We did also hit one other super annoying problem with the new experiments system. Overall we do really like it, it's much easier. But it's far less capable/expressive with complex conditions In many cases we use features more as sources of config with various rules, not just for rollouts etc. A lot of our features look like this - where we have long-standing/maintained behavior that continues after the actual analysis has long-since finished, but we need to preserve the behavior for the exact specific users In a lot of these cases, we also need to have the same experiment apply to multiple features (this seems to work okay) The key problem is that we sometimes need to have combined rules - i.e. additional conditions beyond those in the experiment targeting. e.g. if the experiment is on this variant, and the user is on the web version of the app, use this value. Whereas if they're on another version, use another value Ideally it'd be possible to add additional conditions on experiment rules, like the legacy ones allow

My company is using the paid version of growthbook and we discovered the js-sdk doesn’t work with esm. See https://github.com/growthbook/growthbook/issues/509 for details I opened PR #1719 to fix the issue. Please have someone review it so we can continue with our integration. Thank you!

Hi, just a few thoughts/suggestions on feature flags while I'm working through my first experiment setup: • When using a feature flag in an experiment, it seems to create a two-way link between experiment and feature flag, causing the feature flag to have unpublished changes and revert to draft mode. You then have to go and republish the feature flag, even though nothing about the feature flag itself changed. That's kind of confusing and cumbersome. Why does the feature flag need to know or care how it's being used? (It's nice to see within the feature flag page how it may be overridden by an experiment, but to me that doesn't feel like a change to the feature flag itself, that should require a separate publish, if that makes sense.) • Relatedly, if an experiment uses a feature flag and then that experiment is deleted, the feature flag still has a broken reference to the deleted experiment, which has to be manually removed. • When using a feature flag in an experiment, it'd be nice if the control variation defaulted to the feature flag's default value. E.g., if I'm using a feature flag named

trial-length

with a default value of 7, an experiment using

trial-length

should default to 7 for the control value Thanks!

Also it'd be great if we could pause an experiment to make changes to e.g. linked features and visual editor changes, especially if it hasn't yet gone live on all environments. Right now it seems like you have to stop the experiment, duplicate it, delete the old one, make the changes, fix the experiment key, delete the old broken experiment links out of the feature, then start the new experiment... (Unless I'm missing an easier way to go about this.) I understand it's not ideal to change an experiment partway through running it, but sometimes it's necessary to start it in like a dev environment to make sure everything's working, and make changes accordingly

This message was deleted.

Just setup metric reporting on GA4 data streaming into BQ. After following GB's progress in the background for a few years, I was excited to give this a run. My first impressions: 🤯 ❗❗❗ The reporting interface is beautiful. The controls on everything are robust. I LOVE being able to spin up metrics in seconds. I've only scratched the surface here, but 👏 for what y'all have done.

Oh, sorry, was I supposed to give critical feedback, here? 😛

Hi, I think I’ve spotted a bug regarding the Metric Conversion Windows option in Experiment Settings. I use a ratio metric counting the ratio between new paying users on new free trial users. The user has 3 days to become a paying user after he started the free trial, so there is a conversion window of 72 hours. BUT he might take the free trial as soon as he subscribe, hence the conversion window for the free trial is only 3 hours. When I compute my ratio metric, I very little difference on the denominator part in the UI, either with include or exclude in progress conversions. --> the user count from the denominator might use the 3 hours window (the one from the free trial metric) --> BUT for the metric to be usable, a user should not be counted untill the max of the conversion windows (numerator and denorminator) are elapsed

UX suggestion : The result page of an experiment always opens with Dimension = None. Since we use dimension to split revenues & retention between new and existing users we have to chose the dimension manually each time we go to the results --> please make the result tab open with the previously used dimension already filled. It would make the use of the tool far smoother and help new comers to find their the right numbers.

Hi, we are finding these graphs in the Date Cohorts Dimension kind of useful not also confusing e.g Why are the colours the same for some of the variants? Also, it seems some of the lines are missing in that screenshot I don't see the line for the Control which happens to be the same colour as Great Service?

Hi Team, Thanks for Growthbook, you guys are doing a great job. We are integrating growthbook into our application, and are currently using pro version. We have found couple of probable blockers 1. Probably this is a bug, spotted an issue which is preventing us from publishing a feature flag, I have tracked it here 2. Also we have a feature request, I can see that currently audit logs do not track saved groups, I would be nice if they track changes to them - created a feature request here I would be nice if I can get a ETA on the above two. Thank You.

Bug report since last major version deployment: I create a new experiment from the Experiments menu. Then I add features and configure it. I click

view feature

to check if all is setup but ... the experiment override rule appears twice. It does that for 2 weeks or so, this is very confusing and I manually remove the duplicated experiment from each feature at every test.

We've just found that one of our SDK connections' project filter has disappeared? Caused quite a major bug for all users visiting our site. Has anyone else seen anything like that? ID sdk_19g61pldef9nwj (I have since re-set the project filter)