Hi, a general question. what’s this

suspicious result

means? Is there a explanation doc for these kind of labels?

Hello 👋 Did anyone managed to integrate growthbook with nextjs middleware + Segment? Trying to utilise the trackingCallback but it seems like

analytics-node

and middleware might not be so compatible.. Been patiently waiting for the follow up on this article: https://dev.to/jdorn/ab-testing-with-the-new-nextjs-12-middleware-54j6 😄 thanks!

hi all! is there a recommended way of handling the case when self-hosted (or any) growthbook service goes down and other services rely on it? the naive solution is to hardcode fallback values in the codebase but is there any other option?

Hi there! Can I use PostgreSQL instead of MongoDB to store internal information (information about experiments, feature flags, screenshots, rules, etc)? In a YouTube video, it was said that GrowthBook uses MongoDB to store internal information.

Hi for using bigquery using env variable do we need to give OAUTH permission to the service account?

Hello! Our team has been running Growthbook tests for awhile now and are starting to switch over to the Visual Editor tool. Our frontend platform is Gatsby 5. During the experiment set up, I am able to click the "Open Visual Editor" button which leads me to a window with the Visual Editor panel where I make my edits. That all works fine. But when I start the experiment, it never seems to actually run. I've created a very simple Gatsby repo and link to test this out. Here are all the relevant URLs if needed. I also installed the Growthbook Chrome Extension too. The attached screenshot is how I set up the SDK. My GrowthBook Experiment Page Build Link that we're targetting to (hosted by Netlify) GitHub Repo Any help / insight would be appreciated!

Hello, I'm currently implementing Vue.js SDK with composition API in our Nuxt SPA application. Feature flags works fine, I'm able to fetch them. When it comes to Visual editor it seems like I can't see changes done in expirement so probably I missed something. Before my implementation, we have had growthbook implemented by GTM described in this doc: https://docs.growthbook.io/guide/google-tag-manager-and-growthbook. In this case expirements created with Visual editor works well. My question is: Is it possible to use Visual editor in expirements in SPA application using only Vue SDK? Why feature flags works well, while I cant see eny expirements in response configured in panel? Unfortunately visual editor is very important tool for my organization so Im wondering if we really should implement it in our codebase or leave it in gtm? Appreciate any advices, thanks

Hi GB team. We are using Clickhouse as our data source. And we found a timezone issue about the experiment query. Bellow is a query sample about experiment. It used

toDateTime

to filter the timestamp. And we use the UTC time in the experiment settings. but by default Clickhouse is using the server timezone which is not UTC. so the fix should be add UTC in the toDateTime function like this

toDateTime('2023-02-24 17:00:00', 'UTC')

Can anyone help on this?

__experiment as (-- Viewed Experiment
    SELECT
      e.user_id as user_id,
      toString(e.variation_id) as variation,
      e.timestamp as timestamp,
      e.timestamp as conversion_start,
      
      dateAdd(hour, 720, e.timestamp) as conversion_end
    FROM
        __rawExperiment e
    WHERE
        e.experiment_id = 'SOME_EXP_ID'
        AND e.timestamp >= toDateTime('2023-02-24 17:00:00')
        
    )

We’re running an A/A experiment (random assignment to two variants, but user sees the same product). Typically there’s been small variation for metrics in Growthbook but most of the violin plots are centered at close to 0 now that ~50,000 users have been assigned. Our assignments should be random. But one of our core binomial metrics is showing a 2.3% percent drop in the “experiment” group. Growthbook computes a 3.68% chance that the variant will beat the control, when I’d expect ~50% chance, since the product is the same. This is a ratio metric, with the denominator set to one of the other states in our funnel. The denominator’s metric shows the expected 0% change. There’s a conversion window on the metric, but I tried copying the metric and 1) setting the denominator to all experiment users and 2) modifying the conversion window, and the percent size / probability distribution is roughly the same. This experiment contains has been running for 4 months. I created a new experiment phase and looked at just data from the last 1 month, and I wasn’t able to reproduce this. Seeing a 0% percent change for the new phase. I had a few questions, and I’m open to any advice people have on interpreting the results of A/A tests in Growthbook: • What could help explain this outcome? Are these false positives expected, even after such a long experiment length? • Could this suggest a flaw in our assignment logic or our metric definition in any way? Again, the difference in most other metrics is close to 0, and the second phase showed a 0% difference, so the first result (on 4x the users) is confusing. • Does this suggest that I should be skeptical of any experiment result that moves the metric less than ~2.3%? Is the A/A test helping me understand the underlying natural variance for the metric in any way?

what kind of API rate limit do you have? in your hosted version?