Hi I could use some help with a simple URL redirect experime GrowthBook Users #ask-questions

Hi. I could use some help with a simple URL redire...

straight-agent-66253

08/06/2024, 7:25 PM

Hi. I could use some help with a simple URL redirect experiment using the HTML snippet installation and Segment. I'm running this on a Shopify store. Here's the code I have on the site:

Copy code

<head>

    ...

    <!-- Growthbook -->
    <script>
      const devMode = localStorage.getItem('devMode')?.toLowerCase();

      window.growthbook_config = window.growthbook_config || {};
      window.growthbook_config.enableDevMode = window.Shopify?.theme?.role !== 'main' || devMode === 'true';
    </script>
    <script
      async
      data-client-key="sdk-QnM1u88NYaRA75z"
      data-use-sticky-bucket-service="cookie"
      src="<https://cdn.jsdelivr.net/npm/@growthbook/growthbook/dist/bundles/auto.min.js>"
    ></script>
</head>

And in my redirect experiment, I'm getting a sample mismatch error. The test is a 50/50 split, but looking at the counts, it looks like there's more traffic hitting the control than the variant, and not evenly split. I'm not sure if I'm doing something wrong in my set up or what.

rhythmic-agent-34208

08/06/2024, 8:06 PM

Hi Sameer! This can sometimes happen if the redirect happens before the tracking callback has a chance to fire, which prevents users from getting counted in the experiment. To counteract this, you can add a delay like this:

Copy code

window.growthbook_config.navigateDelay = 300; // Adjust this value as needed

straight-agent-66253

08/07/2024, 1:26 AM

Wouldn't this cause a long and noticeable wait before the redirect happens?

rhythmic-agent-34208

08/07/2024, 1:23 PM

It should be negligible, but testing will confirm. Another technique you can use to improve experiment results is to set up redirects for the control and variant. This way, the UX is the same.

👍 1

straight-agent-66253

08/08/2024, 5:43 AM

I relaunched the test with the navigation delay set. I tested using the Segment debugger to ensure the “Experiment Viewed” event fires. And I’m still seeing this error. The split for this test is now 70% for control and 30% for the variant.

straight-agent-66253

08/09/2024, 1:43 AM

@strong-mouse-55694 Bumping this. Any other ideas?

rhythmic-agent-34208

08/09/2024, 4:15 PM

I think there could be a couple of things. - Is your total users around 800? - Can you share the whole code for GB implementation script? - We do have an SDK update coming soon that improves redirect behavior.

straight-agent-66253

08/12/2024, 4:57 PM

Hey @strong-mouse-55694 - The code I provided in the beginning of this thread is all the code I have set up for the GB implementation. Per the GB docs, GB automatically calls Segment with an "Experiment Viewed" event if window.analytics is available. And I see GB calling that

rhythmic-agent-34208

08/12/2024, 7:34 PM

Thanks for sharing that. I checked with our engineers. The only known cause for this happening is the latency mentioned above—where the redirect happens before the tracking callback fires. Currently, the only solution is to bump the navigation delay. As I mentioned, there's an update coming that'll improve this, but it hasn't shipped yet.

55 Views

Open in Slack

Previous Next