https://www.growthbook.io/
Join Slack
Hi. I could use some help with a simple URL redirect experiment using the HTML snippet installation ...
# ask-questions
s
Hi. I could use some help with a simple URL redirect experiment using the HTML snippet installation and Segment. I'm running this on a Shopify store. Here's the code I have on the site:
Copy code
<head>

    ...

    <!-- Growthbook -->
    <script>
      const devMode = localStorage.getItem('devMode')?.toLowerCase();

      window.growthbook_config = window.growthbook_config || {};
      window.growthbook_config.enableDevMode = window.Shopify?.theme?.role !== 'main' || devMode === 'true';
    </script>
    <script
      async
      data-client-key="sdk-QnM1u88NYaRA75z"
      data-use-sticky-bucket-service="cookie"
      src="<https://cdn.jsdelivr.net/npm/@growthbook/growthbook/dist/bundles/auto.min.js>"
    ></script>
</head>
And in my redirect experiment, I'm getting a sample mismatch error. The test is a 50/50 split, but looking at the counts, it looks like there's more traffic hitting the control than the variant, and not evenly split. I'm not sure if I'm doing something wrong in my set up or what.
r
Hi Sameer! This can sometimes happen if the redirect happens before the tracking callback has a chance to fire, which prevents users from getting counted in the experiment. To counteract this, you can add a delay like this:
Copy code
window.growthbook­_config.navigateDelay = 300; // Adjust this value as needed
s
Wouldn't this cause a long and noticeable wait before the redirect happens?
r
It should be negligible, but testing will confirm. Another technique you can use to improve experiment results is to set up redirects for the control and variant. This way, the UX is the same.
đź‘Ť 1
s
I relaunched the test with the navigation delay set. I tested using the Segment debugger to ensure the “Experiment Viewed” event fires. And I’m still seeing this error. The split for this test is now 70% for control and 30% for the variant.
@strong-mouse-55694 Bumping this. Any other ideas?
r
I think there could be a couple of things. - Is your total users around 800? - Can you share the whole code for GB implementation script? - We do have an SDK update coming soon that improves redirect behavior.
s
Hey @strong-mouse-55694 - The code I provided in the beginning of this thread is all the code I have set up for the GB implementation. Per the GB docs, GB automatically calls Segment with an "Experiment Viewed" event if window.analytics is available. And I see GB calling that
r
Thanks for sharing that. I checked with our engineers. The only known cause for this happening is the latency mentioned above—where the redirect happens before the tracking callback fires. Currently, the only solution is to bump the navigation delay. As I mentioned, there's an update coming that'll improve this, but it hasn't shipped yet.
60 Views
Open in Slack
PreviousNext
Powered by

GrowthBook Users

Open source platform for stress free deployments, measured impact, and smarter decisions.

Powered by