Hello everyone! 👋 In our first A/B experiment using GrowthBook, we noticed an interesting phenomenon. Around 20K users seemed to be experiencing both variants, even on the same day. It’s worth mentioning that we hadn’t altered the targeting conditions or the traffic distribution for the variants. The only unusual occurrence was an accidental phase change and temporary pause in the experiment. Can anyone shed some light on why these users might have received different variants? 🤔

Hey David - Sorry to hear! Can you tell me what stack you're using? (Which SDK & which data source type?). Can you also share what

Assignment Attribute

you're using for targeting? Are you seeing multiple exposures for these users over multiple days, or just the day of the phase change?

In this case it was using the JS SDK and the Redshift data source. For targeting attribute we use the user id which is a string. We saw most of the multiple exposures during the day of the phase change, but a few more the rest of the days. Does it make sense that changing phases change also the variation that they get?

No, the only time a user could change variations is if the number of variations changed as a result of the new phase, or if the

Experiment Key

changed as we use a combination of the

Assignment Attribute

and the

Experiment Key

when doing bucketing.

Changing the phase stops the experiment but it doesn’t promote a specific variation automatically, right?

Changing the phase doesn't automatically release a specific variation - but that is an option if an experiment is stopped.

Did you change the split percentages perhaps?

Nope, we didn’t

We're facing this same

seed

vs tracking key issue. How did you guys overcome this?