https://www.growthbook.io/ logo
#ask-questions
Title
# ask-questions
c

clean-river-69825

10/24/2023, 7:59 AM
Hello everyone! 👋 In our first A/B experiment using GrowthBook, we noticed an interesting phenomenon. Around 20K users seemed to be experiencing both variants, even on the same day. It’s worth mentioning that we hadn’t altered the targeting conditions or the traffic distribution for the variants. The only unusual occurrence was an accidental phase change and temporary pause in the experiment. Can anyone shed some light on why these users might have received different variants? 🤔
b

billions-xylophone-11752

10/24/2023, 1:05 PM
Hey David - Sorry to hear! Can you tell me what stack you're using? (Which SDK & which data source type?). Can you also share what
Assignment Attribute
you're using for targeting? Are you seeing multiple exposures for these users over multiple days, or just the day of the phase change?
c

clean-river-69825

10/24/2023, 2:00 PM
In this case it was using the JS SDK and the Redshift data source. For targeting attribute we use the user id which is a string. We saw most of the multiple exposures during the day of the phase change, but a few more the rest of the days. Does it make sense that changing phases change also the variation that they get?
b

billions-xylophone-11752

10/24/2023, 2:04 PM
No, the only time a user could change variations is if the number of variations changed as a result of the new phase, or if the
Experiment Key
changed as we use a combination of the
Assignment Attribute
and the
Experiment Key
when doing bucketing.
c

clean-river-69825

10/24/2023, 2:07 PM
Changing the phase stops the experiment but it doesn’t promote a specific variation automatically, right?
b

billions-xylophone-11752

10/24/2023, 2:11 PM
Changing the phase doesn't automatically release a specific variation - but that is an option if an experiment is stopped.
f

fresh-football-47124

10/25/2023, 1:34 AM
Did you change the split percentages perhaps?
c

clean-river-69825

10/25/2023, 6:44 AM
Nope, we didn’t
I’m checking the experiment audit log and I found a weird seed when we restarted the experiment changing an attribute condition. Apart from adding the condition it changed the
seed
. In previous changes the seed was the tracking key, in this change is a uuid. Does it make sense? We didn’t change the tracking key 🤔
w

worried-jackal-47620

03/26/2024, 2:29 PM
We're facing this same
seed
vs tracking key issue. How did you guys overcome this?
14 Views