#announcements
Title
d
dry-eye-65136
05/10/2023, 4:06 PMHi everyone , if i'm already running an a/b test experiment and i decided to start using namespaces with the same experiment , would it cause users to switch variants ?
f
fresh-football-47124
05/10/2023, 6:05 PM
no, they won’t switch, but some people who were included may be excluded
d
dry-eye-65136
05/11/2023, 9:10 AMyes exactly that's what i'm asking about for people who were already included
so may be the challenge here , at least from what i've seen is that the decision to either use namespaces or not is always based on the next experiment, which is not necessarily be known at this time
just asking if any sort of practice to do so i'm able may be to make this decision later on?
may be would i use the namespaces from day 1 , set the experiment exposure to 100% , and use the namespace splitting as the experiment exposure , not sure if this would work @fresh-football-47124 ?
@fresh-football-47124 not sure if the above practice could work
f
fresh-football-47124
05/13/2023, 5:44 PM
you’re doing 100% of the namespace or exp exposure?
d
dry-eye-65136
05/13/2023, 5:46 PMNo I mean to solve the problem of deciding if I want to include other experiments in the same namespace later ( if I don't know now if I want to ) , to always use namespace anyways and use it as the experiment exposure instead
The problem is that when running an experiment, its hard to decide if I want to run another experiment within the same namespace now
I only know when I plan for the next experiment to run , and discuss with the rest of stakeholders either to run it exclusively or not
f
fresh-football-47124
05/13/2023, 5:49 PM
right
we find that most people over estimate how much experiments will interact
meaningful interaction effects are usually rare
d
dry-eye-65136
05/13/2023, 5:54 PMThis is interesting!
Perfect, so the above approach would work even if I'm not adding other experiments later
f
fresh-football-47124
05/13/2023, 5:56 PM
how much of the namespace are you using for each experiment?
d
dry-eye-65136
05/13/2023, 5:59 PMCurrently I have one experiment without using namespaces at all , I'm thinking of an approach be able to decide later while pushing more experiments to be able to decide either to run it exclusively or not
The question is if for example the current experiment exposure is 20% without namespaces , if I changed to use a namespace with the same key and allocated 20% of the namespace to that experiment, then set the experiment exposure to 100% , would this still cause issues making people switch variants ?
f
fresh-football-47124
05/13/2023, 6:02 PM
ah, yes, I think it would - I think we use a different hashing seed for the namespace on purpose
But let me check the code
👍 1
ya, when hashing namespaces we change the seed
d
dry-eye-65136
05/14/2023, 5:32 AMGreat , this is good to know thanks a lot
4 Views