Good morning, does anybody have an idea how a %50-...
# ask-questionsf
famous-battery-55941
08/19/2024, 8:58 AMGood morning, does anybody have an idea how a %50-%50 experiment could distribute %15-%85, for a single day? Screenshot in 🧵
famous-battery-55941
08/19/2024, 8:59 AMIt has been distributing well but then yesterday seems to have gone off the rail
famous-battery-55941
08/19/2024, 9:15 AM• If there were non-Growthbook errors, I'd expect the distribution numbers to be roughtly equal but cumulatively less.
• If there were something with the algorithn/setting I would expect it be consistently distributed at a particular ratio
So not really sure where to start investigation for it...
famous-battery-55941
08/19/2024, 3:18 PMGraph seems to be sorting itself out for today as well
r
rhythmic-agent-34208
08/19/2024, 8:17 PMThere are several variables at play. Are you still seeing an unequal distribution?
d
dry-ambulance-7044
08/20/2024, 8:15 AMWe saw an almost 50-50 distribution yesterday and saturday despite seeing almost exactly a 33-66 distribution on sunday - no changes to experiment or product along the way
r
rhythmic-agent-34208
08/20/2024, 9:28 PMWhat's your setup? How are you hosting GB? And, @dry-ambulance-7044, are you on the same instance as @famous-battery-55941? Or did you both independently see these variations on the same day?
f
famous-battery-55941
08/21/2024, 12:45 PMWe are on the same instance. We have a web page which evaluates the variants on page load. It seems to be 50%-50% with the exception of Sunday. (Checked today as well)
r
rhythmic-agent-34208
08/21/2024, 8:00 PMIt's hard to say for sure (there aren't any obvious answers), but as long as your experiment health looks good, then you're still in shape.
It does look lie it's the variant in each graph that fell short, which usually means something is preventing the trackingCallback occurring for the variant.
50 Views