Hi All, Just started using GB and loving it so fa...
# ask-questions
p
Hi All, Just started using GB and loving it so far. I’ve noticed a very slight drift in ratios with all of the experiments I’ve been running, thought its not enough to cause an SRM alert or past 0.01 at the moment. I’m using react/next with Mixpanel as my data source. Screenshot attached. The drift always seems to overweight the control, which makes me think its an implementation issue as if something isnt loading in time and more users are defaulting into the control. Growthbook debugger says everything is working normally. For guardrail metrics, its making all variants look worse due to lower conversions. I guess two questions. Is this normal?, and if not can someone point me where to go to to further debug or investigate why it might be happening.
👀 1
h
The drift always seems to overweight the control,
I'd love to see other evidence of this, but I'm not seeing anything too crazy here. This kind of imbalance in that screenshot is fairly normal. The associated p-value is around 0.14.
For guardrail metrics, its making all variants look worse due to lower conversions.
It shouldn't cause lower conversions on average which is what our metrics look at. Sometimes chance imbalance causes spurious results, but just a slight imbalance in numbers shouldn't cause the average conversion rate to be different.
p
Amazing thanks @helpful-application-7107 . I’ll keep an eye on it and keep a log of consistent results where the drift keeps going into control and come back here.