This message was deleted.
# ask-questionsw
worried-planet-2191
10/26/2023, 11:42 AMThis message was deleted.
👀 1
h
helpful-application-7107
10/26/2023, 3:15 PMThe drift always seems to overweight the control,I'd love to see other evidence of this, but I'm not seeing anything too crazy here. This kind of imbalance in that screenshot is fairly normal. The associated p-value is around 0.14.
helpful-application-7107
10/26/2023, 3:16 PMFor guardrail metrics, its making all variants look worse due to lower conversions.It shouldn't cause lower conversions on average which is what our metrics look at. Sometimes chance imbalance causes spurious results, but just a slight imbalance in numbers shouldn't cause the average conversion rate to be different.
p
prehistoric-king-13950
10/27/2023, 9:24 AMAmazing thanks @helpful-application-7107 . I’ll keep an eye on it and keep a log of consistent results where the drift keeps going into control and come back here.
140 Views