Hi guys~ I noticed this SRM warning. but I’m not q...
# announcements
h
Hi guys~ I noticed this SRM warning. but I’m not quite understand why it warns out. I feel the actual split show not big difference than expected.
f
Hi Herb. is this a new experiment?
has it been running long?
h
Kind of new. Has been running for few days
f
how large are the numbers in each variation?
Copy code
Every experiment automatically checks for a Sample Ratio Mismatch and will warn you if found. This happens when you expect a certain traffic split (e.g. 50/50) but you see something significantly different (e.g. 46/54). We only show this warning if the p-value is less than `0.001`, which means it's extremely unlikely to occur by chance.
🙌 1
h
currently the numbers are
19245/2291
, and the warning is gone. so when number of variations is lower than a magic number it will show the warning?