Hi guys~ I noticed this SRM warning. but I’m not quite understand why it warns out. I feel the actual split show not big difference than expected.
05/16/2022, 7:56 AM
Hi Herb. is this a new experiment?
has it been running long?
05/16/2022, 7:57 AM
Kind of new. Has been running for few days
05/16/2022, 9:09 AM
how large are the numbers in each variation?
Every experiment automatically checks for a Sample Ratio Mismatch and will warn you if found. This happens when you expect a certain traffic split (e.g. 50/50) but you see something significantly different (e.g. 46/54). We only show this warning if the p-value is less than `0.001`, which means it's extremely unlikely to occur by chance.
05/16/2022, 3:39 PM
currently the numbers are
, and the warning is gone. so when number of variations is lower than a magic number it will show the warning?