Hi guys~ I noticed this SRM warning. but I’m not quite understand why it warns out. I feel the actual split show not big difference than expected.
f
fresh-football-47124
05/16/2022, 7:56 AM
Hi Herb. is this a new experiment?
fresh-football-47124
05/16/2022, 7:56 AM
has it been running long?
h
helpful-carpenter-3208
05/16/2022, 7:57 AM
Kind of new. Has been running for few days
f
fresh-football-47124
05/16/2022, 9:09 AM
how large are the numbers in each variation?
fresh-football-47124
05/16/2022, 9:23 AM
Copy code
Every experiment automatically checks for a Sample Ratio Mismatch and will warn you if found. This happens when you expect a certain traffic split (e.g. 50/50) but you see something significantly different (e.g. 46/54). We only show this warning if the p-value is less than `0.001`, which means it's extremely unlikely to occur by chance.
🙌 1
h
helpful-carpenter-3208
05/16/2022, 3:39 PM
currently the numbers are
19245/2291
, and the warning is gone. so when number of variations is lower than a magic number it will show the warning?