https://www.growthbook.io/ logo
#ask-questions
Title
# ask-questions
i

incalculable-elephant-73095

12/15/2023, 2:48 PM
Hi. I`ve got a problem with the test:
Copy code
Sample Ratio Mismatch (SRM) detected. P-value below 0.001
Sounds like a bug. I went through the dimensions, and it seems the most crucial point here is traffic with referral = t.co from the UK Previously, there were no issues, and now I have no idea what I can do and how to fix this problem. Can I exclude somehow this anomaly from the existing test?
h

helpful-application-7107

12/15/2023, 3:11 PM
Your dimensions can't always reveal where the problem is coming from and if some dimensions are too small they won't show the issue at all. Normally the issue comes up in your SDK implementation where for some reason you aren't tracking experiment exposures right after looking up feature values so some users in one test variant are being bucketed more than others.
i

incalculable-elephant-73095

12/15/2023, 3:13 PM
So, should I check SDK installation on the website in terms of bucketing?
h

helpful-application-7107

12/15/2023, 3:14 PM
Yes, that's what I'm suggesting. Sometimes SRM warnings are false positives, though, and if this issue has never happened on other experiments with similar set ups, and the difference in the buckets isn't that much from what you expect, then maybe it is a false positive. You can exclude a group from the test using the Custom SQL filter in the analysis settings for your experiment, but I don't think that will solve the problem if there truly is one.
❤️ 1
i

incalculable-elephant-73095

12/15/2023, 3:27 PM
Ok, we`ll check it. How to exclude them from the test?
r

rhythmic-agent-34208

12/20/2023, 1:06 AM
Hi Volodymyr, are you specifically wanting to exclude users ​*from the experiment *​who are in the UK and who were also referred by t.co? If so, you can use a Targeting Condition to exclude them from the experiment. If you want to exclude those users ​*from your analysis*​, Luke's suggestion to use the Custom SQL filter will do that for you.
i

incalculable-elephant-73095

12/20/2023, 8:27 AM
Got it. Appreciate your help!
b

brief-honey-45610

12/20/2023, 3:01 PM
You're very welcome! Let us know if you have any other questions. 🙂
👍 1
4 Views