We have recently started using GrowthBook for A/B tests, and our initial tests were successful and error-free. However, we encountered an issue with our latest test, as we keep receiving a Sample Ratio Mismatch error message ~2 days after launching the test.
To address the SRM error, we made some updates and restarted the test (Phase 3) on Tuesday, 18th. Unfortunately, we have noticed that the same SRM error message has appeared again.
We understand that there could be several potential reasons for this issue. Can anyone suggest areas to investigate first, considering that the error message consistently appears around 2 days after the start of a new phase?
Any assistance or advice would be most appreciated!
f
fresh-football-47124
07/20/2023, 10:03 AM
what is the sample size?
fresh-football-47124
07/20/2023, 10:04 AM
have you changed the split at all while the experiment has been running?
fresh-football-47124
07/20/2023, 10:05 AM
sometimes cacheing serves users one side of an experiment
r
rich-noon-84967
07/20/2023, 10:10 AM
Thanks Graham, I think we have just now discovered the bug in our implementation 🤞 this will do the trick. Appreciate your speedy response though
w
white-fireman-22476
07/20/2023, 10:17 AM
@rich-noon-84967 I'm curious what the bug was in case other people have similar bugs in the future, we might be able to give better advice. Do you mind sharing?
r
rich-noon-84967
07/20/2023, 10:23 AM
I will report back on the bug issues if we find that the test reports error free now