Hi, I'm hoping to get more information on how Sample Ratio Mismatches are detected. We have experiments that are flagged as having an SRM but the displayed splits are rounded to a 10th of a percent whereas the true splits should be irrational numbers (6 variants with equal allocation of the full audience) so I'm wondering if the SRM flag has something to do with rounding that is happening before the proportions test. Any info is much appreciated! (cc @ambitious-apartment-58735)

yeah, that's likely the cause. We take the total test traffic and multiply by the expected split to use in our comparison. So it's expecting

.167*total

instead of

.166666*total

. With enough traffic, that could cause a false positive for the SRM test

Thank you! @ambitious-apartment-58735 Thoughts on if we can still use this feature for QA?

I think we’d end up running the check independently — or we can insist on numbers of cells that lead to rational splits 😆. @future-teacher-7046 would it be easy for you to increase precision there? Is our data (sometimes millions of users/cell) atypically large for growthbook users?

Yeah, we can increase precision pretty easily. Millions of users is not atypical for GrowthBook, but 6-way splits are less common.

Ah, makes sense. Thanks!