https://www.growthbook.io/ logo
#announcements
Title
# announcements
l

loud-hydrogen-50191

08/14/2023, 4:59 AM
Hi team GB, it's my first time encountered an SRM ..
Copy code
SRM happens when the actual traffic split is different from what you expect.
For this test, the p-value of the SRM check is 6.8e-7.That means there's only a 1 in 1470588 chance the observed traffic split happened by random chance.
May i know how GB determines this SRM for a single experiment, like is there an sql available for me to run ? Is it affected by the
Data Sources
->
Join Tables
or perhaps
Experiment Assignment Queries
?
f

fresh-football-47124

08/14/2023, 5:37 AM
it could be caused by the assignment attribute of the experiment being different than the assignment in the experiment report
but usually they are caused by changing of the split mid experiment
l

loud-hydrogen-50191

08/14/2023, 5:46 AM
hi @fresh-football-47124 thanks for the quick reply. I have checked that 1. the hashed attribute did not change, 2. the experiment start time is above the last feature change i can see in the audit log. 3. split was not modified
f

fresh-football-47124

08/14/2023, 7:01 AM
what split do you have?
l

loud-hydrogen-50191

08/14/2023, 8:38 AM
hi Graham, sorry for the late reply, it's 50/50
Copy code
{
  "defaultValue": false,
  "rules": [
    {
      "condition": {
        "deviceId": "test-forced-on"
      },
      "force": true
    },
    {
      "condition": {
        "loginStatus": 0
      },
      "variations": [
        false,
        true
      ],
      "coverage": 1,
      "weights": [
        0.5,
        0.5
      ],
      "key": "MyExptKey",
      "hashAttribute": "deviceId"
    }
  ]
}
Copy code
Warning: Sample Ratio Mismatch (SRM) detected. We expected a 50/50 split, but observed a 50.4/49.6 split (p-value = 9.2e-7). There is likely a bug in the implementation.
1. Sorry, i misunderstood your question. It's 50.4/49.6 split. 2. And i assume GB is able to calculate the split number is because of all of the queries (i currently have 10 of them) for this particular experiments 3. And i'm wondering which field is used to calculate the split number ? I can see
count, main_sum, main_sum_square
9 Views