loud-hydrogen-50191
08/14/2023, 4:59 AMSRM happens when the actual traffic split is different from what you expect.
For this test, the p-value of the SRM check is 6.8e-7.That means there's only a 1 in 1470588 chance the observed traffic split happened by random chance.
May i know how GB determines this SRM for a single experiment, like is there an sql available for me to run ?
Is it affected by the Data Sources
-> Join Tables
or perhaps Experiment Assignment Queries
?fresh-football-47124
loud-hydrogen-50191
08/14/2023, 5:46 AMfresh-football-47124
loud-hydrogen-50191
08/14/2023, 8:38 AM{
"defaultValue": false,
"rules": [
{
"condition": {
"deviceId": "test-forced-on"
},
"force": true
},
{
"condition": {
"loginStatus": 0
},
"variations": [
false,
true
],
"coverage": 1,
"weights": [
0.5,
0.5
],
"key": "MyExptKey",
"hashAttribute": "deviceId"
}
]
}
Warning: Sample Ratio Mismatch (SRM) detected. We expected a 50/50 split, but observed a 50.4/49.6 split (p-value = 9.2e-7). There is likely a bug in the implementation.
1. Sorry, i misunderstood your question. It's 50.4/49.6 split.
2. And i assume GB is able to calculate the split number is because of all of the queries (i currently have 10 of them) for this particular experiments
3. And i'm wondering which field is used to calculate the split number ? I can see count, main_sum, main_sum_square