prehistoric-beard-84272
09/20/2023, 7:42 AM{
"status": 200,
"features": {},
"experiments": [
{
"key": "test_aa_v2",
"status": "running",
"variations": [
{
"css": "",
"js": "console.log(\"Version0\");",
"domMutations": []
},
{
"css": "",
"js": "console.log(\"Version1\");",
"domMutations": []
}
],
"hashVersion": 2,
"hashAttribute": "deviceId",
"urlPatterns": [
{
"include": true,
"type": "regex",
"pattern": ".*\\/products\\/.*"
}
],
"weights": [
0.5,
0.5
],
"filters": [],
"seed": "test_aa_v2",
"phase": "0",
"coverage": 1,
"meta": [
{
"key": "0"
},
{
"key": "1"
}
]
}
],
"dateUpdated": "2023-09-18T16:50:28.162Z"
}
This test runs on a PDPs and I just compare how often the ATC button is clicked, which statistically should be very even across a high number of events.
Any ideas why results lean to one side?fresh-football-47124
helpful-application-7107
09/21/2023, 4:47 PMhelpful-application-7107
in that thread. Please take a look and let me know if you have follow-up questions.prehistoric-beard-84272
09/22/2023, 8:13 AMhelpful-application-7107
09/25/2023, 4:25 PMv1
that you ran?prehistoric-beard-84272
09/25/2023, 4:39 PMhelpful-application-7107
09/25/2023, 4:42 PMprehistoric-beard-84272
09/25/2023, 4:42 PMhelpful-application-7107
09/25/2023, 4:45 PMprehistoric-beard-84272
09/25/2023, 4:45 PMhelpful-application-7107
09/25/2023, 4:45 PMprehistoric-beard-84272
09/25/2023, 4:47 PMhelpful-application-7107
09/28/2023, 4:12 PMprehistoric-beard-84272
09/29/2023, 6:13 AMhelpful-application-7107
09/29/2023, 3:35 PMbusy-air-96466
10/09/2023, 9:23 AMhelpful-application-7107
10/10/2023, 2:37 PMI have an existing A/B experiment, and I modified it to use a namespaceThis is an unsafe operation and it can be difficult to make guarantees about how the user experience will change over time. We should make it clear in the app that there isn't a safe way to do this without creating new features/re-randomizing if we don't already.