This message was deleted.
# announcementsw
worried-planet-2191
04/12/2023, 12:52 PMThis message was deleted.
f
fresh-football-47124
04/12/2023, 3:14 PM
Hi Nora
fresh-football-47124
04/12/2023, 3:14 PM
that is interesting
fresh-football-47124
04/12/2023, 3:15 PM
roughly how much traffic was included in each group?
h
helpful-application-7107
04/12/2023, 3:54 PMChance differences like this can definitely occur; or the randomization could have been problematic. To figure which is the case, I agree looking at the sample size in each condition could be useful as our SRM test is pretty conservative. Then you could look at other metrics that are not totally correlated with this metric.
But this kind of plot is great and building them in to GrowthBook is on our radar.
f
flaky-dog-36449
04/14/2023, 9:03 AMHi! There were 61326 users in the control group and 62203 users in the treatment group. We didn't get any warnings from growthbook that the groups weren't balanced in the main results. After removing some users that were assigned to both groups, there were 61275 users in the control group and 62145 in the treatment group.
Definitely, my next step is to check uncorrelated metrics. Could be that the groups are unbalanced in terms of activity level by chance, but it's just so strange that the difference disappears right when the experiment starts 🤔
141 Views