I have a test experiment that I m running with seed data How

Question

I have a test experiment that I m running with seed data However I m getting an error of `A Sample Ratio Mismatch SRM was detected with p value of 0 ` I inserted 2000 experiment viewed records split 50 50 across the 2 variations From the 1000 users that saw variation 0 300 went onto convert Of the 1000 users that saw variation 1 700 went onto convert 1000 users did not convert at all I m trying to understand why this is giving a SRM

clean-crayon-32883 · Answer

The help doc on this says SRM happens when the actual traffic split is different from what you expect but in our case the experiment viewed is exactly 50 50

clean-crayon-32883 · Answer

The metric is 30 70 which is what I would expect but this is what GB is complaining about

clean-crayon-32883 · Answer

I think the reason for this is because of this

clean-crayon-32883 · Answer

It seems like any user that doesn t convert to a subscription is excluded

clean-crayon-32883 · Answer

Which seems strange and I m probably missing something obvious here Why would you totally exclude users that don t convert to a subscription

future-teacher-7046 · Answer

Activation Metric is used to completely filter people out of the analysis For example if you are assigning variations on page load but the experiment doesn t actually happen until a modal pops up you could use a Viewed modal event as the activation metric

future-teacher-7046 · Answer

Most of the time you don t need to set an activation metric It s only for cases where the experiment query returns extra users that didn t actually see the experiment and you need to filter those out to get cleaner data

clean-crayon-32883 · Answer

Oh got it Thank you < future teacher 7046> I removed the activation metric and it looks much better