Hi, is there a way to exclude a user from an experiment based on their actions/events?

There are two ways currently: If the user's actions are known to the SDK, they can be set as an attribute, and you can target the feature based on that. Alternatively, you can exclude a user or set of users when running the experiment analysis (using segments, or custom filters).

What do you mean by custom filters?

if you go to customize an experiment analysis, there is a custom SQL filter at the bottom

Ah, ok. I see it. That's definitely useful. 🙂

yes, for that one experiment

Hm, I'm not sure that would work. In our case the experiment assignment (A/B for example) triggers first, and then based on the next step we'd either exclude you (if you click a link to login - meaning you're not new user) or include you (do nothing) if you continue.

you could add an activation metric for that event

What does that mean? Add additional criteria to metric SQL?

Activation metrics are useful if your assignment and experiment viewing are separate - like if you had an AB test in a modal that not every user saw, but you had to render the test before hand. You could then fire an event on modal open, and use that event as an activation metric, to only include users who actually saw the experiment

I saw that in the docs but didn't quite understand it.

yes

Ok, that would work yep. Thanks.

the number of activated users will be less than the total bucketed - and as long as its a random assignment, it should be the same percentage split.

Ok. But the problem is that we don't have that exclusion rule (link) on both versions. 🤔

Hard to A/B test if the versions are symmetrical in the metrics

I think we can try figuring it out by adding an SQL filter and maybe adjusting metrics to not count excluded people. If we fail miserably, I'll definitely reach back to you. 🙂 Thanks