This message was deleted.

Hi Serhii - there are two ways to solve this. 1. bucket users as close to the exposure as possible - ie, on the Product page. 2. If that is unavoidable, you can use activation metrics to filter out from the experiment results, and only include users who saw the activation event (in this case, the event would be ‘viewed product page’). Since the assignments are random, it is unlikely to bias your sample by those amounts. We have built in SRM checks to alert you about any significant imbalances.

Hi @fresh-football-47124. Thank you for your response. 1. Does it mean that we need to generate for each experiment unique user_id and exactly on the experiment page? Otherwise I don’t understand your point. For example, the library returns for the user_id 1234 the variant A on the Main page. The same value it returns on the experiment page. So, the final result will be the same no matter what the page it is. If I understood this correctly 2. As I understand activation metric is used when we have already got the split Yeach, I see that you have alerts about imbalances, we just try to avoid this, because it is not clear what to do in that case and experiment may fail because of this

I am interested on the solution as well. Does using inline experiment on the experiment page solve this problem?

No, you should not use a new id per experiment - you should always use the same id for the user if you can. The way we bucket users is to hash together the User id (or whatever you have as your hash attribute) and the experiment name/tracking key. This way users will get new assignments for every new experiment