01/10/2022, 3:36 PM
Hi everyone. I'm checking the underlying queries that calculate the experiment results and have a question. It looks like the conversion attributes to the earliest pageview found for a particular user_id, in which case this scenario applies in case of the conversion window being 24 hours.
Copy code
1.user views a test variation 
2. user views a test variation again after 3 hours
+ user converts after 27 hours after viewing the variation for the first time.

conversion doesn't get attributed to the variation.
Just want to check my assumption at the time, but it looks like it makes sense for the conversion window to get updated each time the user views the variation (although I am not an expert in a/b tests)


01/10/2022, 3:46 PM
You're correct, it only looks at the first exposure date when calculating conversion windows. I think that's the right default behavior, but I could see some value in being able to change it. Attribution is hard and there's no perfect answer. Imagine you have an experiment on a pricing page. The user lands there, gets put into an experiment, but doesn't convert. Then a few days later they click a promo in an email, go to the pricing page (sees the experiment again), and does convert. Should the A/B test get credit for the conversion or the promo email?


01/10/2022, 7:10 PM
Yes, you are correct. While my approach can definitely work and give results in a shorter time, it adds complexity that can results in lots more things going wrong, so yeah, can totally relate. Sometimes simpler is better.