hey, I have a question about conversion window. Do...
# ask-questions
m
hey, I have a question about conversion window. Does it work from the very first exposure to the experiment only or does it work from any exposure to the experiment? Let's say that we have 24h conversion window, customer gets exposed to the experiment for the first time, then all conversions that this user does in next 24 hours for that metric are taken into account. Now if the user comes back in a couple of days, there could be two scenarios (1) is not exposed to the experiment; (2) is exposed to the experiment. From the visual attached (https://docs.growthbook.io/app/metrics#metric-windows), it looks like in both (1) and (2) user's conversions won't be taken into account on the subsequent experiment activation. Is that correct? Or in (1) the conversions won't be taken into account and in (2) they will be taken into account for the next 24hr from this subsequent exposure to the experiment? I think a better approach would be to have it cover all exposures rather than the very first one only. There are experiments e.g. in my account page, where users might not visit that often, then each exposure should be indicator that user has been potentially affected by the changes, so the the conversion action is relevant. What would be the rationale behind having the window start from the first exposure only? Also activation metric could be related to that. We often have experiments where user is activated to the experiment only once, at the start of the application, even if the have not experienced the changes that are part of the experiment. In that case activation metric could act similarly to how experiment exposure works. Then the conversion window could be calculated from the the particular instance of activation metric event timestamp. Please let me know what you think.
c
Two things: • You should never have a situation where the same user gets different experiment exposures during the test (unless an anonymous user clears cookies, in which case you would treat it as a separate user since you have no way to tell that it’s the same user anyway). If a user gets the experiment version on day 1, the same user should still get the same experience on day 7 of the test. • It sounds to me that what you want is to use “None” as the option for metric window. In that case you get what you seem to be looking for, that conversion counts regardless of when during the test the user was exposed to the experiment.
m
@cool-balloon-15377 thanks for your feedback. • I was talking about scenario where the same user gets the same experiment exposure more than one time. • I'd like to see the start of the window being attached to latest experiment exposure time.
h
I think a better approach would be to have it cover all exposures rather than the very first one only. There are experiments e.g. in my account page, where users might not visit that often, then each exposure should be indicator that user has been potentially affected by the changes, so the the conversion action is relevant.
this is much more complicated to implement in the SQL and makes queries much slower and more costly. I'd consider using no experiment window as the other user suggested.
👍 1