I’m running a drip campaign experiment now: 50% of users get a simple welcome email, 50% of users get the welcome email + a new drip campaign that sends them 4 additional emails spaced out over a couple of weeks. Our hypothesis is that users in the experiment group will see a higher rate of success on a few metrics related to our funnel.
I’m having trouble defining the conversion delay and window here. Since users in the exposed group get 4 emails over many days, they could convert after the first email, or the fourth. My intuition tells me I should set a conversion delay of 2 days after exposure (when the first email in the drip campaign is sent), and a window of 14 days (a few days after the last email is sent). If the user converts from any email, the experiment is a success. We’d then modify specific emails in future experiments to tease out the effects of one email over the other. Does that logic sound right, or would you approach it differently?
The challenge I’m having is that I want to define this delay / window only on the metrics for this experiment. The delay / window for other experiments would be different. I can only see a way to define this logic in the metric itself, and not for a (metric, experiment). How do people handle this?
06/11/2022, 7:34 PM
Yes, that logic sounds right. We're working on a way to override conversion windows on a per-experiment basis. Until then, you could duplicate the metric and change conversion rates just for this experiment.