Hi folks, great product! 🤩 I’m wondering (this may be a silly question), for the experiments query, do you expect all experiment & variation assignment for the same-user to be returned (it would be the same assignment occurring at a different times) ? I’m thinking you would because the same user could convert more than once ? (e.g. an e-commerce user buys two items in separate occasions during the duration of an experiment affecting the buy flow … or the same user may drop at checkout the 2nd time). Thanks in advance!

Hi Alexei

If you return multiple rows for the same user/experiment, we only count the earliest timestamp. From that point, the user has 72 hours (by default) to convert. You can override that 72-hour conversion window on a per-metric basis if needed.

Hi Graham / Jeremy, thank you for the clarifications. From a scalability perspective, we were hoping only the earliest data point needs to be tracked, so this is great! Just to be really sure, a conversion (or lack of it) is counted only once (first time) per user for the duration of an experiment ?

That's true for binomial (conversion) metrics. We group by users first before grouping by variation and a conversion just means >=1 rows for a user. For count, duration, and revenue metric if there are multiple rows for a user, we sum the values together. So if a user is put into an experiment and then makes two $10 orders. A "Purchased" binomial metric would count that user as converted. A "Revenue per User" metric would count that user as having $20

Thank you very much for the detailed explanation, it makes perfect sense! I will likely be in-touch as I’m exploring building on GrowthBook to put in place the components needed to support an experimentation culture in the context of central data & search teams serving the needs of multiple product teams in several two-sided marketplaces.

Hey! Just on this topic. Would you add something like this explanation to your documentation on conversion windows too?

If you return multiple rows for the same user/experiment, we only count the earliest timestamp. From that point, the user has 72 hours (by default) to convert.

It was causing a bit of confusion at ours as not all scrutinize the SQL query generated. And also would be nice with an explanation of why 72 hours is the default 🙌

Thanks for that feedback, we can add that to the docs