https://www.growthbook.io/ logo
e

enough-ambulance-15689

08/26/2021, 3:07 AM
Hi folks, great product! šŸ¤© Iā€™m wondering (this may be a silly question), for the experiments query, do you expect all experiment & variation assignment for the same-user to be returned (it would be the same assignment occurring at a different times) ? Iā€™m thinking you would because the same user could convert more than once ? (e.g. an e-commerce user buys two items in separate occasions during the duration of an experiment affecting the buy flow ā€¦ or the same user may drop at checkout the 2nd time). Thanks in advance!
f

fresh-football-47124

08/26/2021, 3:57 AM
Hi Alexei
Do you have the experiment/user mapping in multiple tables?
I'm trying to understand that use case - the idea is that you return the user id with the experiments they saw in the experiments query, then the metric queries return the conversion or event data with user id and date. we can then map back to the experiments and do the statistics
āœ… 1
f

future-teacher-7046

08/26/2021, 9:38 AM
If you return multiple rows for the same user/experiment, we only count the earliest timestamp. From that point, the user has 72 hours (by default) to convert. You can override that 72-hour conversion window on a per-metric basis if needed.
e

enough-ambulance-15689

08/26/2021, 3:35 PM
Hi Graham / Jeremy, thank you for the clarifications. From a scalability perspective, we were hoping only the earliest data point needs to be tracked, so this is great! Just to be really sure, a conversion (or lack of it) is counted only once (first time) per user for the duration of an experiment ?
f

future-teacher-7046

08/26/2021, 4:03 PM
That's true for binomial (conversion) metrics. We group by users first before grouping by variation and a conversion just means >=1 rows for a user. For count, duration, and revenue metric if there are multiple rows for a user, we sum the values together. So if a user is put into an experiment and then makes two $10 orders. A "Purchased" binomial metric would count that user as converted. A "Revenue per User" metric would count that user as having $20
We'd love to hear if you expected or want different a behavior. There are a lot of opportunities for us to add tweaks and settings to customize how this works.
e

enough-ambulance-15689

08/26/2021, 4:13 PM
Thank you very much for the detailed explanation, it makes perfect sense! I will likely be in-touch as Iā€™m exploring building on GrowthBook to put in place the components needed to support an experimentation culture in the context of central data & search teams serving the needs of multiple product teams in several two-sided marketplaces.
f

full-island-88199

06/07/2022, 2:36 PM
Hey! Just on this topic. Would you add something like this explanation to your documentation on conversion windows too?
If you return multiple rows for the same user/experiment, we only count the earliest timestamp. From that point, the user has 72 hours (by default) to convert.
It was causing a bit of confusion at ours as not all scrutinize the SQL query generated. And also would be nice with an explanation of why 72 hours is the default šŸ™Œ
f

fresh-football-47124

06/07/2022, 2:42 PM
Thanks for that feedback, we can add that to the docs
šŸ™Œ 1
(added)
šŸ™ 1
4 Views