https://www.growthbook.io/
Join Slack
Menu
Home
Archive
Hi! Quick question related to metrics and `conversion window` parameter: if I set up this `conversio...
# announcements
g
Hi! Quick question related to metrics and 
conversion window
parameter: if I set up this 
conversion window = 48 (hours)
, in a binary variable as an example, what would happen is: • All the distinct users assigned to the experiment in the 
Experiment Assignment Query
will be counted in the denominator. • Just the users with a conversion, with a timestamp falling in the conversion_window (
experiment assignment timestamp
+ 48h) will be counted in the numerator. However, there will be some users added into the denominator, which still have time to conver (imagine they just joined the experiment, they still have 48h to convert). This means that the metric will not be matured until 48h after of finishing the experiment. Is this reasoning correct? Many thanks!
f
Yes, that's correct. If you configure the analysis settings, you can control this behavior by specifying what to do with "in-progress" conversions.
The downside of waiting for metrics to "mature" is that data is delayed by the conversion window, so it can take a while to see results after starting the experiment. That's why we don't default to that setting.
g
Thanks a lot!
Hi Jeremy, a follow up question here: I’m seeing that binary metrics don’t have this 
"in-progress behaviour"
parameter. This means that, even if the 
Conversion Window
is set, for example, to 48 hours, as soon as a new entry/user is assigned to the experiment, it will be counted in the denominator of the metrics, regardless of the maturity of this return didn’t reach the conversion window yet. This is correct? Thanks again.
f
If that's happening it sounds like a bug. I can take a look. Binomial metrics should behave the same as the others.
84 Views
Open in Slack
PreviousNext
Powered by

GrowthBook Users

Open source platform for stress free deployments, measured impact, and smarter decisions.

Powered by