Hi Quick question related to metrics and `conversion window`

Question

Hi Quick question related to metrics and `conversion window` parameter if I set up this `conversion window = 48 hours ` in a binary variable as an example what would happen is All the distinct users assigned to the experiment in the `Experiment Assignment Query` will be counted in the denominator Just the users with a conversion with a timestamp falling in the conversion window `experiment assignment timestamp` + 48h will be counted in the numerator However there will be some users added into the denominator which still have time to conver imagine they just joined the experiment they still have 48h to convert This means that the metric will not be matured until 48h after of finishing the experiment Is this reasoning correct Many thanks

future-teacher-7046 · Answer

Yes that s correct If you configure the analysis settings you can control this behavior by specifying what to do with in progress conversions

future-teacher-7046 · Answer

The downside of waiting for metrics to mature is that data is delayed by the conversion window so it can take a while to see results after starting the experiment That s why we don t default to that setting

green-jordan-83609 · Answer

Thanks a lot

green-jordan-83609 · Answer

Hi Jeremy a follow up question here I m seeing that binary metrics don t have this ` in progress behaviour ` parameter This means that even if the `Conversion Window` is set for example to 48 hours as soon as a new entry user is assigned to the experiment it will be counted in the denominator of the metrics regardless of the maturity of this return didn t reach the conversion window yet This is correct Thanks again

future-teacher-7046 · Answer

If that s happening it sounds like a bug I can take a look Binomial metrics should behave the same as the others