Hi all, quick question about binomial funnel metrics in GrowthBook. I've defined two metrics: • Application Start Rate (based on a specific start event) • Application Completion Rate, which uses the start metric as its denominator However, there's a discrepancy in the scorecard results (see the attached screenshot): I'm expecting the numerator of the start metric (top row) to match the denominator of the completion metric (bottom row), since the latter is defined as a funnel step from the former - but the counts don't align. Any idea what might be causing this mismatch? Happy to share the raw SQL GrowthBook is using for both metrics if that would help. Thanks in advance!

huh

hey @fresh-football-47124, yeah I have both underlying SQL queries available. I could share them here if you think it would be helpful. Some additional context/information: I’ve set conversion windows (3 days) for all funnel metrics from time of experiment exposure. I’ve been able to identify the 10 users who were included in the denominator of the completion rate metric but were not counted in the numerator of the start rate metric. These 10 users all triggered the start event more than 3 days after exposure. So they were correctly excluded from the numerator of the start rate metric but incorrectly included in the denominator of the completion rate metric. So it looks like the conversion window wasn’t respected when chaining metrics. Could you help clarify if GB enforces the conversion window from the previous step when determining the denominator for chained binomial metrics? If not, what’s the recommended way to ensure consistent funnel definitions?

okay, let me check with the data science team

Thank you! 🙏

Could you help clarify if GB enforces the conversion window from the previous step when determining the denominator for chained binomial metrics?

We do not enforce that the numerator for start rate equals the denominator for completion rate. If both start rate and completion rate have conversion windows, and start rate is an activation metric for completion rate, then we use the sum of those windows in the analysis. However, as you saw in your data, a user can still be included in the start rate denominator and completion rate numerator.

If not, what’s the recommended way to ensure consistent funnel definitions

If having the start rate denominator and completion rate numerator exactly match is important, then you can increase the conversion window for completion rate. I do not think that excluding these 10 users affects your results much, so you could also keep them.

Thanks Luke!