Does the low base rate problem apply with Bayesian A/B testing? We have always done the “lazy assignment” suggested by this related article and assigned users to the experiment as late as possible in the funnel. Now that we’re using Growthbook, I’m curious if that applies. Total number of conversions would still be the same regardless of the assignment time, so I’m just reasoning through whether the difference in rates will matter

I believe it does still matter. All of those users who are included in the analysis, but didn't see the experiment add noise and variance to the results. So it will take more data to get a meaningful signal.

Thanks, that makes sense

You can use activation metrics in GrowthBook if lazy evaluation is not practical for an experiment. Let's you capture everyone, and filter it down during analysis to only those who saw the experiment.