Huge thanks to Growthbook community for such a stylish & functional tool.
I was wondering if there’s a way to add a metric that should be cohort-level, not user-level (e.g. overall CTR across the cohort or average (across the cohort) position in the ranked list where conversion happens); i.e. aggregation should not happen before summing values for all users.
04/29/2022, 3:38 PM
There is not currently. I've found that tends to be very susceptible to outliers - one user with extreme values can have huge effects on the results. Do you have a specific use case for this?
04/29/2022, 4:02 PM
For instance, we would like to assess CTR across the cohorts in an A/B test. The proper way to do it assumes that the denominator contains the total number of impressions, not the number of users (since users in different cohorts could generate different number of views).
With Growthbook, we can track the total number of impressions and the total number of clicks. Unfortunately, from what you mentioned I get that we could not compare cohorts in terms of their ratio.
04/29/2022, 4:08 PM
That makes sense. Currently the denominator is always the number of users. We do plan to support denominators coming from other metrics (e.g. impressions) in the future.
05/06/2022, 1:30 PM
Hey, I had a query similar to what Alexey had, Its pretty good to know that CTR is being considered as a metric! From a feature standpoint I was wondering if we can have it as a 'hybrid' metric which is just transformation on existing metrics?
05/06/2022, 1:36 PM
HI Vinod, do you have a specific example in mind? There are a few ways we're considering approaching these types of metrics. It's good to have a variety of use cases so we know we're solving them all.
05/07/2022, 3:30 PM
Hi Jeremy, My use case would be same as Alexey, A CTR metric across dimensions. That is just getting total number of clicks divided by total number of impressions.