melodic-solstice-24408
07/26/2023, 8:42 AMAVG
, which makes sense in general but for my specific case could lead to some noisy results given some annoying outliers in the metric distribution. Therefore, I would like to use MEDIAN
to aggregate the metric across users, and as there’s no option to do so when creating the metric I was wondering if there’s a workaround that people use or if it will be possible in the future to also select the aggregation method across all users.white-fireman-22476
07/26/2023, 8:47 AMmelodic-solstice-24408
07/26/2023, 8:54 AMpercentile capping
available from a specific version on?white-fireman-22476
07/26/2023, 8:55 AMmelodic-solstice-24408
07/26/2023, 8:56 AMwhite-fireman-22476
07/26/2023, 8:57 AMmelodic-solstice-24408
07/26/2023, 9:03 AMhelpful-application-7107
07/26/2023, 4:28 PMHowever, the only enforced aggregation across users isYou're correct that the enforced aggregation across users is, which makes sense in general but for my specific case could lead to some noisy results given some annoying outliers in the metric distribution.AVG
AVG
however, the field you showing is the user-level aggregation.
Normally you actually want to SUM(value)
as per the default so that your user-level total is the sum of all of their values, and then in analysis we divide by the N users which gives you the average per user (so the AVG
happens internally in our stats engine). If you choose AVG(value)
as your custom aggregation, you're doing something very different, where each user will get the average of the rows that are accorded to them. Instead of looking at "duration per user" you're looking at "average duration per user" and comparing those across variations.Therefore, I would like to useThis is a very different statistical test than averaging across users, which as I described above, is what we do above. This is not supported by growthbook, and the test you're describing is often called "quantile testing"to aggregate the metric across usersMEDIAN
Hi Fabrizio. It sounds like you might be interested in winsorization which Growthbook does have the ability to do: https://docs.growthbook.io/app/metrics#capped-valueJames is correct that to deal with your outliers you probably want to work with percentile winsorization. However, if you care about the "duration per user" and not "average duration per user" I suggest you user percentile winsorization and the
SUM(value)
default aggregation.SUM(value)
per user and trim them to be the same as the 95th percentile.white-fireman-22476
07/26/2023, 7:52 PM