hello folks, when running experiments in GrowthBook, how are outliers treated? so for example something like spend (similar to order value) that is extremely high and happens only on A, is that something that is excluded when calculating exp results?
f
fresh-football-47124
07/12/2022, 7:20 AM
Hi Yaniv. We don't automatically correct for outliers, but we do have the ability for you to adjust for this. You can add a capped value to adjust for this: https://docs.growthbook.io/app/metrics#capped-value
you can also do any custom filtering if you're using a SQL data source from the experiment configuration
l
late-oyster-63304
07/12/2022, 7:22 AM
got it, thanks!
ideally I would prefer to have something that removes values based on percentiles, rather than caps.
we can achieve that in the metric queries as well @fresh-football-47124
this is a tricky topic, this can cause experiments to flip based on these values (percentiles or caps), as a user of GrowthBook, this is part of the magic i'd love to get from the product 🙂
f
fresh-football-47124
07/12/2022, 7:31 AM
thanks for the feedback. It's really tricky to make something work for all cases. I'll bring it up with the team
l
late-oyster-63304
07/12/2022, 7:32 AM
absolutely, though if it's not supported, this is something relatively hard to figure out for small orgs with limited proficiency in statistics 🙂