Thinking about an experiment we’re running right n...
# give-feedback
t
Thinking about an experiment we’re running right now but take this idea with a grain of salt. It would be extremely interesting to see a histogram of how many times each user generates a metric. We have a “number of exercises completed” metric where the
metric_count / user_count
is something like 5000 / 500. Would love to see a graph when I hover over that metric that shows me if the distribution is normal, skewed or just has a few outliers skewing the count (say 1 user is responsible for 1000 activities pulling up the average for everyone else). This is definitely possible in our data warehouse (mixpanel) but would be interesting to have it in the platform.
f
That's an interesting idea. We support metric "caps" right now where you can set a max value any single user can have. That basically stops huge outliers from screwing up results, but it still doesn't let you visualize the distribution.
🙌 1
We've talked about adding histograms to the metric overview pages, but not in the context of an experiment.
🙌 1
t
What’s useful about the experiment context is it respects all the targeting condition
the metric caps are certainly useful but they force you to know/guess what would be an unreasonable value. Also a visualization would give you a sense of skew to answer the question of “is this driven by power users?” which is always a nagging thought for these count metrics.