Hello. I have been using Growthbook for our analysing our experiments and one of the analysis seem to behave weird. This experiment has run for a few days, but the violin plot is just one line at the center, the Chance to Beat is always 50%, and the risk of choosing either control or variant is always zero. We already have quite a number of users and the mean of the metric is quite different for the variant as compared to the control. I wonder how I can troubleshoot if there’s anything we did wrong in the way we set this up.

can you share the numbers or screen shot?

Control: 33502 users, mean 161.790 (5420287/33502) Variant: 6178 users, mean 177.191 (1094683/6178)

ya, that seems right - you have very high traffic - what kind of metric is it?

The type is revenue, the rest is set as default. A user may have multiple values, and the values are averaged.

can you just make sure the metric definition is set correctly to revenue and has the right aggregation?

Yeap it was correctly set. It seems I figured out why. There were crazy outliers that skewed the standard deviation (the Control’s STDDEV is 3141 and Variant’s is 3868). After I cap the values in the metric and updated the data, the violin plots and risk of choosing started showing non-zeros.

cool

yeah. i may need to discuss with the stakeholders on what to cap it to, but it seems to work well. thanks for the help.