12/21/2022, 1:59 PM
Negative Metric Values giving weird "Chance to Beat Control" & "Percent Change" Results Hey! I'm using a metric (see screenshot) that gives a negative value due to subtracting some costs. As a result of the negative value (I assume), the "chance to beat control" returns 50% and "percent change" returns 0% even though there is clearly a difference between the two results (i.e. treatment performing less negative which is good). Is this a bug or is it something on my side?


12/22/2022, 1:00 PM
Our stats engine doesn't currently support negative means. Someone made a PR to fix this, but we haven't validated or tested it yet.


12/22/2022, 1:03 PM
Thanks for the feedback @future-teacher-7046! I assumed this was the problem. Any suggestions to mitigate this for the time being?
Hey @future-teacher-7046! Coming back to this question. My metric has "turned positive", meaning negative values are no longer an issue. However, I have an uplift of +176% which seems to cause the chance to beat control to remain at its default value (50%) - see screenshot. The
max percent change
is set to 1000% for this metric, so this shouldn't be the issue. I've also tried removing the metric, updating the data, and then adding the metric again (to ensure a refresh) but to no prevail. Is this another bug? How can it be fixed?