hey Team. I have experienced a situation where one metric in an experiment shows drop but the relative change % shows uplift? It is a guardrail metric. See attached. What might be the reason for it?

By "drop" do you mean not significant? If so the mean is +4, but the lower bound is -4.94%, meaning that it's not significant.

@helpful-glass-76730 no, look at the values control 2479 and variation 2446, but the % change shows +4.04% which is incorrect, should be -1.33%

@most-iron-81885 My bad! Tried to do some calculations, but can't find anything that would justify the lift showed. I hope someone from the GrowthBook team can help you!

looks like this is an issue for count metric type

Hi Krzysztof, thank you for bringing this to our attention. I am getting a rounded value of -1.33% too when I use the 100 ­* ((13916534/5689)-(13595330/5484)) / (13595330/5484) = -1.32597504232 calculation. I will now escalate this to the Engineering team to further investigate. Thank you for bringing this to our attention and apologies for any inconvenience caused. As soon as I have a response I will be in contact via this thread.

@most-iron-81885 are you seeing the miscalculation with the % change when CUPED is disabled?

Hi Krzysztof, I have good news, its not a bug, its how CUPED works. CUPED works by adjusting the means. There's a throwaway line at the bottom of this section https://docs.growthbook.io/app/experiment-results#difference-types Definitely confusing from a UI standpoint and a usability gap has been pushed forward to the Engineering team for this to be more intuitive. Apologies for the confusion here.

yes, this is it! Thanks @flaky-noon-11399!!

My pleasure to assist 🙏