Hey everyone I am trying to reproduce the Change % measure but I am not sure about the right equatio...

cool-diamond-12331

07/22/2024, 8:32 AM

Hey everyone I am trying to reproduce the Change % measure but I am not sure about the right equation The UI shows Change % = 1.21% The related metric Configuration

• Conversion Window = 72 hours

• event count

Current Metric values that I am using • Control Group => 0.349 • Test Group => 0.353 However, When I follow the Relative equation by just using the showed values by growthbook

Change % = 100 * ((0.353-0.349)/0.349) = 1.146%

Anyway, i see some information like these which I am not sure if they influence the calculations in a way or another

Copy code

95% Credible Interval: [x%, y%]
Chance to Win: z%
risk: a%
b / user

Any help is appreciated 🙏

rhythmic-agent-34208

07/22/2024, 9:20 AM

Hi Obadah, Good morning and thank you for your response. I will now escalate this to the Data Scientist team to further investigate. As soon as we have an update, we will follow up via this thread. Thank you for your ongoing patience :)

🙏 1

cool-diamond-12331

07/22/2024, 9:30 AM

Good Morning Natasha, Thank you for your quick response 🙏

youre welcome 1

rhythmic-agent-34208

07/22/2024, 9:53 AM

For the team to have all the information needed, please can you take a full screenshot of the affected results table?

rhythmic-agent-34208

07/22/2024, 11:19 AM

Thank you, I'll relay this assisted information over to them and get back to you shortly :)

🙏 1

rhythmic-agent-34208

07/22/2024, 4:27 PM

Hi Obadah, Thank you for your patience. The Data Scientist have had a look over this and have confirmed the calculation to be correct: 100 * ((2789/7898) - (3016/8644)) / (3016/8644) = 1.207991, rounded to 1.21 We can also see this another way if we just add one decimal to our rounded mean values: `3016/8644 = 0.3489` `2789/7898 = 0.3531` `100 * (0.3531 - 0.3489) / 0.3489 = 1.203` So adding one more decimal at least gets us to 1.20, and if we kept going we'd get to 1.207991! Please note, if you were to enable CUPED (which you haven't in this case, but wanted to share information incase you d in future), the exact lift % can sometimes differ from a manual calculation of the difference in means because CUPED does some magic under the hood that changes the means used in estimation (so it could actually be a little different from 0.349 and 0.353. Hope this helps to confirm the calculation for you :)

🙌 1

cool-diamond-12331

07/22/2024, 7:52 PM

oh nice! I missed this in my validation, thank you so much! Thanks for sharing more info and for your help and support 🙌

rhythmic-agent-34208

07/23/2024, 10:35 AM

It was our pleasure to assist 🙏🌼

44 Views

Open in Slack

Previous Next

GrowthBook Users

Open source platform for stress free deployments, measured impact, and smarter decisions.