> An experiment that changes the metric by less...
# announcementsw
wooden-country-60054
03/09/2023, 2:28 PMAn experiment that changes the metric by less than this percent will be considered a draw (default 50)Just curious why such large number was chosen for the default value, isn't 50% quite a big move for practically any metric?
f
fresh-football-47124
03/09/2023, 2:37 PM
An experiment that changes the metric by less than this percent will be considered a draw (default 0.5)Where do you see 50? We have a max percent change to solve for Twyman’s Law that is 50…
w
wooden-country-60054
03/09/2023, 2:40 PMGrowthBook is locally deployed
l
late-dentist-52023
03/09/2023, 4:05 PM@wooden-country-60054 you can (or an admin can) set the default. I also thought it changed for the default for new metrics when we recently bumped our version to 1.9.0, but wasn’t sure if it got modified/adjusted inadvertently by someone on the team.
w
wooden-country-60054
03/09/2023, 4:11 PMThank you. Maybe someone on our side accidentally updated the defaults. I apologize, it seems that it could happen.
r
refined-helicopter-60
03/09/2023, 6:20 PMI came here to investigate this as well, ours was set as default to 50% which confused me. We just started using growthbook and I don't think anyone set that manually to 50% — it seems out of the box.
refined-helicopter-60
03/09/2023, 6:33 PMChecking with our engineer who set up the first experiment, but I'm seeing some metrics with
Min percent change : 0.5%Min percent change : 50%refined-helicopter-60
03/09/2023, 6:37 PMUpdate: we never changed any of the defaults or settings in the metrics queries, so likely a bug. Running the web app
Build: 33916f5 (2023-03-09)f
fresh-football-47124
03/09/2023, 6:38 PM
okay, thanks, we’ll take a look
k
kind-monkey-39734
03/09/2023, 6:41 PMscreenshot from growthbook cloud from an older metric we created
h
helpful-application-7107
03/09/2023, 6:41 PMThere used to be a bug that that would, in certain cases on local deployment, multiply that number by 100 and you'd end up with 50% to 5000% as the min and max respectively.
helpful-application-7107
03/09/2023, 6:42 PMWe landed a fix for that bug, and hopefully it won't happen again going forward, but you may need to reset the metrics that were affected by this as a (hopefully) one time change.
r
refined-helicopter-60
03/09/2023, 6:43 PMGreat, thanks for the quick fix. Just to clarify since we're new, out of the box is min .5% and max 50%?
h
helpful-application-7107
03/09/2023, 6:43 PMYes, those should be our defaults.
👍 2
helpful-application-7107
03/09/2023, 6:46 PMIf you still notice 50 and 5000% for some reason on newly created metrics, please let us know.
r
refined-helicopter-60
03/09/2023, 6:49 PMStill there on
Build: a453ee8 (2023-03-09)h
helpful-application-7107
03/09/2023, 6:50 PMThis is from a fresh set-up?
r
refined-helicopter-60
03/09/2023, 6:50 PMI created a new metric, yeah
h
helpful-application-7107
03/09/2023, 6:50 PMOh, but are the organization defaults still 50 and 5000?
k
kind-monkey-39734
03/09/2023, 6:51 PMah they are. i’ll switch them
r
refined-helicopter-60
03/09/2023, 6:51 PMAh I see now, yeah. We'll fix that.
h
helpful-application-7107
03/09/2023, 6:51 PMYeah, sorry so the bug would result in the org defaults getting x100, and then that would propagate through to the metrics.
💡 1
helpful-application-7107
03/09/2023, 6:51 PMHopefully now that won't happen to the org settings any more, but you'll need to fix existing metrics and definitely fix your org defaults.
helpful-application-7107
03/09/2023, 6:52 PMSorry for the noisy 🐛
r
refined-helicopter-60
03/09/2023, 6:53 PMFYI on the settings page w/ a narrower browser window.
h
helpful-application-7107
03/09/2023, 6:54 PM@fresh-football-47124 didn't you push a fix for that overlap? I feel like the save should be floating at the bottom (e.g. save is always visible), and then we could move it to the left as well.
helpful-application-7107
03/09/2023, 6:55 PMBut anyways, yeah we'll improve that. Thanks for the heads up.
r
refined-helicopter-60
03/09/2023, 6:56 PMyou got it!
31 Views