https://www.growthbook.io/ logo
#announcements
Title
# announcements
w

wooden-country-60054

03/09/2023, 2:28 PM
An experiment that changes the metric by less than this percent will be considered a draw (default 50)
Just curious why such large number was chosen for the default value, isn't 50% quite a big move for practically any metric?
f

fresh-football-47124

03/09/2023, 2:37 PM
An experiment that changes the metric by less than this percent will be considered a draw (default 0.5)
Where do you see 50? We have a max percent change to solve for Twyman’s Law that is 50…
w

wooden-country-60054

03/09/2023, 2:40 PM
GrowthBook is locally deployed
l

late-dentist-52023

03/09/2023, 4:05 PM
@wooden-country-60054 you can (or an admin can) set the default. I also thought it changed for the default for new metrics when we recently bumped our version to 1.9.0, but wasn’t sure if it got modified/adjusted inadvertently by someone on the team.
w

wooden-country-60054

03/09/2023, 4:11 PM
Thank you. Maybe someone on our side accidentally updated the defaults. I apologize, it seems that it could happen.
r

refined-helicopter-60

03/09/2023, 6:20 PM
I came here to investigate this as well, ours was set as default to 50% which confused me. We just started using growthbook and I don't think anyone set that manually to 50% — it seems out of the box.
Checking with our engineer who set up the first experiment, but I'm seeing some metrics with
Min percent change : 0.5%
and some at
Min percent change : 50%
(this is the current default) So it could easily be a number formatting bug.
Update: we never changed any of the defaults or settings in the metrics queries, so likely a bug. Running the web app
Build: 33916f5 (2023-03-09)
f

fresh-football-47124

03/09/2023, 6:38 PM
okay, thanks, we’ll take a look
k

kind-monkey-39734

03/09/2023, 6:41 PM
screenshot from growthbook cloud from an older metric we created
h

helpful-application-7107

03/09/2023, 6:41 PM
There used to be a bug that that would, in certain cases on local deployment, multiply that number by 100 and you'd end up with 50% to 5000% as the min and max respectively.
We landed a fix for that bug, and hopefully it won't happen again going forward, but you may need to reset the metrics that were affected by this as a (hopefully) one time change.
r

refined-helicopter-60

03/09/2023, 6:43 PM
Great, thanks for the quick fix. Just to clarify since we're new, out of the box is min .5% and max 50%?
h

helpful-application-7107

03/09/2023, 6:43 PM
Yes, those should be our defaults.
👍 2
If you still notice 50 and 5000% for some reason on newly created metrics, please let us know.
r

refined-helicopter-60

03/09/2023, 6:49 PM
Still there on
Build: a453ee8 (2023-03-09)
— LMK when the fix is posted and I can try again & report back.
h

helpful-application-7107

03/09/2023, 6:50 PM
This is from a fresh set-up?
r

refined-helicopter-60

03/09/2023, 6:50 PM
I created a new metric, yeah
h

helpful-application-7107

03/09/2023, 6:50 PM
Oh, but are the organization defaults still 50 and 5000?
k

kind-monkey-39734

03/09/2023, 6:51 PM
ah they are. i’ll switch them
r

refined-helicopter-60

03/09/2023, 6:51 PM
Ah I see now, yeah. We'll fix that.
h

helpful-application-7107

03/09/2023, 6:51 PM
Yeah, sorry so the bug would result in the org defaults getting x100, and then that would propagate through to the metrics.
💡 1
Hopefully now that won't happen to the org settings any more, but you'll need to fix existing metrics and definitely fix your org defaults.
Sorry for the noisy 🐛
r

refined-helicopter-60

03/09/2023, 6:53 PM
FYI on the settings page w/ a narrower browser window.
h

helpful-application-7107

03/09/2023, 6:54 PM
@fresh-football-47124 didn't you push a fix for that overlap? I feel like the save should be floating at the bottom (e.g. save is always visible), and then we could move it to the left as well.
But anyways, yeah we'll improve that. Thanks for the heads up.
r

refined-helicopter-60

03/09/2023, 6:56 PM
you got it!
18 Views