https://www.growthbook.io/
Join Slack
> An experiment that changes the metric by less than this percent will be considered a draw (defa...
# announcements
w
An experiment that changes the metric by less than this percent will be considered a draw (default 50)
Just curious why such large number was chosen for the default value, isn't 50% quite a big move for practically any metric?
f
An experiment that changes the metric by less than this percent will be considered a draw (default 0.5)
Where do you see 50? We have a max percent change to solve for Twyman’s Law that is 50…
w
GrowthBook is locally deployed
l
@wooden-country-60054 you can (or an admin can) set the default. I also thought it changed for the default for new metrics when we recently bumped our version to 1.9.0, but wasn’t sure if it got modified/adjusted inadvertently by someone on the team.
w
Thank you. Maybe someone on our side accidentally updated the defaults. I apologize, it seems that it could happen.
r
I came here to investigate this as well, ours was set as default to 50% which confused me. We just started using growthbook and I don't think anyone set that manually to 50% — it seems out of the box.
Checking with our engineer who set up the first experiment, but I'm seeing some metrics with 
Min percent change : 0.5%
and some at 
Min percent change : 50%
(this is the current default) So it could easily be a number formatting bug.
Update: we never changed any of the defaults or settings in the metrics queries, so likely a bug. Running the web app 
Build: 33916f5 (2023-03-09)
f
okay, thanks, we’ll take a look
k
screenshot from growthbook cloud from an older metric we created
h
There used to be a bug that that would, in certain cases on local deployment, multiply that number by 100 and you'd end up with 50% to 5000% as the min and max respectively.
We landed a fix for that bug, and hopefully it won't happen again going forward, but you may need to reset the metrics that were affected by this as a (hopefully) one time change.
r
Great, thanks for the quick fix. Just to clarify since we're new, out of the box is min .5% and max 50%?
h
Yes, those should be our defaults.
👍 2
If you still notice 50 and 5000% for some reason on newly created metrics, please let us know.
r
Still there on 
Build: a453ee8 (2023-03-09)
— LMK when the fix is posted and I can try again & report back.
h
This is from a fresh set-up?
r
I created a new metric, yeah
h
Oh, but are the organization defaults still 50 and 5000?
k
ah they are. i’ll switch them
r
Ah I see now, yeah. We'll fix that.
h
Yeah, sorry so the bug would result in the org defaults getting x100, and then that would propagate through to the metrics.
💡 1
Hopefully now that won't happen to the org settings any more, but you'll need to fix existing metrics and definitely fix your org defaults.
Sorry for the noisy 🐛
r
FYI on the settings page w/ a narrower browser window.
h
@fresh-football-47124 didn't you push a fix for that overlap? I feel like the save should be floating at the bottom (e.g. save is always visible), and then we could move it to the left as well.
But anyways, yeah we'll improve that. Thanks for the heads up.
r
you got it!
71 Views
Open in Slack
PreviousNext
Powered by

GrowthBook Users

Open source platform for stress free deployments, measured impact, and smarter decisions.

Powered by