https://www.growthbook.io/ logo
p

purple-cat-95990

01/19/2023, 9:23 AM
Hey, interested if you’re developing features for high level reporting of experiments (# run, failure rate etc etc), with tag filters would be good too. Some things in the dashboard management view, but for some reason we only see current month?
f

fresh-football-47124

01/19/2023, 9:24 AM
have you been running for more than 1 month?
p

purple-cat-95990

01/19/2023, 9:25 AM
Yes, running for almost year. The north stars are over longer period
f

fresh-football-47124

01/19/2023, 9:25 AM
We debated putting the win/failure rate - but we’re not sure that’s a metric that helps with a healthy experimentation culture
probably a bug then, we’ll take a look (about the one month thing)
p

purple-cat-95990

01/19/2023, 9:28 AM
Interested re. the debate, I can hypothesise what was discussed. We likely want a discussion on whether our failure rate is high enough. i.e. reasoning over whether experiments are testing/proving or just measuring. (I assume the debate may have been the other way) Obvs this would go further down, to understand hypotheses better.
f

fresh-football-47124

01/19/2023, 9:30 AM
ya, we feel that a higher experimentation rate is more important than the win rate, and if you’re optimizing for win rate, you might become risk adverse with more controversial or potentially risky experiments
p

purple-cat-95990

01/19/2023, 9:34 AM
Yeh, thats what we want to foster (keeping risk and learning high). i.e. it has to be about learning, win rate too high means we likely do not have the balance of learning vs just proving our apriori. Hard to reason over without some data on it to then dive into the detail
f

fresh-football-47124

01/19/2023, 9:34 AM
oh interesting
I like that
learning rate
any ideas how we would quantify that?
I guess we could show the win rate, and give some context if its over some rate, perhaps we could make that adjustable
p

purple-cat-95990

01/19/2023, 9:39 AM
Yeh my simplified version was win-rate, perhaps on metrics/areas (we tag teams, they own areas of product) Very limited additional thought, is how learning rate works in NN. i.e. about the step you make on the optimisation path for a metric. Likely over-complicated, but is kinda what we’re talking about. i.e. are you taking right steps to explore the space, to get to an optimum (which is harder for these metrics vs ML, but think the analogy holds)
f

fresh-football-47124

01/19/2023, 9:41 AM
That is cool- I think there maybe some ways to do this in the UI
I’ll bring it up with the team
h

happy-autumn-40938

01/31/2023, 8:16 PM
hi @purple-cat-95990, about the specific "Experiments by month" issue where you're only seeing the current month, would you be able to do a screen share session? We can't seem to recreate the issue it on our end. Thanks!
Also, would you be able to DM me the email address you're using to log into GrowthBook?
3 Views