Hi! I am wondering whether you plan to introduce support for meta-analysis of finished experiments at some point?
02/23/2023, 3:47 PM
Can you explain what you mean by that?
02/24/2023, 12:00 PM
Hi! Thank you for clarifying! In short, meta-analysis implies aggregation of effect sizes and other metrics from already ran experiments tackling similar problem to make a more reliable/powerful conclusion + extract tones of other learnings. It is an established methodology that can be automatised. In our company we do it by ourselves and I am wondering whether you are thinking about providing support for it in GB? To me it looks like a natural feature to add that will bring tones of value to GrowthBook as a product.
@fresh-football-47124, did the clarification help? 🙂
02/27/2023, 3:40 PM
hi Ekaterina. We're definitely considering a range of approaches to analyze across experiments. however, in general there are many different use cases across our customers and experimentation in general for how to use meta analyses, so it's not clear which product would have the biggest impact. we're particularly interested in enabling analysis at the metric level (understanding what kinds of experiments have contributed to metric lift, for example) and at the project/tag level.
nonetheless, our first step is building out our REST API so that we can make analyses like the one you're doing easier to automate.
just to help me understand your particular use case, what is the analysis you all currently do with gb experiments?