Hello GrowthBook! And thank you again for a great tool! 🎉 We have a couple of use cases for metrics that are a bit tricky to handle right now, and were wondering about your thoughts on these. 1. Firstly, sometimes we want to analyze on a lower level than users. Examples could be sessions, pages or specific event types. For the latter, we have for example the type “search event” where the search team wants to minimize the number of typed letters before reaching the desired search result. We could technically do this using your brilliant new mapping tables of users to event_ids, but we would easily violate statistical assumptions, like very well written here. The article suggests a couple of methods to correctly estimate the variance in such cases (like the delta method and bootstrapping). Is this something you have thought about before, or that others need also? Anyone else here that have similar needs, and found a good work-around? 2. Secondly, sometimes we want to analyze and optimize on shares of something. An example: We sell groceries and recipes along with them. The products in an order can be added from our recipes or just from the users other needs. We ran an experiment on tagging the recipes that were climate friendly. We wanted to improve the share of recipes that were climate friendly, not just the # of such recipes sold. I.e. wanted the denominator to be total recipes sold, not users. Optimizely does something like this for changing denominators. Any thoughts on this? (btw, we tried to use recipes as activation metric, but we quickly ran into SRM as our treatment increased also the # of any recipe sold). 3. (also just a sidenote - as our currency is not dollars, it would be a nice to have to not have the $ sign on the revenue metric as it confuses some users 😊 ) Thanks again! 😊

We're currently working on first class support for ratio metrics (picking a custom denominator) while correcting for analysis/randomization unit variance. It's one of our most requested features.

Thanks for your response! Really great to hear! 🙌 🎉 What about when randomization unit is not the same as the analysis unit? 🤔

I think that is also solved by ratio metrics, but maybe I'm misunderstanding your specific use case.