Hi, I have a question on how to use a `Retuning Visitors` as metric in GrowthBook. I use Firebase An...

red-fall-3528

08/17/2022, 12:56 AM

Hi, I have a question on how to use a

Retuning Visitors

as metric in GrowthBook. I use Firebase Analytics and currently I get almost the metrics business want to see but

Retuning Visitors

. What I stuck right now is, I can get a user count which represents the

Retuning Visitors

but it is divided by

All Experiment Users

and I get ratio instead of actual count. Any help is aprreciated.

red-fall-3528

08/17/2022, 8:44 AM

that helps a lot. thank you so much!

red-fall-3528

08/17/2022, 9:03 AM

I have few questions to ask for your suggestion. They are related to metrics. We plan to use GrowthBook for 2 types of experiments • domain expertises -led: experiments suggested by products owner from their observations. We’’ll use specific metrics, for example conversion ratio, or CRT to measure the impact. • business-led: experiments originate from business team. These experiments may come from business team’s intuition. The only focus is number of

retuning visitors

. for the first type of experiments, I think they’re kind of straightforward from experiments perspective. But for the Business-led experiments, I’m not sure if

returning visitors

, and to the extent

retention rates

, are proper metrics.

red-fall-3528

08/17/2022, 9:09 AM

I would like to ask your suggestion on the business-led experiments. Have you run the experiments with the similar setup? Could you share any pitfalls with these experiments.

future-teacher-7046

08/17/2022, 11:57 AM

I would be a little careful with using click through rate as a primary metric. It's fairly easy to increase the click through rate while actually hurting the UX and long-term metrics like revenue (e.g. giant buttons, bright colors, "free kittens" CTA, etc.) Usually companies will have a set of core business metrics that they use for every experiment, such as revenue, retention, engagement. Then, depending on the experiment you might add some secondary metrics like CTR that are more for debugging and exploration.

red-fall-3528

08/18/2022, 6:14 AM

that makes sense. We’re using DAU as a long-term metrics and ideally we want to add it into metrics on GrowthBook as well. I am not sure how can I add that into GrowthBook because all metrics have

All Experiment Users

as denominator by default.

future-teacher-7046

08/18/2022, 12:16 PM

Yeah, for DAU, we typically see companies make metrics like "day 2 retention" or "day 7 retention". Basically measuring what percent of experiment users come back after a certain amount of time. To do that you would make a binomial metric, select an "app open" event (or similar) in your SQL, set a conversion delay (24 hours for day 2, 168 hours for day 7, etc), and finally set a conversion window time of 24 hours.

red-fall-3528

08/19/2022, 1:07 AM

thanks so much. I think retention rates and the use of conversion delay. and with conversion delay, that means I can use the same query and just change to conversion delay to get retention rates from different time frame, does it?

red-fall-3528

08/19/2022, 10:44 AM

cc: @bland-tiger-91293

bland-tiger-91293

08/19/2022, 10:54 AM

I did following this: https://github.com/growthbook/growthbook-kotlin

future-teacher-7046

08/19/2022, 12:28 PM

Yes, conversion windows only have an effect inside an experiment, not on the metric preview page. Can you share the list of feature rules that you expected to trigger the tracking callback? The experiment might not be set up properly there.

red-fall-3528

08/19/2022, 2:11 PM

sure. we’re using these config rules: https://cdn.growthbook.io/api/features/key_stag_9cf44e2478194106

bland-tiger-91293

08/24/2022, 1:16 AM

trackingCallback is working fine now, Thanks Jeremy

148 Views

Open in Slack

Previous Next

GrowthBook Users

Open source platform for stress free deployments, measured impact, and smarter decisions.