Hi, I have a question on how to use a `Retuning Vi...
# announcementsr
red-fall-3528
08/17/2022, 12:56 AMHi, I have a question on how to use a
Retuning VisitorsRetuning VisitorsRetuning VisitorsAll Experiment Usersred-fall-3528
08/17/2022, 8:44 AMthat helps a lot. thank you so much!
red-fall-3528
08/17/2022, 9:03 AMI have few questions to ask for your suggestion. They are related to metrics.
We plan to use GrowthBook for 2 types of experiments
• domain expertises -led: experiments suggested by products owner from their observations. We’’ll use specific metrics, for example conversion ratio, or CRT to measure the impact.
• business-led: experiments originate from business team. These experiments may come from business team’s intuition. The only focus is number of
retuning visitorsreturning visitorsretention ratesred-fall-3528
08/17/2022, 9:09 AMI would like to ask your suggestion on the business-led experiments. Have you run the experiments with the similar setup? Could you share any pitfalls with these experiments.
f
future-teacher-7046
08/17/2022, 11:57 AM
I would be a little careful with using click through rate as a primary metric. It's fairly easy to increase the click through rate while actually hurting the UX and long-term metrics like revenue (e.g. giant buttons, bright colors, "free kittens" CTA, etc.)
Usually companies will have a set of core business metrics that they use for every experiment, such as revenue, retention, engagement. Then, depending on the experiment you might add some secondary metrics like CTR that are more for debugging and exploration.
r
red-fall-3528
08/18/2022, 6:14 AMthat makes sense. We’re using DAU as a long-term metrics and ideally we want to add it into metrics on GrowthBook as well. I am not sure how can I add that into GrowthBook because all metrics have
All Experiment Usersf
future-teacher-7046
08/18/2022, 12:16 PM
Yeah, for DAU, we typically see companies make metrics like "day 2 retention" or "day 7 retention". Basically measuring what percent of experiment users come back after a certain amount of time.
To do that you would make a binomial metric, select an "app open" event (or similar) in your SQL, set a conversion delay (24 hours for day 2, 168 hours for day 7, etc), and finally set a conversion window time of 24 hours.
r
red-fall-3528
08/19/2022, 1:07 AMthanks so much. I think retention rates and the use of conversion delay.
and with conversion delay, that means I can use the same query and just change to conversion delay to get retention rates from different time frame, does it?
red-fall-3528
08/19/2022, 10:44 AMcc: @bland-tiger-91293
b
bland-tiger-91293
08/19/2022, 10:54 AMI did following this: https://github.com/growthbook/growthbook-kotlin
f
future-teacher-7046
08/19/2022, 12:28 PM
Yes, conversion windows only have an effect inside an experiment, not on the metric preview page.
Can you share the list of feature rules that you expected to trigger the tracking callback? The experiment might not be set up properly there.
r
red-fall-3528
08/19/2022, 2:11 PMsure. we’re using these config rules: https://cdn.growthbook.io/api/features/key_stag_9cf44e2478194106
b
bland-tiger-91293
08/24/2022, 1:16 AMtrackingCallback is working fine now, Thanks Jeremy
58 Views