https://www.growthbook.io/ logo
#give-feedback
Title
# give-feedback
b

better-waiter-76960

06/08/2023, 4:26 PM
Hi team, is there a way to run experiments on different level? Can leverage GrowthBook so that it runs user level experiments as well as page level experiments? I haven't found relevant materials from the documentation cc: @gorgeous-needle-6147
f

fresh-football-47124

06/08/2023, 4:28 PM
yes, I believe so…
can you describe what the difference is?
b

better-waiter-76960

06/08/2023, 4:30 PM
I think Growthbook is defaults all of the experiment analysis at the user level
However we want to add an experiment that happens at the page level. Page views might be something we are interested in understanding
l

late-dentist-52023

06/08/2023, 4:46 PM
we’ve used Ratio Metrics to get at this, but it does require creating 2x as many metrics to be able to get things at a device/event level
f

fresh-football-47124

06/08/2023, 4:48 PM
you can add a ‘pageurl’ as an attribute, and then target experiments at that
do you have a lot of urls you’re looking to test?
h

helpful-application-7107

06/09/2023, 12:03 AM
This is a complex area with a variety of solutions. One possible solution is to just focus on your unit of interest: pages. In general, GrowthBook doesn't care what unit you are working with. You could work with pages, devices, users, etc. all the same, so long as you are firing experiment impressions and bucketing on that ID AND have metrics set up that are at that ID level. For example, you could randomize pages by some page ID, and then analyze user behavior on that page that gets tracked at the page ID level. This will require you to create a page ID, fire a tracking callback whenever you randomize the page into one treatment or another, get features using that page ID, and then connecting GrowthBook to your data warehouse with a
page
ID type. You then would need to create metrics that were at this
page
level, whether it was aggregating over user behavior on that page or some other metric, but basically you need to create a metric on that same
page
ID level. There are some statistical nuances I could get in to here, but if you wanted to randomize at the page level (e.g. load version X or Y for all pages that get loaded), then this is one possible approach.
@late-dentist-52023 I'd be curious to learn more about your approach if you have time to explain how you set up your system.
l

late-dentist-52023

06/09/2023, 9:11 PM
@helpful-application-7107 it isn’t exactly the same in that for us it isn’t a
page
that we are evaluating. Also growthbook is our analysis tool, but is not serving features so I don’t know how much that would alter what someone else could implement easily. I don’t have the time today, but will try to summarize the setup for you and some of the tradeoffs we are willing to accept.
8 Views