I have a case where I need to override the metric ...
# announcements
s
I have a case where I need to override the metric definition for every A/B test (So that I measure a page load time only for the page the experiment is launched on). The way I see it, I need to add the “WHERE” filter to the SQL associated with retrieving the data for the metric (the same way it’s done for the experiment users in the Configuration window). Are there any plans for adding this functionality in the future? Setting up a separate metric for every page on the website would be quite irrational. Here’s how the underlying metric query may look like:
SELECT
user_pseudo_id, created_at AS timestamp, time_millis / 1000 AS value FROM
sendpulse_normalized.events
WHERE event_name = “FCP” AND DATE(created_at) >= “2023-08-10” AND (page_location = “https://somewebsite.com/some-page”) The part in the brackets is the filter applied in the metric overrides.
l
we have something similar (we analyze experiments in growthbook but don't use the feature flags etc) Our experiments get defined with some additional attributes that can indicate what they are targeting. In your case if the experiment was id 1234 we would end up with a table that has a structure like (this is simplified we handle arrays e.g. an experiment could target one or many page_locations): table: experiment_target_map
Copy code
experiment_id  | target_page_location
-------------------------------------
1234           | <https://somewebsite.com/some-page>
then your metric becomes:
Copy code
SELECT
  user_pseudo_id,
  created_at AS timestamp,
  time_millis / 1000 AS value
FROM
  `sendpulse_normalized.events`
WHERE
  event_name = "FCP"
  AND DATE(created_at) >= "2023-08-10"
  AND (page_location in (select target_page_location from experiment_target_map where experiment_id = '{{ experimentId }}'))
now this metric can be used as a "targeted page performance" metric and you don't need to build it for each page, you just need to populate a table that links experiments to their targeted pages
h
Some kind of experiment level template variable would be nice, but unlikely in the near term. Scott's solution seems best for now.
s
Looks like an interesting approach, but I can’t figure out how it works. Looks like we still need to somehow dynamically substitute `'{{ experimentId }}`with the real experiment id? It looks like a query with parametrization, but I don’t know if growthbook supports it and it looks like BigQuery which I use for data warehouse doesn’t support it
h
We support some SQL Template variables: https://docs.growthbook.io/app/datasources#sql-template-variables, including
'{{ experimentId }}'
s
Great! Didn’t know you guys had something like that. This should also improve the query execution and probably save some $ in BigQuery
@late-dentist-52023 Great solution! I am currently trying to figure out the best way to approach the problem of using the page load speed as a guardrail metric. There are tons of pitfalls though. Sharing this paper with you just in case (Section 5.1) https://exp-platform.com/Documents/2017-08%20KDDMetricInterpretationPitfalls.pdf I can share with you my approach in GrowthBook when or if I manage to figure it out.
l
@some-planet-44104 we've been using it for a while in different scenarios, we generally do a join on the mapping table and have some additional processing in dbt pipelines that prepare metrics etc for performance benefit. So can be optimized for speed and cost specific to your setup.