early-tent-72446
08/25/2021, 7:57 AMSELECT
*
FROM
__variations
WHERE
-- Skip experiments with fewer than 200 users since they don't have enough data
users > 200 -- Skip experiments that are 5 days or shorter (most likely means it was stopped early)
AND datediff(day, start_date, end_date) > 5 -- Skip experiments that start of the very first day since we're likely missing data
AND datediff(day, '2020-08-24 18:53:16', start_date) > 2
ORDER BY
experiment_id ASC,
variation_id ASC
while this filter AND datediff(day, start_date, end_date) > 5
makes sense in general, it excludes some important edge cases, e.g.
1. Dates filter: imagine an email experiment where different versions of the email go on the same day to the users. The datediff(day, start_date, end_date) > 5
will make this experiment impossible to analyze using Growthbook because in this case start_date = end_date ("experiment_view" event happens on the same day for everyone)
2. Imagine an experiment on an event-specific webpage (e.g. convention etc.) - same problem with all experiment exposures on the single day here
Suggestion - can the values for these filters be made as optional parameters in the config.yml
? Smth like minimum_days = 6
by defaultfuture-teacher-7046
config.yml
if you are using that.early-tent-72446
08/27/2021, 7:38 AM