some-planet-44104
11/17/2023, 2:20 PMrhythmic-agent-34208
11/17/2023, 2:20 PMfresh-football-47124
helpful-application-7107
11/21/2023, 6:32 PMy = b0 + b1 * t1 + b2 * t2 + b3 * t1 * t2
where t1
is 0/1 for a/b in ab test 1 and t2
is 0/1 for a/b in ab test 2. Then if b3
is different from 0 there is evidence of an interaction effect.
There's an analog in ANOVA but I find that people describe anova models in different ways that makes communicating with them harder than communicating with regression models.some-planet-44104
11/23/2023, 3:17 PMold_control, new_control, old_treatment, new_treatment
Sorry if something sounds not exactly clear - I’m still relatively new to statistics and just trying to find my way aroundhelpful-application-7107
12/01/2023, 4:05 PMI heard that same results can be obtained through regression though.Yes, I'm just stating it in regression format because I find it easier to communicate about given my backgorund. Multi-factor anova will be identical to regression in this setting.
but also for testing the presence of novelty and primacy effects? In case I decide to segment my users by new vs. returning and see if the effect of my change is the same for both new and returning users (by testing these four combinations)Generally yes, this is the rough approach to estimating "heterogeneous treatment effects" where you look at treatment effects within dimension slices, and then specifically run a test for whether the effect in group A is different from the effect in group B.