Is there a way to automatically stop an experiment...
# give-feedbacks
some-planet-44104
02/06/2023, 12:12 PMIs there a way to automatically stop an experiment when certain number of users has been reached? We want to make sure that experiments don’t run longer than needed and automatically stop when the required sample size has been reached.
f
fresh-football-47124
02/06/2023, 1:17 PM
Hi Yevhen - not currently.
fresh-football-47124
02/06/2023, 1:17 PM
we do have that on our roadmap
fresh-football-47124
02/06/2023, 1:18 PM
the default statistics package is Bayesian, which doesn’t have that fixed horizon
fresh-football-47124
02/06/2023, 1:19 PM
would you want it to be turned off and automatically implement one of the cases, or just off and back to the control?
s
some-planet-44104
02/06/2023, 10:29 PMIn my case turning off is quite enough, but I guess directing all the traffic to the winning variant is the usual case with experimenting platforms
some-planet-44104
02/06/2023, 10:31 PMI understand the Bayesian approach is somewhat different from Frequentist one, but I’m still not completely sold on it so I still apply frequentist concepts to it, just to be on the safe side.
26 Views