Open source feature flagging and experimentation platform for stress free deployments, measured impact, and smarter decisions.

GrowthBook Users

I would go with B. A might be interesting, but the interaction effects can be hard to test. Detecting AAA vs AAB vs ABA vs ABB... (etc) will be hard and from my experience is unlikely to be interesting

for B - you just have to set up a feature flag that is not a boolean to get multiple variants

Awesome thanks graham - is it possible to add new variants whilst an experiment is in flight ?

Similarly what about adding new metrics ? Would these be filled out historically for the duration of the experiment?

You can add variants - but its not really recommended as people might switch variants (unless you have sticky bucketing enabled), or it can cause SRM warnings

metrics can be added / removed as needed - those are on the analytics side only, and don't affect assignment