We have an A/B/C test and we want to kill one of the groups and keep running the experiment with only two groups. Should we assign 0% of traffic to C to accomplish this? Is there a way to reassign users in group C to either A or B? If we do a new A/B experiment with a different tracking key we are at risk of users that got B before getting A now, right?
a
adventurous-computer-41819
06/27/2023, 1:20 PM
@gray-crayon-34592 have you hunted through their documentation? I think I remember reading something about this.
Other ideas..
• Do you have high sample sizes?
• Can you leave C running? How important is it to kill variation C?
◦ If C is >5% chance to be best, or has a low sample size, I suggest to just keep it running - as it makes analysis far simpler and less room for error.
• Could you do a workaround where you call C a loser, then edit the code/experience of C to match the control for the rest of the experiment - to prevent ongoing negative effects of C (and report on C using the legitimate data/timeframe)