We are adopting GB for our product and I have a question, but let me know if it is not the right place to ask.
I wanted to start with an A/A test to make sure that everything works well. At the same time, we don't want to "waste time" waiting for the A/A test to conclude. So, I proposed that we run an A/A test and our first A/B test at the same time in parallel. Then, we will wait for the pre-determined sample size for each test, and look at them separately, if the A/A test concluded that we are doing everything right, then we trust the results A/B test. Otherwise, we investigate what happened. Is there something wrong with this strategy? Is there something better? other than an A/A/B test since it will take 50% more time to conclude. Thank you in advance!
08/31/2023, 8:25 AM
That’s an okay strategy
A/A tests should be a periodic thing you run and you should also be comfortable running tests in parallel
08/31/2023, 8:30 AM
Thank you Graham for you reply. Yes, I intend to start a new phase of the A/A test every month as a monitoring method