A/A test
An A/A test is a type of experiment wher...
# experimentationm
mysterious-addition-37059
09/26/2024, 9:31 AMA/A test
An A/A test is a type of experiment where two identical versions of something (e.g., a website, app feature, or system process) are compared against each other. It is conducted to test the reliability and validity of the testing platform or methodology itself.
So we were running A/A test for some time, and here some results in the thread
mysterious-addition-37059
09/26/2024, 9:31 AMIMG_20240926_122919_768.jpg
mysterious-addition-37059
09/26/2024, 9:34 AMAs we can see - results of variants are not identical. But as for me - it should be.
Any investigations, ideals of how to understand better "why" results are not identical?
Maybe someone is also runned A/A with growthbook and could share results?
p
plain-apartment-23496
09/26/2024, 9:40 AMI would say that the sample size is not big enough. Give the test some more time, and it should become closer to 0% change for all metrics. Currently these results could easily happen by chance 🙂
So I see no issues, as you should get a bigger sample size before analyzing the A/A test
🔥 2
37 Views