Hello Growthbook team. I have a question about the interpretation of the results table. We have set up our first A/A test to see how the data comes in and looks. To do this, we used "number of experiments" as the metric. I understand the denominator (number of users), but I don't understand the numerator and why there is such a big difference between the baseline and variant 1. Where does this number come from and why is there such a big gap between the two variants?

⭐ Our official support hours are 6:30am - 5pm Pacific Time, Monday through Friday. You may occasionally hear from us outside of these hours. Your support request has been logged in our system. Our support team will get back to you very soon!

Hi Stefan, our office is closed today for the US national holiday, but we’ll follow up on this as soon as we’re back online. Thank you!

Hi again, Stefan, It seems like the traffic is indeed splitting roughly 50/50 so we are properly seeing the tracked experiment events in your warehouse as 50/50. It's likely the metric is set up in a way that is depending on the experiment values (generally somewhat dangerous for all metrics) so it would be helpful to see what it looks like. Could you send a screenshot?

Hey August. Thanks for your reply. The set up of the metric looks like this: We called this metric "Count of experiments - Prod".

So the ​`value`​ column here seems like it will return 0 for control users and 1 for treatment users. So naturally it will be 1 for all treatment users and 0 for all control users (the fact that there is any 1 in your control is interesting, probably some bug with the id logging)

What we do now for COUNT metrics is we

sum(value)

and since you're passing the int_value as

value

, we'll end up summing a bunch of zeros for your control users.

Hey Luke. It was exactly as you said. We added zeros, so the numerator was 1 (we don't know why exactly 1 and not 0, either). That was our A/A test. So we wanted to see that there were no changes or increases between the treatment and the control user. We just wanted to see the sum of the "experiment viewed" event (regardless of the test group). We changed the user value aggregation to "Count(*)" and now it looks the way we wanted it to: Thanks for your help!