A/B test ramp up question. It seems that increas...
# announcements
c
A/B test ramp up question. It seems that increasing the sampling % in the experiment underlying feature doesn't increase the experiment sampling % ? We need to adjust the experiment sampling % directly? Would there be any trouble if the underlying feature sampling % and Experiment sampling % are differnt ?
g
@alert-portugal-86208 @astonishing-airline-55247
f
No issues with the statistics - the overall exposure (at the moment) in the experiment reports is used for records only
if your split percentage changes, you’ll get SRM errors
c
Didn't quite get this
"the overall exposure (at the moment) in the experiment reports is used for records only"
f
it’s just for recording the over all exposure on the experiment report
if you need to do a new phase, that could be what changed - the statistics uses the split percentages, not the overall exposure
c
right now if we change the sampling % in the feature, it didn't show up in the experiment. so I was confused there are two places that we need to adjust the sampling % for the experiment to change sampling ?
f
ya, we’re workng on fixing that experience
c
want to make sure I understand this correctly. • changing sampling % on feature will actually change the experiment sampling. The new sampling % is in effect even the "experiment report" page setting didn't change • changing sampling % on the "Experiment tab" is just a display on the experiment report (as a marker of different phases) so before this disconnection get fixed on the growthbook side, to change the sampling %, we need to : 1. change sampling % on the feature 2. do the same change on epxeriment so it reflects the correct % on the report (just a label, it didn't impact underlying data)
f
correct
even if you don’t do step 2, it won’t affect the statistic and results.
🙏 1
c
Got it. the experiment page change is only for better reporting/documenting purpose. Thank you !