This message was deleted.

Hi Nishant - did you turn off the experiment or change the winning variation to 100%?

Steps that i followed: 1. Stopped the experiment and declared treatment as winner. 2. Disabled the A/B experiment rule under the feature flag. 3. Set the default behaviour of the feature flag to New feature.

okay, cool

No

but yes, it’s possible that some users who were in the control, will now have events caused by the rolled out new variation

We have different conversion window for different metrics. One that we focus on has 2 day and 7 day conversion window.

there are a couple of ways around this

The feature is a clear winner in our case. The discussion is around the actual uplift. The day of conclusion the lift was 27% with Prob(to beat control) was 98% and risk to be around 0.01%. After few days of feature roll-out the initial gain that we had has reduced to 15% uplift with Prob(to beat control) is 95% with risk of 0.1%.

you could change the exp end date back 7 days… but it might not give the same results

The discussion is around the gain we see in the control ->is it due to feature roll-out to 100% of users and some users from the control are taking the test and converting quickly hence minimising the gap between variation and control.

yes, it seems like it

I will look at all the suggestions provided by you. Thanks a ton for clarifying the query.