Hello there
I looks like we overlooked something a...
# ask-questionsn
nutritious-airport-83990
04/26/2023, 4:23 PMHello there
I looks like we overlooked something and it goes wrong.
General question: should we change something in the code in order to use Namespaces function?
Active test
Groups A (33,3%), B (33,3%), C (33,3%). Namespace: [0, 0.5]
Disabled test
Groups A, B, Namespace [0.5, 1]
In the Active test we have A: 80k users, B: 20k users, C: 20k users. (based on analytics)
The split is not correct. Could you please advise where we can find the issue?
f
fresh-football-47124
04/26/2023, 4:30 PM
Hi Alexander - did you always have that same split or did you adjust it at some point?
n
nutritious-airport-83990
04/26/2023, 4:31 PMHi Graham - Sometimes we have two groups for this feature flag, sometimes we have three groups.
f
fresh-football-47124
04/26/2023, 4:31 PM
oh, for the same experiment name?
n
nutritious-airport-83990
04/26/2023, 4:32 PMyes. But it was disabled before the latest run
f
fresh-football-47124
04/26/2023, 4:32 PM
the ‘track by’ is the same for both rules?
n
nutritious-airport-83990
04/26/2023, 4:35 PMDo you mean this?
nutritious-airport-83990
04/26/2023, 4:37 PMIn the Active test we have one key (feature flag) and in the disabled test we have another key (feature flag)
nutritious-airport-83990
04/26/2023, 4:37 PMThe namespace has the same name in both tests
nutritious-airport-83990
04/26/2023, 4:41 PMAnd we user deviceID for the splits on the device.
nutritious-airport-83990
04/26/2023, 4:43 PMBased on numbers I can assume that
Group A = Group A from the Namespace: [0, 0.5] + the whole Namespace [0.5, 1] - 80k
Group B = Group B from Namespace: [0, 0.5] - 20k
Group C = Group C from Namespace: [0, 0.5] - 20k
f
fresh-football-47124
04/26/2023, 4:52 PM
Did you add a new phase to the experiment to ignore past users?
fresh-football-47124
04/26/2023, 4:52 PM
(It will set a date range)
n
nutritious-airport-83990
04/26/2023, 4:55 PMHow can I add a new phase?
We have very low retention rate, so the majority of the users are new.
nutritious-airport-83990
04/26/2023, 4:56 PMAnd we don't want to ignore the past users.
I thought GB splits every user every time it sees him.
nutritious-airport-83990
04/26/2023, 4:59 PMI removed namespaces and I'm using 100% of the traffic. Now I have a nice 33% split for every group in the active test again
f
fresh-football-47124
04/26/2023, 5:00 PM
ok
n
nutritious-airport-83990
04/26/2023, 5:04 PMBut it doesn't solve the initial issue. 😭 We'd like to have mutually exclusive tests. I would greatly appreciate your advise, Graham.
f
fresh-football-47124
04/26/2023, 6:30 PM
Can you DM me a screen shot of the rules? Something seems a bit off
n
nutritious-airport-83990
04/26/2023, 6:32 PMDM'd
79 Views