Hi everyone I occasionally need to run tests on us...
# ask-questions
w
Hi everyone I occasionally need to run tests on users whom I cannot explicitly attribute. To address this, I implemented the 32-bit FNV-1a hash algorithm for deterministic hashing of string values, such as user IDs, to ensure an even distribution of hash values. I then used this hash function to assign users to A/B test variations, assigning each user to a specific group based on their hash value and the experiment parameters (coverage, number of variations, weights). This approach ensures a deterministic and controlled assignment of users to groups, replicating the user splitting logic similar to what is used in the Growthbook platform. However, I encountered a question while validating my manual split. Using a randomly selected portion of historical data, I ran 10,000 splits and: • For a set of binomial metrics, I checked via a z-test for proportions that the A/A test did not exceed the 5% threshold (i.e., the p-value exceeded the threshold no more than 500 out of 10,000 times). • I then examined the p-value distribution to ensure it was uniform. • In the second stage, I performed a homogeneity check for categorical metrics using a χ²-test. Additionally, I extracted the salt used in a conducted test and applied it to my function—this resulted in matching group distributions. Despite this, I still have concerns and would like to ensure the correctness of my group splitting approach. Could you kindly provide further advice or recommendations?