Hi :wave: I’m looking into GB here for the first ...
# ask-questions
m
Hi 👋 I’m looking into GB here for the first time, and I have what’s maybe a less conventional use case. My company already has a digital testing tool (Adobe Test). We want something that we can do omnichannel testing in, for a wider set of use cases, like marketing and membership tests. This means that for some tests, Growthbook won’t be the trigger at all….which I think is fine. If we have an experiment_id and a variant_id for a given entity ID (like user_id), we can import and analyze a past experiment. My real question is: Can we import even simpler incremental/directional/pre-post tests, with just a date, an experiment_id, and variant_id--no user_id? Put another way, I’m thinking of pre/post tests at a higher level than user.
It occurred to me that I could simply enter a single static user_id value, say ‘0’, which would be functionally the same as no user_id. The metrics are naturally already in our DW so that should fulfill the other end of the exp
f
ya, you can make customize the SQL to make it work with before an after testing
and its not uncommon to use GrowthBook for just the reporting side
and let us know if you have any thoughts on making a transition from Adobe Test and Target to GrowthBook easier.
m
I only wish it were up to me lol. Thanks for your help!
Any guidance on before/after testing would be helpful. My hacks so far are unsuccessful. My report still wants to pull exposures and only takes the first record of a user_id, so using a single static ID is not really working. I don’t even really care about exposures. Would I treat this scenario like a holdout experiment?
Ok, after dissecting the queries, it’s clear that the opposite approach is better, to seed an exposures table with all known user IDs. I’m using a single timestamp for all of them, one experiment name of course, and I tried one variation name, and ran the queries manually in my DW, so now I’m getting a main_sum and main_sum_squares. But when I rerun the exp results in GB, I still get ‘no data’ for all fields in my goal metrics regardless of statistics engine (probably b/c I’m only including one variation? I am ignoring conversion windows.) I know there are more queries or calcs happening behind the scenes, but I’m wondering if I do need to include two variation_ids in my seed exposures table after all, and how that will play out for this type of scenario given my variations are essentially time-based. Seems to borrow bits from feature flag and holdout experiments without fitting neatly into either. (And, FTR, I totally get it, this is not the standard use case, and we will undoubtedly have more standard use cases/exps that do follow more normal setups.)
f
ya, probably because of the one variation
if you click on the 'view queries' you should see the results that GrowthBook is getting back
m
I’ll try inserting one exposure for the other ‘variation’ and see what happens 😂