Hey @fresh-football-47124, just had a few follow-up questions on GrowthBook:
• What is the binary size of the iOS and Android's SDKs
• Are there any limitations on how many experiments we can run at the same time?
• Do you show the number of estimated users remaining until the experiment can reach a specific confidence level?
• Can you tell me more about the Segment integration - couldn’t find any documentation on it.
• Can we have 1 feature flag for an iOS and Android experiment, so we can run 1 experiment on both platforms
• Do you all support these types of experiments?
◦ Multi-Armed Bandit
▪︎ Dynamically allocate traffic to variations that are performing well with machine learning
◦ Mutually Exclusive Experiments
▪︎ Create experiments that have mutually exclusive visitors
◦ Multivariante Testing
▪︎ Experiment on multiple elements at once to find the best combination
10/03/2022, 1:22 AM
1. Let me check on that and get back to you - but none of the SDKs are very large, typically just a few k.
2. No limitations on the platform, usually the limitations are on the amount of traffic you have to test with.
3. We do show the estimated time remaining until you reach the number of conversions required.
4. We work with Segment’s data structure, but we’re not a destination for them. You can set up Segment to export your data to a warehouse, and then you can connect GrowthBook to that warehouse. Since we know how Segment stores data, when you set that up, GrowthBook will automatically create the SQL queries.
5. Yes - you can either create separate environments per mobile OS, or use projects per OS, or set up an “OS” attribute and use that for targeting on the flags.
6. Experiment types:
a. Multi-arm bandits: not yet, but it’s on our roadmap
b. Mutually exclusive experiments - yes, currently supported with namespaces
c. Multivariate testing - sort of - you can add N number of variations, but we compare each to the control, instead of doing direct comparisons between each. In effect this tends to give the same result.
10/03/2022, 2:21 PM
@fresh-football-47124 Thanks for this, super helpful.
• Could you please tell me the limitation on the amount of traffic? I don't follow this
• Do you know the timeline on the Multi-arm bandit experiment?
10/03/2022, 2:24 PM
The traffic comment just means that you need enough users to reach statistical significance. Each additional test you run in parallel adds a little variance to results, so you need a little more traffic to compensate for that.
For multi-arm bandits, we don't have a specific timeline yet. It's not one of the top priority projects we're working on right now, but we move pretty fast and priorities change based on how many users are asking for specific features.