We've just hit a serious bug in Growthbook's handling of invalid JSON in conditions. If there's invalid JSON in the targeting for an experiment, the UI accepts it, but the ultimate output of the CDN completely omits the condition ‼️
e.g. the trailing comma in this JSON is technically not valid
It's then brought into this feature as an experiment rule
Then in the final output from the CDN - entirely missing the
I think this is quite a serious problem. Obviously yes the UI should be validating it, but also there must be some kind of exception/error being thrown on the backend that should surely prevent the entire CDN output from being generated, and causing alerts
of course we've been confused about this not behaving as expected for weeks 🙃 but it's nice to have figured it out now
12/11/2023, 2:40 PM
Yeah, we should be validating/preventing this up front in the targeting modal. We didn't want a typo in one feature to prevent the entire payload from rendering in the CDN, which is why we're trying to recover gracefully there. We're defaulting to omitting the condition, but I can see why that might cause issues.
We have a pretty good UI workflow for JSON feature flags that I think we can re-use for advanced targeting conditions. If you make a small, easily correctable typo (e.g. extra comma), we fix it automatically and ask you to confirm the fix and save again.
12/11/2023, 2:42 PM
We would significantly prefer things to break more seriously if something isn't working - at least the entire feature. Hidden/silent failure is incredibly dangerous
12/11/2023, 2:45 PM
yes, agreed. That's what we do for most things, not entirely sure why conditions were treated differently. We should have, at the very least, defaulted to a condition that always evaluated to false, in essence forcing the rule to skip