Hey GrowthBook team - we noticed our mongodb deplo...
# give-feedback
Hey GrowthBook team - we noticed our mongodb deployment's CPU usage had gotten really high (I think this happened over a few months time), to the point of causing timeouts/aborted requests - I took a look at some of the mongo logs and it seemed to mostly be from queries that agenda makes to find/update jobs. It seems the
jobs weren't being cleaned up properly (or something along these lines):
Copy code
test> db.agendaJobs.find({name: 'eventCreated'}).count()
a majority of these had a
of `null`:
Copy code
test> db.agendaJobs.find({name: 'eventCreated', nextRunAt: null}).count()
I ended up deleting these to see if it helped and our mongo CPU usage dropped pretty much back down to 0. I haven't dug into why so many are created or whether something should actually be cleaning them up, but just thought I'd share here in case this is a known issue. It's possible its related to our own setup, although I don't think we have any changes that would affect this job type specifically
Thanks, I think we might be missing some indexes on that collection. So those un-indexed queries were doing full scans which get really expensive as the size grows. In addition to adding missing indexes, we should be able to add a cronjob to clean up those old jobs pretty easily.
my not-so-comfortable-with-node/typescript guess is that it's because
is used in the constructor of the `EventNotifier`: https://github.com/growthbook/growthbook/blob/1a9e129bfae401e62bde002fb4679b5dc9de956d/packages/back-end/src/events/notifiers/EventNotifier.ts#L[…]6 and in all usages of
a new EventNotifier is created
creating new docs for every event is the proper behavior since the event id is part of the unique key. We're just missing the cleanup part
got it, that makes sense
Added a GitHub issue to track this - https://github.com/growthbook/growthbook/issues/2116
🙏 1