https://www.growthbook.io/ logo
Join Slack
Powered by
Hi all, for the last few days we’ve been seeing “l...
# ask-questions
r
Hi all, for the last few days we’ve been seeing “license server error” message show up on our growthbook instance (its deployed on-prem). can someone help us figure out what is going on?
its just a 502 bad gateway error, no other iniformation
👀 1
the experiments seem to be running fine, and we’re also able to add new members. the only issue i’m seeing is I am unable to give environment level access to new members, it says “purchase a commercial license to enable per env and per project permissions”. We already have a commercial license ofcourse
w
Is it possible something changed in your internal network? The license server needs to be whitelisted, and perhaps that got removed?
r
What is the license server? I don’t remember needing to add that when I set it up
w
If you are running your application behind a firewall you will need to whitelist 
75.2.109.47
to be able to have your license verified.
r
that’s a public IP ?
w
yes
r
Is this a new requirement? I don’t believe we’ve whitelisted this ever and we’ve had the license for about a year
w
No it is not a new requirement
Is there perhaps a new firewall on your network, or changed settings?
r
ok lemme check. this needs ingress to our growthbook instance?
w
No. egress
r
okay
we don’t have any egress limited
Copy code
egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
    description = "Allow all outbound traffic"
  }
w
hrmm.
Perhaps it is limited in your firewall settings not in your network settings
r
Ok I can check with the IT team, lemme get back to you
Nothing has changed with the firewall settings in a while
Can it be something on your end?
Could you try to simulate the response if i give you the license number
w
It could be possible, but the server doesn't return 502s and the load balancer returned only 1 502 one week ago, but GB checks the license server daily.
so I don't think it is our end. If you give me your license key, or the name of your org I can try and look into it more.
r
Engine by MoneyLion
w
I do see successful requests made to the license server. So it looks like your instance is in a strange error state. Can you let me know what version of Growthbook you currently are on?
r
looks like we pull the latest image from docker 
FROM growthbook/growthbook:latest
w
Thanks
r
maybe i can try to redeploy. I also noticed it says license expires feb 22,2025 — it is set to autorenew right?
w
correct
You can try redploying, I don't think it will help. I'll try and figure out what is going on, or give you a workaround soon.
r
sounds good, thank you!
w
Sorry for your troubles!
actually redeploying may help.
r
no problem, thanks for your quick response
w
I see that it is saying you are on a version from 2024-04-11. So that is probably the last time you redeployed
If you go to your /settings page and click on "Refresh" in the license section do you see any issues?
r
huh
I’ve tried that a few times before and nothing changed, but just tried it now and the error disappeared
did you do something on your end?
w
no
r
oh jk, it’s back
weirdly though - it says license is active 😕
w
it should be active, right?
r
yes ofcourse. I figured it would say something else since the license isnt being verified
w
We do see the verification on our license server as if everything is normal.
r
im going to try to redeploy and report back
w
ok. I'm glad that things seem to be working now and hopefully it won't happen again. We haven't had any other reports from any of our other users, so whatever weird error state you might have been in might be hard to figure out. Do definitely let us know if it happens again.
r
To be clear, its still ongoing. Let me see if redeploying solves it
w
Ok. I'm confused as to what broken thing you are seeing. If you want to screen shot your license section on the General settings page that might help.
Or a screenshot of wherever you are seeing a 502 error message.
r
Screenshot 2025-01-30 at 10.11.13 AM.png,Screenshot 2025-01-30 at 10.11.30 AM.png,Screenshot 2025-01-30 at 10.11.39 AM.png
We’ve been able to give our environment level access for the last year, but started seeing this issue recently. Have there been changes to the pro plan, and is that different from the “commercial” plan ?
w
no. There was a bug where the error message for a single bad license request wasn't being cleared that existed for a day. I'm not sure if that happened in April last year. But it is now entirely possible that your redeploy will fix the situation.
r
i just redeployed
w
Have you tried clicking on the license refresh again?
r
yah, still the same
w
im sorry, that is unfortunate. I'll try and see if I can recreate your issue.
It does seem though that you are still on a version from 2024-04-11.
So it looks like your redeploy didn't get the latest version from growthbook for some reason.
r
ok lemme push a new image to ECR
👍 1
w
I can confirm that that old version you were on had a bug where if there was a random network error to the license server one time then the error wouldn't clear even though subsequent requests worked. That was fixed before our major version release. I think you must have been the one 502 request that failed a week ago. Once you push your new ECR image the error should go away.
Sorry that that bug ended up impacting you 9 months later!
r
okay I see, good to know! I’ll report back after we redploy 🙂
thanks for the quick response on this so far, appreciate it
w
Sure thing. Let us know if you run into any other issues.
r
ok sweet, the redeploy seems to have fixed it! thank you 🙂
🙌 1
24 Views
Open in Slack
PreviousNext
Powered by