brief-traffic-36675
08/17/2023, 3:02 PMk8s
)
OTEL_EXPORTER_OTLP_ENDPOINT: <http://prometheus-pushgateway:9091/metrics>
OTEL_SERVICE_NAME: growthbook
Our prometheus-pushgateway
is running and is exposed as an internal k8s service. From within a growthbook pod:
curl <http://prometheus-pushgateway:9091/metrics>
# HELP go_gc_duration_seconds A summary of the pause duration of garbage collection cycles.
# TYPE go_gc_duration_seconds summary
.
.
.
However, what I am seeing in the growthbook
logs is the following
Process is not running on K8S Error: Failed to load page, status code: 403
at IncomingMessage.<anonymous> (/usr/local/src/app/node_modules/@opentelemetry/resource-detector-aws/build/src/detectors/AwsEksDetector.js:192:32)
at /usr/local/src/app/node_modules/@opentelemetry/context-async-hooks/build/src/AbstractAsyncHooksContextManager.js:50:55
at AsyncLocalStorage.run (node:async_hooks:319:14)
at AsyncLocalStorageContextManager.with (/usr/local/src/app/node_modules/@opentelemetry/context-async-hooks/build/src/AsyncLocalStorageContextManager.js:33:40)
at IncomingMessage.contextWrapper (/usr/local/src/app/node_modules/@opentelemetry/context-async-hooks/build/src/AbstractAsyncHooksContextManager.js:50:32)
at IncomingMessage.emit (node:events:525:35)
at IncomingMessage.emit (node:domain:489:12)
at endReadableNT (node:internal/streams/readable:1358:12)
at processTicksAndRejections (node:internal/process/task_queues:83:21)
{
"stack": "OTLPExporterError: Not Found\n at IncomingMessage.<anonymous> (/usr/local/src/app/node_modules/@opentelemetry/otlp-exporter-base/build/src/platform/node/util.js:103:39)\n at /usr/local/src/app/node_modules/@opentelemetry/context-async-hooks/build/src/AbstractAsyncHooksContextManager.js:50:55\n at AsyncLocalStorage.run (node:async_hooks:319:14)\n at AsyncLocalStorageContextManager.with (/usr/local/src/app/node_modules/@opentelemetry/context-async-hooks/build/src/AsyncLocalStorageContextManager.js:33:40)\n at IncomingMessage.contextWrapper (/usr/local/src/app/node_modules/@opentelemetry/context-async-hooks/build/src/AbstractAsyncHooksContextManager.js:50:32)\n at IncomingMessage.emit (node:events:525:35)\n at IncomingMessage.emit (node:domain:489:12)\n at endReadableNT (node:internal/streams/readable:1358:12)\n at processTicksAndRejections (node:internal/process/task_queues:83:21)",
"message": "Not Found",
"name": "OTLPExporterError",
"data": "404 page not found\n",
"code": "404"
}
Any suggestions?future-teacher-7046
/metrics
part in your endpoint. That is added automatically by the OpenTelemetry library we're using./metrics/metrics
now and that's why it's throwing a 404 errorbrief-traffic-36675
08/17/2023, 3:58 PMroot@growthbook-api-web-c4f677c8-wh749:/usr/local/src/app# env | grep -i otel
OTEL_EXPORTER_OTLP_ENDPOINT=<http://prometheus-pushgateway:9091>
future-teacher-7046
brief-traffic-36675
08/17/2023, 4:08 PMOTEL_EXPORTER_OTLP_ENDPOINT
if I understand correctly?future-teacher-7046
brief-traffic-36675
08/17/2023, 4:13 PMProcess is not running on k8s
seems to be related to a check of whether the app is running specifically on EKS. I have no experience on node.js
but could the library be useful only on EKS? (don't think so but just asking)