#844600078948630559 Hi guys, Am trying to change the endpoint for opentelemtry tracing by using the env variables suggested here https://github.com/jzelinskie/cobrautil/pull/3 But seems to be that during the initialization, the spicedb is not taking the exporters and endpoints that have been passed as ENV variables and takes the default values Passing a different endpoint using this variable "OTEL_EXPORTER_OTLP_ENDPOINT", but doesn't work Any suggestions ?

👋🏻Have you tried with

SPICEDB_OTEL_PROVIDER=otlpgrpc
SPICEDB_OTEL_ENDPOINT=your_endpoint

am able to initialize now, but do we have proxy endpoint that we can use . Could you please suggest

sorry I'm not sure I follow.

SPICEDB_OTEL_ENDPOINT

should be your collector endpoint

When i use the localhost endpoint, its tailing with a ":" although i dont provide this is the endpoint variable. The same happen while using the newrelic endpoint as well. Not sure if its expecting any variables . Please suggest. Thank you

@Shyam 👋🏻 I'm afraid I need more context in order to understand what's going on. Can you provide us with the values you are using for the env variables? (obfuscating any sensitive parts). We have an example setup to demonstrate OTEL tracing in our examples repository: https://github.com/authzed/examples/tree/main/tracing The docker-compose file will stand up the otel-collector, tempo, grafana and SpiceDB. I was for example able to do

docker-compose up otel-collector tempo grafana

and then run SpiceDB in my IDE with the following env variables:

SPICEDB_OTEL_ENDPOINT=0.0.0.0:4317
SPICEDB_OTEL_PROVIDER=otlpgrpc
SPICEDB_OTEL_INSECURE=true

(Do not use insecure flag unless you are testing locally)

@vroldanbet I am using newrelic endpoint and these are the variables I have set. SPICEDB_OTEL_ENDPOINT=otlp.nr-data.net:4317 SPICEDB_OTEL_PROVIDER=otlpgrpc SPICEDB_OTEL_INSECURE=true I get the following error rpc error: code = Unavailable desc = connection error: desc "transport: Error while dailing dail tcp ; i/o timeout"

is otlp.nr-data.net:4317 a newrelic endpoint? if so, I assume TLS must be enabled?

yes, this has been set to false , but still I get timeout error.

hrm 😕 returning a timeout because the API key is missing would seem a surprising API response

@vroldanbet I have referred to the example repo(https://github.com/authzed/examples/tree/main/tracing) and it has been mentioned that this setup cannot be used for production observability setup. Could you please let me know if there is any cons with this approach or if any other approach to achieve observability. Thanks

Hi, that's is right, this is an examples repository and it is NOT meant to be used in production. I only pointed it out so you could troubleshoot locally with your client application.

@vroldanbet Thanks for the clarification, I have tried the setup from examples repo, but still I get the connection error as below . "lookup otel-collector on 172.x.x.x:53: no such host"

Thats the name the collector gets in the network created by docker. Your client app won't be able to resolve it. Instead you need to check port mapping as shown with the command "docker ps"

Am using this setup in kubernetes and I have given the same name for my 'otel-collector' deployment. So referring to this in the endpoint.

Is the

Service

you created also named that way?

I did try to use the service name in the endpoint, but when I use the service name it fails with "connection refused error"