https://authzed.com logo
Title
s

Shyam

01/05/2023, 11:07 AM
#844600078948630559 Hi guys, Am trying to change the endpoint for opentelemtry tracing by using the env variables suggested here https://github.com/jzelinskie/cobrautil/pull/3 But seems to be that during the initialization, the spicedb is not taking the exporters and endpoints that have been passed as ENV variables. "OTEL_EXPORTER_OTLP_ENDPOINT" Any suggestions ?
v

vroldanbet

01/05/2023, 12:58 PM
👋🏻Have you tried with
SPICEDB_OTEL_PROVIDER=otlpgrpc
SPICEDB_OTEL_ENDPOINT=your_endpoint
s

Shyam

01/05/2023, 3:46 PM
am able to initialize now, but do we have proxy endpoint that we can use . Could you please suggest
v

vroldanbet

01/05/2023, 3:49 PM
sorry I'm not sure I follow.
SPICEDB_OTEL_ENDPOINT
should be your collector endpoint
s

Shyam

01/05/2023, 5:52 PM
When i use the localhost endpoint, its tailing with a ":" although i dont provide this is the endpoint variable. The same happen while using the newrelic endpoint as well. Not sure if its expecting any variables . Please suggest. Thank you
v

vroldanbet

01/09/2023, 11:49 AM
@Shyam 👋🏻 I'm afraid I need more context in order to understand what's going on. Can you provide us with the values you are using for the env variables? (obfuscating any sensitive parts). We have an example setup to demonstrate OTEL tracing in our examples repository: https://github.com/authzed/examples/tree/main/tracing The docker-compose file will stand up the otel-collector, tempo, grafana and SpiceDB. I was for example able to do
docker-compose up otel-collector tempo grafana
and then run SpiceDB in my IDE with the following env variables:
SPICEDB_OTEL_ENDPOINT=0.0.0.0:4317
SPICEDB_OTEL_PROVIDER=otlpgrpc
SPICEDB_OTEL_INSECURE=true
(Do not use insecure flag unless you are testing locally)
s

Shyam

01/10/2023, 1:41 PM
@vroldanbet I am using newrelic endpoint and these are the variables I have set. SPICEDB_OTEL_ENDPOINT=otlp.nr-data.net:4317 SPICEDB_OTEL_PROVIDER=otlpgrpc SPICEDB_OTEL_INSECURE=true I get the following error rpc error: code = Unavailable desc = connection error: desc "transport: Error while dailing dail tcp ; i/o timeout"
v

vroldanbet

01/10/2023, 1:42 PM
is otlp.nr-data.net:4317 a newrelic endpoint? if so, I assume TLS must be enabled?
if so please do not set OTEL_INSECURE to true
s

Shyam

01/10/2023, 1:55 PM
yes, this has been set to false , but still I get timeout error.
and Is there a possible way that I can pass the "api-key" which is the password for my newrelic account ?
Also, I have tried http provider as below SPICEDB_OTEL_ENDPOINT=otlp.nr-data.net:4318 SPICEDB_OTEL_PROVIDER=otlphttp Below is the error. 2023/01/10 14:46:42 Post "http://otlp.nr-data.net:4318/v1/traces": dial tcp 162.x.x.x:4318: i/o timeout (Client.Timeout exceeded while awaiting headers) It is expecting the "api-key" for the newrelic account, to which the data needs to be pushed to.
v

vroldanbet

01/10/2023, 4:04 PM
hrm 😕 returning a timeout because the API key is missing would seem a surprising API response
s

Shyam

01/16/2023, 8:07 AM
@vroldanbet I have referred to the example repo(https://github.com/authzed/examples/tree/main/tracing) and it has been mentioned that this setup cannot be used for production observability setup. Could you please let me know if there is any cons with this approach or if any other approach to achieve observability. Thanks
v

vroldanbet

01/16/2023, 8:12 AM
Hi, that's is right, this is an examples repository and it is NOT meant to be used in production. I only pointed it out so you could troubleshoot locally with your client application.
s

Shyam

01/16/2023, 8:16 AM
@vroldanbet Thanks for the clarification, I have tried the setup from examples repo, but still I get the connection error as below . "lookup otel-collector on 172.x.x.x:53: no such host"
v

vroldanbet

01/16/2023, 8:19 AM
Thats the name the collector gets in the network created by docker. Your client app won't be able to resolve it. Instead you need to check port mapping as shown with the command "docker ps"
s

Shyam

01/16/2023, 8:23 AM
Am using this setup in kubernetes and I have given the same name for my 'otel-collector' deployment. So referring to this in the endpoint.
v

vroldanbet

01/16/2023, 9:55 AM
Is the
Service
you created also named that way?
s

Shyam

01/16/2023, 11:34 AM
I did try to use the service name in the endpoint, but when I use the service name it fails with "connection refused error"