Hi All I am running spicddb 1 17 on AWS
# spicedb
k
Hi All, I am running spicddb 1.17 on AWS EKS cluster and i am trying to push traces from spicedb application to AWS Xray using aws-otel-collector sidecar, while the traces are reaching the aws-otel-collector sidecar (i can verify it by the logs) it not getting pushed to X-Ray service and not visible on the aws console...any inputs on resolving the same will be very useful...i dont see any errors getting logged...
v
Hi! I'm afraid we haven't experimented with XRay, do you run other services with OTEL sidecar collector and it works? We have a local docker-compose example that is end-to-end, in case that helps troubleshooting locally. https://github.com/authzed/examples/tree/main/tracing
k
yes i do run other servies and I am able to push prometheus metrics to cloudwatch, but for some reason, the traces are not getting pushed to x-ray...i do see the traces getting logged in the sidecare logs though
v
From the docs I see that they need their own propagators: https://aws-otel.github.io/docs/getting-started/go-sdk/manual-instr
are other apps in your organization instrumented with the XRay propagator?
k
for the java apps we do make use of aws opentelemetry agent and auto instrumentation so i guess the traces are inline with xray specs..for spicedb as its using opentelemetry and not aws distribution of the sdk, i believe their is a mismatch in the header spec which is probably causing the issue
is their a way that i can set xray propogator & xray id generator?
v
I'm afraid not without modifying the code, I'd suggest you build SpiceDB with the modification to verify this is actually the problem, then we can discuss ways of upstreaming it so that SpiceDB can support XRay out of the box
in order to test it out, you'll have to modify @jzelinskie
cobrautil
and set the xray stuff there : https://github.com/jzelinskie/cobrautil/blob/main/cobraotel/cobraotel.go#L208-L212 Then you import that modified cobrautil into SpiceDB code, build the container, and run it in AWS to confirm it works
3 Views