https://authzed.com logo
Join Discord
Powered by
I upgraded my client this week from 0 10
# spicedb
b
I upgraded my client this week from 0.10.0 to 0.12.1; and the SpiceDB infra with it to the latest version - and have narrowed down a dependency issue with grpc-js (ver >1.6.9) which seems to cause random spikes for simple CheckPermission queries.
v
We haven't had reports so far. What versions > 1.6.9 did you test with? Any chances you can test with the latest and greatest?
b
did that this afternoon; with no success.
everything went to the latest, and I've been working my way back down to find stuff that worked.
heres the perspective from my client side: (note the big spikes are 70000ms response) - the drop is me downgrading grpc.

https://cdn.discordapp.com/attachments/1144226808305303562/1144244657413443625/image.png▾

heres the same time period; except from the spicedb trace data (note 1000ms peak)

https://cdn.discordapp.com/attachments/1144226808305303562/1144245056262385695/image.png▾

What i'm basically seeing is if my grpc-js resolves to a version thats above 1.6.9; the connection appears to hang on my clients end but spicedb seems to indicate that it returned the data in a reasonable/consistent time. So to me that seems like its not detecting connection close or something like that.
I will note though; that in my implementation I'm recycling the same client for every call (I open one connection in my 'SpiceService' constructor and then just use that via proxy functions)
j
@brentpi_35990 you see this behavior even if you keep the authzed-node client the same version, but downgrade just grpc-js?
b
correct, the only difference between 'bad' 0.10.0 and 'good' 0.10.0 was the grpc-js that was resolved.
j
okay
that's quite useful to know
just making sure that its isolated to the underlying lib
looks like 1.7.0 of grpc-js enabled a feature called outlier detection by default: https://github.com/grpc/grpc-node/pull/2221/files
its backed by an ENV var: might be worth trying to set it to force that off and see if it helps
there are a few other changes in the client at the version, but that might be something to look at
b
thanks heaps, i'll have a look tomorrow.
p
We had something similar happen. Unfortunate version bump of local grpc-js caused constant connection drops and increase in spicedb latency (presumably because the connection was dropped before it could return the data?). Broke stuff for us. Had to revert back to the previous version. Will add the version numbers in a bit if that's any help
j
@perseus29 that would be good to know
we might revert our grpc-js dep
if it is a wide issue
p
Sure, lemme see. It was the most recent grpc-js version released at the time
Bump was to 1.9.0 that broke it. We were at 1.8.12
j
hmmm
PreviousNext
Powered by