depends on the query and your scale
# spicedb
j
depends on the query and your scale
b
Current volume is in the low hundreds per minute. The queries are pretty simple, evaluating 1 or 2 relationships.
j
and how many nodes are you deployed on?
b
4 instances in kubernetes
j
I'd check the following: - CPU of those nodes - Mem of those nodes - CPU of the database
its possible you're overloading Aurora - SpiceDB doesn't support read replicas
so its likely hitting a single Aurora node
b
I'm considering running separate read-only SpiceDB's that connect to aurora read nodes. We have a service in front of SpiceDB that would route traffic based on consistency level.
But thanks for the insight. I'll take a look at those metrics. If it seems like we're overloading aurora we'll want to scale up the number of aurora nodes then?
j
likely, yeah
and running a different SpiceDB is indeed an option
there is also that tracking issue for having SpiceDB doing the routing
v
would also suggest using
zed
with the debug flag to understand where time is being spent. Alternatively you can use OTel tracing, which now supports granuarity down to the SQL query
(if you want to get the SQL granuarity in the OTel traces, I suggest getting a nightly build since it's not yet in a release)
4 Views