Hey! We see

unexpected consistency middleware error

logged every now and then. What are the possible causes for this error?. EDIT: Nvm, I see the log also has an

error

specifying the actual error.

this typically happens when you are providing malformed zedtokens, or you've manipulated your datastore in a way that has broken SpiceDB's own MVCC mechanism (e.g. truncating transactions table and the like)

Thanks for the reply! But it was actually just a cancelled context due to a timeout. The curious thing now though is why we timeout. The timeout is 5 seconds, but we seem to spend more time than that on the span

OptimizedRevision

. I'm thinking it might be locked on https://github.com/authzed/spicedb/blob/f813abe7188a8aa5f97268cb1f10c956dad0769f/internal/datastore/revisions/optimized.go#L77.

make sure you have enough SpiceDB nodes and your DB has a sufficient number of available connections

there is open telemetry tracing on, and if you use Postgres, there is also tracing down to the postgres driver. It would allow you to see where is time being spent