Hey SpiceDBers, wanted to share some results of a performance test on 4 EC2

c5a.xlarge

machines, 8 SpiceDB pods (default configs, except

SPICEDB_DATASTORE_REVISION_QUANTIZATION_INTERVAL = 1s

and

SPICEDB_DATASTORE_REVISION_QUANTIZATION_MAX_STALENESS_PERCENT = 0

with PostgreSQL datastore on

db.m6g.xlarge

Test is 500 requests per second - 90% checkPermission, 8% lookUpResources, 2% provisionRelations done by JMeter We managed to receive around 74% cache hit ratio, uniform SpiceDB pod requst-handling distribution by injecting LinkerDNS, and a 95 percentile of checkPermission was around 400ms, we managed as low as 300ms in one of our tests. The schema we have is roughly Account has agent to read and write, Account has customer to read, Agents belong to some agent-group Just wondering, from a fresh perspective, what would you do to improve latency? We tried setting

SPICEDB_DATASTORE_REVISION_QUANTIZATION_INTERVAL

to 10s, 60s, with no significant change to latency. Or if this maybe is just the expected behavior given the compute resources? Or if there are any areas where more info would shed some light on bottlenecks? Thank you! https://cdn.discordapp.com/attachments/844600078948630559/1158442925655588915/image.png?ex=651c438c&is=651af20c&hm=8bd845609d0f3659b1e1f5fbddd9e8dd781b5a83da4d3e27004b6cb9f5848c22& https://cdn.discordapp.com/attachments/844600078948630559/1158442925932429392/image.png?ex=651c438c&is=651af20c&hm=a01251f128bf38f44e87d4244ed9a9ce04ed8349d554a9e9eefde8efbad96aca& https://cdn.discordapp.com/attachments/844600078948630559/1158442926507044955/image.png?ex=651c438c&is=651af20c&hm=804a43123958244fb4a92711fce5dea4b37fed44a619eec360de1f77a97aa23c&