Pablo
07/25/2025, 9:25 AM--enable-experimental-watchable-schema-cache
and set --datastore-schema-watch-heartbeat=300s
and saw a big spike in reads and writes to Postgres db (specially writes) which made it reach 100% cpu capacity.
We have two separate workloads of spicedb in K8S: one that is read only and another that is read+write. We enabled the watchable schema on the read-only instance but not the read+write, is this a wrong approach? Thanks a lot 🙂
edit: spicedb version is v1.38.0, postgres 16.8, deployed to AKS: read-only has 3 replicas and read+write 2 replicas
spicedb read-only args were:
- --grpc-enabled=true
- --http-enabled=false
- --dispatch-cluster-enabled=true
- --metrics-enabled=true
- --telemetry-endpoint=""
- --datastore-engine=postgres
- --dispatch-upstream-addr=kubernetes://spicedb.namespace:50053
- --datastore-conn-pool-read-max-open=230
- --datastore-conn-pool-read-min-open=115
- --datastore-gc-window=1h
- --datastore-gc-max-operation-time=3m
- --log-level=warn
- --enable-experimental-watchable-schema-cache=true
- --datastore-schema-watch-heartbeat=300s