willthornton.
11/28/2023, 10:55 AMvroldanbet
11/28/2023, 11:33 AMdatastoreRevisionQuantizationMaxStalenessPercent
parameter. Increasing will also help in situations where there is contention on the connection pool.
We also recommend running with PG >= 15, as it is necessary to get the query planner to select some of the new indexes. We've also observed some minor performance improvement by moving to it.
We also definitely recommend making the jitter basically same or close to max connection lifetime, as it helps amortize the impact of connections being removed from the pool due to reaching max lifetime.
There is exhaustive connection pool metrics, please refer to the /metrics
endpoint to see the documentation around each metricsashayakovtseva_46690
11/28/2023, 1:21 PMrate(pgxpool_acquire_duration_ns{pool_usage="read"}[$__rate_interval]) / rate(pgxpool_acquire_count{pool_usage="read"}[$__rate_interval])
). keep that in mind when analyzing spicedb_datastore_query_latency
which is a histogram
2) there seem to be a 3x difference between sum(spicedb_datastore_query_latency_count{operation="QueryRelationships"})
and sum(pgxpool_acquire_count)
which I can't get my head around.
QueryRelationships doesn't use cache (does it), so whenever sum(spicedb_datastore_query_latency_count{operation="QueryRelationships"})
is incremented -> we actually issued a datastore query -> we acquired and then returned connection from/to the pool. so I assume the above sum should be equal to (or at least be close to, taking into account other db operation) sum(pgxpool_acquire_count)
. but with my local docker-compose installation I get like x3.3 difference (with sum(pgxpool_acquire_count)
being smaller).willthornton.
11/28/2023, 2:09 PMvroldanbet
11/28/2023, 2:28 PMvroldanbet
11/28/2023, 2:45 PM