I am getting this in the logs:

│ spicedb-5bc6d7f6d5-9g8p7 {"level":"warn","error":"timeout: context deadline exceeded","time":"2023-01-05T13:53:24Z","message":"error attempting to perform garbage collection"}                                 │
│ spicedb-5bc6d7f6d5-dx8sb {"level":"error","module":"pgx","args":[4124334],"err":"timeout: context deadline exceeded","pid":25001,"sql":"WITH rows AS (SELECT namespace, object_id, relation, userset_namespace, │
│  userset_object_id, userset_relation, created_xid, deleted_xid FROM relation_tuple WHERE deleted_xid < $1 LIMIT 1000)\n\t\t  DELETE FROM relation_tuple\n\t\t  WHERE (namespace, object_id, relation, userset_n │
│ amespace, userset_object_id, userset_relation, created_xid, deleted_xid) IN (SELECT namespace, object_id, relation, userset_namespace, userset_object_id, userset_relation, created_xid, deleted_xid FROM rows) │
│ ;\n\t","time":59989.653851,"time":"2023-01-05T13:53:24Z","message":"Exec"}

There is probably a lot of garbage because I was using TOUCH instead of CREATE consistently, and unaware of https://github.com/authzed/spicedb/issues/901. This has been fixed, but I would like to clean the existing garbage. Is it possible to increase this deadline through an environment variable? For context, the datastore is Postgres, in a rather beefy RDS instance of Postgres 14 (4 vCPU).

👋🏻 yeah, there is an env variable for that:

--datastore-gc-interval duration                           amount of time between passes of garbage collection (postgres driver only) (default 3m0s)
      --datastore-gc-max-operation-time duration                 maximum amount of time a garbage collection pass can operate before timing out (postgres driver only) (default 1m0s)
      --datastore-gc-window duration                             amount of time before revisions are garbage collected (default 24h0m0s)
      --

you should use

--datastore-gc-max-operation-time

Thanks! Would the equivalent env var to the

--datastore-gc-max-operation-time

command-line arg be

SPICEDB_DATASTORE_GC_MAX_OPERATION_TIME

, after the other ones? We are using environment variables instead of command-line args for configuration in our deployment.

yup! any CLI option you can turn into an env var by prepending

SPICEDB_

and turning it into snake case