Hi all, my company uses SpiceDB with a Postgres da...
# spicedb
x
Hi all, my company uses SpiceDB with a Postgres datastore alongside our Postgres app DB. We sync data from the app DB to SpiceDB periodically (on production promotions), fetching tenant data in batches and using
WriteRelationships
with
Touch
to create relationships. Previously, we inserted relationships without checking for duplicates or removing stale ones. Now, we want the sync to delete outdated relationships from SpiceDB and possibly skip inserting existing ones. Stale relationships accumulated from past entity deletions in the app DB without corresponding SpiceDB updates, and we want to ensure future potential errors in our application / app DB don’t leave stale relationships in SpiceDB. We plan to use
ReadRelationships
to fetch all relationships for an entity, compare them to the app DB, and determine which to delete or possibly even skip inserting. The sync may run multiple times daily, mostly
Touch
-ing existing relationships (>99% of the time). Should we avoid this by using
ReadRelationships
(as it will already be used for deletions) to skip existing relationships, or is frequent
Touch
calls to existing relationships fine? Checking for existence requires waiting for all
ReadRelationships
for the entity to finish before doing any
Touch
, forcing us to hold all relationships in memory per entity/tenant, which we are afraid might be too much memory (though we limit how many tenants are processed in parallel, a couple tenant's tables could be really large leading to lots of relationships potentially). Since insertions are rare, checking first might be better, but
Touch
-ing regardless is simpler and that way we think we should be able to implement only ever holding a particular batch size of relationships/entities in memory at once. Do you have thoughts on what approach might be better?
10 Views