Hey all, gm -- quick question: is there
# spicedbb
Ben
09/27/2024, 2:18 PMHey all, gm -- quick question: is there a way to run
zed backup create ...v
vroldanbet
09/27/2024, 2:29 PMunfortunately zed backup create is not supported in Authzed Serverless because it does not support bulk export API
vroldanbet
09/27/2024, 2:29 PMThe alternative would be to use
ReadRelationshipsReadSchemab
Ben
09/27/2024, 2:31 PMGood deal -- I figured I'd have to roll my own 🤙 We have a hydrate-on-read capability in our platform api, but our internal users aren't always great about signing in and hitting the needed endpoints during hydration windows 😂
Ben
09/27/2024, 2:31 PMThanks @vroldanbet!
v
vroldanbet
09/27/2024, 2:33 PMcould you expand more on what hydrate on read means here? trying to see if I can help in some other way
j
Joey
09/27/2024, 2:34 PM@Ben if you contact us, we can likely get you a backup file
b
Ben
09/27/2024, 2:34 PM@Joey - right on! Should I email or dm or something else?
Ben
09/27/2024, 2:36 PM@vroldanbet - all of our resources and subjects have metadata stored in our KVaaS metadata store, so I wrote some in-app functionality a while back to write relationships to SpiceDB whenever APIs that write or get from the metadata store fire off. But, it requires those APIs to be hit and, ofc, it makes it slow the first time they're hit during a hydration window
Ben
09/27/2024, 2:36 PMwe dog food our own metadata store and use it as a "has this been hydrated" cache too, but that first time can be slow going lol
v
vroldanbet
09/27/2024, 2:39 PMso what you are saying is that there is a delay between things getting synced to the kv and then getting synced to SpiceDB?
vroldanbet
09/27/2024, 2:40 PMif KVaaS is written by API, do you synchronously write to SpiceDB?
b
Ben
09/27/2024, 2:43 PM> there is a delay between things getting synced to the kv and then getting synced to SpiceDB
Not really, just a delay for page loads when we're writing everything we read from the KV store to SpiceDB
Otherwise, to your point, we double write to the KV store and SpiceDB during normal operations
Ben
09/27/2024, 2:45 PMThe nice thing is we're technically still pre-launch so we only have a few dozen users and 99% of the relationships we have are for the 10s of thousands of data assets the system maintains for them -- I think I'll be able to knock out a custom export script in short order
Ben
09/27/2024, 2:47 PM(sorry, I may not be explaining it well 😬 a lot going on for a Friday morning)
v
vroldanbet
09/27/2024, 4:40 PMso without the Watch API supported in serverless, it's hard to do a migration without manually reconciling some of the data. Consider that you will take a snapshot of the system (you read schema and all relations at the same revision). You can move that to your internal SpiceDB, but you need to also write new items after you took the snapshot.
You could do dual write to both SpiceDBs, but the moment you restore the snapshot, you could be recreating a relationship that was already deleted.
vroldanbet
09/27/2024, 4:41 PMalso if you have caveats, you could be recreating a relationship with a different caveat context
b
Ben
09/27/2024, 5:01 PM> You can move that to your internal SpiceDB, but you need to also write new items after you took the snapshot
Yep, that's pretty much the plan at this point 💯 I'm going to run a background migration shortly to start hydrating all data at-present (and get a timing idea), then we'll either have a DEV outage while we deploy with new target details and run the migration 1 more time or we'll allow our hydrate-on-call process to take over w/o any more outage than app deployments
Ben
09/27/2024, 5:01 PMthanks as always for all of the help 🙌
v
vroldanbet
09/27/2024, 5:32 PManytime!
7 Views