https://authzed.com logo
It s not really a SpiceDB question but
w

williamdclt

05/25/2023, 9:56 AM
It's not really a SpiceDB question, but it's related and you might have ideas/experience 😄 We have an offline-first application. It loads a bunch of resource, and the user has various permissions on each of these informing the UI (we won't show the "delete" button if you don't have the permission to edit the thing). Problem: that's a lot of permission. A few hundreds resources, ~20 permissions, that's thousands of permissions that needs to be loaded in the app: that means a huge load on spicedb if we need to do a check for each of these. It would only be ~20 lookupResources, but it's not working perf-wise (~30s per lookup). I'm not expecting much, but just in case: is there an obvious approach I'm missing?
v

vroldanbet

05/25/2023, 10:51 AM
even with the recently merged cursored LookupResources, you are bound to whatever order in which your app shows the elements. If you use the stable order coming from SpiceDB, you could fetch the first 100 elements for
permissionA
, but it would be a different first 100 elements for
permissionB
, leading to a disjoint set, forcing you to ask for the next 100 elements until you find the same elements (or you've reached the conclusion that is not in the set based on the order of elements). I think this use-case is better served by asynchronously loading all those UI elements after
PermissionCheck
and the power of HTTP2 pipelining. Whatever huge load you believe that its, it would still be more efficient than LookupResources in its current incarnation (even with cursors). If the permissions share common branches, those would be deduplicated by SpiceDB's hot spot caching. We've scaled SpiceDB to billions of tuples and a million requests per second, so I have difficulties to understand why this wouldn't scale merely based on checks.
w

williamdclt

05/25/2023, 12:02 PM
> We've scaled SpiceDB to billions of tuples and a million requests per second, so I have difficulties to understand why this wouldn't scale merely based on checks. I'm not expressing myself well: I have no doubt I can make it work. I'd just rather avoid burning money on scaling the database and the spicedb cluster if there's a better way 😄 > I think this use-case is better served by asynchronously loading all those UI elements after PermissionCheck I have to get all these permission checks ahead-of-time because this app is offline-first: I need to know what are the permissions of the current user to know what to show whent they're offline. Doesn't sound like I'm missing anything obvious to make my problem go away, fair!
v

vroldanbet

05/25/2023, 12:09 PM
>I'm not expressing myself well: I have no doubt I can make it work. I'd just rather avoid burning money on scaling the database and the spicedb cluster if there's a better way 😄 Depending on your schema, a baseline cluster for say 1K RPS differs. That's key to Zanzibar: throw compute to solve complex authorization problems. >I have to get all these permission checks ahead-of-time because this app is offline-first: I need to know what are the permissions of the current user to know what to show whent they're offline. That's fine, firing off
PermissionChecks
concurrently even if it's to collect them before going is still a good strategy worth exploring. Did you have the chance to test the cursored LR?
w

williamdclt

05/25/2023, 12:17 PM
> That's fine, firing off PermissionChecks concurrently even if it's to collect them before going is still a good strategy worth exploring I have that ready to test this afternoon indeed! > Did you have the chance to test the cursored LR? I did not, not yet
v

vroldanbet

05/25/2023, 12:22 PM
I'd also suggest you look at changes we recently did that have very positive impact to throughput: https://github.com/authzed/spicedb/pull/1285 https://github.com/authzed/spicedb/pull/1310/ The first one helps a ton when there is DB contention around the time the quantization window elapses. It helps spread out the potential thundering herd of requests to database. It's by default set to 10%, I'd suggest playing around with higher values.