https://authzed.com logo
Join Discord
Powered by
Proposal: Query Planner · Issue #1573 · ...
# spicedb
a
Any general performance advice for very popular objects? Is this just handled well? e.g. I'm worried about a schema design where an object has potentially millions of user relationships to it, and having to scan all those for checks or lookup subjects. I'm assuming based on the discussion in [this issue](https://github.com/authzed/spicedb/issues/1573) that it doesn't perform well.
The only "trick" I have thought of is basically encoding the schema in reverse. Maybe this is a bit of a hack but it could work? e.g. rather than checking if an object has a "getter" relation to a user, check if a user has a sort of "supplier" relation to the object. 😅
y
can you give a more concrete example?
on a related note, at my previous company we started implementing admin functionality by having a 
platform
and then admins that could be connected to that singleton
but because that singleton was connected to all of the objects in our system, when you called 
LookupResources
you'd get a massive list and that had a tendency to knock us over
a
ah yeah we are doing that pattern too. good point.
y
so we instead made it something where admin users would have their permissions checked on the singleton for admin interfaces, but if they wanted to interact with an object as a user, they had to give themselves the relevant permissions on that object
an elevation workflow rather than "admins can always see everything"
it happened to be something that wasn't difficult to make a business case for
but i'd be curious to hear more about your problem
a
basically for this case there is a file that millions of users can download, but not all users. and in some cases there are files with an attribute which may be more restricted. e.g. there is a "download any file" kind of role, vs a "download safe files" kind of role (imagine the files have an attribute "safe" vs "not safe"). so i'd like the system to be able to integrate with just check against the file id, download permission, and the user id. but i'm worried about the cardinality.
(i would model "safe or not" using something like a not_safe relation to a user:*)
y
wide relations can definitely be a problem
negation can be helpful if the list of users who shouldn't be allowed to do something is much smaller than the list of users who should
lookupSubjects is just always going to be a problem here
a
oh yeah interesting
y
bulkCheck can help work around the lookupSubjects issue
a
fortunately the main use case is check here. getting a million users back isnt useful anyway.
y
yeah many-to-one relations aren't a problem for checks
but one-to-many can be
a
ack ty for confirming
y
afaik 🤔
a
any experience with "reversing" the traversal in the schema? it seems maybe like a hack but, the glory of not having a built in user type allows us to model things more creatively 🤷‍♂️
so i'd write relations from the user to the files (very few), instead of the files to the users (very many)
y
checks are a walk from the object to the subject
so i don't think there's going to be a way to reverse it and retain check semantics
a
What I mean is reversing the schema. So instead of a relation from the file to users, add a relation on the user to the files. Then check using the user as the resource. It makes it lower cardinality at the cost of stretching the domain model. The object (e.g. the file) has a "supplier" kind of relationship to the user instead of the typical user having a consumer/role kind of relationship to the object. On my phone otherwise I'd give a concrete schema to be clearer
y
ohh gotcha. and then the "subject" is the file and the "resource" is the user
hmm... if it's something i could prototype easily i'd try it and see what the differences were
but i'd also be hesitant because that sounds like the kind of optimization that could bite me later
(i know i'm supposed to be speaking as an expert here but i haven't considered this approach before and i don't feel like i know the ins and outs of the implementation well enough to speak directly to it)
one concrete issue right now is that lookupSubjects doesn't support cursoring
we're working on enabling that but it's not there yet
a
ack
y
or concrete difference, anyway
a
If we had the query planner feature that knew which direction to traverse on its own, we could avoid coupling the schema to that more "operational" concern. Changing the schema "back" to the "right" direction later would be breaking, so it would be nice to get all the tuples and queries the way you really want to begin with.
In other words it would be nice to describe the schema more naturally, and let which direction to traverse be an optimization detail that didn't affect tuples & schema
I wonder if a simple/low-cost approach to flipping the query strategy could be to annotate the schema in some way for this?
Copy code
definition file {
    // hint:reverse
    relation downloader: user
    permission download = downloader
}
j
we plan to do it
its on the roadmap and reversing wide checks into smaller lookups is like the second use case (after super basic type-based optimizations as the first)
but its not yet scheduled
and yes, we could in theory use a hint model to help before we have formal stats
it should also be noted that if you have a million direct subjects, its already fast
i.e. 
relation member: user
and you're looking for a specific user
we don't load all the users in that case; instead, the system looks for the one matching user
the problem comes in when you have indirect, like 
relation viewer: group#member
and a million groups
now we have to load all the groups to walk to find the subject
an LR here would likely be faster, hence the idea of reversing
a
Ty!
14 Views
PreviousNext
Powered by