yup, i recalled setting that up

Yep, there's no open source way to protect admin/write/read like authzed.com has right now. We're looking to gather more requirements for open source tenancy though, since we've just been trying to scratch our own itch with what we've built privately.

We'd appreciate input on the github issue that I linked though, as it'll help us understand where users would like to set boundaries. The admin/write/read for Authzed.com will not handle all use cases.

hey michael

👋

Hello 👋 I have a quick question about usage and how Zanzibar might or might not be aimed at solving it. Our current permissions implementation is done through a view querying across a few tables and doing a few switch cases to determine whether something is reachable. We are reaching a limitation on the speed to answer our calls when we go from "do I have permission X on Y" to "list me all entities on which I have permission X, ordered by criteria". I am wondering if Zanzibar could answer something like this. My understanding is it wouldn't really be used in that way, but I might have missed something so here I am 🙂

Zanzibar (and SpiceDB) has an API call Expand which resolves all of the entities with particular permission/relation. You'd then take that output and feed it into the SQL query -- a process we call "pre-filtering" because you are attempting to filter results before you ever get them from your database

when the number of entities becomes larger than you can fit as a literal in a SQL query, that's when you'd start exploring some of our proposals that are not yet implemented for scaling this operation https://github.com/authzed/spicedb/issues/207

That means possibly loading thousands of values to feed in our database though, correct? Because zanzibar has no idea about the ordering criteria I want to apply I'd need to load everything (possibly big both in number of values and therefore in size) and then feed it into a database to apply ordering. Is that the proposal?

Haha, the first section of this blog post perfectly summarizes my last few weeks https://authzed.com/blog/acl-filtering-in-authzed/

Yep, but thousands can often fit in a query. It's probably when you get to tens of thousands that the query is too large to be sent over the wire. The proposal is actually for generating datastructures like bitmaps or bloom filters that can be used instead when you do have too many results for a pre-filter query.

We're exploring the space a bunch, so we'd love to hear about what would make/break the solution for you. For example, you can do a lot with custom postgres extensions, but they wouldn't be available in hosted database platforms.

Yes, I figured it would have to do with something like that indeed. At this stage though for a set of 30k values I'm seeing the same performances doing the query with the hard coded list of ids than I would with the normal SQL query so I'm not seeing the value of switching just yet. I guess there is a threshold at which it becomes very clear that it is worth making the switch.

I'd be happy to discuss our use case and possible implementations if that might help 🙂

Describing it in that github issue would be most appreciated

so that we don't lose it in the discussion in discord

Sadly we're using a hosted database (gcloud postgresql) so extensions are not an option at the moment 😦

yep! that's why i asked haha

Will do, it might benefit others as well

thanks for that

Yeah makes sense 🙂 I've been trying to turn this issue around in so many ways but am not able to find the right solution. I can find solutions that will work for now, but am not exactly super certain how they will scale and that saddens me a lot 😅

there are some interesting things out there, for example, RDS on AWS allows foreign data wrappers of other things that implement that postgres wire protocol -- which would be a large undertaking, but would potentially give us portability to present a table to postgres

The blog post (https://authzed.com/blog/acl-filtering-in-authzed/) seems to be very close to answering what we want to do, until we introduce the

order

part 😅

is ordering not something you can do after the fact?

Yes, it's just bringing me back to the performance hit I am seeing at 30k entries already in which querying for order given a hard coded list of ids will bring me the exact same performance as just querying my database through my view directly

So at the moment I'm not seeing a value in adding a new complex system into our stack since I'm at "scale" (whatever that means for us at the moment and the foreseeable future) not seeing benefits

(Thanks for pointing me to those issues/blogposts, it gives me great insights into that field where I'm lacking some knowledge)

Indeed for now, we're still coming up with a good strategy for solving this problem in a generic fashion -- knowing you have requirements like ordering will help us a lot with our final design. There are other benefits to introducing SpiceDB, but I understand you keeping your eyes on the problem at hand.