Google-Scale Authorization: Getting to 1...
# spicedb
x
hi, spicedb blog shared a dataset generator in Go that took the number of users, pieces of content, and interactions as input, created relationships using a Pareto distribution, and outputted an Apache Avro file with the relationships represented in SQL. how can i access this test dataset? https://authzed.com/blog/google-scale-authorization#using-pareto-distribution-to-shape-the-graph
v
This is work that was done for a customer and unfortunately cannot be shared as is. We could sanitize the script with some unrelated schema, if it helped, but I'm not sure I'm going to have the bandwidth. Perhaps we could do that as part of the effort to opensource our internal load-testing tool, thumper?
x
I eagerly anticipate the open-sourcing of these two tools, which would facilitate a more effective assessment of whether the deployed SpiceDB meets user requirements.
v
I'd say perhaps the best place to start is to start modeling your business domain in SpiceDB schema language. From there you can probably use k6, we've used it in the past. And regarding data generation, it's a simple script, I'll see if I can clean it up a little bit. Are you expecting to run a multi-region SpiceDB? I ask this to understand which database to recommend.
x
Indeed, our ultimate goal is to deploy a multi-region database to enhance data reliability. However, the initial deployment might be limited to a single region, with plans to migrate to a multi-region setup subsequently.
v
But does processes from one region access to data in another region? If the access is yes then you need a database like CockroachDB / Spanner. If not, you can probably get away with a SpiceDB cluster per region with its own DB.
x
For a scenario involving 15 million relationships and a throughput of 15,000 qps what would be the recommended configuration for SpiceDB?
v
The answer, as usual, is "it depends". Depends on how complex your schema is. But those are not big numbers and I'd say a postgres would work here. But more important is if you want to have a single state of the world for all regions, or you are fine with multiple SpiceDB's each one with its own database, because then you want to look into databases that are designed for global distribution like Cockroach/Spanner in case you need the former
x
thanks a lot
4 Views