Building a SpiceDB Datastore
# spicedb
g
is there any documentation or material around creating datastore and database access patterns ?
v
There is no documentation on how to build a SpiceDB Datastore. The Datastore API definition (https://github.com/authzed/spicedb/blob/main/pkg/datastore/datastore.go#L331-L385) is the right place to start. SpiceDB Datastore must comply with the following traits in order to support the SpiceDB API, these are the most important aspects to bear in mind while building the system: - supports snapshot reads. This allows SpiceDB asking for the state of the system at a specific time. - support at least intra-region external-consistent transactions (roughly equivalent to https://jepsen.io/consistency/models/strict-serializable). If it does not support cross-region external-consistent transactions, this must be properly documented, as SpiceDB follows Zanzibar architecture and is meant to provide external-consistency transactions across the globe. Right now Spanner and CockroachDB datastores support this. As far as I can tell from the docs, DynamoDB does not support external-consistent transactions cross-region: > Strongly Consistent Reads > Read operations such as GetItem, Query, and Scan provide an optional ConsistentRead parameter. If you set ConsistentRead to true, DynamoDB returns a response with the most up-to-date data, reflecting the updates from all prior write operations that were successful. Strongly consistent reads are only supported on tables and local secondary indexes. Strongly consistent reads from a global secondary index or a DynamoDB stream are not supported. > Global tables read consistency > DynamoDB also supports global tables for multi-active and multi-Region replication. A global table is composed of multiple replica tables in different AWS Regions. Any change made to any item in any replica table is replicated to all the other replicas within the same global table, typically within a second, and are eventually consistent. For more information, see Consistency and conflict resolution.
If DynamoDB provides a means to wait for replication to happen then it would be possible to also support Global Tables. For example MySQL offers GTIDs and I believe there are means to wait for a GTID to be replicated to a replica. A similar strategy may be possible for Postgres. We have plans in the roadmap to support replicas in Postgres/MySQL.
SpiceDB has a test-suite for datastores. The goal would be to have the DynamoDB datastore pass that test suite. As far as I can tell AWS provides a DynamoDB emulator (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.html) so it should be possible to spin new instances locally for the test suite.
If no snapshot reads are supported (I couldn't tell after a cursory look) you'll need to implement your own MVCC like the Postgres / MySQL datastore do
g
this is really helpful thank you
47 Views