Hey guys. I'd like to know what options I have to deal with the consistency problem. I know it's likely the most common question, but still. I'm using stream commits to relationships in SpiceDB, so any changes that happen in my database are streamed to a special service which creates respective relationships in SpiceDB. Let's say frontend makes POST request to create record A, and by design it immediately redirects users on the page to see information about record A and pulls information about records B and C, which depend on record A. In the current design, subsequent immediate requests to get records B and C, after record A is created, fail since there is a gap when record A streamed to SpiceDB, so read after write requests fall with 403 status code since relationship for record A not exists in SpiceDB and SpiceDB returns false on check permission request. Since relationships in SpiceDB are eventually consistent, we can't rely fully on some kind of delay or polling. We are considering following options: - Create relationship directly in the transaction within creating record A. 2PC approach, but I'm worried about responsibility here, cuz relay service and backend will write almost the same things - Changing UI. But all other clients must be aligned within it which adds costs on maintaining and may confuse in the future - Websocket events, but cost of implementation is high here. Not all clients may support websockets events and its adds complexity to backend and frontend I'm wonder are there any common recommendations how to deal with cases when strong consistency is required? Also, I'm more interested about it in terms of how Google most likely solved this issue? Let's say in Google Drive, you create a folder, and then you want immediately to see what's inside this folder. Currently, Google Drive UI doesn't redirect users immediately to created folder, but I believe Google won't rely on it that someone will be slow enough to let relationships be created, right?

What you are describing is "read your writes". This is addressed with zedtokens. But your are streaming changes to SpiceDB async. The way to address this is to have your application consume events that acknowledge the write happened in SpiceDB, so you'll "block" the redirect until the write to SpiceDB took place, and the zedtoken was emitted via an event your application waits for. Google solves this easily: they use the same underlying database for application and Zanzibar, so it all happens in the same transaction. 2PC is subject to inconsistencies, because it's two different transactional boundaries, and it will almost certainly lead to incosistent data. Another option we've been using recently is durable workflows. We've

go-workflows

in our project https://github.com/authzed/spicedb-kubeapi-proxy, but there exist many alternatives for other languages. I guess the most popular service in this space would be https://temporal.io/. I like

go-workflows

because you don't have to rely on a third-party service, but you use your own database to keep track of the state of the workflow.

Thanks a lot for such a comprehensive answer! Our main language is JavaScript, so build backend on NodeJS and NestJS. Regarding durable workflows, at first glance it's implementation of an orchrestration saga, right? So, the idea behind durable workflows, run distrubited transaction, wait when all services completes their local transaction and only after this return success response to client on the POST request, right? Regarding streaming approach, where exactly we should block "redirect". Do you think it's something that should be implemented on frontend side? I mean, backend must return success response immediately, but frontend should wait before relay service emits success creation of relation, right?

yeah, there's various implementations of durable workflows that have different semantics. the most useful variety for this case will return a value when the workflow completes, which lets you pretend that the distributed transaction is happening synchronously, so it could happen inside of the request flow. for a streaming approach you're entering the territory of needing a websocket or something equivalent to recieve and then handle a completion message, which would be handled by the frontend.

> Regarding durable workflows, at first glance it's implementation of an orchrestration saga, right? So, the idea behind durable workflows, run distrubited transaction, wait when all services completes their local transaction and only after this return success response to client on the POST request, right? It's similar to SAGA, but confined to a single service that executes multiples transactions. The idea is that at each step of the distributed transaction you are recording it's state. E.g. - Write row to main DB - Record in main DB that row was written - Write to SpiceDB - Record that SpiceDB write was completed There is some inversion of control, where the framework handles calling the different steps, and handles the recording of state for you. And if things fail, it takes care of retrying. Of course, it forces you to thinking about partial failures, recovery / compensation, and idempotency.

I see. So, basically watch API must be called before SpiceDB emits an event. Also, thanks for sharing idea behind durable workflow. So far, I think I got enough to evaluate what would be better in my case. Thanks a lot!