Hi, we are currently running Spicedb on a Nomad cluster with Consul Connect, in a single instance capacity. We want to make the service redundant without Kubernetes. However, we cannot find how to do this. How can we get this working? If it's not clear how to do this on Nomad with Consul Connect, is it possible on Docker Compose? Just so we have an example outside of Kubernetes. Also, is there some kind of technical overview of how multiple instances communicatie with each other?

Hey Gert 👋, you can scale out SpiceDB just like you would with any stateless application. Just add more "replicas" (whatever concept Nomad uses for that). What you will not get out of the box, compared to running in Kube, is the horizontal scalability properties of SpiceDB which you get when having all those replicas clustered together. On top of that, I'd warn you about using a service mesh, we've had reports of performance degradation when running SpiceDB on Istio. There are various options around Nomad and Docker-Compose. You just need to find a way to hook into SpiceDB, which for now only supports: kube API, DNS or XDS. We can also consider adding native support for other protocols / APIs if it proves valuable for the community. So all that SpiceDB needs is something to feed with information about it's peers. Consul Connect does not seem to have support for XDS, but does have support for DNS: https://developer.hashicorp.com/consul/docs/services/discovery/dns-overview. I found this entry about Consul Connect XDS supports https://github.com/hashicorp/consul/issues/10339. Nomad documentation (https://developer.hashicorp.com/nomad/docs/networking/service-discovery) shows there are 2 different primitives to support service discovery. > Services are registered using the service block, with the provider parameter defining which catalog to use. Nomad stores the IP and port of all allocations associated with the service in the catalog and keeps these entries updated as you schedule new workloads or deploy new versions of your applications. re: Docker Compose, you can use

replicas

functionality from compose: https://docs.docker.com/compose/compose-file/deploy/#replicas. For service discovery you could use their built-in DNS nameserver: https://docs.docker.com/network/#dns-services Here is technical overview on how instance talk to each other, we refer to it as "dispatching": https://authzed.com/docs/spicedb/concepts/dispatch

Thanks! So you are able to run multiple instances for redundancy purposes, but i assume updates of a permission in one of these instances will then not be propagated to other instances. So an update will result in inconsistent data. Is this assumption correct?

no, this is not correct. SpiceDB is a stateless service, it stores it's state into a database (currently supported: Postgres, MySQL, Spanner and CockroachDB). So writes will go to the same database and be observed by other instances.