I've started working on a statefulset based Helm chart for spicedb. You can find it here: https://github.com/jonwhitty/helm-charts/tree/master/charts/spicedb The goal is to make it more fully featured. Right now it just supports an absolutely minimum viable Helm chart to install a multi-node spicedb cluster. It's been used by our team already to manage a multi-node spicedb cluster with dispatching. More work is coming to wire up the TLS settings and everything, but hopefully it helps some of ya'll out there in the community already 👍 Feel free to contribute as well :).

what specifically is broken? that issue is about some unwanted log lines, I don't think it should affect any behavior

🤦‍♂️ For some reason I took those logs as error outputs and mistakenly thought the request failed. I'm sorry about that. False alarm 🤣

whew! no worries

@User we're also considering running a cluster, if I may ask, what's the reason for the statefulness? As far as I can see a simple deployment would also do, right?

Yeah, a deployment should work fine as well because underneath the cover kuberesolver is used to establish the cluster membership. I'm just used to these types of "clustered" deployments being a statefulset, and so that's the only reason. If there are any dependencies on node1 coming online before node2, node3, etc.. then it'll be nice as a statefulset. But yeah, it could be a deployment. What do you think? Should I just change it to a deployment?

Yes, changing it to a deployment would probably be a good idea 🙂 Not sure if there are any spicedb internals that would profit from statefulness though. (i.e. cache on disk?)

What's your reasoning though? I don't really see much value over a deployment vs a statefulset for a spicedb rollout. In fact, a statefulset gives us more options in the event that a gradual rollout or cache on disk etc.. become a dependency.

(I'm not from authzed, just another engineer interested in the same) My thoughts: - it's easier to argue about the state in general since everything is ephemeral - it's straight forward to understand and deploy, there will be no questions around if/what of the state might need a backup - a stateful-set with only 1 replica can not do zero-downtime deployments (arguably not that important if you anyways run a cluster)

for the record, we run using a deployment as well

as of right now, SpiceDB nodes are not stateful outside of restarts

@User is correct that may change sometime down the road (although we have no plans currently to do so)

👍 cool. Good to know. I'm happy to switch it over to a deployment. I just thought there was a good chance it might go stateful because of any codependencies on startup etc.. Most cloud native clustered deployments use statefulsets, so that's the rationale I had. We can always change it if the topology for spicedb changes to more of a stateful paradigm. At that point it'd be a pretty large change warranting a new major Helm chart anyways. I'll switch it over to a deployment 🙂

Thanks for the background as well Joey.

yeah, the goal until now has been to keep SpiceDB more as cattle, rather than pets

(to borrow the terminology)

😆

@User updated. It's now a deployment.

the value is still called

statefulset

😉 https://github.com/jonwhitty/helm-charts/blob/master/charts/spicedb/values.yaml#L56

Good catch. My bad.

Setting up a Helm registry through GitHub pages right now as well. So soon you'll be able to

helm repo add jonwhitty ...
helm repo update
helm upgrade --install ... spicedb

nice!

Now the spicedb chart can be installed through the GitHub pages Helm registry hosted through that repository. For more instructions see: https://jonwhitty.github.io/helm-charts/charts/spicedb/ We can iterate on that chart as a community and fine tune it as needs be. I welcome contributions.

Hey all! A question reg. helm charts/clustering: As far as I understand instances may dispatch to other instances in the cluster. Now with a Kubernetes deployment using the service as the

dispatch-upstream-addr

there's a good chance that the instance will call itself. Is there any easy way to prevent that?

why would you want to prevent that?

Isn't the purpose of the

dispatch-upstream-addr

to call another instance in the hopes that it has it cached? Maybe I just don't understand what the dispatching is for 🙂

that's totally what it's for, but it's a sub-problem, so it's possible that this instance is the one with the sub-problem cached

ah, I see. But it then fully relies on the Kubernetes service to load balance that usefully right?

this takes the load balancing out of kubernetes's hands, it does rely on kubernetes networking to give us a route back to the current pod when that's the one selected

makes sense, thanks for the clarifications!