I tried updating the SpiceDB cluster SpiceDB #spicedb

I tried updating the SpiceDB cluster

theconductor

05/30/2025, 3:30 PM

I tried updating the SpiceDB cluster port using the

grpc-addr

flag, but now the health check is failing. Is there anywhere else I have to update the port? (Error in 🧵)

Copy code

apiVersion: authzed.com/v1alpha1
kind: SpiceDBCluster
metadata:
  name: spicedb-cluster
  namespace: spicedb
spec:
  config:
    image: authzed/spicedb:v1.37.1
    grpcAddr: ':9090'

theconductor

05/30/2025, 3:30 PM

Copy code

establishing connection
timeout: failed to connect service "localhost:50051" within 1s
12m (x2 over 14m)      Normal    Created             Pod/spicedb-cluster-spicedb-7b6bfd8f95-zvwvf    Created container spicedb
12m (x2 over 14m)      Normal    Pulled              Pod/spicedb-cluster-spicedb-7b6bfd8f95-zvwvf    Container image "authzed/spicedb:v1.37.1" already present on machine
4m31s (x77 over 14m)   Warning   Unhealthy           Pod/spicedb-cluster-spicedb-7b6bfd8f95-zvwvf    Readiness probe failed: parsed options:
> addr=localhost:50051 conn_timeout=1s rpc_timeout=1s
> tls=false
> alts=false
> spiffe=false

Joey

05/30/2025, 3:34 PM

why are you changing the port internally?

theconductor

05/30/2025, 3:35 PM

A particular customer's security requirements restricts which ports we can use for gRPC

Joey

05/30/2025, 3:35 PM

you should be able to just change the port mapping?

Joey

05/30/2025, 3:36 PM

here, I believe you're changing the port being used in the container

theconductor

05/30/2025, 3:54 PM

Oh okay, that makes sense. Since the SpiceDBCluster manages the Service, would I do that through a patch? This wasn't working for me:

Copy code

apiVersion: authzed.com/v1alpha1
kind: SpiceDBCluster
metadata:
  name: spicedb-cluster
  namespace: spicedb
spec:
  config:
    image: authzed/spicedb:v1.37.1
  patches:
    - kind: Service
      patch:
        spec:
          ports:
            - name: grpc
              port: 9090
              protocol: TCP
              targetPort: 50051

theconductor

05/30/2025, 3:55 PM

I got readiness probe errors from the operator itself:

Readiness probe failed: Get "http://10.244.1.134:8080/healthz": dial tcp 10.244.1.134:8080: connect: connection refused

ecordell

05/30/2025, 5:21 PM

haven't seen that before - can I see the operator logs?

ecordell

05/30/2025, 5:23 PM

you're probably running into an issue because 9090 is already defined

ecordell

05/30/2025, 5:23 PM

Copy code

- name: grpc
    port: 50051
    protocol: TCP
    targetPort: 50051
  - name: gateway
    port: 8443
    protocol: TCP
    targetPort: 8443
  - name: metrics
    port: 9090
    protocol: TCP
    targetPort: 9090
  - name: dispatch
    port: 50053
    protocol: TCP
    targetPort: 50053

ecordell

05/30/2025, 5:23 PM

these are the ports on the service

ecordell

05/30/2025, 5:23 PM

also - the dispatch API is also grpc and needs to be on a different port

ecordell

05/30/2025, 5:23 PM

which may conflict with those port requirements

theconductor

05/30/2025, 5:28 PM

Good call, I looked at the events, not the logs. Here's the log error. I guess I duplicated the grpc port instead of replacing it?

Copy code

E0530 17:27:48.949494       1 handler.go:31] "requeueing after error" err="Service \"spicedb-cluster\" is invalid: spec.ports[1].name: Duplicate value: \"grpc\"" syncID="M/G5O" controller="spicedbclusters" obj={"name":"spicedb-cluster","namespace":"spicedb"}

theconductor

05/30/2025, 5:29 PM

And thanks for pointing that out. What is the
dispatch
API for? nvm, I see the docs for it, definitely an important endpoint: https://authzed.com/docs/spicedb/concepts/dispatch

theconductor

05/30/2025, 6:02 PM

I'll add that I switched to 9091 just to test before sending that log ^^

29 Views

Previous Next