SpiceDB is an open source, Google Zanzibar-inspired, database system for creating and managing security-critical application permissions.

SpiceDB

Hi all. We are using the SpiceDB operator for our deployment. We're also using an nginx ingress infront of the spiceDB cluster. What we are observing is if a pod is shutdown and a new pod spun up to take its place, for a few minutes after, we are seeing errors like 

`rpc error: code = DeadlineExceeded desc = received context error while waiting for new LB policy update: context deadline exceeded`

We have the gRPC shutdown grace period set to 5s currently.

so my guess is that when the upstream becomes unavailable, the http connection is still open, and you're having to wait until the gRPC connection ages out (which is a different configuration value than the shutdown grace period)

we use envoy for ingress in our own infra for this reason

I believe nginx works with gRPC. We have the following config:
```
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
```

it saying that it supports gRPC doesn't mean that it handles it well - we've seen the same problem with folks using AWS ALBs, which also claim to support gRPC