Ended up with (in case anyone wants to do this in the future):

healthcheck:
      test:
        ["CMD", "/usr/local/bin/grpc_health_probe", "-addr", "spicedb:50051"]
      interval: 30s
      timeout: 3s
      retries: 5

The fact that grpc_health_probe is baked into the spicedb container image (https://github.com/authzed/spicedb/blob/main/Dockerfile) is 👌