Re: this issue I'm still hitting this with v1.11.0. I've dug a little bit: - The

grpc_health_probe

is returning

NOT_SERVING

(screenshot 1) - On SpiceDB side, I'm seeing this log:

{"level":"debug","datastoreReady":false,"dispatchReady":true,"time":"2022-08-08T12:20:53Z","message":"completed dispatcher and datastore readiness checks"}

. That's with LOG_LEVEL=trace. My old service on

v1.7.1

has no problem: pods report healthy and can connect to the DB. Spinning up a new pod on

v.1.7.1

works as well. I can't see anything wrong database-side. How can we know why does SpiceDB think that the datastore isn't ready?

The datastore will report

ready=false

if the migration level it's talking to doesn't equal the latest migration - when you upgraded, did you run

spicedb migrate

?

I was looking at that indeed, my migration hook is a post-upgrade so that makes sense. What's the recommended way to run migrations? Is it a

pre-upgrade

hook?

Yeah, run

spicedb migrate head

from the version you're updating to before rolling out the new version

I can do that quickly

BTW, we are planning to open source a kube operator that (if you use it) will mean you won't have to worry about migrations, since it will handle all of that

The improved migrations system will also make this more apparent