the behavior is: * on startup, spicedb checks to see if the head migration matches the migration it wants. if it doesn't match, it throws an error. * it doesn't check while operating, so an existing SpiceDB instance will continue to work if a new instance performs a migration, provided the database shape expected by the existing instance is compatible

many minor versions contain no migrations so they're compatible by default

Is there a way to make sure migrations wait if another instance is already running them? I hit a race condition with 1.4 where different instances started fighting over the new index.

That is, one started creating the index. Another came online and deleted it to level set before creating. Chaos and crashlooping ensured.

the issue you're describing is something that we've seen recently and have been working to address

i don't remember where we fixed it, though

let me get back to you

My current workflow is to run migrations and then call "serve" on each pod that comes up.

but i'm asking about what we did for the multiple migrations problem

are you running them as an initcontainer?

what environment is this in?

But with an organization custom set of tooling around it. It may be possible. It may not.

I'll have to check with the teams running that stuff.