Hello! I am currently working on modeling some hypothetical scenarios, and I wanted to ask about proper datastore version handling. Do you have any recommendations for how to perform downgrades in the unlikely event that a version containing a datastore migration needs to be rolled back? I did not see any information on this in the datastore migration documentation: https://authzed.com/docs/spicedb/concepts/datastore-migrations It looks like the migration changes I have seen are backwards compatible in regards to neighboring versions, so maybe this isn't an issue. I'm sure the chances of this happening are extremely unlikely, but I would appreciate your insights or recommendations for this type of scenario if you have any. Thank you

Rolling back datastore migrations is not something that has first-class support in the migration framework, so you are on your own. Migrations are typically backward compatible, but some are not. For example version 1.14 introduced a 2 phase migration for postgres datastore which makes rolling back not possible. There have been other 2 phase migrations recently in some of the datastores.

When I've needed to rollback (hasn't happened since 1.14), I needed to check that the migration was backward-compatible and manually update the migration table (

alembic_version

IIRC), otherwise the pods on the old version won't start due to unknown migration

That's right, that's what I tried to refer to "you are on your own" - you need to manually tinker with the migration system. But other than manually updating the stored migration version, as William said, you should be able to rollback so long the previous migration is backward compatible. You can have a look at the GitHub release pages to identify the versions that have a 2-phase migrations.