nope; haven't had time to give it more thought today

Ok

are you using

TOUCH

or

CREATE

?

okay; there is an improvement that should land sometime next month that makes TOUCH significantly faster on PG

is this something I can pull in directly without having to wait for a release? (just for the sake of my migration, there won't be nearly as many writerelationship calls after this, especially not ones that overlap)

there isn't a PR for it yet

ah damn. okay. sequential migration (processing 1 resource at a time) takes ~10hours for me and seems to be the only way I don't get errors right now unfortunately 🥲

that doesn't seem right

wait actually since i've run the script multiple times, re-adding would be happening via

TOUCH

.

ah

so you might need to run GC on the system then

ahhh. can i force that?

start up SpiceDB against it and let it run the GC pass (should start fairly quickly)

but that's a good point

we should probably just have a command for it

what would the impact be? from my understanding, the GC would get rid of tombstone rows (rows that are deleted as the result of a TOUCH operation). this would have a significant impact in future TOUCH actions?

yes

because the index would be smaller

not to mention it would lower the transactions table size

assuming there are any transactions older than 24 hours

ahhh i see.

there wouldn't be anything older than 24hrs yet. i started doing this around 18 hours ago

so then only the tombstones then

gotcha. i could run one spicedb pod with a zero gc interval and it should invoke immediately?

yeah

and you're inserting via only one write call at a time, correct?

now, in the sequential runs, yes. one write call containing multiple relations

that should still work without hitting serialization issues; let's hope the GC helps there

sequential runs are working fine, the problem is the time it takes - i have around 1.9m resources for which I need to write relations. if i'm doing them one at a time, it takes ~10h (+/- 1h depending on the rps)