I'm not sure how much data exactly i'm loading into the test server. It might be a lot. I'm working on a not so small project that has lots of tests, and i've enabled syncing of some resources into spicedb. so almost all test case will use spicedb.

I'd first try to see if it is due to a single test or all of them running at the same time; since the tester is in-memory, it is not super optimized and each test only gets a few workers to run with

@Joey question i have there - does it tear down the in-memory relations when a given connection with a given secret key disconnects, or do they live for the lifetime of the instance?

lifetime

@Joey it doesn't seem to be related to a specific test. I'm having this issue even when running tests with one worker. I'll probably opt to disable spicedb sync on all tests except for tests that really need it. But will have to handle the issue in the future when we will totally depend on spicedb...

"disable spicedb sync"

we are using sequelize ORM, so i've added some hooks on create, update and delete operations for some selected tables.

are you issuing a bunch of writes ops or?

got it

we've considered adding a way to make it GC

yeah the only thing i can think is ref counting based on connections but that's kind of an opaque behavior

well, and it wouldn't meet the current goals of allowing additional tests to just "come along" and use it

yeah in our usage every test is getting its own connection

@Idan let me know if you're doing parallelized writes or something else to fill in the relationships

my tests now should not have parallelized writes. but running the tests at first caused another critial issue - the spicedb requests sometimes just hang. after adding a deadline to the request, i got back this error "serialization max retries exceeded; please reduce your parallel writes". So now each test should have one spicedb request in parallel. And each test suite has its own token so it will be isolated. Similar to GC, I've also tried to reuse the same spicedb token, deleting all relations, updating the schema, creating initial relations, and only then run the test suite, but this also had the same performance issue...

you shouldn't be using the same SpiceDB token if you're doing distinct tests

I know. I wasn't using the same token for all tests. I used same token for each worker - each worker runs tests in serial not in parallel. I hoped that reusing the datastore would have a good impact on performance. I tried this approach to see if it would be better than having different tokens for each test suite.

ah

hi! I was wondering if there are some suggestions on improving this flow. Currently, setting up a spicedb schema per-test is pretty slow since the schema writing part takes around 12s per test. I tried reusing the same authzed client across all tests (fine for us because they all use the same schema + use unique uuids to effectively isolate everything) but then I get the "serialization max retries exceeded; please reduce your parallel writes" error. So i'm not really sure how to proceed from there without writing some custom code/wrapper. Wondering if I'm missing something though