We don't currently generate the devtools API as a part of authzed-go -- which might make it difficult to use directly.

well, we do -- it's just in v0

that's ok, it's ok if it breaks. We want to start by having that feedback loop in CI. Our schema grows large, and playground is becoming challenging to manage

Maybe it makes more sense for us to export a package for our consistency tests so that users can easily write their own go tests that configure a schema and execute the consistency test

@User I'd also be curious to know what about the playground is becoming challenging, so we can address that separately

so far we've been relying on it and it's been pretty helpful. The developer workflow is right now pretty artisanal and relies on "yep, is green in playground" rather than seeing tests run in our CI pipeline. the fixtures are essentially a flat list, and our schema is growing quickly. The editor does not allow to provide any structure to separate by business domain. The same happen with the assertions. I'd be OK running the playground assertions in our CI (not sure if there is API to do that right now) but that still does not address the fact the fixtures and assertions are a large wall of text without structure

I love expected relations, allows us to see if unexpected things show up as the schema evolves

yeah, we have plans to make fixtures and assertions have a UI view

to make it easier

The "consistency tests" you refer to, if I understood it correctly, would be also of help, as it kind of "dynamically explores all the paths in your schema". Not sure how it figures out "what is correct and what is not"

are you just using yaml comments to separate them now? haha

@User if there was a command like...

spicedb validate mystuff.yaml

and it ran the same validation as Playground (assertions, expected relations, etc)

would that be sufficient for your CI?

if we ignore the fact that it's getting difficult to manage fixtures and assertions, yes, it would be sufficient!

in the end, the development happens in playground. But it's just one part of the development lifecycle

you might be able to have a bunch of separate schemas that you just concat together -- so you can maintain them individually, but test them together

FWIW I really like the workflow of defining my schema tests as go tests, and then just talking to the test server with different access key per test

how would various schemas help if everything is an interconnected graph?

(curious)

It was more a suggestion that you could organize things how you please and then "generate" a final schema from it.

Is your problem that because things are interconnected it's hard to organize or is it that it's hard to know what is being done in different parts of the schema/test data?

or is it something else entirely

I think the schema, at least for now, is manageable. we are essentially spiking out all the bits of the github business domain that we think are "challenging" to make sure the paradigm would support it.

it's the tests and fixtures. It does not have structure to say "these are the fixtures involved for these scenarios, and these are the test-cases for that business domain"

so it sounds like, hypothetically, if there was more structure to the validation file format and a

spicedb validate

, it might be sufficient? (just probing for answers :))

yeah, that's right

I tried some. For example started separating with mmultiple blocks of "assertTrue" and "assertFalse" separated by comments. Parser didn't like that

not quite though -- you want to test various relationships/assertions all with the same schema right?