https://authzed.com logo
Join Discord
Powered by
`FullyConsistent` And you don't need to do anythi...
# spicedb
j
FullyConsistent
And you don't need to do anything special to hit this issue: by default, SpiceDB uses mode 
minimizelatency
, which will hit the caches. If the data has just been written (within ~5s), its possible with that mode to get answers that do not reflect the recently written relationships, unless a zedtoken is used
w
I see, and that includes relationships written by --load-configs when using a new auth token?
j
it can
serve-testing
is just running an in-memory SpiceDB
if you want to be "sure", you could throw a few second timeout between starting that and making your API requests in the tests
w
To make sure I understand, then- for any connection using a new auth token in serve-testing mode, there's a risk of having the schema but no data for the first few seconds starting from the first API request?
serve-testing and --load-configs, I mean
j
I don't believe for a new auth token, no
but you'd have to confirm that
w
Is there a way to shorten that time when using serve-testing? These tests pass > 99% of the time, so it can take a while to reproduce the error (last time, I had to rerun a test 167 times before it failed.)
j
I don't believe we have a setting to do so for serve-testing
I'll have to look
doesn't look it
I'll see if we can address this on our side
can you repeat the error you're seeing here?
w
I don't think we have logs from SpiceDB from a failed test run (due to the volume of test runs it usually takes to reproduce it), so I can't say if there's anything there, but no error is returned to the client- the tests just fail due to relationships they expected to be seeded from the YAML not being present.
I could try to put together a self-contained repro project that's smaller than the repo linked above if that would help.
j
yeah
it would
w
Alright, made. This project spins up a SpiceDB container with a simple bootstrap file with one relationship and then pings it a thousand times with different auth tokens to query whether that relationship exists, logging the attempt number for each failure: https://github.com/wscalf/rare-initialization-race-repro
Some other findings from messing around with it today: setting the request to fullyconsistent does make the test pass but isn't really what we're trying to do there- the idea is that when using a startup file, that represents the initial state/earliest snapshot of the system for the sake of testing things. But it seems that isn't correct? And adding waits (5s or 10s) between instantiating the client and making the call does not seem to make the test pass.
j
@wscalf if you're willing to make a code change to a checked out SpiceDB, I can send you a one line change that might solve it
and then you could test against that
w
I'm game
j
change that to 
0
see if that solves it
w
It does.
Or at least, the one-off build of SpiceDB does what we need and passes the test. As far as solving our problem, is the recommendation to run a fork of SpiceDB in our integration tests? Or was this more to help you confirm the issue?
j
Just confirming the issue
If you can’t reproduce it with this change
Then we know what change to make
w
Sounds great! Yeah, the test run hit it a thousand times, and it gave the expected result each time, same as when running with fully consistent mode.
Is there a GH issue or anything I can reference from our side?
j
I'll just issue a PR
@wscalf is it okay if I use your repro as a test case in SpiceDB?
w
Sure!
j
I derived my test from yours
3 Views
PreviousNext
Powered by