spicedb
- j
Joey
12/21/2022, 2:04 AMI'm saying even 100 or 1000 rels within a single write should work - j
Joey
12/21/2022, 2:04 AMif it isn't, its because something is timing out in PG - j
Joey
12/21/2022, 2:04 AMbecause without any other transactions running, that seems like something is off - j
Joey
12/21/2022, 2:05 AMthe error is:ERROR: could not serialize access due to read/write dependencies among transactions - j
Joey
12/21/2022, 2:05 AMif there is a single transaction running, on the other hand, there should be no issue serializing access - p
Perseus
12/21/2022, 2:06 AMright. each call was sending all the relations 1 resource had. I can increase that and accumulate relations across multiple resources in 1 call, but still keep it sequential. - p
Perseus
12/21/2022, 2:06 AMill give that a go after running the GC - j
Jake
12/21/2022, 2:10 AMwith batches of 1000 relationships I was able to backfill at about 3k relationships per second without the TOUCH upsert improvement - j
Jake
12/21/2022, 2:10 AMwith the upsert improvement and multiple workers I was able to get that up to 21k rels/sec - j
Jake
12/21/2022, 2:10 AMwhat are you seeing? and on what hardware? - p
Perseus
12/21/2022, 2:15 AMrps does not seem to be the limiting factor for me. the datastore instance is a 2core 8gb memory RDS instance running postgres have 3 spicedb pods running (1.6cpu cores, 1gb ram each) from my understanding, whats limiting me is the 'overlapping' of subjects if i try to process in parallel. I can have multiple resources in a batch that an accessor has a relation with, which is creating the serializing error in pg - j
Jake
12/21/2022, 2:17 AMyeah, the deletes that go out as part of a TOUCH event are locking too aggressively AFAICT, the upsert fix fixes that as well - j
Jake
12/21/2022, 2:17 AMif you're willing to accept a diff and make your own build, I can give you that now - p
Perseus
12/21/2022, 2:27 AMyeah i can do that if the gc doesnt make things better - could you send the commitid? ill figure out the building part - p
Perseus
12/21/2022, 4:54 AMthanks for the help. seems to be holding strong at batches of 2k with no errors right now (peaks at around 200 rps with many relations in each request) - p
Perseus
12/21/2022, 4:56 AMquick q regarding the otel integration - is there a direct way to propagate traceids from my application to spicedb? i looked through the dependency tree, there was a b3 propagator present, just wanted to make sure that the propagation of traceids was possible - u
Unhinged
12/21/2022, 8:10 AMHi are there any examples of how to call the java client equivalent of the sandbox's "Expected Relations" lookup? - v
vshar178
12/21/2022, 8:33 AMHi, We do not see NewRelic in the integrations page, can you please confirm is NewRelic integration is available via Go/Java Agents? https://authzed.com/integrations - j
Jake
12/21/2022, 2:59 PMCan you comment with your results on the upsert tracking issue: https://github.com/authzed/spicedb/issues/901 - d
DocD2
12/21/2022, 3:00 PMAre there any case studies of companies using spicedb? - d
dsieczko
12/21/2022, 3:09 PMHi @DocD2 ! We’re working with current customers and OSS users to publish case studies as part of our brand refresh; expect to see case studies in Q2 but we can share customer and user references if needed in the meantime. Feel free to DM me here or send me an email at damian@authzed.com if you'd like references. - d
DocD2
12/21/2022, 3:11 PMWould you mind dming me. I don't seem to be allowed to dm you. - v
vshar178
12/21/2022, 6:27 PMHi, Please suggest. - v
vroldanbet
12/21/2022, 6:29 PM👋🏻 native integration via NewRelic SDKs is not supported. NewRelic can be integrated via prometheus and open-telemetry with SpiceDB - p
pdewilde
12/21/2022, 8:27 PMI'm using the ReadRelatonships grpc https://github.com/authzed/authzed-go/blob/main/proto/authzed/api/v1/permission_service_grpc.pb.go#L27 I'm using a loop that calls Recv() on the permissionsServiceReadRelationshipsClient until I get an EOF or other error, but my loop in looping forever returning the same relation and a NIl errorreadClient, err := spiceClient.ReadRelationships(c, &myrequest) if err != nil { // handle err } else { for resp, err := readClient.Recv(); err != io.EOF; { if err != nil { // handle non-eof err } // handle response } } - j
Joey
12/21/2022, 8:28 PMyou should useerrors.Is - j
Joey
12/21/2022, 8:28 PMlet me get an example - j
Joey
12/21/2022, 8:29 PMfor { resp, err := lookupClient.Recv() if errors.Is(err, io.EOF) { break } if err != nil { // do something } // handle response here } - j
Joey
12/21/2022, 8:29 PMthis is what we use in all our tests - p
pdewilde
12/21/2022, 8:32 PMThanks, still learning go.