844600078504951838 #spicedb

Channels

Jake

10/14/2022, 12:26 AM

if you're trying to do a big import, my suggestion would be to 1) use touch instead of create, it's idempotent, and 2) try to make sure that your data truly doesn't overlap for concurrent things, or do your backfill serially in batches of 1000

symion

10/14/2022, 12:27 AM

its not really that big

symion

10/14/2022, 12:28 AM

[
  {
    "ID": 1,
    "name": "Readonly",
    "rules": {
      "reader": ["alerts", "views"],
      "writer": [],
      "manager": []
    }
  },
  {
    "ID": 2,
    "name": "Basic",
    "parent_role_ids": [1],
    "rules": {
      "reader": [],
      "writer": ["alerts", "views"],
      "manager": []
    }
  },
  {
    "ID": 3,
    "name": "Admin",
    "parent_role_ids": [2],
    "rules": {
      "reader": [],
      "writer": [],
      "manager": ["alerts", "views"]
    }
  }
]

symion

10/14/2022, 12:28 AM

there is a small loop that translates this in to spicedb relations.

symion

10/14/2022, 12:28 AM

its not terrible

Jake

10/14/2022, 12:29 AM

are you doing user defined roles?

Jake

10/14/2022, 12:29 AM

if these rules are static they could just be in schema

Jake

10/14/2022, 12:30 AM

even in UDR, it looks like that should only translate to like 8 relationships

Jake

10/14/2022, 12:30 AM

if you're getting serialization errors writing 8 relationships that clearly don't overlap, something seems very wrong

symion

10/14/2022, 12:32 AM

yes effectively. we're doing user defined roles with some system level roles. these are the system level roles. The database is empty otherwise

symion

10/14/2022, 12:32 AM

spicedb=# select * from relation_tuple;
 namespace | object_id | relation | userset_namespace | userset_object_id | userset_relation | created_xid |     deleted_xid     
-----------+-----------+----------+-------------------+-------------------+------------------+-------------+---------------------
 resource  | alerts    | reader   | role              | readonly          | member           |         787 | 9223372036854775807
(1 row)

spicedb=# select * from relation_tuple_transaction;
         timestamp          | xid | snapshot 
----------------------------+-----+----------
 1970-01-01 00:00:00        |   1 | 1:1:
 2022-10-14 00:26:23.658744 | 785 | 785:785:
 2022-10-14 00:27:37.423208 | 787 | 787:787:

symion

10/14/2022, 12:33 AM

you think wrong in how we're trying to setup the relations?

Jake

10/14/2022, 12:33 AM

no, just think your script is generating the same relationship twice, and you're using CREATE to insert it

Jake

10/14/2022, 12:34 AM

for whatever reason

symion

10/14/2022, 12:34 AM

only when I run it personally...

symion

10/14/2022, 12:36 AM

Is there a variable I can set to get spicedb to output all the sql queries it is running? I'm just getting errors from postgres.

Jake

10/14/2022, 12:36 AM

you can run it with

--log-level debug

symion

10/14/2022, 12:37 AM

SPICEDB_LOG_LEVEL=debug ?

Jake

10/14/2022, 12:37 AM

yep, if you're using env vars

symion

10/14/2022, 12:37 AM

ok. I'll do this and add some better output to our script in the morning

symion

10/14/2022, 1:12 AM

This is the entire log from startup to error

symion

10/14/2022, 1:13 AM

compose-spicedb-1            | {"level":"debug","module":"pgx","args":["resource","alerts","reader","role","readonly","member","resource","views","reader","role","readonly","member"],"commandTag":"SU5TRVJUIDAgMg==","pid":86,"sql":"INSERT INTO relation_tuple (namespace,object_id,relation,userset_namespace,userset_object_id,userset_relation) VALUES ($1,$2,$3,$4,$5,$6),($7,$8,$9,$10,$11,$12)","time":2.139288,"time":"2022-10-14T00:56:57Z","message":"Exec"}

symion

10/14/2022, 1:14 AM

Thats the only write I see

Jake

10/14/2022, 1:14 AM

And that’s the same database that the select was from earlier?

symion

10/14/2022, 1:15 AM

no, its a clean one

symion

10/14/2022, 1:15 AM

always starting over from zero

symion

10/14/2022, 1:16 AM

spicedb=# select * from relation_tuple;
 namespace | object_id | relation | userset_namespace | userset_object_id | userset_relation | created_xid |     deleted_xid     
-----------+-----------+----------+-------------------+-------------------+------------------+-------------+---------------------
 resource  | alerts    | reader   | role              | readonly          | member           |         787 | 9223372036854775807
 resource  | views     | reader   | role              | readonly          | member           |         787 | 9223372036854775807

Jake

10/14/2022, 1:19 AM

i see it twice

Jake

10/14/2022, 1:20 AM

compose-spicedb-1            | {"level":"debug","module":"pgx","args":["resource","alerts","reader","role","readonly","member","resource","views","reader","role","readonly","member"],"commandTag":"SU5TRVJUIDAgMg==","pid":86,"sql":"INSERT INTO relation_tuple (namespace,object_id,relation,userset_namespace,userset_object_id,userset_relation) VALUES ($1,$2,$3,$4,$5,$6),($7,$8,$9,$10,$11,$12)","time":2.139288,"time":"2022-10-14T00:56:57Z","message":"Exec"}

Jake

10/14/2022, 1:20 AM

that one that succeeds