hello) i have problems with memory on
# spicedb
y
hello) i have problems with memory on spice db. The memory usage is not stable and jumping all time from 60% to 90+ %. i think that i should play with garbage collector settings. Could someone help me with it?)
@vroldanbet 🙏 take a look, please
v
there is little I can do with this information. You'd need to capture a pprof memory dump at the right time to determine what's going on. what version of SpiceDB are you using? How much memory are you giving it? What kind of RPC calls are your doing? Are you doing LookupResources that return big lists of resources? I recommend setting
GOMEMLIMIT
to the value of your kubernetes deployment memory limit
y
version - 1.31 memory - 5gb LookupResources - yes, we are doing it. we have users with >1000 resources
v
are you setting the optional limit in
LookupResources
and using cursoring?
y
i don`t see any limits or cursors in our code
thank you for answers)
v
that's likely the problem. When you don't set the limit, SpiceDB has to load everything in memory before sending it, which explains the spikes. It's unfortunate but it had to be left as the default because otherwise we would have broken the API.
y
hello) i just tested limit/cursor parameters in my code. I have user with 20k resources and i should lookup all his resources. When i tested it without limit/coursor memory grows was up to 5%, but when i use limit=200 memory grow is up to 20%.
mb i`am doing something wrong. i am new in golang , and mb i should clean memory after each request?
@vroldanbet
v
can you please run more tests to make sure it's not an outlier. If the behaviour persists, I'd suggest you to open an issue.
y
i tested a lot of times, and on 2 environments
mb, spicedb save coursor in memory
or how it works with coursor? because as we know in mysql offset is not the best practice, because its still scan all table mb spice doing the same
v
wait, are you using mysql?
it's not offset, it's doing keyset strategy
55 Views