hello, I did more investigation on our problem with spice (version 1.9.0, but it was the same on 1.8.0) memory usage on Lookup. I created a synthetic dataset with a simple schema: User member of Customer, Resource owned by User. Resource can be shared with other Users and Customers. Backend is PostgreSQL. Spice is single node.
0. spice initial memory usage: <30MB.
1. create 8k Resources, owned by User u1: memory usage still very low
2. query lookup non-concurrently, repeatedly (every 2s), for user u1: memory spikes to >200MB for a few seconds, then down to <100MB
3. while querying, I create thousands more relations on other users and customers: memory spikes as soon as I run the creation script, then it tops to >500MB
4. I stop the concurrent relation creation (it's still doing Lookup), memory usage 300MB
5. stop Lookup and wait a few minutes: memory usage 240MB, I'd expect it to be <100MB
6. restart spice: <30MB
If I do the relation creation (point 3) non-concurrently, before running the repeated lookup script (point 2), memory never exceeds 120MB.