Hey everyone! 👋 I was reading [dispatch/caching/...
# spicedb
Hey everyone! 👋 I was reading [dispatch/caching/caching.go](https://github.com/authzed/spicedb/blob/main/internal/dispatch/caching/caching.go) while looking into a memory-use problem with our spicedb containers. Specifically, I was looking at how the cache limits its memory use. Some relevant snippets: https://github.com/authzed/spicedb/blob/a2e9e40fae53b49a6fb183aa0da338011e1eaba6/internal/dispatch/caching/caching.go#L44-L55
Copy code
type checkResultEntry struct {
    response *v1.DispatchCheckResponse

type lookupResultEntry struct {
    response *v1.DispatchLookupResponse

var (
    checkResultEntryCost       = int64(unsafe.Sizeof(checkResultEntry{}))
    lookupResultEntryEmptyCost = int64(unsafe.Sizeof(lookupResultEntry{}))
Copy code
cd.c.Set(requestKey, toCache, checkResultEntryCost)
I am new to Go and could be missing something, but I think that the cost passed to the cache for each item ends up being '8', the size of a struct containing a single pointer field. If the cache max cost is meant to be in terms of bytes of memory used, then I think this would be wrong because the pointed-to
could be using much more than 8 bytes of memory. So, a cache that is set to use no more than 16MB of memory, could actually take up whatever memory is used by 2 million
structs. The cache config code does state that max size is in terms of bytes: https://github.com/authzed/spicedb/blob/a2e9e40fae53b49a6fb183aa0da338011e1eaba6/pkg/cmd/server/cacheconfig.go#L45-L51 Should the actual memory cost of the
struct, including referenced memory, be computed to accurately enforce max cost? Thank you!