https://authzed.com logo
#spicedb
Title
# spicedb
b

bigben

05/16/2022, 2:07 PM
Hey everyone! 👋 I was reading [dispatch/caching/caching.go](https://github.com/authzed/spicedb/blob/main/internal/dispatch/caching/caching.go) while looking into a memory-use problem with our spicedb containers. Specifically, I was looking at how the cache limits its memory use. Some relevant snippets: https://github.com/authzed/spicedb/blob/a2e9e40fae53b49a6fb183aa0da338011e1eaba6/internal/dispatch/caching/caching.go#L44-L55
Copy code
go
type checkResultEntry struct {
    response *v1.DispatchCheckResponse
}

type lookupResultEntry struct {
    response *v1.DispatchLookupResponse
}

var (
    checkResultEntryCost       = int64(unsafe.Sizeof(checkResultEntry{}))
    lookupResultEntryEmptyCost = int64(unsafe.Sizeof(lookupResultEntry{}))
)
https://github.com/authzed/spicedb/blob/a2e9e40fae53b49a6fb183aa0da338011e1eaba6/internal/dispatch/caching/caching.go#L222
Copy code
go
cd.c.Set(requestKey, toCache, checkResultEntryCost)
I am new to Go and could be missing something, but I think that the cost passed to the cache for each item ends up being '8', the size of a struct containing a single pointer field. If the cache max cost is meant to be in terms of bytes of memory used, then I think this would be wrong because the pointed-to
DispatchCheckResponse
could be using much more than 8 bytes of memory. So, a cache that is set to use no more than 16MB of memory, could actually take up whatever memory is used by 2 million
DispatchCheckResponse
structs. The cache config code does state that max size is in terms of bytes: https://github.com/authzed/spicedb/blob/a2e9e40fae53b49a6fb183aa0da338011e1eaba6/pkg/cmd/server/cacheconfig.go#L45-L51 Should the actual memory cost of the
checkResultEntry
struct, including referenced memory, be computed to accurately enforce max cost? Thank you!