Doing some load testing, we were running into oomkilled and realized we have some questions about how the cache works. Is there any cache eviction other than evicting old quantization buckets?
Alec
08/04/2023, 6:11 PM
The scenario was mostly experimental as we get the set up going, and so I'm not terribly worried about real world implications (e.g. we only have a single pod running currently) but it did leave us wondering what exactly happened
j
Joey
08/04/2023, 7:28 PM
all cache entries are TTLed to approximately 2x the quantization window
Joey
08/04/2023, 7:28 PM
but they are also evicted if over the allocated memory
Joey
08/04/2023, 7:28 PM
so you should not be ghetting OOM if you set the cache sizes to be below your mem limit