On podbor-minuta.ru some database queries are heavy, so we keep their results in a cache: apartment lists, aggregate numbers for listing pages. While the value is in the cache, everything is fast. The trouble starts the moment it expires. One day under load we saw a batch of 500 responses timing out. The database was alive, but queries to it suddenly queued up and hit the connection limit. The answer was in how we refreshed the cache. The logic was naive: if the value is not in the cache, ...