Caching & Cache Invalidation

Use Case: Caching & Cache Invalidation

The everyday problem: The same expensive query runs thousands of times a minute — a product catalogue, a user's permissions, a currency rate, a rendered dashboard. Each call hits the database, recomputes the same answer, and adds latency. Caching stores the result so subsequent calls return instantly. But then the famous hard part arrives: the data changes and the cache still serves the old value. "There are only two hard things in computer science: cache invalidation and naming things." The challenge is twofold — store expensive results to serve them fast, and keep the cache correct when the underlying data changes.

Six approaches: the storage tiers (in-memory, distributed, hybrid), the patterns for keeping them correct (cache-aside, write-through), and the protection against the failure mode that takes down systems (stampede). Most real systems combine several.

---

Way 1: In-Memory Cache (IMemoryCache)

Best used for: Single-instance apps, or per-server caching of data that is small, read constantly, and tolerant of brief staleness — reference data, config, lookup tables, computed values. The fastest possible cache.

The concept: IMemoryCache stores objects directly in the application's process memory. Reads are a dictionary lookup — nanoseconds, no serialization, no network. You set an expiry so entries refresh periodically.

builder.Services.AddMemoryCache();

public class ProductCatalog(IMemoryCache cache, AppDbContext db)
{
    public async Task<List<Category>> GetCategoriesAsync(CancellationToken ct)
    {
        return (await cache.GetOrCreateAsync("categories", async entry =>
        {
            entry.AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(10); // refresh every 10m
            entry.SetSize(1);                                                 // for size-limited caches
            return await db.Categories.AsNoTracking().ToListAsync(ct);
        }))!;
    }
}

// Bound the cache so it cannot consume unbounded memory
builder.Services.AddMemoryCache(o => o.SizeLimit = 1024); // requires SetSize on every entry

Why it works: The data lives in the same process as the code, so a cache hit is a raw memory access — orders of magnitude faster than any network call. GetOrCreateAsync gives the cache-aside pattern in one call: return the cached value or compute, store, and return it.

Benchmark note: In-memory hits are ~nanoseconds versus ~1 ms+ for a database round-trip — effectively free. Two limits define its scope. First, memory pressure: unbounded caches cause out-of-memory crashes, so always set SizeLimit and per-entry sizes. Second, and more important, it does not scale across servers: with 5 instances you have 5 independent caches that can hold 5 different versions of the same data, and invalidating one does not invalidate the others. For multi-instance apps, in-memory cache is fine for immutable/reference data but wrong for anything that must be consistent across the farm — that needs Way 2.

---

Way 2: Distributed Cache (Redis)

Best used for: Multi-instance applications needing a shared cache — session state, user permissions, any data that must be consistent across all servers and survive individual instance restarts.

The concept: A central cache server (Redis) holds the data; all application instances read and write the same shared store over the network. Invalidate once and every instance sees it. The trade-off versus in-memory is a network round-trip and serialization, in exchange for a single consistent cache across the farm.

builder.Services.AddStackExchangeRedisCache(o =>
{
    o.Configuration = builder.Configuration.GetConnectionString("Redis");
    o.InstanceName  = "myapp:";
});

public class UserPermissionService(IDistributedCache cache, AppDbContext db)
{
    public async Task<Permissions> GetPermissionsAsync(int userId, CancellationToken ct)
    {
        var key = $"perms:{userId}";

        // Cache-aside over a distributed store
        var cached = await cache.GetStringAsync(key, ct);
        if (cached is not null)
            return JsonSerializer.Deserialize<Permissions>(cached)!;

        var perms = await db.LoadPermissionsAsync(userId, ct);

        await cache.SetStringAsync(key, JsonSerializer.Serialize(perms),
            new DistributedCacheEntryOptions { AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(15) }, ct);

        return perms;
    }

    // Invalidate across ALL instances at once — the win over in-memory
    public Task InvalidateAsync(int userId, CancellationToken ct)
        => cache.RemoveAsync($"perms:{userId}", ct);
}

Why it works: One shared cache means one source of cached truth. When a user's permissions change, you remove one key and every server immediately stops serving the stale value — the consistency that per-server in-memory caches cannot provide.

Benchmark note: A Redis hit is ~0.2–1 ms (network + deserialization) — slower than in-memory's nanoseconds but still 5–50× faster than recomputing from the database, and it scales to enormous read volumes. The costs: serialization overhead (cache complex objects as compact JSON or MessagePack, not bloated formats), and a new dependency whose outage you must handle — decide fail-open (cache miss → go to DB, higher load but available) which is almost always correct. Redis also gives you atomic operations, pub/sub for invalidation signalling, and TTLs for free. This is the backbone cache for any serious multi-instance system.

---

Way 3: HybridCache (.NET 9 — best of both tiers)

Best used for: Wanting in-memory speed and distributed consistency together, with stampede protection built in — the modern default that supersedes hand-rolling the two-tier (L1/L2) pattern.

The concept: HybridCache (new in .NET 9) layers a local in-memory cache (L1) in front of a distributed cache (L2). Reads check L1 first (nanoseconds), fall back to L2 (milliseconds), and only compute from source on a full miss. It also deduplicates concurrent calls for the same key (stampede protection, Way 6) automatically, and unifies the API so you write one call instead of orchestrating two caches.

builder.Services.AddHybridCache(o =>
{
    o.DefaultEntryOptions = new HybridCacheEntryOptions
    {
        Expiration      = TimeSpan.FromMinutes(15),  // L2 (distributed) lifetime
        LocalCacheExpiration = TimeSpan.FromMinutes(2) // L1 (in-memory) lifetime — shorter
    };
});
// Wire up the L2 backing store (Redis); without it, HybridCache is L1-only
builder.Services.AddStackExchangeRedisCache(o => o.Configuration = redisConn);

public class ProductService(HybridCache cache, AppDbContext db)
{
    public async Task<Product> GetProductAsync(int id, CancellationToken ct)
    {
        return await cache.GetOrCreateAsync(
            $"product:{id}",
            async token => await db.Products.FindAsync([id], token), // only runs on a full miss
            cancellationToken: ct);
        // Concurrent callers for the same id share ONE computation — no stampede
    }

    public async Task InvalidateAsync(int id, CancellationToken ct)
        => await cache.RemoveAsync($"product:{id}", ct); // clears L1 and L2
}

// Tag-based invalidation — drop whole groups at once
// await cache.RemoveByTagAsync("catalog", ct);

Why it works: Most reads are served from L1 at memory speed; L2 provides cross-server consistency and survives instance restarts; the source is hit only on a genuine miss. The built-in request coalescing means a thousand simultaneous misses for the same key trigger one computation, not a thousand — solving the stampede problem you would otherwise have to build yourself.

Benchmark note: You get L1 hit latency (nanoseconds) for hot keys and L2 consistency for everything else, with stampede protection included — previously this required careful hand-rolled L1/L2 plumbing plus a SemaphoreSlim per key. The main caveat is L1/L2 coherence: a local L1 entry on server A is not invalidated when server B updates L2, so keep LocalCacheExpiration short (seconds to a couple of minutes) to bound staleness, and use RemoveAsync/tag invalidation for changes that must propagate promptly. For new .NET 9+ projects this is the recommended caching entry point.

---

Way 4: Cache-Aside vs Write-Through (the consistency patterns)

Best used for: Deciding when the cache is populated and invalidated relative to writes — the pattern choice that determines how stale your cache can get.

The concept: Two dominant patterns. Cache-aside (lazy): the application manages the cache — read checks cache, on miss loads from DB and populates; on write, update the DB and invalidate (delete) the cache key so the next read repopulates. Write-through: writes go through the cache, which updates both itself and the DB, so the cache is always current. Cache-aside is simpler and more common; write-through trades write latency for read freshness.

// CACHE-ASIDE — invalidate on write (most common, simplest, robust)
public async Task UpdateProductAsync(Product product, CancellationToken ct)
{
    await db.UpdateAsync(product, ct);           // 1. write to the source of truth
    await cache.RemoveAsync($"product:{product.Id}", ct); // 2. delete the key (do NOT update it)
    // Next read repopulates from the DB — guaranteed fresh.
    // Deleting (not updating) the key avoids caching a value that a concurrent write changed.
}

// WRITE-THROUGH — cache and DB updated together on write
public async Task UpdateProductWriteThroughAsync(Product product, CancellationToken ct)
{
    await db.UpdateAsync(product, ct);                          // write DB
    await cache.SetAsync($"product:{product.Id}",
        JsonSerializer.SerializeToUtf8Bytes(product), _options, ct); // and refresh cache
    // Reads are always warm and fresh, but every write pays the cache-write cost.
}

Why it works: Cache-aside keeps the database authoritative and uses the cache as a disposable accelerator — invalidating (deleting) on write is the safest default because the next read always rebuilds from truth, sidestepping the "I updated the cache with a value that was already stale" race. Write-through keeps reads perfectly warm at the cost of write latency and is best when reads vastly outnumber writes and freshness matters.

Benchmark note: The subtle correctness rule: on a write, delete the cache key rather than updating it. Updating invites a race — two concurrent writers can interleave and leave the cache holding the loser's value while the DB holds the winner's. Deletion forces a clean reload from the authoritative source. Even with deletion, a tiny race exists (a read repopulates with old data microseconds before a write's delete lands); for most data this sub-second window is harmless, but for strict consistency you need short TTLs as a backstop or versioned keys. Cache-aside with delete-on-write plus a modest TTL covers the vast majority of real applications.

---

Way 5: Output / Response Caching (cache the whole response)

Best used for: Entire HTTP responses that are identical for many users — public product pages, published articles, anonymous API responses — where you can skip not just the data fetch but the entire request-handling pipeline.

The concept: Rather than caching data inside your code, you cache the rendered HTTP response itself. ASP.NET Core's output caching middleware (.NET 7+) intercepts requests, and if a fresh cached response exists for that request, returns it without invoking your endpoint at all. CDN/edge caching extends this to the network edge, serving responses geographically close to users.

builder.Services.AddOutputCache(o =>
{
    o.AddBasePolicy(b => b.Expire(TimeSpan.FromSeconds(30)));
    o.AddPolicy("products", b => b
        .Expire(TimeSpan.FromMinutes(5))
        .SetVaryByQuery("category", "page")  // separate cache entry per query combination
        .Tag("catalog"));                    // for grouped invalidation
});

var app = builder.Build();
app.UseOutputCache();

app.MapGet("/api/products", GetProducts).CacheOutput("products");

// Invalidate a tagged group when the catalogue changes
app.MapPost("/admin/products", async (Product p, IOutputCacheStore store, CancellationToken ct) =>
{
    await SaveProductAsync(p, ct);
    await store.EvictByTagAsync("catalog", ct); // drop all cached product responses at once
});

Why it works: Caching at the response level skips the entire pipeline — routing, model binding, data access, serialization — returning a pre-rendered response in microseconds. For content identical across users, this is the highest-leverage cache: one cached response serves everyone. CDN edge caching pushes it further, serving from a node near the user with zero origin load.

Benchmark note: Output caching can take an endpoint from, say, 50 ms (full pipeline) to sub-millisecond (cached response), and offloads enormous read volume from origin servers — a CDN can absorb millions of requests your servers never see. The hard constraint: it only works for responses that are the same for many requests. Personalised responses (per-user data) cannot be output-cached without varying by user, which fragments the cache and erodes the benefit. Use VaryBy carefully — varying by too many dimensions multiplies cache entries. Output caching is for public, shareable content; use data caching (Ways 1–3) for personalised paths.

---

Way 6: Stampede Protection (prevent the thundering herd)

Best used for: High-traffic keys where a cache expiry can trigger a flood of simultaneous recomputations that overwhelm the database — the failure mode that turns a cache miss into an outage.

The concept: When a popular cached key expires, every concurrent request misses at once and all of them try to recompute it simultaneously — a "cache stampede" or "thundering herd" that can hammer the database with thousands of identical expensive queries in an instant, sometimes taking it down. The fix: ensure only one caller recomputes while the others wait for that single result.

// Per-key locking so only ONE caller recomputes; others await the same result
public class StampedeProtectedCache(IMemoryCache cache)
{
    private readonly ConcurrentDictionary<string, SemaphoreSlim> _locks = new();

    public async Task<T> GetOrCreateAsync<T>(string key, Func<Task<T>> factory, TimeSpan ttl)
    {
        if (cache.TryGetValue(key, out T? cached)) return cached!; // fast path: hit

        var keyLock = _locks.GetOrAdd(key, _ => new SemaphoreSlim(1, 1));
        await keyLock.WaitAsync();
        try
        {
            // Double-check: another caller may have populated it while we waited
            if (cache.TryGetValue(key, out cached)) return cached!;

            var value = await factory();           // ONLY ONE caller reaches here
            cache.Set(key, value, ttl);
            return value;
        }
        finally
        {
            keyLock.Release();
            _locks.TryRemove(key, out _);
        }
    }
}
// .NET 9 HybridCache (Way 3) does this automatically — prefer it for new code.

// Complementary technique: probabilistic early expiration — refresh BEFORE expiry
// so a key is recomputed by one request slightly early rather than by all at the moment it expires.

Why it works: The per-key lock funnels all concurrent misses for the same key through a single recomputation — one database query instead of thousands. The double-check after acquiring the lock means callers who waited get the value the first caller just computed, rather than redundantly recomputing it themselves.

Benchmark note: Without stampede protection, a single popular key expiring under high traffic can spike the database with thousands of simultaneous identical queries — a self-inflicted denial of service that commonly takes systems down at exactly peak load. With protection, that becomes one query plus a brief wait for the rest. The lock adds negligible overhead on the hot (hit) path since it is only taken on a miss. HybridCache includes this automatically, which is a strong reason to adopt it; if you are on an older stack, the per-key semaphore pattern above is the standard fix. Pair it with slightly randomised TTLs so many keys do not all expire at the same instant.

---

Choosing the right approach

• Single instance, reference data, max speed → In-memory cache (Way 1)
• Multi-instance, shared + consistent → Distributed Redis (Way 2)
• Want both tiers + stampede protection, .NET 9+ → HybridCache (Way 3) — modern default
• Deciding when to invalidate → Cache-aside with delete-on-write (Way 4)
• Whole responses identical across users → Output / CDN caching (Way 5)
• Hot keys under high traffic → Stampede protection (Way 6) — built into Way 3

The mental model: caching is two separate problems — storage (where the cached value lives: memory, Redis, both, or the HTTP response) and invalidation (keeping it correct: TTL, delete-on-write, tags). Pick storage by your topology: in-memory for single-instance or immutable data, distributed for multi-instance consistency, HybridCache to get both. Pick invalidation by your freshness need: a short TTL alone is often enough for tolerant data; delete-on-write (cache-aside) for data that must reflect changes promptly. Layer output caching on top for public shareable responses. And on any high-traffic key, add stampede protection — or use HybridCache, which gives you the modern storage tiers and stampede protection in one API. The honest truth the famous quote captures: getting the value into the cache is easy; getting stale values out at the right moment is the part that requires thought.

Note: this is an informational engineering overview. Latency figures are illustrative order-of-magnitude estimates that depend on data size, serialization, network, and infrastructure. Always measure against your own workload.