I Cached One Page and Stored It 248 Times

I turned on output caching for a report endpoint that was getting hammered, patted myself on the back, and went to lunch. When I came back the server was still doing the exact same amount of work. Same CPU, same database chatter, same everything. The cache was on. I could see it in the config. It just wasn't catching anything.

The endpoint was dead simple — a paginated report, /report?page=1, the kind of thing that's identical for every visitor on the same page. Perfect candidate for caching. So why was the handler running on basically every request?

Here's the setup, trimmed down to nothing:

builder.Services.AddOutputCache(); var app = builder.Build(); app.UseOutputCache(); app.MapGet("/report", (int page) => { Interlocked.Increment(ref defaultMisses); return Results.Text($"page={page}"); }).CacheOutput();

That counter is the whole trick. A cache hit never calls the handler, so the counter only moves when the cache misses. If caching works, the number should barely climb.

I had a guess about what was wrong, and I was wrong about it. My assumption was that the default policy caches one body per path and ignores the query string, so ?page=1 and ?page=2 would collide and everyone would see page 1. That's the bug I went looking for. It's not the bug I had.

The default policy varies by the entire query string. Every distinct query gets its own cache entry. ?page=1 and ?page=2 are correctly separate. Good. But so are ?page=1&utm_campaign=spring and ?page=1&utm_campaign=twitter. And ?page=1&t=8831. And every other combination of marketing params, cache-busters, and random junk that real traffic drags along behind it.

My report was linked from newsletters and tweets and who knows what else, so in practice almost no two requests had the same query string. Same page, same bytes, and every single one of them a brand new cache entry.

So I measured it instead of guessing. I fired 300 requests at the endpoint, all for page=1, varying only the tracking noise the way real links do:

string[] campaigns = { "spring", "newsletter", "twitter", "reddit", "email", "" }; for (int i = 0; i < 300; i++) { var utm = campaigns[rng.Next(campaigns.Length)]; var q = utm.Length == 0 ? "page=1" : $"page=1&utm_campaign={utm}&t={rng.Next(10000)}"; await http.GetStringAsync($"{baseUrl}/report?{q}"); }

Then I read the counter:

300 requests to /report, all page=1 handler runs (cache misses): default=248

248 out of 300. The cache wasn't broken — it was doing precisely what it was told. It just got told to key on stuff that didn't matter. Every unique t= value minted a fresh entry, served it to exactly one request, and sat there taking up memory until it expired. That's not a cache. That's a memory leak with a hit rate.

The fix is one line. You tell the policy which query values actually change the response:

app.MapGet("/report-varied", (int page) => { Interlocked.Increment(ref variedMisses); return Results.Text($"page={page}"); }).CacheOutput(p => p.SetVaryByQuery("page"));

SetVaryByQuery("page") means: build the cache key from page and nothing else. The utm_campaign, the t, the fbclid some platform stapled on, all of it ignored. Same 300-request burst against this one:

handler runs (cache misses): varied=1

One. The first request for page=1 ran the handler, filled the cache, and the other 299 never touched it. That's the cache I thought I'd turned on an hour earlier.

The part that stung a little is that the default isn't wrong, exactly. Varying by the full query string is the safe choice for a framework. It can't know which of your params are load-bearing and which are noise, so it assumes they all matter and never serves you a stale-for-the-wrong-reason response. Correct, conservative, and quietly useless for any endpoint that lives on the public internet where URLs collect barnacles.

A caveat, because this cuts both ways. Once you pin SetVaryByQuery("page"), you own that list. Add a real filter — ?sort=date, ?region=eu — and forget to add it to the vary list, and now you've got the opposite bug: two genuinely different responses sharing one cache entry, and someone sees the wrong data. That failure is a lot more embarrassing than a low hit rate. So the honest rule is: list every query key that changes the body, no more and no less, and treat that list as part of the endpoint's contract. When an endpoint has a lot of legitimate filters, output caching might just not be worth the bookkeeping, and I'll cache deeper in the stack instead.

My opinion after this one: I don't trust a cache I haven't watched miss. The config looked perfect. The hit rate was garbage. A counter on the handler and a /stats endpoint took ten minutes to wire up and told me more than any amount of staring at CacheOutput() would have. Numbers here are from a Release build in a container, best-effort, not a lab — I care about the ratio, 248 to 1, not the milliseconds.

Here's the whole thing if you want to watch it miss yourself. Drop it into a fresh dotnet new web project on .NET 10 and run dotnet run -c Release -- demo:

using System.Diagnostics; var builder = WebApplication.CreateBuilder(args); builder.Services.AddOutputCache(); var app = builder.Build(); app.UseOutputCache(); var defaultMisses = 0; var variedMisses = 0; // Default policy: varies by the ENTIRE query string. app.MapGet("/report", (int page) => { Interlocked.Increment(ref defaultMisses); return Results.Text($"page={page}"); }).CacheOutput(); // Pinned to the one query value that changes the body. app.MapGet("/report-varied", (int page) => { Interlocked.Increment(ref variedMisses); return Results.Text($"page={page}"); }).CacheOutput(p => p.SetVaryByQuery("page")); app.MapGet("/stats", () => Results.Text($"default={defaultMisses} varied={variedMisses}")); if (args.Contains("demo")) { _ = Task.Run(async () => { var baseUrl = "http://127.0.0.1:5090"; while (true) { try { using var ping = new HttpClient(); await ping.GetStringAsync($"{baseUrl}/stats"); break; } catch { await Task.Delay(100); } } using var http = new HttpClient(); string[] campaigns = { "spring", "newsletter", "twitter", "reddit", "email", "" }; var rng = new Random(42); for (int i = 0; i < 300; i++) { var utm = campaigns[rng.Next(campaigns.Length)]; var q = utm.Length == 0 ? "page=1" : $"page=1&utm_campaign={utm}&t={rng.Next(10000)}"; await http.GetStringAsync($"{baseUrl}/report?{q}"); await http.GetStringAsync($"{baseUrl}/report-varied?{q}"); } Console.WriteLine($"\n300 requests each, all page=1"); Console.WriteLine($"handler runs (cache misses): {await http.GetStringAsync($"{baseUrl}/stats")}"); await app.StopAsync(); }); app.Urls.Add("http://127.0.0.1:5090"); } app.Run();