Vary Header Pitfalls with Fingerprinted Assets

A fingerprinted file is already content-addressed: app.a1b2c3d4.js names exactly one byte sequence, forever. Vary tells every shared cache that the same URL can legitimately return different bytes depending on a request header — which is the opposite claim. When both are true at once, the edge does the only thing it can: it stores one copy per header value, and the hit ratio for that asset collapses while nothing about the file has changed.

Symptom: The Hit Ratio Falls Without a Deploy

The failure has a distinctive signature. Origin egress rises, the Age header on asset responses stays suspiciously low, and edge analytics show a hit ratio that dropped on a day with no asset changes. Nothing in the build pipeline is at fault — the hashes are stable, the TTLs are a year, and a curl of the URL returns exactly the right bytes. The key simply stopped being one key.

Confirm it in one command before investigating anything else:

curl -sI https://example.com/assets/app.a1b2c3d4.js | grep -iE "^(vary|cache-control|age|cf-cache-status):"

Any Vary value other than Accept-Encoding on a hashed path is a defect. The rest of this page explains what each directive costs, how the three major edges interpret it, and how to narrow it back down.

Why Vary Multiplies a Key That Is Already Unique

The edge cache key is a composite of request attributes; the cache key reference covers how scheme, host and path combine. Vary extends that composite with response-declared dimensions. Crucially it is multiplicative, not additive: three encodings and forty distinct User-Agent buckets produce 120 objects, each of which must be populated by its own cold origin fetch before it can serve a hit.

There is a second, quieter cost. Most edges apply a per-object size or count ceiling to the variant set they will track for a single URL. Past that ceiling, the edge stops caching the URL altogether rather than storing more variants — so an over-broad Vary does not merely reduce the hit ratio, it can drive it to zero for that path while every other asset looks healthy.

Directive-by-Directive Reference

The rule that falls out of the table: a fingerprinted asset should declare the narrowest Vary that still describes a real difference in the response body. For JavaScript, CSS, fonts and media that is Accept-Encoding and nothing else. For negotiated images it is Accept, Accept-Encoding. There is no third case.

How Cloudflare, Fastly and CloudFront Treat Vary

The three edges diverge enough that a Vary value which is merely wasteful on one is actively destructive on another.

Cloudflare does not honour arbitrary Vary on cacheable assets. Compression negotiation is handled internally rather than through the header, and Vary: User-Agent is discarded rather than turned into distinct objects — so a stray directive costs you nothing at the edge but still misleads every browser and corporate proxy downstream. The one directive it does act on is Cookie: a response varying on it is treated as private and skips the cache entirely. Cache Rules are where you correct all of this, as covered in the Cloudflare cache rules reference.

Fastly implements Vary faithfully. Whatever the origin declares becomes a real bucketing dimension in the object store, so Vary: User-Agent produces genuinely thousands of objects unless you normalise the header in vcl_recv before the hash is computed. Fastly is therefore the edge where an accidental directive costs the most, and also the one where the fix is most direct: rewrite the request header to a small enumerated set.

CloudFront ignores the origin’s Vary for key construction and uses the cache policy instead. Headers enter the key only if the policy lists them, and compression variants only if EnableAcceptEncodingGzip and EnableAcceptEncodingBrotli are set. The trap is the inverse of Fastly’s: an origin emitting Vary: Accept-Encoding under a policy that has both flags off will serve one compressed variant to every client regardless of what they asked for.

Accept and Image Content Negotiation

Accept is the one non-encoding directive with a defensible use on a static path. An origin that inspects Accept: image/avif and returns AVIF, WebP or JPEG from a single URL is genuinely returning different bytes, and it must declare Vary: Accept or shared caches will serve AVIF to clients that cannot decode it.

The cost is that raw Accept values are nearly as diverse as User-Agent. Browsers send long, version-specific lists, so varying on the unmodified header fragments the cache almost as badly. Normalise before the key is built — collapse the header to a two- or three-value enumeration — and emit Vary: Accept only on the negotiated image prefix.

The alternative, and the better default for a fingerprinted pipeline, is to skip negotiation entirely: emit hero.a1b2c3d4.avif and hero.a1b2c3d4.webp as separate hashed files and let <picture> choose. One object per format, zero variance, no directive required. Negotiation is worth its complexity only when the markup cannot be changed.

Emitting the Narrowest Possible Vary

Nginx origin. Set the header on the fingerprinted prefix and strip whatever the application layer added. proxy_hide_header removes the upstream value before add_header writes the replacement; without it, both end up on the response.

# Fingerprinted assets: exactly one Vary dimension
location ~* ^/assets/[a-z0-9._-]+\.[a-f0-9]{8,}\.(js|css|woff2)$ {
    root /var/www/current;
    gzip_static  on;
    brotli_static on;

    proxy_hide_header Vary;
    add_header Vary          "Accept-Encoding" always;
    add_header Cache-Control "public, max-age=31536000, immutable" always;
    add_header Access-Control-Allow-Origin "*" always;
}

# Negotiated images: the only place Accept is allowed
location ~* ^/img/[a-z0-9._-]+\.[a-f0-9]{8,}\.(avif|webp|jpg)$ {
    root /var/www/current;

    proxy_hide_header Vary;
    add_header Vary          "Accept, Accept-Encoding" always;
    add_header Cache-Control "public, max-age=31536000, immutable" always;
}

Cloudflare Cache Rule. Pin the key to path plus encoding for the asset prefix so no origin header can widen it. Terraform keeps the rule versioned alongside the deploy:

resource "cloudflare_ruleset" "asset_vary_guard" {
  zone_id = var.zone_id
  name    = "Narrow cache key for fingerprinted assets"
  kind    = "zone"
  phase   = "http_cache_settings"

  rules {
    action = "set_cache_settings"
    action_parameters {
      cache = true
      edge_ttl {
        mode    = "override_origin"
        default = 31536000
      }
      cache_key {
        cache_by_device_type = false
        custom_key {
          query_string { include = [] }
          header {
            include        = ["accept-encoding"]
            exclude_origin = true
          }
          cookie {}
          user {
            device_type = false
            geo         = false
            lang        = false
          }
        }
      }
    }
    expression  = "(http.request.uri.path matches \"^/assets/.*\\.[a-f0-9]{8,}\\.(js|css|woff2)$\")"
    description = "Path plus encoding only"
    enabled     = true
  }
}

exclude_origin = true is the important line: it tells the edge to build the key from the listed headers alone and disregard whatever Vary the origin declared.

CloudFront cache policy. The policy is the whole story — the origin’s Vary is advisory. This one caches on path and the two compression flags, and forwards no headers, cookies or query strings:

{
  "CachePolicyConfig": {
    "Name": "assets-narrow-vary",
    "Comment": "Fingerprinted assets: path plus encoding only",
    "DefaultTTL": 31536000,
    "MaxTTL": 31536000,
    "MinTTL": 31536000,
    "ParametersInCacheKeyAndForwardedToOrigin": {
      "EnableAcceptEncodingGzip": true,
      "EnableAcceptEncodingBrotli": true,
      "HeadersConfig": { "HeaderBehavior": "none" },
      "CookiesConfig": { "CookieBehavior": "none" },
      "QueryStringsConfig": { "QueryStringBehavior": "none" }
    }
  }
}

aws cloudfront create-cache-policy --cache-policy-config file://assets-narrow-vary.json

For negotiated images, clone the policy with "HeaderBehavior": "whitelist" and a single item, Accept. Do not attach that variant to the JS or CSS behaviour.

Verification with curl -I

One command tells you whether the header is right, and a second tells you whether the edge is acting on it:

# 1. What does the origin declare, and how is the edge storing it?
curl -sI https://example.com/assets/app.a1b2c3d4.js \
  | grep -iE "^(vary|cache-control|cf-cache-status|x-cache|age):"

# Expected on a healthy fingerprinted asset:
#   vary: Accept-Encoding
#   cache-control: public, max-age=31536000, immutable
#   cf-cache-status: HIT
#   age: 21600

# 2. Prove the encodings are separate objects and nothing else is
for enc in br gzip identity; do
  printf '%-9s ' "$enc"
  curl -sI -H "Accept-Encoding: $enc" https://example.com/assets/app.a1b2c3d4.js \
    | grep -iE "^(content-encoding|cf-cache-status)" | tr -d '\r' | paste -sd' '
done

# 3. Prove User-Agent does NOT split the cache
for ua in "Mozilla/5.0 (Windows NT 10.0)" "Mozilla/5.0 (iPhone)" "curl/8.5.0"; do
  curl -sI -A "$ua" https://example.com/assets/app.a1b2c3d4.js \
    | grep -iE "^(cf-cache-status|age):" | tr -d '\r' | paste -sd' '
done

Step 2 should show each encoding returning its own content-encoding with a HIT — three objects, all warm. Step 3 should show the same non-zero age for all three agents. If the age resets to 0 when the agent string changes, User-Agent is in the key and the cache is fragmenting.

Detecting Fragmentation from Hit-Ratio Data

Header inspection catches the problem on a URL you already suspect. Hit-ratio data catches it before anyone reports slowness, and the shape of the regression is unmistakable: a step change on a single day, no recovery, no corresponding asset change.

Three signals separate a Vary regression from ordinary cache churn:

• The drop is a step, not a slope. Traffic-mix shifts and TTL expiries move the ratio gradually. A new key dimension applies to every subsequent request at once.
• Origin egress rises proportionally while unique URLs stay flat. Your logs show the same set of hashed paths being fetched, just far more often. Group origin requests by path and compare the count of distinct paths against the request total — a rising ratio of requests per unique path is the fingerprint.
• The plateau is stable. Once every variant is populated the ratio stops falling and holds at whatever fraction the variant set allows. A ratio that has been flat at an implausible number for days is a fragmented key, not a warming cache.

Cross-check against your TTL configuration before concluding: a genuinely short max-age produces a similar plateau for a completely different reason, and the TTL tuning guide covers how to tell the two apart.

When to Reconsider

Narrowing Vary is right for content-addressed files, but the directive exists for good reasons and there are paths where widening it is correct:

• HTML entry points. These are not fingerprinted and legitimately differ by locale or session. Vary: Cookie on a personalised document is correct — the alternative is serving one user’s rendered page to another.
• Fonts served cross-origin under a restrictive CORS policy. If Access-Control-Allow-Origin is echoed per requesting origin rather than fixed, Vary: Origin is mandatory. The cleaner fix is a single fixed value, since a hashed font has nothing worth restricting.
• A legacy origin you cannot modify. If the application server insists on emitting Vary: User-Agent, override the key at the edge rather than fighting the origin. Cloudflare’s exclude_origin and CloudFront’s cache policies both let the edge ignore the declaration while it still reaches browsers.
• Deliberately differentiated bundles. If you really do ship different JavaScript to different client classes, give each its own hashed path. Distinct URLs are cheaper, more debuggable, and observable in logs in a way that variant buckets never are — the reasoning is the same as for naming assets by content.

Frequently Asked Questions

How do I clear the variant objects a bad Vary already created?

Correct the origin header first, then purge the affected URLs by path. A purge removes every variant stored under that key, so one purge per URL is enough regardless of how many buckets accumulated. If the fragmentation spans a whole release, purge by cache tag or surrogate key rather than enumerating paths — the tag matches the object, not the variant. Skipping the purge leaves the old buckets resident until their TTL expires, which on an immutable asset means a year of stale variance.

Is Vary: Accept-Encoding still needed if the CDN compresses on the fly?

Emit it anyway. Cloudflare and CloudFront both handle negotiation without reading the header, but the header is not only for them: browser caches, corporate proxies and any origin shield in the path use it to decide whether a stored response may be reused for a differently-encoded request. Omitting it is the classic cause of a brotli-only client receiving gzip bytes from an intermediate cache that stored a single variant.

Does Vary affect the browser cache as well as the edge?

Yes, and this is why a directive Cloudflare discards is still worth removing. A browser will not reuse a stored response when a header named in Vary differs from the stored request. Vary: User-Agent therefore forces a re-download after any browser update, defeating immutable on a file whose bytes never changed. The edge damage is recoverable with a purge; the browser damage is silent and shows up only as unexplained repeat downloads in field data.

Related

• Cache key architecture — parent reference: how the edge assembles a key and which components belong in it
• Fingerprinting in HTTP headers — the full response-header contract for immutable assets
• Cloudflare cache rules and purge — overriding key construction and clearing fragmented variant sets
• Cache-Control immutable and TTL tuning — separating a short TTL from a fragmented key when diagnosing a low hit ratio
• Static asset fingerprinting fundamentals — section overview