Cache-Control Headers

Most SEOs spend weeks obsessing over keyword placement and completely ignore how their server delivers files. If your origin server has to regenerate the entire HTML document every time Googlebot requests a page, you are actively burning your crawl budget to the ground.

The Core Directives Every SEO Should Know

Stop letting your DevOps team guess what the caching policy should be. You need to understand the basic directives because they directly impact Core Web Vitals (specifically LCP and TTFB).

• max-age: This tells the browser exactly how many seconds it can use a cached version of a file before it needs to ask the server for a fresh one. Cache-Control: max-age=31536000 is the gold standard for static assets like logos or fonts (that equals one year).
• s-maxage: The "s" stands for shared. This directive is strictly for CDNs and proxy caches (like Cloudflare or Fastly). If you want Cloudflare to cache a page for a week, but force users' browsers to check every day, you use s-maxage=604800, max-age=86400.
• no-cache: This is the most misunderstood header in web development. It does not mean "do not cache this." It means the browser can physically store the file, but it must revalidate with the server (using an ETag) to ensure the file hasn't changed before showing it to the user.
• no-store: This is the nuclear option. It means absolutely nothing is allowed to save a copy of this file. Use this exclusively for checkout pages, bank dashboards, and internal user profiles.

The "stale-while-revalidate" Cheat Code

If there is one caching directive you should bully your engineering team into implementing, it is stale-while-revalidate.

Introduced a few years ago, this directive allows a CDN or browser to immediately serve a slightly stale (expired) version of a page to a user, while silently fetching the updated version in the background.

Cache-Control: max-age=600, stale-while-revalidate=86400

1. For the first 10 minutes (max-age=600), the page is fresh. Anyone asking for it gets the cached version instantly.
2. If someone requests the page at minute 11, the cache is technically stale.
3. Because of stale-while-revalidate, the server instantly hands the user the 11-minute-old page (zero latency, massive TTFB win).
4. In the background, the server asynchronously rebuilds the page and updates the cache. The next user gets the fresh version.

This completely masks backend database latency from both Googlebot and your users.

Pro-Tip: Cache Busting Your CSS Never use a short max-age on CSS or JavaScript files just because you update your site frequently. Set the max-age to a full year. When you push an update to your CSS, change the filename (e.g., main-v2.css or main.css?v=234). This forces the browser to fetch the new file while letting you keep the massive performance benefits of aggressive caching.

Cache-Control vs. Crawl Budget

Googlebot respects standard HTTP caching headers. If you have an e-commerce category page with a max-age of 5 minutes, Googlebot realizes the content is volatile and may attempt to crawl it constantly.

Conversely, if you leave default server settings that apply a no-cache header to your entire blog, Googlebot is forced to download the entire HTML payload from scratch every single time it crawls your site. For a million-page site, this will instantly exhaust your crawl rate limit, leaving your newly published articles unindexed for days.