How to do ISR and advanced caching with Astro

Incremental static regeneration is a powerful pattern for rendering pages on the web, delivering dynamic content at the speed of static. Astro on Netlify lets you implement on-demand ISR with regular HTTP headers, keeping your content fast and fresh with just a few lines of code.

#TL;DR

You can deliver high-performance sites while keeping your content up-to-date by using Astro with Netlify’s advanced cache control features. In this guide, we’ll explain how and why to use different caching options, and show the code you’ll need to get fine-grained control over caching behaviours in an Astro site.

#What is ISR?

Incremental Static Regeneration or ISR is an approach to page rendering that lets you cache a page when first requested, and then control when that page is regenerated - either after a specific period, or by manually revalidating it. It was popularized by Next.js, which implements it internally within the framework, but it’s a pattern that can be implemented with any framework in a standards-based way using HTTP headers and a CDN that supports them.

#Why use ISR

By default Astro renders pages as static HTML, which means they’re super fast and easily cached. However it also means that the whole site needs to be rebuilt whenever content changes, and this can take some time on large content-driven sites. In these cases it’s more efficient to use server-side rendering, which renders the page only when it is requested. This is better for builds, but means that the performance is not as good as a fully static site. Luckily you can have the best of both worlds if you use an advanced CDN cache: pages that are rendered efficiently at runtime with the latest content, but are delivered quickly from the cache.

#How to implement ISR using Astro

A modern CDN like Netlify’s edge network gives you powerful tools to control your site’s cache using just HTTP headers. Astro lets you set headers on a page-by-page basis, so you can control how each page is cached and when it is regenerated. On-demand revalidation on Netlify lets you refresh just the pages that have changed, keeping your site fast and fresh.

When using server-side rendering with Astro you can set headers using the Astro.response.headers object in your page. Setting the Cache-Control header allows you to control how the page is cached, both in the browser and CDN.

---
Astro.response.headers.set("Cache-Control", "public, max-age=300, s-maxage=3600");
// Sets the number of seconds to cache the page in the ☝️ browser and ☝️ the CDN
---

This caches the response in the browser (max-age) for 5 minutes, and in the CDN (s-maxage) for 1 hour.

This is basic caching: if a user loads the page within an hour they will get the cached response, but if it’s more than an hour then it will block while it re-renders.

If the data is unlikely to change, you can set a long expiry time. Don’t worry if you find you need to change it later – any new deploy will clear the CDN cache instantly and your visitors will get the fresh content, without broken asset links. It’s important that when you do this it’s only for the CDN cache, not the browser cache, because otherwise your users may be left with stale content in their cache and you would have no way to invalidate it.

#Optimizing: durable and stale-while-revalidate

What if your page is slow to render? Perhaps it needs to load data from a slow API or database, plus running a serverless function to render a page always involves some extra time vs. serving from cache. Fortunately, two extra cache directives come in really useful here:

First, let’s enable Nelify’s Durable Cache using the durable directive. Without it, each edge node only has its own local cache to consult before having to invoke a serverless function to generate the page. By specifying durable, an additional layer will cache responses from serverless function and make these available to all edge nodes globally.

Second, let’s add a specific optimization for pages that need revalidation: the stale-while-revalidate directive. This does what it says on the box: if the cached page has expired, the edge node will send a stale version while it revalidates the page in the background. The page load is never blocked for the user, though it won’t be perfectly fresh for everyone. Of course, you control exactly how much “staleness” is ok for the page.

We can add these directives to the Cache-Control header, but there is a potential risk here:

While durable is a Netlify-specific optimization which is ignored by any other software, the stale-while-revalidate directive might also affect caching in the browser, so you may end up with conflicting versions or double-revalidations taking place. Luckily, there is a standard solution for this: the CDN-Cache-Control header, which is used only by CDN providers and never by browsers. And since some customers have a less-common setup where Netlify is running behind another CDN (typically due to a gradual migration process), any further caching conflicts between these can be avoided by using the Netlify-specific header Netlify-CDN-Cache-Control, as we’ll do in the example below.

Using the CDN-only header also helps solve another caching issue. When a new version of a site is deployed, Netlify’s CDN automatically invalidates all related files, but how would the browser know? As long as it uses cached pages, it might ask for additional assets that either don’t exist anymore, or do exist but have deprecated contents. The solution lies in separating concerns between browser and CDN:

• The browser only uses the regular Cache-Control header, so we’ll change its value to ensure that the browser always checks the freshness of its cache against the server. Thanks to conditional requests, only content that has actually changed would be downloaded.

• Netlify would handle the actual caching logic for your page, including all revalidation cases.

Here’s an example:

---
// Tell the browser to always check the freshness of the cache
Astro.response.headers.set("Cache-Control", "public, max-age=0, must-revalidate");

// Tell Netlify's CDN to treat it as fresh for 5 minutes, then for up to a week return a stale version
// while it revalidates. Use Durable Cache to minimize the need for serverless function calls.
Astro.response.headers.set(
  "Netlify-CDN-Cache-Control", "public, durable, s-maxage=300, stale-while-revalidate=604800"
);
---

#On-demand revalidation with cache tags

While s-maxage and stale-while-revaidate are a nice and easy way to implement ISR on Astro, we’re not really using the power of Astro and the CDN. Up until now we’ve just been caching for a fixed amount of time. This is OK if you don’t know when a data source has updated, but if we can get notifications when it has changed, we can revalidate the cache for only the page where it is needed. This is because Netlify supports the Cache-Tag header, with fine-grained revalidation by tag. Netlify also supports the Netlify-Cache-Tag header with the same meaning but specific to just Netlify’s CDN, and which will not be visible to the browser. Most CMSs and ecommerce platforms support webhooks to report content changes, so you can use this to trigger a revalidation of the cache for only the pages that need it.

To see how cache tags work, let’s look at an example. Cache tags let you mark each page with a tag that can then be used to invalidate the cache for any page with that tag. One of the most powerful uses for this is to tag a page with the IDs of the data that it uses. This means that when that data is changed, you can invalidate only the pages that use that data. This is a powerful way to do ISR, because you can keep the cache fresh for only the pages that need it, and not waste time re-rendering pages that haven’t changed. If you edit one entry in the CMS, the cache will be invalidated just for pages that use that data and not the whole site.

#How to revalidate your pages on demand with cache tags and webhooks

This example shows how to use Contentful webhooks to do on-demand revalidation, but you can use a similar approach with any CMS.

The first step is to tag each page with the data IDs. Here’s an example of how you might do this with Contentful using the Contentful Astro Starter:

---
import { client } from "../../lib/contentful";

const { slug } = Astro.params;
let book;

try {
  book = await client.getSingleBook(slug);
} catch (error) {
  return Astro.redirect("/404");
}
// The browser should always check freshness
Astro.response.headers.set("cache-control", "public, max-age=0, must-revalidate");

// The CDN should cache for a year, but revalidate if the cache tag changes
Astro.response.headers.set("netlify-cdn-cache-control", "s-maxage=31536000");

// Tag the page with the book ID
Astro.response.headers.set("netlify-cache-tag", slug);
---

This sets the cache tag to the page slug, which in this example is the object ID. When the book is updated, the cache for this page is invalidated.

Next you can use a webhook to revalidate the cache for this page when the book is updated:

import { purgeCache } from "@netlify/functions";

export async function POST({ request }) {
  const body = await request.json();

  // See below for information on webhook security
  if (request.headers.get("X-Contentful-Webhook-Secret") !== process.env.CONTENTFUL_WEBHOOK_SECRET) {
    return new Response("Unauthorized", { status: 401 });
  }

  await purgeCache({ tags: [body.sys.id] });

  return new Response(`Revalidated entry with id ${body.sys.id}`, { status: 200 });
}

This uses the purgeCache function from the @netlify/functions package to invalidate the cache for the book and all pages that use it. You can then set up a webhook in Contentful to call this function when a book is updated.

Next, create a webhook that posts to /api/webhook.json. Choose “Select specific triggering events” and choose “Entry”.

#Invalidating collection pages

A page doesn’t need to just return a single cache tag: it can return multiple tags separated with commas if it uses multiple data entries. In this demo the index page might will to be invalidated every time any book is changed, because it lists all of them. Here we could tag it with every book ID, but on most sites that would be a lot of tags. Instead, we can add a different tag for the whole data type. Because tags have no semantic meaning, you can call this whatever you want, but it’s simplest to use the data type name. We can show this on the index page:

---

import { client } from "../../lib/contentful";

const books = await client.getAllBooks();

// The browser should always check freshness
Astro.response.headers.set("cache-control", "public, max-age=0, must-revalidate");

// The CDN should cache for a year, but revalidate if the cache tag changes
Astro.response.headers.set("netlify-cdn-cache-control", "s-maxage=31536000");

// Tag the page with the content type
Astro.response.headers.set("netlify-cache-tag", "books");
---

Let’s update the webhook to invalidate the cache for the whole content type as well as the ID of the book:

await purgeCache({ tags: [body.sys.id] })
await purgeCache({ tags: [body.sys.id, 'books'] })

This will now purge the index page (because it’s tagged with “books”) as well as the individual book page when a book is updated.

#Fine-grained cache control

So far we’ve been caching each page as the same for each visitor. This is probably what you want for most public pages, but not always. The Vary header can be used to store different versions of a page according to the request headers. However this can be quite an inflexible instrument, so Netlify extended this with the Netlify-Vary header.

An important case will be to control how query parameters are treated. The strict spec-compliant way that a cache will handle these is to treat the whole URL complete with query as the cache key, meaning that ?foo=bar and ?foo=baz will be treated as different pages. While this is what you want in some cases, there are a lot of times when this will break caching entirely. A very common example for this is social sharing. Links from social media will often have tracking parameters added automatically, and you don’t want to cache a different copy of the page for each visitor. This is also the case if you are using ads with tracking IDs. Should ?fbclid=nnnn... be stored as a separate page than ?fbclid=mmmm...? Probably not. However you might be using the query param for something that you do want to use for the cache.

A typical example might be something like sorting, filtering or pagination. Netlify lets you control exactly which query params are used for the cache key using the Netlify-Vary header. Here’s how you can use it on a listings page:

Astro.response.headers.set("Netlify-Vary", "query=sort|page|per_page");

In this example it will treat pages with different values for sort, page and per_page as different pages, but will ignore any other query params. This is a powerful tool for controlling your cache, and can be used in combination with cache tags for even finer control.

#Cookies and caching

If there are user preferences that change how a page is displayed they needn’t break caching. You probably shouldn’t vary on the whole Cookie header, because this will mean that every user need their own version of the page cached, but you can use the Netlify-Vary header to control exactly which cookies are used for the cache key. Here’s an example using a cookie to control the theme:

Astro.response.headers.set("Netlify-Vary", "cookie=theme");

You can use this to choose cookies with low cardinality (i.e. ones that have a small number of possible values) to vary the cache on, which means you can have customized pages while still caching them. If there is a light theme and a dark theme, each of these will then be cached separately. You shouldn’t use it for cookies that have high cardinality, like session or user IDs.

#Debugging your cache

Advanced cache tools like this can be tough to debug. The Cache-Status header can help, by showing whether the CDN had a had a hit, miss or stale response. Netlify supports this header, and you can use it to see what’s happening with your cache.

Astro gives you powerful tools to control your cache, when combined with a platform like Netlify that supports advanced caching features. This gives you the power to control your cache in a way that’s standards-based and uses the web platform.

#Get started

If you’d like to see a real example, take a look at the Astro bookshelf demo, which is based on the main Astro Contentful starter but with the addition of advanced cache control.

To create your own Astro site to deploy to Netlify, you can get started by running npm create astro@latest and then npx astro add netlify to add Netlify to your project.

Or if you prefer, you can also deploy an example directly to Netlify from here by clicking the button below.

For more details see the Astro on Netlify docs.