Static Sites without the Static

Chris BallTuesday, December 31, 2019

A technique to get peak performance without waiting for rebuilds and preview URLs

Static sites, produced with tools like Gatsby, Hugo, and others, are popular for a reason. They deploy almost anywhere, are extremely fast, and don’t compromise on the developer experience. When paired with a CMS though, new sites must be built each time an editor makes an update in the UI.

This isn’t always a problem since static site rebuilds are fast, but it does mean that that tooling needs to be inserted after changes are published to perform the rebuild. Most CMS providers offer webhooks to power this functionality.

Example webhooks from Contentful

Recently, a number of services that have sprung up recently to offer “preview’ or staging versions of a site, and some that attempt to keep the entire preview client-side as part of the editor. But as good as these are, it does mean that an editor (who is often a non-technical user) needs to sit through a rebuild process to see changes they have made.

Recently, I set out to explore a technique to allow instant updates from a CMS-backed site on staging sites, while keeping the production version static. Let’s dive in and see how that technique works.

A look at the tools we’ll use

Next.js

Supports Server Side Rendering out of the box
Has simple “API” support via an /api folder. While we won't leverage it for this article, most CMS-backed sites will need to process simple forms and this is a big selling point.
Can optionally export a fully static site. More on this at the end of the article.

graphql-request

A simple wrapper for fetch to make GraphQL queries

Now (optional)

Provides multi-cloud redundancy out of the box
Supports a CDN caching layer, which we can intelligently set via Server Side Rendering

Start with the dynamic, Server Side Rendered page

First, you’ll need to setup a Next.js site. If you’re starting from scratch, I’d recommend using create-next-app.

Then, set up a GraphQL client using graphql-request:

import { GraphQLClient } from 'graphql-request';
const endpoint = process.env.API_URL;
const token = process.env.API_TOKEN;

export const graphQLClient = new GraphQLClient(endpoint, {
  cache: 'no-cache',
  credentials: 'include',
  mode: 'cors',
  headers: {
    authorization: `Bearer ${token}`,
  },
});

After that, set up a Next.js page that fetches data from the CMS using GraphQL.

import React from "react";
import Head from "next/head";
import { NextPageContext } from "next";
import { graphQLClient } from "../../config/graphql";
import { SeoMetaTags } from "../../graphql/fragments/SeoMetaTags";

const PAGE_DATA_QUERY = `
  query ServicesQuery {
    servicesPage {
      ${SeoMetaTags}
      name
    }
  }
`;

interface Props extends NextPageContext {
  data: Record<string, any>;
}

/**
 * A page that lists all services we offer
 */
const Services = ({ data }: Props) => {
  const tags = data._seoMetaTags;
  const pageData = data.servicesPage;

  return (
    <>
      <Head>
        <SeoMetaTags tags={tags} />
      </Head>

      {pageData.services.map(s => (
        <h3>{s.name}</h3>
      ))}
    </>
  );
};

Services.getInitialProps = async ({ res }) => {
  const data = await graphQLClient.request(PAGE_DATA_QUERY);

  return { data };
};

In a staging environment, we need to point graphql-request to our CMS's "draft" endpoint (most CMS's support this workflow). Whenever an editor saves a change to the CMS, they can reload the page to see the changes.

Remove the need to reload

Reloading the page to see updates works well, but gets tedious for the user that is previewing the changes. What if we could build in a “live reload” feature, that automatically refreshes the staging site after changes are saved? To accomplish a live reload feature, we can:

leverage a SSR response header to determine if page data has changed
add a focus hook that will asynchronously fetch the page in the background and compare the new version header

Set the version header

To set the version header, we will create a hash of the data after it is fetched during Server Side Rendering. We’ll create a utility to keep things clean.

// utils/setVersionHeader.ts

import { ServerResponse } from 'http'

/**
 * Sets an `X-version` header that is used to determine when CMS data has changed.
 * @param data The page data
 * @param res The response from the SSR server
 */
export function setVersionHeader(data: Record<string, any>, res: ServerResponse): string {
  // calculate a hash based on the passed data
  const pageDataHash = require('crypto')
    .createHash('md5')
    .update(JSON.stringify(data))
    .digest('hex');

  if (res) {
    res.setHeader('X-version', pageDataHash);
  }

  return pageDataHash;
}

Add hooks that implement focus and reload behavior

The first hook we will add will simplify the handling of browser focus events.

// hooks/useFocus.ts

import { useState, useEffect } from 'react';

/**
 * Hook that returns true if the window is focused and
 * false if not.
 */
export function useFocus() {
  const [state, setState] = useState(null);

  const onFocusEvent = () => setState(true);
  const onBlurEvent = () => setState(false);

  useEffect(() => {
    window.addEventListener('focus', onFocusEvent);
    window.addEventListener('blur', onBlurEvent);

    return () => {
      window.removeEventListener('focus', onFocusEvent);
      window.removeEventListener('blur', onBlurEvent);
    };
  });

  return state;
}

And a hook that wraps useFocus that checks the version header and reloads the page if necessary:

// hooks/useFocusReload.ts

import { useEffect } from 'react';
import { useFocus } from './useFocus';

/**
 * This works with the useFocus hook to reload the page if data has changed.
 * It works by fetching the current page async, and comparing the X-Version
 * header to the current one (passed in as pageDataHash)
 */
export function useFocusReload(pageDataHash: string) {
  const focused = useFocus();

  useEffect(() => {
    async function handleFocus() {
      if (focused) {
        const res = await fetch(window.location.href, {
          headers: {
            pragma: 'no-cache',
          },
        });

        if (res.ok && res.headers.get('X-version') !== pageDataHash) {
          window.location.reload();
        }
      }
    }

    handleFocus();
  }, [focused]);
}

Here’s how to use the hooks we just put in place:

// pages/services.tsx

import React from "react";
import Head from "next/head";
import { NextPageContext } from "next";
import { graphQLClient } from "../../config/graphql";
import { SeoMetaTags } from "../../graphql/fragments/SeoMetaTags";
import { useFocusReload } from "../../hooks/useFocusReload";
import { setVersionHeader } from "../../utils/setVersionHeader";

const PAGE_DATA_QUERY = `
  query ServicesQuery {
    servicesPage {
      ${SeoMetaTags}
      name
    }
  }
`;

interface Props extends NextPageContext {
  data: Record<string, any>;
}

/**
 * A page that lists all services we offer
 */
const Services = ({ data, pageDataHash }: Props) => {
  useFocusReload(pageDataHash);

  const tags = data._seoMetaTags;
  const pageData = data.servicesPage;

  return (
    <>
      <Head>
        <SeoMetaTags tags={tags} />
      </Head>

      {pageData.services.map(s => (
        <h3>{s.name}</h3>
      ))}
    </>
  );
};

Services.getInitialProps = async ({ res }) => {
  const data = await graphQLClient.request(PAGE_DATA_QUERY);
  const pageDataHash = setVersionHeader(data, res);

  return { data, pageDataHash };
};

export default Services;

To recap what is going on:

1. An x-version header will be set during Server Side Render, based on the CMS API response data. 2. When the browser receives a focus event, the current page is fetched again in the background 3. If the header has changed, we reload the page. If not, we do nothing.

Note: This same technique can be leveraged on any Single Page App to notify users that a new version has been released, and that they should reload the page. In that case, you would typically add a confirmation prompt before reloading on their behalf.

After deploying, you should be able to verify the x-version header was set in the browser.

an example of the x-version header

(an example of the x-version header)

Moving to production on Now

Up to this point, we’ve created a Server Rendered Next.js app that will automatically reload on focus if the page data has been updated in our CMS. The final step is to bring the high performance of a static site to our page.

To do that, we will leverage a built in feature built into the Now platform called Serverless Pre-Rendering. Similar to the header approach above, we’re going to set a special cache header to cache our pages at the CDN layer. In practical terms, that means our page will be as fast as a static site for the duration that it is cached.

Now’s CDN will send a cached version of the page to the user. If the cache time has expired, it will asynchronously fetch and re-cache the page using a Serverless lambda in the background. This is awesome, because it avoids the “slow for 1 user every x seconds” problem you encounter with many caching strategies.

Let’s add a util to set the proper cache header during Server Side Rendering.

// utils/cachePageFor.ts
import { ServerResponse } from "http";

/**
 * This sets a cache header for the specified amount of time. Time is in seconds.
 */
export function cachePageFor(cacheTime: number = 60, res: ServerResponse) {
  if (cacheTime && res) {
    res.setHeader(
      "Cache-Control",
      `s-maxage=${cacheTime}, stale-while-revalidate`
    );
  }
}

The important bits here are s-maxage and stale-while-revalidate. we set s-maxage to the number of seconds we'd like the CDN to serve a cached page, and stale-while-revalidate powers the async cache update strategy mentioned above.

To use the util, we set a cache time on a per-page basis. For example, we might want our homepage to only be cached for a max of one minute before refetching, but our contact page could be cached significantly longer.

Services.getInitialProps = async ({ res }) => {
  const data = await graphQLClient.request(PAGE_DATA_QUERY);
  const pageDataHash = setVersionHeader(data, res);
  cachePageFor(1200) // 20 minutes

  return { data, pageDataHash };
};

Once you deploy this code to staging, you should be able to verify that a cached response was returned:

an example of the x-version header

What happens if you deploy a code update during this time?

Now will intelligently clear caches for you every time you deploy! See https://zeit.co/docs/v2/network/caching/#cache-invalidation for more detail.

Alternate approach: use Next.js’s static export

What if you can’t or don’t want to host on Now? As I mentioned earlier, Next.js also supports exporting a static site. Using this, we can change our build pipeline to export a static version for production.

Unfortunately, this requires adding some configuration to your app, especially if you have dynamic pages. Here’s an example config that filters out dynamic pages and populates sub-pages based on the results of a GraphQL query:

// next.config.js

const GraphQLClient = require('graphql-request').GraphQLClient;
const endpoint = process.env.API_URL;
const token = process.env.API_TOKEN;

const graphQLClient = new GraphQLClient(endpoint, {
  headers: {
    authorization: `Bearer ${token}`,
  },
});

const DYNAMIC_PAGES_QUERY = `
  query services {
    allServices {
      detailPage {
        slug
      }
    }
  }
`;

module.exports = {
  exportPathMap: async function(defaultPathMap, { dev, dir, outDir, distDir, buildId }) {
    // filter out all dynamic pages (ex: services/[slug].tsx)
    const filteredKeys = Object.keys(defaultPathMap).filter(key => !/\[/.test(key));
    let filteredPathMap = {};
    filteredKeys.forEach(key => (filteredPathMap[key] = defaultPathMap[key]));

    // Fetch data for dynamic pages
    const pages = await graphQLClient.request(DYNAMIC_PAGES_QUERY);

    // dynamically add services pages that have a detail page
    const servicesPages = pages.allServices.reduce((obj, service) => {
      const slug = service.detailPage ? service.detailPage.slug : null;

      if (slug) {
        obj[`/services/${slug}`] = { page: '/services/[slug]', query: { slug } };
      }

      return obj;
    }, {});

    return {
      ...filteredPathMap,
      ...servicesPages,
    };
  },
};

You’ll also want to add a hook from your CMS that triggers an export and deployment. Generally, it’s best to leverage a script in package.json to keep things clean.

example of setting up a hook to build the production site statically

(example of setting up a hook to build the production site statically)

You could also leverage the custom webhook to run a workflow on GitHub Actions that accomplishes the same thing.

Enjoy your non-static static site!

We now have what seems to be the best of both worlds. Our staging site acts like a traditional server that reflects changes as soon as they are saved in the CMS. Due to the CDN caching, our production site is just as fast as a static site, with the added bonus of never being out of date for longer than our cache time. We never need to wait on a site rebuild.

We’re currently rebuilding the Echobind site, which leverages this technique. Look for it to be released soon!

Thanks to ZEIT

Massive props to the team at ZEIT for not only adding this feature to the Now platform, but also for writing a blog post on Serverless Pre-Rendering that demonstrated the focus reload technique.

Static Sites without the Static

A technique to get peak performance without waiting for rebuilds and preview URLs

A look at the tools we’ll use

Next.js

Now (optional)

Start with the dynamic, Server Side Rendered page

Remove the need to reload

Set the version header

Add hooks that implement focus and reload behavior

Moving to production on Now

What happens if you deploy a code update during this time?

Alternate approach: use Next.js’s static export

Enjoy your non-static static site!

Thanks to ZEIT

Custom React Hook Guide: Build a Kanye Quote app with React Native

Running Cypress tests in parallel in GitHub Actions

GraphQL — the Dev Edition

Dynamic CMS-driven Redirects with Next.js

Getting Started with Next.js, GraphQL and React Query (1/3)

The Graphcool Framework: A path to instant GraphQL greatness

Interested in working with us?