10 Migrating to the Jamstack

This chapter covers

• Understanding migration challenges
• Solving build-time issues on large sites
• Strategies to migrate existing site functionality
• Exploring popular third-party Jamstack services
• Tips and strategies from the Jamstack community for making a migration

You’ve made it to the last chapter, and hopefully, by now you’re convinced about the benefits of using the Jamstack for your web development projects. You may even want to migrate an existing project to the Jamstack. The good news is that this is entirely feasible and, in fact, quite common. The bad news is that there’s no one-size-fits-all tool or wizard that can handle the migration; it is, for the most part, a customized solution for every migration.

There are so many factors that go into a migration, each one unique. This chapter aims to give you some guidance for planning your migration, as well as some resources and tools that can help ease the process. This process is more than just a technical challenge. It can involve a lot of stakeholders and other organizational considerations that you’ll have to manage, but we will primarily focus on the technical migration aspects.

10.1 What kind of site are you migrating?

Often developers want to just jump in, pick a tool, and start building—especially when they are excited about utilizing new technologies—but it’s critical to first take a step back and evaluate your existing site. This will guide you in choosing the right tools for the job. Jamstack is a fast-moving area of development, with lots of new tools that generate a lot of developer interest. But just because a particular solution has a lot of buzz doesn’t mean it is the right tool to build your application. However, just because a Jamstack tool has been around for a while doesn’t disqualify it as a solution, either.

Let’s look at three broad categories of sites that your project may fall into:

• Content-heavy sites

• Web applications

• Large sites

Keep in mind that your project may fall into more than one category. For instance, you may have large, content-heavy sites, or different parts of your site may fall into unique categories. For example, you may have a content-heavy information site that is also a web application accessible after the user authenticates.

10.1.1 Content-heavy sites

Many, if not most, websites are focused on delivering content rather than functionality. This category could include everything from a blog, product, or service documentation; a large corporate site with a lot of informational pages; or a news site that constantly updates. Here are some things to consider when migrating a content-heavy site:

• Do I need to migrate existing content? This depends on what you are migrating from and what you need to migrate to. For instance, in chapter 9 we looked at how you can integrate the headless WordPress API into an existing Jamstack site. Drupal, another popular CMS, can also be used as a headless CMS (http://mng.bz/XWyp). In fact, many monolithic CMSs can be used as headless backends, so be sure to see whether that is an option for you.

• Are there tools to help with the migration? If you are migrating to an API-based headless CMS, some offer migration tools for popular monolithic CMSs. Be sure to explore the viability of that option first. If you are migrating to file-based content that can be stored in a Git repo and managed by a Git-based headless CMS, there are tools that can help with that as well, though many are community-maintained. For example, Hugo maintains a list of community-built migration tools (https://gohugo.io/tools/migrations/), including from sources like Joomla, Drupal, and WordPress.

• Does our current CMS implementation include a lot of customization? Despite these tools, it is possible (if not likely) that you’ll need to write a custom migration script in order to handle all the edge cases. In my experience, years of CMS use often build up a legacy of “shortcodes,” plug-ins, and other highly customized usages of the CMS that can require extra care during a migration. Take stock of any plug-ins and shortcodes that may be used to modify your content before writing your migration script. A common example is that WordPress automatically renders a YouTube video player from a YouTube URL. If there are a lot of plug-ins and shortcodes but the existing CMS has an API providing the rendered HTML, it may be easier to transition the rendered content rather than the raw content.

10.1.2 Web applications

A web application is defined not by its content, though it will likely have content considerations, as well, but instead by its functionality. Here are three things to consider:

1. Do you need to migrate existing functionality, or can you leverage existing backend APIs? If your backend code is already accessible via APIs that you can call, then you may not need to transition it (at least not immediately anyway). If you need to migrate, you’ll need to decide whether to do so via an existing third-party Jamstack service or write custom code (e.g., as serverless functions deployed to the cloud and called from the application). We’ll deal with these questions more in section 10.2.

2. Do you need a frontend JavaScript framework? What requirements are there for your frontend? Will the application benefit from building on one of the SSGs that are built on frameworks like React, Vue, and Angular? The answer for many web applications may be yes, but not always. There are still a lot of web applications that can save a lot of overhead on the client by using plain JavaScript instead of a frontend framework. Frontend frameworks can do some heavy lifting in terms of loading data on the client and managing the state on the application, making it easier to build many UI interactions, but they come with overhead in terms of the amount of JavaScript your visitors are required to download.

3. What can you prerender? Keep in mind that one of the key things that differentiates the Jamstack approach is prerendering wherever possible. When it comes to web applications, it can be easy to default to rendering everything at run time by making server/API calls. Some creative thinking can help illuminate portions of the site that may not be as dynamic as they seem. New techniques, such as distributed persistent rendering (DPR) or incremental static regeneration (ISR) (which we’ll talk more about in section 10.1.3), can also help address things like user-generated content. Keep in mind that a page does not need to be fully prerendered (during the build) or fully client-side rendered. You can render parts of the page during the build, and other parts can load on the client via JavaScript.

10.1.3 Large sites

Large sites have one unique problem when it comes to the Jamstack: build times. This may seem trivial, but if you are building a site with thousands or even hundreds of thousands of pages (as some e-commerce sites have), build times can get especially lengthy. Even a build time that is only a few minutes can become a real impediment to developing and testing a site. They can also interfere with content updates. Imagine the reaction of a marketing team waiting on a 15-minute build to push a small but critical correction to the site, for example.

Sean C. Davis looked at this issue in an article for CSS Tricks in late 2020 (so it doesn’t account for some recent updates to the SSGs) titled “Comparing Static Site Generator Build Times” (http://mng.bz/y4Dy). You can see the performance he found in his tests for large sites in figure 10.1.

Figure 10.1 This chart shows how each SSG scales in build times for large sites. Some, like Hugo, have a linear increase, while others increase exponentially as the number of pages increases.

As figure 10.1 shows, the more “traditional” SSGs—those not based on a JavaScript framework and that focus entirely on generating static files—tend to have faster build times that increase linearly. The JavaScript-framework-based SSGs tend to take longer to build and increase exponentially as sites get particularly large.

Thus, the simplest solution to the issue of build time performance is to choose a SSG that is optimized for fast build times, such as Hugo or Eleventy (although Jekyll also performed surprisingly well in Davis’s tests). Hugo is built in Go and is designed for extremely fast build times—even for sites with tens of thousands of pages. Eleventy is built in JavaScript and, while not as fast as Hugo, it is still extremely fast for large sites and includes some performance-related tooling to help debug long-running build processes.

Both Hugo and Eleventy are “traditional” SSGs, though. This means that they focus on generating static pages, don’t require a framework, and don’t run on both the client and the server (in the case of Jamstack, the “server” is a serverless function). If you cannot choose one of these options, there are solutions built into some of the other SSGs that can help.

Incremental Builds

Gatsby supports incremental builds (http://mng.bz/M2Ko). In simple terms, this means that Gatsby tracks the impact of any changes and only rebuilds the pages that are impacted rather than regenerating the entire site on every rebuild. Gatsby says that its tests show this reduces a 20-minute build time to under a minute for the first build and under 10 seconds for subsequent builds. Support for incremental builds in Gatsby, though, depends on your deployment provider. For instance, Gatsby’s own Gatsby Cloud obviously supports incremental builds, as does Netlify (http://mng.bz/aDzo). It is worth noting that Jekyll also has experimental support for incremental builds (see http://mng.bz/g4Dx).

Incremental Static Regeneration (ISR)

ISR is an optimization supported by Next.js (see http://mng.bz/5KEZ for a detailed description). The easiest way to understand how it works is to compare it to incremental builds. Gatsby’s incremental builds happen during the build after each deploy by regenerating only updated pages, meaning that at some point all the pages need to be generated before they can be tracked and, if necessary, updated. However, using ISR, you’d only generate a subset of pages that you initially specify and update missing pages after they are deployed, when they are first requested by the user.

Essentially, ISR is a way of deferring the generation of noncritical pages to when a user requests them. The requested pages are added to the build, and each subsequent request will be served the static version, meaning only the initial visitor will see any delay in receiving the content. This can occur in one of two ways:

• With a fallback—The user requests a page that has not yet been generated. They will receive a static page in a loading state (i.e., any content and data has not been populated) and the page will update once the server responds with the generated page. All subsequent requests will receive the completed page via the static file, not the loading state.

• Without a fallback—This is the recommended alternative. The user requests a page that has not yet been generated. The server will respond to the user when the page is rendered. Subsequent requests will serve the static file.

One important distinction of ISR is that you can set a time to revalidate. This means that Next.js will show the stale version but regenerate the page in the background. When the page has generated, it will invalidate the cache and show the updated page to the current visitor. All subsequent visitors will see the updated page served from the static file until the time to revalidate has passed again. While this is similar to server-side rendering (SSR), the key difference is that an SSR page will hit the server on every request, while an ISR page will only hit the server when a page is stale and will serve the static file otherwise. Functionally, it is very similar to SSR plus a cache.

ISR will allow a large site to generate only the critical pages, reducing the number of pages that need to be built initially, and thereby reducing the build time. For instance, an e-commerce site with thousands of products could generate the most frequently trafficked pages and defer rendering for the others in order to drastically reduce their overall build time. Currently, ISR for Next.js is only supported when hosted on Vercel (http://mng.bz/0wxz), although Netlify supports a similar variation called distributed persistent rendering, which we’ll discuss next.

Distributed Persistent Rendering

Netlify has always supported what they call atomic and immutable deploys.

• Atomic deploys—A deploy is made as a single, complete unit with no interim state. This means that the prior deploy exists until the new deploy is complete and nothing gets updated in between. Thus, if a deploy fails, the site doesn’t go down, as the prior deploy was never replaced in whole or in part.

• Immutable deploys—Once a deploy is complete, it cannot be changed after the fact. This allows you to roll back to a prior deploy, for instance, without anything breaking.

As Netlify saw it, ISR breaks these contracts, which they consider a critical benefit of Jamstack deployment (though there is disagreement in the community as to how critical they are to the Jamstack). To address this, Netlify proposed a standard they call distributed persistent rendering (DPR) (http://mng.bz/6ZrZ).

The key differences between ISR and DPR are as follows:

• DPR was proposed as a standard. (You can view the RFC here: http://mng.bz/KBYn.) The idea is that, in doing so, it can be adopted and supported both by multiple SSGs and deployment platforms.

• DPR does not support an invalid/stale state for a page. Deferred pages are rendered once upon first request and added to the build, thereby becoming a persistent asset of the build. This means that rolling back to a prior build would include any deferred pages that had been previously rendered as they become part of the deploy. It also means there is no question as to the state of the rendered page; it is either rendered or not yet rendered but never out of date. Netlify believes this avoids hard-to-debug issues related to the state of a build when using ISR.

Ultimately, DPR solves the same problem as ISR by allowing large sites to defer rendering pages and decrease their build time. Currently Netlify supports DPR for Next.js sites via their Essential Next.js build plug-in (http://mng.bz/9K28), which installs automatically for Next.js deployments. As of this writing, only Eleventy has publicly shared that it is actively working to support DPR as well via Eleventy Serverless (http://mng.bz/jyG9).

Deciphering the Terminology

I just threw a lot of terminology at you: incremental builds, ISR, and DPR. It can be tough to know how to solve the problem of long builds for large sites. Let’s break it down:

• If you are building a large site, explore the possibility of using either Hugo or Eleventy. While neither includes a JavaScript framework by default and neither currently supports incremental builds, ISR, or DPR, both are optimized for building large sites. If you’ve determined that your project requires a JavaScript framework, then it is worth exploring other options (but remember that even though they haven’t been built on the backend using a framework, you are still free to include a framework on the frontend with either solution).

• If you are using Gatsby, be sure to choose a deployment provider that supports incremental builds.

• If you are using Next.js, your choice of ISR or DPR is largely dictated by your deployment platform (e.g., Vercel or Netlify). They are both functionally similar and will achieve similar results in reducing build times, but DPR will not allow you to specify a time to revalidate. This means that, once generated, a page cannot be updated when using DPR.

• If you are using an SSG other than Hugo, Eleventy, Gatsby, or Next.js, your options for reducing build times on large sites are limited at the moment. Most of the techniques described here are new, and other popular SSGs are working on adding support, so this may change in the near future.

10.2 What functionality do you need to migrate?

Jamstack sites rely on JavaScript and API calls to enhance the frontend with dynamic functionality, which promotes the use of discreet services to accomplish these tasks. In some cases, this is backend code you write yourself as cloud (i.e., serverless) functions. These functions may contain custom business logic that your site requires, or they may make API or database calls and then transform the data for consumption by the frontend. These custom functions can be deployed as Netlify functions, Cloudflare functions, AWS Lambda, and more. The choice generally depends on your chosen stack and your chosen deployment provider.

But migrating to a Jamstack site does not necessarily require that you convert all your existing server-side code into custom cloud functions. Jamstack generally encourages the use of purpose-built third-party services that are designed to meet many common site requirements like authentication, search, data storage, and more. In many cases, these can save you significant time and effort when building a project.

In order to determine the best available options, start by breaking down the functionality of your existing site into services. Then consider the following questions:

• Can we leverage existing backend code via an API? Moving to the Jamstack doesn’t mean rewriting everything from scratch. If some of your existing backend code already exposes an API or can be easily modified to expose an API that can be consumed in your Jamstack application, this may be the best available option because requires the least development effort and allows other stakeholders to continue utilizing existing systems. Nonetheless, you might consider transitioning to another option anyway if the Jamstack migration presents an opportunity to refactor and improve legacy code.

• Can I use a third-party service? Even if you may not be able to leverage existing backend code, you don’t need to build your backend from scratch. There are a plethora of services that can be easily used in Jamstack sites. The main considerations should be whether they meet your requirements in terms of features and pricing. Many services have free developer accounts that scale up in cost based on usage, so it’s important to consider pricing before making a change. See section 10.2.1 for a description of a number of popular options.

• Does the third-party service offer migration tools? If you are choosing a third-party service, be sure to explore whether they offer a migration path from your existing backend. This is especially common when migrating content from popular monolithic CMSs, like WordPress or Drupal, to a headless CMS. However, other services may offer migration tools depending on what you are migrating from (e.g., authentication providers often offer migration paths), so be sure to do some research before doing the work yourself.

• How do I break down custom code into discrete services? The way existing server-side code in your monolithic web application can translate to serverless cloud functions may not be immediately obvious. Break down the functionality of your site and take an inventory of what needs to be written. In particular, look for opportunities to create reusable pieces of code. Try to keep the purpose of each cloud function simple, reusable, and discrete.

10.2.1 Popular third-party Jamstack services

There are far more options available for nearly every category of service than I can cover here, but it is worth exploring some of the available options to give you a better sense of the sort of third-party solutions that may be available to you.

• Content management

  • Contentful—An enterprise-focused, API-based headless CMS that emphasizes ease of content modeling and a polished editing experience.

  • Sanity—An API-based headless CMS that focuses on the flexibility and customizability in both its content modeling and editing experience.

  • Agility CMS—An enterprise-focused, API-based headless CMS that offers a large number of prebuilt integrations with popular SSGs.

• E-commerce

  • Snipcart—An add-on headless e-commerce solution that is designed to be extremely easy to integrate into any Jamstack site with minimal effort.

  • Commerce.js—An API-based headless e-commerce solution that is designed for creating highly customized e-commerce experiences.

• Authentication/user management

  • Auth0—An all-in-one authentication and user management solution that integrates with most existing third-party login services as well as custom options.

  • Netlify Identity—A solution that can work with external providers and is designed to integrate quickly and easily within any site deployed to Netlify.

• Forms

  • Netlify Forms—Captures the contents of any form submission and designed to be easy to add to any custom form on sites deployed to Netlify.

  • Formspree—A flexible form option that can integrate easily into added to HTML forms or integrated via React components or the API.

• Search

  • Algolia—A hosted search index and API that provides prebuilt integrations for most popular frontends (and backends).

  • Lunr—An open source client-side Jamstack search library built into JavaScript that can quickly perform searches against a generated index file.

• Media management/processing

  • Cloudinary—Provides media storage and APIs that can modify, transform, and optimize images and video before displaying them on your site.

10.3 Making the move

You’ve audited your site to figure out the different pages, templates, and functionality that you’ll have to accommodate. You’ve explored SSGs and headless CMS options, and you’ve researched the supporting services that you may want for things like authentication/user management, search, forms, and more. Finally, you’ve sold your boss and any stakeholders on the value of this project, and now they are looking to you for next steps. While I can’t give you a project plan, as it depends on far too many factors for me to accurately cover here, I can give you a few more recommendations and guidance with a little help from some Jamstack community experts.

10.3.1 Don’t move everything at once

Start small to learn and experiment first. List all the features you currently have and see how it would translate in a Jamstack world. It’s not just about picking a SSG, sometimes you’ll need to employ many services to achieve a similar result. Evaluate the feasibility and cost.

—Frank Taillandier (personal communication)

Most sites are bigger than you think, with lots of hidden or weird features. Also, most older sites are monolithic. Don’t try to recreate EVERYTHING at once—you can do things in a phased approach as you split up frontend, backend, content, APIs, etc. Use something Netlify or Vercel rewrites to combine old pages or URL paths into a single domain.

—Joel Varty (personal communication)

This is critical: you can adopt the Jamstack incrementally. The entire site doesn’t have to be converted and moved to the Jamstack all at the same time. If you’re dealing with a particularly large site, break it down into pieces that can be implemented individually. You can leverage redirects and proxies, which most deployment providers, such as Netlify and Vercel, support, so that the transition is seamless for your users; they don’t need to know which parts of the site exist on the provider’s CDN and which exist on your legacy servers. Or you could host the Jamstack portions of your site on your existing infrastructure since it is just static HTML, JavaScript, and CSS files; there’s no additional infrastructure required to support it.

In addition, you can begin transitioning services and deploy cloud functions that can be used on your existing site and your new site simultaneously. This may require a bit more work up front by adding two different site integrations but can save you headaches in the long run. For example, transition some server-side logic into an AWS Lambda function that can be used both by the new Jamstack sections deployed and by the preexisting portions of the site. Starting small can have the added benefit of giving you time to experiment and learn, particularly if you are new to the Jamstack, without overburdening yourself.

10.3.2 Pick a headless CMS up front

Don’t forget about your marketing and editing teams who will need to use your solution for ages and iterations—make sure they can create and edit pages, that they have modules, that they can manage URLs and sitemaps, that they have a reliable preview and functional text editor, that they can search and find content easily, that they can reuse modules, that they can create forms and landing pages—you don’t build sites just for 100 speed score to show off on your Twitter. You build them for humans who need to feel efficient, creative, and independent.

—Olga Voigt (personal communication)

You may be tempted to simply manage your content as files in Markdown. Markdown is easy. Git already versions files for you. Why bother with the added complication and expense of moving to a headless CMS?

If you have multiple content contributors/editors or separate teams working on site code versus site content, this will, in my opinion, ultimately be a mistake that reveals itself over time. Even worse, it becomes more difficult to fix as time goes on because the volume of content that needs to be transitioned to a CMS only grows.

In my experience, the problem occurs because, unless you are very careful, the processes for deploying content and code become mixed when deploying content as files. Code changes get mixed in with content changes, causing problems on both ends. Developers can’t push their code because of content changes that aren’t ready to go live, while content editors are often blocked from getting critical content out because of code changes that are not ready to be released.

While an API-based headless CMS obviously solves this, even a Git-based headless CMS allows for a publishing process that doesn’t require mixing processes. For instance, Netlify CMS supports a simple workflow process for content approvals, allowing you to push content changes directly into production once they have been reviewed and approved. This keeps the workflow for content separate from the regular deployment flow for code changes, allowing each to work independently.

10.3.3 Consider building templates from scratch rather than porting

If you’re coming from WordPress, I would export the content and build from scratch or start with a Jamstack template. Scraping a WordPress theme and converting it to an SSG isn’t going to be a fun time.

—Mike Neumegen (personal communication)

It may be tempting to think you can port or scrape your existing templates into the templating system supported by your SSG, but this can be painful and not take advantage of the frontend flexibility that Jamstack offers. Instead, you may want to consider building new templates from scratch (perhaps using the opportunity for a design refresh) or use one of the countless themes available for nearly all SSGs.

For example, if you were using WordPress, you might consider porting each template from PHP or even scraping the generated HTML and converting that to a template. The idea is to save yourself a lot of time and effort, but the porting process can be difficult and time-consuming. For instance, some templating solutions supported by popular SSGs, like Liquid or Handlebars, don’t include logic that translates easily from a language like PHP. Even in cases where the templates do support that kind of logic, for example JavaScript/JSX, translating them can be extremely time consuming, negating the benefit of doing it in the first place.

Plus, when you port a template, you are less likely to properly take advantage of converting reusable portions of your templates into components or partials. Or you may not take full advantage of the componentized content offered by your headless CMS when you are converting templates designed for page-based content. For example, it is not uncommon to find elements of design and layout within the content for a page stored in WordPress. When moving to a headless CMS, it is best practice to separate the design and layout elements from the content by building this page using multiple content elements rather than a single block of content.

10.3.4 Keep as much as possible

I would advocate for keeping as much as possible. Like a lot of people might still want to use WordPress. Good for them, if that’s what they’re comfortable with. Use the WPGraphQL plug-in and consume the GraphQL output through Gatsby or combine with another service with an API mesh.

—Jaden Baptista (personal communication)

The decoupled nature of Jamstack means that you can, and in many cases should, keep parts of your existing backend in place. There is no requirement that you move everything to a new Jamstack service or move everything upfront. There may be parts of your backend that either meet a very specific need and will be difficult to port or that already support being used in a headless manner and are easy to port.

For example, you may be using WordPress, Drupal, or another monolithic CMS. If your team is happy with the content management and editing experience, it is worth considering using your existing CMS in a headless manner. On the other hand, your site may integrate data from a specialized backend that stores data in a database, CRM (customer relationship management) system (e.g., Salesforce), or in an ERP (enterprise resource planning) system (e.g., SAP). While this would be complex to port into another system, explore whether you are able to use existing APIs or, alternatively, connect the database to services that can expose it (e.g., services like Hasura [hasura.io], StepZen [stepzen.com], or TakeShape [takeshape.io] can expose existing databases as GraphQL APIs that your Jamstack frontend can consume).

10.4 What’s next?

Well, we’ve reached the end of our Jamstack journey together, but hopefully just the beginning of yours. If I were to leave you with one final tip, it’s that there is a great community that has been created around the Jamstack and is always willing to help, so you don’t have to go on that journey alone. Start by joining one (or both) of the existing online communities for Jamstack:

• Jamstack Community Discord (https://jamstack.org/discord)—This is a large Discord community that is managed by Netlify but open to everyone; there’s no requirement that you use Netlify.

• The New Dynamic Slack (visit https://www.tnd.dev/ and click the Slack link in the upper-right corner)—This is a community-managed Slack group that has been around for some time. While it is much smaller, its active members include many of the creators of popular Jamstack tools.

Another opportunity to meet and learn from fellow Jamstack developers is via meetups and conferences around the globe. You can search your local area on Meetup.com or browse the list of user groups on https://jamstack.org/community/. There are also a couple of Jamstack-specific conferences, including the Jamstack Conference (https://jamstackconf.com/), organized by Netlify, and TheJam.dev (https://cfe.dev/events/the-jam-2022/), organized by one of our coauthors, Brian, in Jamuary (get it?).

The Jamstack community is, in my experience, one of the strengths of Jamstack in general. Be sure to participate in it, and I will see you there.

Summary

• No two migrations are alike, but there are some common issues that content-heavy sites, web applications, and large sites face when moving to the Jamstack. These include migrating content, migrating functionality, and dealing with long build times.

• Content-heavy sites should decide whether they need to migrate their existing content and, if so, the best headless CMS option to migrate to. Some headless CMS systems have migration tools for moving from popular monolithic CMS solutions. Determine whether your existing CMS supports a headless deployment. If so, determine whether your needs are better met staying with the existing solution or moving to a dedicated headless CMS solution.

• Web applications must take an inventory of what functionality needs to be transitioned and determine what existing backend code can be reused in the Jamstack site, what code needs to be rewritten, and what services may be available to replace existing functionality. This inventory can be used to explore whether you can reuse any existing backend APIs, whether you should migrate to a third-party solution, or whether you need to build your own backend.

• When moving a large site to the Jamstack, developers need to take build times into consideration. The available options for dealing with this are choosing a highly optimized SSG like Hugo or Eleventy, or choosing a solution that supports ISR or DPR, such as Next.js.

• There is a large and helpful community of developers around Jamstack who can be a great resource when you are learning to build Jamstack sites or when you hit a roadblock.