Chapter 6. Programmatic Page Creation

In the previous chapter, we took a close look at how Gatsby enables the retrieval of arbitrary data from a variety of sources, whether that means a local filesystem, an external database or service, or a CMS or commerce platform. Thanks to source plugins, we can easily populate our Gatsby pages and components with external data.

But it’s often the case that we want the creation of our Gatsby pages to be dictated by the data rather than manually creating the pages first and populating them with data later. What happens when the pages in our Gatsby site depend on the data we retrieve in order to exist? How do we take GraphQL data and generate pages based on arbitrary response data?

One of Gatsby’s most important features, and one of the chief reasons for its success among developers, is its capacity to perform programmatic page creation, in which pages depend on and are created according to GraphQL data. For instance, the home page of a blog might be manually created, but its archive pages and individual article pages likely rely on external data. As such, as developers, we can’t predict how many pages there might be, since the number of blog posts is arbitrary.

Thanks to the gatsby-node.js file and the createPages API, we can now create not only manual pages that contain data but also generated pages that depend on external data to exist. Through these mechanisms, we can implement rich Gatsby sites that don’t just contain static content; they also dynamically adjust to the data that we present them, becoming truly data-driven sites in the process.

Programmatic page creation involves a combination of the gatsby-node.js file, the createPages API, and, optionally, Gatsby templates. We’ll cover each of these in turn in this chapter, in addition to introducing another type of Gatsby plugin: transformer plugins. To connect the dots between our explorations of GraphQL and the discussion of source plugins in the previous chapter, let’s first revisit how to create a Gatsby page that depends on GraphQL data. Then, we’ll examine transformer plugin usage before delving into the central file where programmatic page creation takes place: gatsby-node.js.

Traversing GraphQL Data in Pages

In this section, we’ll walk through building a rudimentary Gatsby site using a source plugin instead of using data hardcoded into the Gatsby configuration file (as our site name was). This means we’ll be introducing a page that relies entirely on data made available through a source plugin, rather than from gatsby-config.js or from JSX we write.

To begin, clone a new version of the default starter for Gatsby:

$ gatsby new gtdg-ch6-programmatic-pages gatsbyjs/gatsby-starter-default
$ cd gtdg-ch6-programmatic-pages

Then install the gatsby-source-filesystem source plugin, which will allow you to access your filesystem through Gatsby’s internal GraphQL API (see ““Sourcing Data from the Filesystem”” if you need a refresher). In this walkthrough, we’ll build a basic catalog of the files in our codebase. Fortunately, Gatsby’s default starter comes with gatsby-source-filesystem preconfigured for us.

Next, create a file named file-list.js in the src/pages directory, and copy the contents of src/pages/page-2.js into it. Then change the contents of that file to the following to distinguish it from src/pages/page-2.js:

import React from "react"
import { Link } from "gatsby"

import Layout from "../components/layout"
import SEO from "../components/seo"

const FileListPage = () => (
  <Layout>
    <SEO title="File list" />
    <h1>Hi from the file list page</h1>
    <p>Welcome to the file list page</p>
    <Link to="/">Go back to the homepage</Link>
  </Layout>
)

export default FileListPage

Open GraphiQL in your browser by executing gatsby develop and navigating to https://localhost:8000/___graphql. You can issue a simple test query—the very first one we issued in Chapter 4—to make sure the GraphiQL interface is working properly:

{
  site {
    siteMetadata {
      title
    }
  }
}

Running this query in GraphiQL will give you the following JSON response:

{
  "data": {
    "site": {
      "siteMetadata": {
        "title": "Gatsby Default Starter"
      }
    }
  },
  "extensions": {}
}

Now you have your site title, which is provided by the Gatsby configuration file. Let’s try a new query, this time using the GraphQL data retrieved from the filesystem source plugin. In this example, the fromNow argument indicates that the birthTime (time at which the file was created) and modifiedTime (time at which the file was modified) should be returned as a string, like “8 minutes ago” or “in 8 minutes”:

{
  allFile {
    edges {
      node {
        relativePath
        birthTime(fromNow: true)
        modifiedTime(fromNow: true)
        extension
        prettySize
      }
    }
  }
}

You can see the result of the query we just issued to GraphiQL in Figure 6-1.

Figure 6-1. The result of our GraphiQL request, with the JSON response matching the structure of the query we provided

As you can see, according to the configuration in gatsby-config.js (scan for files in src/images/), we now have a list of the two files returned by the source plugin, gatsby-icon.png (the favicon for Gatsby) and gatsby-astronaut.png (the astronaut image we see on the home page when spinning up any Gatsby default starter):

{
  "data": {
    "allFile": {
      "edges": [
        {
          "node": {
            "relativePath": "gatsby-icon.png",
            "birthTime": "21 minutes ago",
            "modifiedTime": "21 minutes ago",
            "extension": "png",
            "prettySize": "21.2 kB"
          }
        },
        {
          "node": {
            "relativePath": "gatsby-astronaut.png",
            "birthTime": "21 minutes ago",
            "modifiedTime": "21 minutes ago",
            "extension": "png",
            "prettySize": "167 kB"
          }
        }
      ]
    }
  },
  "extensions": {}
}

Back in our file list page, we can now add the query to our file. In order to make GraphQL queries available to our Gatsby pages, we need to declare a dependency on the graphql tag in Gatsby by adding to the Gatsby import statement:

import React from "react"
import { graphql, Link } from "gatsby"

import Layout from "../components/layout"
import SEO from "../components/seo"

Let’s also add a console.log() statement that allows us to preview the data that we’ve just retrieved from the GraphQL API. Here is the rest of the file after the import statements:

const FileListPage = ({ data }) => {
  console.log(data)
  return (
    <Layout>
      <SEO title="File list" />
      <h1>Hi from the file list page</h1>
      <p>Welcome to the file list page</p>
      <Link to="/">Go back to the homepage</Link>
    </Layout>
  )
}

export const query = graphql`
  {
    allFile {
      edges {
        node {
          relativePath
          birthTime(fromNow: true)
          modifiedTime(fromNow: true)
          extension
          prettySize
        }
      }
    }
  }
`

export default FileListPage

As you can see, we’ve now brought our GraphQL query in as an additional export, which Gatsby makes available to our page via the data variable, which we need to add as an argument to the FileListPage function. If you inspect the console at http://localhost:8000/file-list/, you’ll see the JSON object printed there, as illustrated in Figure 6-2.

Figure 6-2. The result of our GraphQL query as a JSON object representing the data variable, inspected in the Firefox developer console

Now, let’s go ahead and replace the page’s contents with some more compelling information about the files themselves, pulling from the data we now have available to us from the GraphQL API. In the process, let’s add a table that identifies our two files. In this example, I only show the const definition of the FileListPage function:

const FileListPage = ({ data }) => {
  return (
    <Layout>
      <SEO title="File list" />
      <h1>Hi from the file list page</h1>
      <table>
        <thead>
          <tr>
            <th>Relative path</th>
            <th>Created</th>
            <th>Modified</th>
            <th>Extension</th>
            <th>Pretty size</th>
          </tr>
        </thead>
        <tbody>
          {data.allFile.edges.map( ({ node }, index) => (
            <tr key={index}>
              <td>{node.relativePath}</td>
              <td>{node.birthTime}</td>
              <td>{node.modifiedTime}</td>
              <td>{node.extension}</td>
              <td>{node.prettySize}</td>
            </tr>
          ))}
        </tbody>
      </table>
      <Link to="/">Go back to the homepage</Link>
    </Layout>
  )
}

Figure 6-3 shows the result of our new file list page.

Figure 6-3. The final result of the file list page, containing a full accounting of the files in src/images/ based on the results of a GraphQL query issued against data from gatsby-source-filesystem

Now that we’ve reviewed how to construct pages that use data in GraphQL queries (like a blog home page), we can begin our examination of how to generate pages based on GraphQL data (like an individual blog post page). The question we now need to answer is: if we wanted to generate pages for each of the files displayed in our file list page, how would we go about doing that, given that there may be an arbitrary number of files in our Gatsby site?

Working with Transformer Plugins

Before we get into working with gatsby-node.js and the createPages API, let’s discuss a category of Gatsby plugin that is critical for a variety of use cases: transformer plugins. In many cases, you’ll find that the raw data brought in by source plugins from filesystems or from external sources isn’t quite appropriate for your needs.

Transformer plugins help Gatsby developers recast that data so it’s more usable. Thus, we can think of programmatic page creation involving externally sourced data as a three-step process: sourcing the data itself through source plugins, transforming it so Gatsby can comprehend it through transformer plugins, and generating Gatsby pages based on those data structures through the gatsby-node.js file.

Adding Transformer Plugins

Transformer plugins are installed in exactly the same way as source plugins and other Gatsby ecosystem plugins:

# If using NPM
$ npm install gatsby-transformer-remark

# If using Yarn
$ yarn add gatsby-transformer-remark

And they are also configured the same way in gatsby-config.js. For example, to use gatsby-transformer-remark, which is responsible for transforming Markdown into HTML, you only need to include the new plugin in your configuration file. Follow these steps:

  1. Add gatsby-transformer-remark as a plugin alongside gatsby-source-filesystem, and then expand the scope of gatsby-source-filesystem to fetch from ${__dirname}/src/, replacing ${__dirname}/src/images/.

  2. Change options.name to src, not images.

Now you’ll be able to retrieve files from anywhere in the filesystem. Here’s what your gatsby-config.js should look like:

plugins: [
  `gatsby-plugin-react-helmet`,
  `gatsby-plugin-image`,
  {
    resolve: `gatsby-source-filesystem`,
    options: {
      name: `src`,
      path: `${__dirname}/src/`,
    },
  },
  `gatsby-transformer-remark`,
  `gatsby-transformer-sharp`,
// ...

Transforming Markdown into Data and HTML

Earlier, we ingested data from the surrounding filesystem by using the gatsby-source-filesystem source plugin. But although we have data about our files, to use what’s contained in those files we need a transformer plugin to interpret their contents. Many blogs and static sites on the web use Markdown, a text format, but in order for Gatsby to construct a page containing Markdown, it needs to be transformed into HTML first.

To try this out let’s create an arbitrary Markdown file, src/posts/lorem-ipsum.md, representing a sample Markdown blog post. Inside, we’ll include some Markdown text. The upper section of the Markdown file containing keys and values is known as frontmatter, and we’ll refer to it this way later:

---
title: "Lorem ipsum dolor sit amet"
date: "2020-11-04"
---

Lorem ipsum dolor sit amet.

Consectetur adipiscing elit.

- Sed et gravida lacus
- Duis lorem massa

Egestas quis sapien fringilla.

Now let’s add a second Markdown file, src/posts/consectetur-adipiscing.md:

---
title: "Consectetur adipiscing elit"
date: "2020-12-05"
---

Donec lacinia vulputate porttitor.

Duis lacinia venenatis mi eget posuere.

- Phasellus rutrum dolor at lectus imperdiet
- At condimentum leo dapibus

Morbi fringilla tincidunt aliquam.

Now, navigate to https://localhost:8000/file-list again and you’ll see all of our files represented, including the Markdown files we just created. Let’s take a quick look at our GraphQL API now that we have these new Markdown files in place. When we write a new query in GraphiQL and open the autocomplete pulldown, we can see two new fields: allMarkdownRemark (for all Markdown files) and markdownRemark (for individual Markdown files). Issue the following query to see our Markdown files represented as JSON, as seen in Figure 6-4:

{
  allMarkdownRemark {
    edges {
      node {
        id
        frontmatter {
          date(fromNow: true)
          title
        }
        excerpt
        html
      }
    }
  }
}

The result of the query also shows the HTML that gatsby-transformer-remark has generated based on the Markdown in our Markdown files.

Figure 6-4. The result of our GraphQL query showing information from the Markdown files as well as their transformation into HTML

Adding a List of Markdown Pages

Now, copy the contents of src/pages/page-2.js to create another page, src/pages/blog.js, where we’ll display a list of Markdown posts. Here, we’ll create a list of Markdown files that will give our example code some characteristics of a blog. Change the contents of src/pages/blog.js to the following:

import * as React from "react"
import { graphql, Link } from "gatsby"

import Layout from "../components/layout"
import SEO from "../components/seo"

const BlogPage = ({ data }) => (
  <Layout>
    <SEO title="Blog" />
    <h1>Blog</h1>
    {data.allMarkdownRemark.edges.map( ({ node }) => (
      <div key={node.id}>
        <h2>{node.frontmatter.title}</h2>
        <p>{node.excerpt}</p>
        <p><em>{node.frontmatter.date}</em></p>
      </div>
    ))}
    <Link to="/">Go back to the homepage</Link>
  </Layout>
)

export const query = graphql`
  {
    allMarkdownRemark {
      edges {
        node {
          id
          frontmatter {
            date(fromNow: true)
            title
          }
          excerpt
        }
      }
    }
  }
`

export default BlogPage

When you save this file with the development server running, you’ll see the blog page update with the contents of your Markdown files, though they’re out of order. To fix this, let’s add a sort criterion to the GraphQL query. Here’s the new export statement for our blog page:

export const query = graphql`
  {
    allMarkdownRemark(
      sort: {
        fields: [frontmatter___date],
        order: DESC
      }
    ) {
      edges {
        node {
          id
          frontmatter {
            date(fromNow: true)
            title
          }
          excerpt
        }
      }
    }
  }
`

The resulting list of Markdown pages, now sorted in descending order by date, is shown in Figure 6-5.

Figure 6-5. Our new blog page, containing information from each Markdown blog post we created, accessed through the filesystem plugin and transformed by the gatsby-transformer-remark transformer plugin

Great! We have the beginnings of a working blog. But we’re still missing one of the critical aspects of any blog: the individual blog post pages for each Markdown page we’ve created. In addition, we need to link to those pages from our list of blog posts. The solution to this problem lies in Gatsby’s programmatic page creation.

Working with gatsby-node.js

The most important element of Gatsby for generating pages programmatically is the gatsby-node.js file, which contains logic for implementing the createPages API, a set of methods that Gatsby developers can interact with to generate pages. When used in combination with Gatsby template files, programmatic page creation in Gatsby represents one of the most important features of the framework, because it allows developers to query data and map that data to pages at build time.

Creating Slugs for Markdown Pages

To create our pages one of the things we’ll need is a slug, a unique name for each post that can identify them in URLs. We’ve already created implicit slugs through the naming of our Markdown files: lorem-ipsum.md and consectetur-adipiscing.md. Some data sources might provide their own slugs as fields in API responses, in which case we don’t need to generate slugs based on the blog post titles.

Using onCreateNode

Every time we create a new page, we need to do two things:

  1. Generate the file path or slug for the page.

  2. Create the page itself, usually based on a template.

To create the slugs, we’ll need to use a new file we haven’t seen before, the gatsby-node.js file, and the onCreateNode API. Gatsby calls each onCreateNode function we export each time a new node is created or updated. To try out onCreateNode, let’s open the currently empty gatsby-node.js file in our codebase and add these lines:

exports.onCreateNode = ({ node }) => {
  console.log(`Node created of type "${node.internal.type}"`)
}

Now, restart your Gatsby development server, which is a required step as you’ve modified the gatsby-node.js file. Watch the log output as it registers each and every node it creates based on the data returned by the filesystem source plugin, as seen in Figure 6-6.

Figure 6-6. Each onCreateNode event is logged in the terminal output when the Gatsby development starter is started, indicating the creation of a new node

Using createNodeField

Now, modify the contents of gatsby-node.js to use createNodeField, one of the actions available in onCreateNode. The createNodeField function allows us to arbitrarily add new fields on nodes that are created by other plugins. If you don’t have control over how the node is provided within the source plugin’s configuration in the gatsby-config.js file, you have to use gatsby-node.js to bolt on additional fields.

We’ll add a new field called slug. In the process, we’ll use a function available in the gatsby-source-filesystem source plugin, createFilePath, to create slugs based on filenames. The createFilePath function automatically accesses the parent File node and creates the slug on our behalf. We’ll use the filenames we gave our Markdown files before and convert them into slugs:

const { createFilePath } = require(`gatsby-source-filesystem`)

exports.onCreateNode = ({ node, getNode, actions }) => {
  const { createNodeField } = actions
  if (node.internal.type === `MarkdownRemark`) {
    const slug = createFilePath({ node, getNode, basePath: `posts` })
    createNodeField({
      node,
      name: `slug`,
      value: slug,
    })
  }
}

Once you’ve made these changes, save gatsby-node.js again, restart the development server, and run the following query in GraphiQL:

{
  allMarkdownRemark {
    edges {
      node {
        fields {
          slug
        }
      }
    }
  }
}

You’ll now see our slugs represented within each individual Markdown node, as shown in Figure 6-7.

Figure 6-7. The result of the preceding GraphQL query, containing a reference to our newly created field within each individual Markdown file node

Now that we’ve created slugs that go with our blog posts, we can create pages programmatically based on those slugs.

Adding a Template

One of the things we need to generate pages programmatically based on GraphQL data is a template through which we can run data. A template in Gatsby is often important, because it indicates to our Gatsby site how data should be rendered within programmatically created pages. It wouldn’t make sense to place all of this logic in gatsby-node.js, especially if we have multiple templates.

Create a new file named src/templates/post.js, which will contain our blog post template. Add the following code—as you can see, we’re using a GraphQL query to get the information we need from our Markdown files:

import React from "react"
import { graphql } from "gatsby"
import Layout from "../components/layout"

export default function Post({ data }) {
  const post = data.markdownRemark
  return (
    <Layout>
      <h1>{post.frontmatter.title}</h1>
      <div dangerouslySetInnerHTML={{ __html: post.html }} />
    </Layout>
  )
}

export const query = graphql`
  query($slug: String!) {
    markdownRemark(
      fields: {
        slug: {
          eq: $slug
        }
      }
    ) {
      html
      frontmatter {
        date(fromNow: true)
        title
      }
    }
  }
`
Note

The dangerouslySetInnerHTML attribute in React indicates that the HTML string defined as the value of __html should be inserted as is. It is named this way to clearly indicate the risk of doing this, which could expose your site to cross-site scripting (XSS) attacks. Because we control the contents of the Markdown files, we can rest assured in this case that it’s safe.

As you can see, one of the main differences between this query and the others is that we’re also retrieving the full HTML generated by gatsby-transformer-remark from our Markdown. Now that we’ve created a template for each individual blog post, we need to write the logic that will generate the pages based on the slugs we’ve now provided to our GraphQL API and the template we’ve just created.

Adding Markdown Pages with createPages

To finish our blog, we need to use Gatsby’s createPages API to generate pages that use our slugs and template. We’ll use the Node.js path library to handle the path to our blog post template, so we need to add it as a dependency at the top of the file:

const path = require(`path`)
const { createFilePath } = require(`gatsby-source-filesystem`)

Below the onCreateNode function we added previously, add a new createPages function, as shown here:

const path = require(`path`)
const { createFilePath } = require(`gatsby-source-filesystem`)

exports.onCreateNode = ({ node, getNode, actions }) => {
  const { createNodeField } = actions
  if (node.internal.type === `MarkdownRemark`) {
    const slug = createFilePath({ node, getNode, basePath: `posts` })
    createNodeField({
      node,
      name: `slug`,
      value: slug,
    })
  }
}

exports.createPages = async ({ graphql, actions }) => {
  const { createPage } = actions 
  const result = await graphql(`
    {
      allMarkdownRemark { 
        edges {
          node {
            fields {
              slug
            }
          }
        }
      }
    }
  `)

  result.data.allMarkdownRemark.edges.forEach( ({ node }) => {
    createPage({ 
      path: `/blog` + node.fields.slug,
      component: path.resolve(`./src/templates/post.js`),
      context: { 
        slug: node.fields.slug,
      },
    })
  })
}

Invoke the individual createPage function from the actions (a collection of functions used to manipulate state in a Gatsby site) available in the createPages API.

Define the GraphQL query we need to generate pages.

Add the page generation logic just below the const definitions, before closing out the createPages function.

When we invoke createPage, we also provide context, which represents data made available as GraphQL query variables in page queries.

Now when you navigate to http://localhost:8000/blog/lorem-ipsum, you’ll see one of our blog posts represented, as illustrated in Figure 6-8.

Figure 6-8. An individual blog post located at /blog/lorem-ipsum, demonstrating that our blog post is correctly using our template via gatsby-node.js

The final step is to link back to our blog posts from the blog page. Let’s go back to src/pages/blog.js and add links to each individual blog post. First, we’ll add a link to each of the blog post teasers that displays on the blog page:

const BlogPage = ({ data }) => (
  <Layout>
    <SEO title="Blog" />
    <h1>Blog</h1>
    {data.allMarkdownRemark.edges.map( ({ node }) => (
      <div key={node.id}>
        <h2>
          <Link to={`/blog${node.fields.slug}`}>
            {node.frontmatter.title}
          </Link>
        </h2>
        <p>{node.excerpt}</p>
        <p><em>{node.frontmatter.date}</em></p>
      </div>
    ))}
    <Link to="/">Go back to the homepage</Link>
  </Layout>
)

Second, we need to update the GraphQL query at the bottom to include our newly created slug field:

export const query = graphql`
  {
    allMarkdownRemark(
      sort: {
        fields: [frontmatter___date],
        order: DESC
      }
    ) {
      edges {
        node {
          id
          frontmatter {
            date(fromNow: true)
            title
          }
          fields {
            slug
          }
          excerpt
        }
      }
    }
  }
`

The result of these changes can be seen in Figure 6-9, and our completed src/pages/blog.js is as follows:

import React from "react"
import { graphql, Link } from "gatsby"

import Layout from "../components/layout"
import SEO from "../components/seo"

const BlogPage = ({ data }) => (
  <Layout>
    <SEO title="Blog" />
    <h1>Blog</h1>
    {data.allMarkdownRemark.edges.map( ({ node }) => (
      <div key={node.id}>
        <h2>
          <Link to={`/blog${node.fields.slug}`}>
            {node.frontmatter.title}
          </Link>
        </h2>
        <p>{node.excerpt}</p>
        <p><em>{node.frontmatter.date}</em></p>
      </div>
    ))}
    <Link to="/">Go back to the homepage</Link>
  </Layout>
)

export const query = graphql`
  {
    allMarkdownRemark(
      sort: {
        fields: [frontmatter___date],
        order: DESC
      }
    ) {
      edges {
        node {
          id
          frontmatter {
            date(fromNow: true)
            title
          }
          fields {
            slug
          }
          excerpt
        }
      }
    }
  }
`

export default BlogPage
Figure 6-9. The completed blog, with each blog post title correctly leading to its individual page based on its slug

There you have it! A blog powered by Gatsby’s programmatic page creation.

Conclusion

Programmatic page creation is one of the most fundamental aspects of Gatsby, and it’s deeply powerful because it’s capable of rendering data originating from disparate sources. In this chapter, we first revisited how to traverse GraphQL data in Gatsby pages to prepare ourselves for programmatic page creation. Another prerequisite was the use of transformer plugins, which are used to make external data understandable to the Gatsby framework. Finally, we used gatsby-node.js to generate a set of pages based on our transformed data.

While data from external sources is critical for the success of many Gatsby sites, there is another type of data that Gatsby’s GraphQL API handles: assets, especially media assets, which may be images, videos, or audio files. To come full circle with our examination of Gatsby’s data processing layer and how our Gatsby pages and components interact with its various elements, we’ll now focus our attention on assets and how to handle these often-fickle files (and file sets) in our Gatsby sites.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
44.200.77.92