Chapter 9. Gatsby Plugins and Starters

There are many ways to extend Gatsby sites in order to incorporate additional functionality or to provide kickstarters for faster development. In particular, plugins and starters are crucial for offering additional capabilities beyond the scope of what Gatsby supplies out of the box. Like Gatsby themes, which we’ll cover in Chapter 10, plugins and starters can both be installed entirely from the Gatsby CLI. In addition to officially supported plugins that represent common use cases, there are many community-supported plugins and starters available on the Gatsby website. We’ve already seen several examples of both in this book, but we haven’t yet covered how to make your own.

In this chapter, we’ll explore not only how to create Gatsby plugins and starters, but also how to make them available to other interested developers by publishing them. In addition to tailor-made starters, I’ll show you how to make your own source plugins, which are responsible for retrieving data from other sources, and transformer plugins, which transform data so Gatsby can work with it.

Creating Gatsby Starters

As we covered earlier in the book, starters in Gatsby are boilerplate example projects that Gatsby developers can employ to spin up a new site for rapid development. So far in this book, we’ve covered three extremely common starters used frequently by Gatsby practitioners as foundations for new sites: gatsby-starter-hello-world, gatsby-starter-default, and gatsby-starter-blog.

There are countless other starters available in Gatsby’s Starter Library, which Gatsby recommends developers of starters browse for inspiration and to ensure that a starter doesn’t already exist that handles their specific use case. In the coming sections, we’ll discuss how to initialize a starter project, enable configuration of your starter, assess performance and accessibility considerations, and finally, license, test, and release your completed starter implementation.

Gatsby Starter Requirements

Every Gatsby starter must contain all of the files and directories listed in Table 9-1 within its source code in order to be installable as a starter.

Table 9-1. Gatsby starter file requirements
File/Directory Description
README.md A file that instructs developers how to install and configure the project, as well as providing any other pertinent information such as available features, project structure, tips and advice, and how to contribute
package.json A file listing all of the project’s Gatsby dependencies and scripts, and any other required JavaScript dependencies
gatsby-config.js A file containing configuration data and a list of additional plugins required by the starter
src/pages A directory for Gatsby page components, which must include at least one JavaScript file named index.js
static A directory containing static assets that will not be processed, such as a favicon.ico file
.gitignore A manifest file indicating to Git which files and directories should be excluded from source control management, such as the node_modules directory, Gatsby .cache and public directories, and environment variable .env files
.prettierrc (optional) An optional configuration file for Prettier, a JavaScript library for linting and formatting JavaScript for Gatsby development
LICENSE A file containing a relevant license for your starter (Gatsby recommends a license file containing the BSD Zero Clause License)

In addition to these requirements, there are some best practices that Gatsby starters should follow in order to be both discoverable and useful for developers. Your Gatsby starters should be:

Located at a stable URL
Without a single, stable URL where the starter is available, such as a GitHub repository or NPM package page, developers won’t be able to discover your starter or use it in their own sites.
Open source
Though many developers write closed-source Gatsby plugins for their employers, in order for a Gatsby starter to be usable by any developer, it needs to have a permissive open source license.
Configurable
The best starters are those that are optimized to accept a variety of configuration settings in its Gatsby configuration file.
Performant
Like Gatsby itself, Gatsby starters should endeavor to maximize their use of Gatsby’s performance optimizations in order to keep loading times minimal.
Web-accessible
In order to provide equitable user experiences for disabled people, Gatsby starters should be evaluated not just for their performance but also for their accessibility.

Remember from our first discussion of Gatsby starters in Chapter 2 that they can be installed by executing a command adhering to the following template, where site-name is the name of the generated Gatsby site and starter-url is the location of the starter:

$ gatsby new site-name starter-url

Here’s an example of a typical command to install the default starter:

$ gatsby new my-gatsby-site 
  https://github.com/gatsbyjs/gatsby-starter-default

As long as your starter is available in a publicly available source code management provider such as GitHub or GitLab, it’s installable.

Enabling Starter Configuration

Because the Gatsby configuration file is often the primary entry point for Gatsby developers upon coming across a new starter, it’s important to leverage metadata in that configuration wherever possible. Some of the most essential metadata you may wish to have starter users configure includes:

Site title
The site title is probably the most frequently overridden configuration in a starter’s configuration file.
Author details
Many Gatsby starters additionally allow for key information such as the site author’s name, contact information, and a brief bio.
Site description
For SEO purposes, it’s best to include a description of the site that can be used to identify it more expressively to users.

In addition, starters that connect with external data sources or third-party services may wish to derive much of this configuration from a source plugin, declared as a dependency in the Gatsby configuration file. And because starters can be reused as themes (discussed in more detail in the next chapter), including a Gatsby theme file for configuration can provide developers using your starter with even more power off the shelf. In short, starters are excellent opportunities to demonstrate to other developers how to satisfy common requirements of Gatsby.

Starter Performance and Accessibility

Starter performance is an important consideration, particularly for developers specifically leveraging the framework for its speed. In order to ensure your starter adheres as closely to performance best practices as possible, it’s strongly recommended to use tools such as Lighthouse or WebPageTest to assess the performance of your starter and to offer recommendations to starter users about how to deal with site performance concerns.

Note

Lighthouse and WebPageTest both provide documentation concerning usage and configuration.

Ensuring starters are web-accessible for people with disabilities is another issue relevant to user experience. The following best practices can help improve the overall accessibility of your starter:

Color contrast
Most sites on the web lack adequate color contrast for users with low vision or colorblindness. WebAIM provides recommendations for implementing color contrast that adhere to Web Content Accessibility Guidelines (WCAG) 2 standards.
Keyboard focus indicators
Many users cannot use a mouse to navigate content and must instead use a keyboard to navigate Gatsby pages, whether due to tremors or a motor disability. WebAIM provides recommendations for keyboard accessibility adhering to WCAG 2 guidelines.
Alternative text
All images and other visual media in your Gatsby starter should have alternative text available that describes the asset in words. WebAIM provides recommendations for alternative text according to WCAG 2 recommendations.
Semantic HTML use
Structuring your markup semantically is crucial for users who use alternative methods to access content. Retaining the semantic value of HTML is also important for search engine optimization. WebAIM provides recommendations for semantic HTML conforming to WCAG 2.
Accessible forms
Form inputs must be labeled, and forms should be both fillable and navigable via the keyboard. WebAIM provides recommendations for accessible forms conforming to WCAG 2.

Because the markup you manipulate in the form of JSX elements in Gatsby is typically not representative of the markup that will appear in the browser and be presented to users, it’s best to test both your starter’s performance and its accessibility in conditions that approximate the experience of real-world users. This means performing a production-ready build of your starter and evaluating it in context.

Note

For additional resources about ensuring the accessibility of your starter, consult the A11y Project, WebAIM, and the Deque Systems article “Accessibility Tips in Single-Page Applications”.

Licensing, Testing, and Releasing Starters

Because Gatsby starters behave just like normal Gatsby projects, you can install a starter and take no further action besides executing gatsby develop (to spin up a local development server) or gatsby build (to perform a full build). If you’re building your starter by executing gatsby build, execute gatsby serve to ensure your starter functions according to your vision in a production environment.

Gatsby recommends that starters contain the BSD Zero Clause License (0BSD) due to its greater permissiveness than the more commonplace MIT License, whose verbiage may render it less appealing for starters that will be used by many different developers.

Note

More information about the BSD Zero Clause License, MIT License, and other licenses is available on ChooseALicense.com.

Once your starter is publicly available in a location such as GitHub, you can add it to the Gatsby Starter Library, where you can tag it with keywords such as drupal or csv to make it even more discoverable by the larger Gatsby community. Before you add your starter to the library, however, ensure that the gatsby new command works with your new starter to avoid any unexpected surprises after release.

Note

Information about how to add your starter to the Gatsby Starter Library is available in the Community Contributions section of the Gatsby documentation.

Creating Gatsby Plugins

As we’ve seen throughout this book, plugins are the primary method of extending Gatsby’s functionality with additional features. In technical terms, plugins are Node.js packages that are installed as dependencies and configured in the Gatsby configuration file. For developers who need a variety of capabilities, they can provide just the right mix of functionality for a powerful Gatsby site.

In the coming sections, we’ll talk about how to create a plugin and make it configurable before moving on to specific types of plugins that differ in their behavior.

Plugin Nomenclature

Gatsby plugins you create must adhere to the naming conventions of other Gatsby plugins in order for Gatsby to recognize them. These naming conventions are outlined in Table 9-2, where * represents a wildcard.

Table 9-2. Gatsby plugin naming conventions
Convention Example Description
gatsby-source-* gatsby-source-sanity Use source plugin nomenclature if your plugin loads data from an external data source, third-party source, or local filesystem.
gatsby-transformer-* gatsby-transformer-remark Use transformer plugin nomenclature if your plugin converts data from a particular format (e.g., YAML or CSV) into a Gatsby-manipulable JavaScript object.
gatsby-{plugin}-* gatsby-remark-images Use nested plugin nomenclature if your plugin is a dependency for an existing plugin by prefixing the plugin name with the name of the plugin it depends upon. This is required if you wish to include a plugin in the options configuration of another plugin.
gatsby-theme-* gatsby-theme-blog Use Gatsby theme nomenclature for Gatsby themes, which are treated as plugins.
gatsby-plugin-* gatsby-plugin-* Use generic plugin nomenclature for generic plugins that don’t fit into any of the previous categories.

Initializing a New Plugin Project

Every plugin requires a package.json file containing metadata about the plugin. NPM also employs this file to display information about the plugin on its dedicated package page. Because all generic plugins, at their core, are JavaScript packages manipulable by NPM, to create one you can simply initialize a new JavaScript package by executing the following commands:

$ gatsby new gtdg-ch9-plugin gatsbyjs/gatsby-starter-default
$ cd gtdg-ch9-plugin
$ mkdir plugins && cd plugins
$ npm init

The NPM command-line interface will then provide a series of selectable options in the terminal to store initial values that are necessary for your package.json file. For this example, set gatsby-plugin-hello-world as the name and index.js as the entry point

As we’ll see in the next section, plugins can serve as implementations of core Gatsby APIs (Gatsby Node APIs, Gatsby Browser APIs, Gatsby SSR APIs) by means of their respective files (gatsby-node.js, gatsby-browser.js, gatsby-ssr.js). Within the gatsby-node.js file inside our plugin (plugin/gatsby-plugin-hello-world), we can implement the createPage, createResolvers, and sourceNodes APIs, all of which manipulate (or create) a given data node in Gatsby.

The most common API that generic plugins utilize is the Gatsby Node API, which we cover in detail in Chapter 14. It facilitates operations like the following, which are common requirements of generic and other types of plugins:

  • Load API keys to issue requests.

  • Issue requests against APIs.

  • Create Gatsby data nodes according to the API response.

  • Create individual pages programmatically using created nodes.

As you can see, in addition to performing programmatic page creation through custom code in our Gatsby site’s gatsby-node.js file, we can undertake the same processes in the context of a plugin.

Tip

Gatsby also makes available a starter for creating plugins, gatsby-starter-plugin, with some information already prepopulated.

Plugin Configuration with Options

As we’ve seen in various Gatsby plugins we’ve explored throughout this book so far, many plugins provide configuration options that are managed by developers to customize how they function within a Gatsby site. For source plugins, this might mean configuring the data source by providing a URL and access token. For other plugins, it might mean configuring how images are processed or how Markdown is transformed.

Accessing and passing plugin configuration options

In the Gatsby configuration file, a typical plugin object (required so that Gatsby recognizes the plugin) contains a resolve string (the plugin name). Optionally, the plugin object can also contain an options object that exposes available configuration. Consider the following example of a “hello world” plugin, which accepts three options:

module.exports = {
  plugins: [
    {
      resolve: `gatsby-plugin-hello-world`,
      options: {
        optionA: `Hello world`,
        optionB: true,
        greeting: `Hello world`,
      }
    },
  ],
}

This allows Gatsby developers to provide arbitrary configuration that is then leveraged by the plugin to perform further work. A Gatsby plugin can access three Gatsby APIs from within its code:

  • Gatsby Node API (gatsby-node.js)

  • Gatsby Browser API (gatsby-browser.js)

  • Gatsby Server-Side Rendering (SSR) API (gatsby-ssr.js)

Suppose that the objective of our plugin is to display a console.log message in the terminal containing our greeting option value, but only if optionB is true. We can leverage the Gatsby Node API by creating a new gatsby-node.js file specific to our plugin under the directory plugins/gatsby-plugin-hello-world:

exports.onPreInit = (_, pluginOptions) => {
  if (pluginOptions.optionB === true) {
    console.log(
      `Logging "${pluginOptions.greeting}" to the console`
    )
  }
}
Note

Gatsby has no opinion on the name of the second argument provided in plugin code that implements the Gatsby Node, Browser, and SSR APIs. For instance, developers building Gatsby themes may wish to use themeOptions instead of pluginOptions as the argument name for readability.

Within the Gatsby Node API, the onPreInit API is among the first to execute when you run gatsby develop or gatsby build. We can now see how any plugin can access its configuration options through the gatsby-config.js file. But what kinds of options can be passed into a typical Gatsby configuration file for use by a plugin?

Fortunately, any JavaScript data type can be passed in to our Gatsby plugins as configuration options, as illustrated in Table 9-3.

Table 9-3. Acceptable JavaScript types for Gatsby plugin configuration options
Data type Example Example plugin
Boolean true gatsby-plugin-sharp
String https://my-backend.io/graphql gatsby-source-graphql
Array [`documents`, `products`] gatsby-source-mongodb
Object
{
  host: `localhost`,
  user: `root`,
  password: `myPassword`,
  database: `user_records`
}
gatsby-source-mysql
Note

Because Gatsby themes, which we cover in the next chapter, are considered by Gatsby to be a type of plugin, themes can receive configuration options from their surrounding site by exporting the contents of gatsby-config.js as a function rather than an object. We’ll return to this in the next chapter, but suffice it to say for this discussion of configuration that plugins cannot do this.

Validating plugin configuration with an options schema

It’s one thing to allow configuration options that dictate how Gatsby plugins should customize their functionality to be passed in, but what happens when a developer provides option values that are not accepted because they are outside the scope of what the plugin can handle as an input? In other words, how do we validate the configuration options passed in to our plugins?

To validate plugin configuration options, we need an options schema against which the options can be compared. Options schemas aren’t required for Gatsby plugins, but in order to enforce correct input and prevent any unintended outcomes, using them is a best practice. Gatsby makes available a pluginOptionsSchema API for defining options schemas.

To accomplish this Gatsby uses Joi, a schema description language and validator for JavaScript. In our gatsby-node.js file, we create a new instance of Joi in order to return a Joi.object schema for the options we expect developers to pass in. Let’s take a second look at the gatsby-plugin-hello-world plugin we built earlier:

module.exports = {
  plugins: [
    {
      resolve: `gatsby-plugin-hello-world`,
      options: {
        optionA: `Hello world`, // String
        optionB: true, // Boolean, optional
        greeting: `Hello world`, // String
      }
    },
  ],
}

Consider a scenario where we want to ensure optionA and greeting are passed in as required options, but optionB is optional and not required. In order to accomplish this with Joi’s schema definition approach, we can add our implementation of the pluginOptionsSchema API to gatsby-node.js within our plugin’s directory:

exports.pluginOptionsSchema = ({ Joi }) => {
  return Joi.object({
    optionA: Joi.string().required().description(`Enables optionA.`),
    optionB: Joi.boolean().description(`Enables optionB.`),
    greeting: Joi.string().required().description(`Greeting logged to 
        console.`),
  })
}

Enforcing a plugin options schema for Gatsby plugins ensures we cover all scenarios where plugin users supply unexpected input. Thanks to Joi’s error handling, if a plugin user supplies options that don’t adhere to the schema, upon the next execution of gatsby develop an error will display instructing them to correct their options accordingly. For example, consider a scenario where a plugin user passes in a Boolean instead of a string to optionA. The following error would appear:

ERROR #11331  PLUGIN

Invalid plugin options for "gatsby-plugin-hello-world":

- "optionA" must be a string
Note

For comprehensive documentation about Joi, how it furnishes schemas, and how it validates data, consult the Joi API documentation.

Best practices for writing options schemas

Many of the best practices for writing options schemas come from Joi itself, but they apply to Gatsby plugins by extension. Gatsby’s best practices for use of the pluginOptionsSchema API include adding descriptions to options, setting default values for options, validating external access where needed, adding custom error messages where desired, and deprecating options in major version releases rather than silently deleting them from the schema. In this section, we’ll cover each of these in turn.

First, be sure to provide a description for each plugin configuration option by using the .description() method to explain its rationale. A schema definition including descriptions can accelerate others’ understanding of your schema and aid tooling that generates plugin options documentation based on the schema you define.

Second, it’s important to set default values for configuration options that allow for fallbacks. For instance, in our “hello world” Gatsby plugin, we could define a default greeting value such that if the user doesn’t provide a custom value, we default to a generic message. In Joi, this can be done with the .default() method, which will then result in the default greeting being logged in all plugin APIs when the user doesn’t supply their own string value:

exports.pluginOptionsSchema = ({ Joi }) => {
  return Joi.object({
    optionA: Joi.string().required().description(`Enables optionA.`),
    optionB: Joi.boolean().description(`Enables optionB.`),
    greeting: Joi.string()
      .required()
      .default(`This is the default greeting.`)
      .description(`Greeting logged to console.`),
  })
}

Third, and particularly important for source plugins, validate external access. Whenever communicating with external services or third-party data sources, it’s typically the case that we need to query APIs that we don’t manage. Because of the risk involved in relying on external APIs, it’s important to validate asynchronously the user’s ability to access the API, which is possible through Joi’s .external() method. By providing more descriptive errors earlier in the process, you can quickly communicate to users that their credentials are invalid.

For example, consider a scenario where we need to write a source plugin that consumes content from a Contentful space (i.e., a content repository). Though gatsby-source-contentful’s plugin options schema is much more complicated than this contrived example suggests, you can see how we can perform access checks against Contentful directly within our Joi schema definition:

exports.pluginOptionsSchema = ({ Joi }) => {
  return Joi.object({
    accessToken: Joi.string().required(),
    spaceId: Joi.string().required(),
    // Additional Contentful options.
  }).external(async pluginOptions => {
    try {
      await contentful
        .createClient({
          space: pluginOptions.spaceId,
          accessToken: pluginOptions.accessToken,
        })
        .getSpace()
    } catch (err) {
      throw new Error(
        `Cannot access Contentful space "${pluginOptions.spaceId}" with the 
         provided access token. Double-check it is correct and try again!`
      )
    }
  })
}

Fourth, it’s a best practice in plugin options schemas to add informative custom error messages in cases where validation fails for a specific field. Joi includes a .messages() method with which plugin developers can overwrite error messages for particular error types. For instance, given a Joi error type of any.required, which indicates that a .required() invocation has failed, we could provide a custom error message for our required optionA value as follows:

exports.pluginOptionsSchema = ({ Joi }) => {
  return Joi.object({
    optionA: Joi.string()
      .required()
      .description(`Enables optionA.`)
      .messages({
        // Override the error message if .required() fails.
        "any.required": `"optionA" needs to be defined as true or false.`,
      }),
    optionB: Joi.boolean().description(`Enables optionB.`),
    greeting: Joi.string().required().description(`Greeting logged to 
        console.`),
  })
}

Fifth, it’s important to deprecate obsolete options once they’re no longer required by your plugin in a new major version release. However, users may be confused if the new major version of your plugin lacks an option they had previously in their plugin configuration. Due to the potential for cryptic error messages that don’t indicate that a plugin option is no longer present in the schema, Joi offers a .forbidden() method that Gatsby plugin authors should use to indicate that the option is no longer necessary. In addition, it’s a best practice to include a custom error message that indicates the deprecation. Consider a scenario where we deprecate optionA as a plugin option:

exports.pluginOptionsSchema = ({ Joi }) => {
  return Joi.object({
    optionA: Joi.string()
      .required()
      .description(`Enables optionA.`)
      .forbidden()
      .messages({
        // Override the error message if .forbidden() fails.
        "any.unknown": `"optionA" is no longer supported. Use "optionB"
                        instead.`,
      }),
    optionB: Joi.boolean().description(`Enables optionB.`),
    greeting: Joi.string().required().description(`Greeting logged to 
        console.`),
  })
}

Because Gatsby developers of any background could be making use of your plugin in their own Gatsby sites, it’s important not only to document your options schema clearly but to provide an excellent developer experience that carries through version releases of your plugins. Adhering to these best practices will ensure a favorable experience for those who are eager to use the plugin you’ve contributed to the Gatsby ecosystem.

Note

For a full accounting of the available error types in Joi’s .messages() method, consult the Joi API documentation.

Performing unit testing on options schemas

Because plugins are commonly used by a wide variety of developers, it’s important to perform unit testing on the options schema to verify that it behaves as you expect. The Gatsby ecosystem offers an official package specifically for unit testing various configuration possibilities and how they validate against an options schema. To perform unit testing on your options schema, install the gatsby-plugin-utils package:

# If using NPM
$ npm install --dev gatsby-plugin-utils

# If using Yarn
$ yarn add --dev gatsby-plugin-utils

The --dev flag will ensure this developer-facing tooling is not included as a user-facing dependency. Now, within the plugin directory you can create a new directory, __tests__, containing a gatsby-node.js file that will perform your unit tests against the Gatsby Node API.

To write a unit test, you can use the testPluginOptionsSchema function available in the gatsby-plugin-utils package together with a test runner like Jest (see Chapter 11 for more on Jest). This function consists of two parameters: the plugin’s Joi schema definition and an example options object to test with. It returns an object containing an isValid Boolean set to true or false based on the test’s success, in addition to an errors array that contains any error messages thrown as the validation fails. Here’s an example of how we can apply unit testing to our recently created Gatsby plugin using Jest:

import { testPluginOptionsSchema } from "gatsby-plugin-utils"
import { pluginOptionsSchema } from "../gatsby-node"

describe(`pluginOptionsSchema`, () => {
  it(`should invalidate incorrect options`, async () => {
    const options = {
      optionA: undefined, // Should be a string
      optionB: `I am a string`, // Should be a Boolean
      greeting: 3.14159, // Should be a string
    }
    const { isValid, errors } = await testPluginOptionsSchema(
      pluginOptionsSchema,
      options
    )

    expect(isValid).toBe(false)
    expect(errors).toEqual([
      `"optionA" is required`,
      `"optionB" must be a string`,
      `"greeting" must be a string`,
    ])
  })

  it(`should validate correct options`, async () => {
    const options = {
      optionA: false,
      optionB: `12345`,
      greeting: `Hello world`,
    }
    const { isValid, errors } = await testPluginOptionsSchema(
      pluginOptionsSchema,
      options
    )

    expect(isValid).toBe(true)
    expect(errors).toEqual([])
  })
})

Performing unit testing on options schemas is always a best practice, especially when your plugin will be used by the Gatsby community and developers who seek robust, well-tested plugins to depend on in their implementations.

Note

For more information about Jest, consult the Jest documentation.

Now that we’ve covered options schemas and plugin configuration, which are concepts that apply to every Gatsby plugin, we can turn our attention to the initial steps required to create each and every plugin, regardless of type. Later in this chapter, we’ll explore how to release and maintain Gatsby plugins for community use.

Interacting with Gatsby Lifecycle APIs

Let’s take a look at an example using the Gatsby plugin we created earlier. In the following example gatsby-node.js file, our plugin implements the sourceNodes lifecycle API as a function. To avoid complexity, we’ve hardcoded the data we need as opposed to performing a request:

exports.sourceNodes = ({ actions, createNodeId, createContentDigest }) => {
  const nodeData = {
    title: "Sample Node",
    description: "Here is a sample node!",
  }
  const newNode = {
    ...nodeData,
    id: createNodeId("SampleNode-testid"), 
    internal: {
      type: "SampleNode",
      contentDigest: createContentDigest(nodeData), 
    },
  }
  actions.createNode(newNode)
}

Create a new node entitled “Sample Node” based on the title parameter.

If this process works, a new top-level field, allSampleNode, will be made available in our GraphQL schema and by extension our GraphQL API internal to Gatsby once we restart the development server.

As you can see, this plugin implements the sourceNodes API to create a new node as functionality beyond what Gatsby core offers. Over the course of creating plugins, you’ll interact with a variety of Gatsby Node APIs like sourceNodes to realize the functionality your plugins provide.

Creating Source Plugins

As we saw in Chapter 5, source plugins are Gatsby plugins that have to do with sourcing data from an external database, a third-party service, or a local filesystem. In this section, we’ll walk through how to create a rudimentary source plugin that consumes an API.

Source plugins retrieve data from remote sources or local filesystems and expose them through Gatsby’s GraphQL API as nodes. In the process, source plugins are therefore responsible for translating remote or local data into a format that Gatsby, and by extension Gatsby developers, can understand. Remember that there are no limitations to populating your Gatsby site with data from multiple source plugins, including multiple source plugins of the same type.

Initializing Projects for Source Plugin Development

If there isn’t a source plugin that matches your use case or fulfills your requirements, it’s time to consider creating one of your own, which you can optionally distribute to the wider Gatsby community and by contributing it to the plugin ecosystem. In this walkthrough, we’ll use an example API to keep things as simple as possible and create a source plugin whose goal is to provide content for a blog.

To demonstrate as many of the available features in source plugins as possible, our source plugin will:

  • Issue requests to an API.

  • Convert response data into Gatsby nodes.

  • Connect certain nodes to enable relationships between blog post authors and blog posts.

  • Accept plugin configuration options.

  • Optimize images recorded as URLs so we can process them with gatsby-image (or, if building a Gatsby v3 plugin, gatsby-plugin-image).

To start, let’s use Gatsby’s “Hello World” starter to spin up a new example website:

$ gatsby new gtdg-ch9-example-site gatsbyjs/gatsby-starter-hello-world

Because our Gatsby site will depend on our source plugin, we need to create a separate Gatsby project for the source plugin. Fortunately, Gatsby provides a plugin starter to quickly spin up a new plugin project:

$ gatsby new gtdg-ch9-source-plugin gatsbyjs/gatsby-starter-plugin

Now, you should have two directories at the same level named gtdg-ch9-example-site and gtdg-ch9-source-plugin.

Though we could create our source plugin directly within the plugins directory of the Gatsby site we just created, maintaining it as a separate project makes releasing later on easier, in addition to benefiting from separate source control management. Remember that directly adding plugin code to your plugins directory makes it a local plugin.

If you open the gtdg-ch9-source-plugin directory, you’ll see the following files represented. This is the typical structure of an initial source plugin project:

/gtdg-ch9-source-plugin
├── .gitignore
├── gatsby-browser.js
├── gatsby-node.js
├── gatsby-ssr.js
├── index.js
├── LICENSE
├── package.json
├── package-lock.json
└── README.md

Most of our modifications to this plugin starter will be located in the gatsby-node.js file, as that is where we leverage the Gatsby Node API to populate our GraphQL API with the data we retrieve. The gatsby-node.js file allows us to customize and extend the default Gatsby build process at various points.

Installing the Source Plugin

In order to test that our source plugin is working as intended, we need to install it into our example Gatsby site, given that we didn’t introduce it as a local plugin earlier. Because Gatsby only recognizes those plugins identified within the gatsby-config.js file, we need to include it there in our example site. Our plugin is not intended for public use yet, so we’ll use the require.resolve() method, which is used for plugins located elsewhere in the filesystem, to add it:

module.exports = {
  plugins: [
    {
      resolve: require.resolve(`../gtdg-ch9-source-plugin`),
    },
  ],
}
Note

For more information about referring to local plugins that aren’t yet installable because they’re still under development, consult the Gatsby documentation on creating local plugins. You can include a plugin by using require.resolve() and a file path, or by using npm link or yarn link to reference the package.

Now, when we run gatsby develop within the gtdg-ch9-example-site directory, we can see our new plugin represented within the terminal output as Gatsby loads plugins found in the Gatsby configuration file. Note that because we have not modified any code within the plugin itself, especially the package.json file, it still carries the name gatsby-starter-plugin:

$ gatsby develop
success open and validate gatsby-configs - 0.032s
success load plugins - 0.076s
Loaded gatsby-starter-plugin
success onPreInit - 0.017s

This output is printed to the terminal thanks to the prepopulated contents of the gatsby-node.js file in our source plugin, which contains the following onPreInit function:

exports.onPreInit = () => console.log("Loaded gatsby-starter-plugin")

Now that we can confirm our plugin is appearing as expected in the gatsby develop terminal output, we can start getting our hands dirty with the plugin code itself.

Creating GraphQL Nodes

Before we turn our attention to the actual querying process that results in the retrieval of data from an external source, let’s use some hardcoded data within our file to populate some initial nodes in Gatsby’s GraphQL API. To do this, we need to invoke the createNode Gatsby function within the sourceNodes API in the plugin’s gatsby-node.js file, just as we did for programmatic page creation. Replace the contents of gatsby-node.js with the following code:

const POST_NODE_TYPE = `Post`

exports.sourceNodes = async ({ 
  actions,
  createContentDigest,
  createNodeId,
  getNodesByType,
}) => {
  const { createNode } = actions

  const data = {
    posts: [
      { id: 1, description: `My first post!` },
      { id: 2, description: `Post number two!` },
    ],
  }

  // Recurse through data and create Gatsby nodes.
  data.posts.forEach(post =>
    createNode({ 
      ...post,
      id: createNodeId(`${POST_NODE_TYPE}-${post.id}`),
      parent: null,
      children: [],
      internal: { 
        type: POST_NODE_TYPE,
        content: JSON.stringify(post),
        contentDigest: createContentDigest(post),
      },
    })
  )

  return
}

We implement the Gatsby sourceNodes API, one of Gatsby’s Node APIs run during the build process, and extract certain Gatsby utilities that make it easy for us to create nodes, such as createContentDigest and createNodeId.

We store our hardcoded data as an array and recurse our way through it, invoking the createNode method for each individual post represented in the array.

We provide certain required fields, such as a node identifier and a content digest, which includes the entirety of the content (in this case, the value of post). Gatsby uses these to track nodes whose content has changed.

If you’ve been following along with this example, you can now run gatsby develop and point your browser to https://localhost:8000/___graphql to test our GraphQL API in GraphiQL. Issue the following query:

{
  allPost {
    edges {
      node {
        id
        description
      }
    }
  }
}

You’ll receive a response that consists of our hardcoded data, as illustrated in Figure 9-1.

Figure 9-1. The result of our allPost query containing the two hard-coded blog posts represented in our custom source plugin

Unfortunately, we’re not quite done, as the data represented in the GraphQL API’s response is static and hardcoded. In other words, it doesn’t reflect the sort of dynamic, evolving data we often need to query from APIs in the wild. But now that the source plugin logic to make those posts available to the GraphQL API is working as intended, we can narrow our focus to solely the querying process.

Querying and Sourcing Remote Data

Because there are so many approaches to querying data from external providers, including not just distinct API specifications but also distinct request clients, it’s far outside the scope of this book to cover the full range of libraries available (like http.get, axios, and node-fetch, all of which are built into Node.js).

In this walkthrough, we’ll use a GraphQL client, which enables our source plugin to query the external API. Optionally, our source plugin could also use subscriptions to preemptively update the data in the site when the data exposed through the API changes. To provide a query mechanism, we’ll use an Apollo client, which we’ll need to install into our source plugin project along with the other dependencies. Remember you can retrieve your data however you see fit using the APIs and libraries most appropriate for your requirements.

To follow along, within the gtdg-ch9-source-plugin directory, execute the following command:

$ npm install apollo-cache-inmemory apollo-client apollo-link  
  apollo-link-http apollo-link-ws apollo-utilities graphql graphql-tag 
  node-fetch ws subscriptions-transport-ws

If you open your source plugin’s package.json file, you’ll see a dependencies section at the end of the file that lists all of the packages you just installed.

Configuring an Apollo client to retrieve data

Now, let’s reopen that gatsby-node.js file within our in-progress source plugin to begin integrating our upstream GraphQL API into our project so that we can use our Apollo client as a means to query authentic, not just hardcoded, data. First, we’ll need to import the dependencies we just introduced. We do this at the start of the file, before the first const statement:

const { ApolloClient } = require("apollo-client")
const { InMemoryCache } = require("apollo-cache-inmemory")
const { split } = require("apollo-link")
const { HttpLink } = require("apollo-link-http")
const { WebSocketLink } = require("apollo-link-ws")
const { getMainDefinition } = require("apollo-utilities")
const fetch = require("node-fetch")
const gql = require("graphql-tag")
const WebSocket = require("ws")

const POST_NODE_TYPE = `Post`

exports.sourceNodes = async ({
// ...

The next step in our process is to add code that will configure our Apollo client to subscribe to data in the upstream GraphQL API. This code comes before we write a sourceNodes function and after we define the POST_NODE_TYPE constant, such that our plugin’s gatsby-node.js file now appears as follows:

const { ApolloClient } = require("apollo-client")
const { InMemoryCache } = require("apollo-cache-inmemory")
const { split } = require("apollo-link")
const { HttpLink } = require("apollo-link-http")
const { WebSocketLink } = require("apollo-link-ws")
const { getMainDefinition } = require("apollo-utilities")
const fetch = require("node-fetch")
const gql = require("graphql-tag")
const WebSocket = require("ws")

const POST_NODE_TYPE = `Post`

const client = new ApolloClient({
  link: split(
    ({ query }) => {
      const definition = getMainDefinition(query)
      return (
        definition.kind === "OperationDefinition" &&
        definition.operation === "subscription"
      )
    },
    new WebSocketLink({
      uri: `ws://localhost:4000`, 
      // or `ws://gatsby-source-plugin-api.glitch.me/`
      options: {
        reconnect: true,
      },
      webSocketImpl: WebSocket,
    }),
    new HttpLink({
      uri: "http://localhost:4000",
      // or `https://gatsby-source-plugin-api.glitch.me/`
      fetch,
    })
  ),
  cache: new InMemoryCache(),
})

exports.sourceNodes = async ({
// ...

We’ll run through the entire file at the end of the process step by step, but for now, the most important thing to understand from the above sample code is that we’ve made available an Apollo client, a type of request client for GraphQL APIs. This Apollo client allows us to invoke query methods to retrieve data from the configured data source. This example demonstrates both local and deployed versions of the upstream GraphQL API.

Note

For more information about the Apollo packages we installed to make our client work, consult the Apollo documentation. Note that Apollo is just one approach to querying data sources, and it’s a best practice to leverage what works most optimally for your requirements.

Querying data from the API

Now that we have our Apollo client configured in adherence with our requirements, we can issue queries on behalf of Gatsby to retrieve the information we need. Our next step is to replace the hardcoded information within our sourceNodes function by defining a GraphQL query and passing it to the Apollo client.

Replace the following section of code:

exports.sourceNodes = async ({
  actions,
  createContentDigest,
  createNodeId,
  getNodesByType,
}) => {
  const { createNode } = actions

  const data = {
    posts: [
      { id: 1, description: `My first post!` },
      { id: 2, description: `Post number two!` },
    ],
  }

with the code shown here—in the process, we’ll retrieve the other information we need for each post from the API:

exports.sourceNodes = async ({
  actions,
  createContentDigest,
  createNodeId,
  getNodesByType,
}) => {
  const { createNode } = actions

  const { data } = await client.query({
    query: gql`
      query {
        posts {
          id
          description
          slug
          imgUrl
          imgAlt
          author {
            id
            name
          }
        }
        authors {
          id
          name
        }
      }
    `,
  })

Before we move forward, let’s take stock of the current state of the gatsby-node.js file in preparation for introducing another node to represent authors from the API. It should contain the following:

const { ApolloClient } = require("apollo-client")
const { InMemoryCache } = require("apollo-cache-inmemory")
const { split } = require("apollo-link")
const { HttpLink } = require("apollo-link-http")
const { WebSocketLink } = require("apollo-link-ws")
const { getMainDefinition } = require("apollo-utilities")
const fetch = require("node-fetch")
const gql = require("graphql-tag")
const WebSocket = require("ws")

const POST_NODE_TYPE = `Post`

const client = new ApolloClient({ 
  link: split(
    ({ query }) => {
      const definition = getMainDefinition(query)
      return (
        definition.kind === "OperationDefinition" &&
        definition.operation === "subscription"
      )
    },
    new WebSocketLink({
      uri: `ws://gatsby-source-plugin-api.glitch.me/`,
      options: {
        reconnect: true,
      },
      webSocketImpl: WebSocket,
    }),
    new HttpLink({
      uri: `https://gatsby-source-plugin-api.glitch.me/`,
      fetch,
    })
  ),
  cache: new InMemoryCache(),
})

exports.sourceNodes = async ({
  actions,
  createContentDigest,
  createNodeId,
  getNodesByType,
}) => {
  const { createNode } = actions

  const { data } = await client.query({ 
    query: gql`
      query {
        posts {
          id
          description
          slug
          imgUrl
          imgAlt
          author {
            id
            name
          }
        }
        authors {
          id
          name
        }
      }
    `,
  })

  // Recurse through data and create Gatsby nodes.
  data.posts.forEach(post =>
    createNode({ 
      ...post,
      id: createNodeId(`${POST_NODE_TYPE}-${post.id}`),
      parent: null,
      children: [],
      internal: {
        type: POST_NODE_TYPE,
        content: JSON.stringify(post),
        contentDigest: createContentDigest(post),
      },
    })
  )

  return
}

Here we configure a new Apollo client, thanks to the availability of the Apollo dependencies we installed earlier.

We define a new GraphQL query that is used by the Apollo client to issue a request against the upstream GraphQL API our Gatsby site relies on.

We create Gatsby nodes by recursing through the API response and extracting the information we need to populate each individual node.

Now, let’s account for the authors, which need to be represented as nodes in their own right within Gatsby:

const { ApolloClient } = require("apollo-client")
const { InMemoryCache } = require("apollo-cache-inmemory")
const { split } = require("apollo-link")
const { HttpLink } = require("apollo-link-http")
const { WebSocketLink } = require("apollo-link-ws")
const { getMainDefinition } = require("apollo-utilities")
const fetch = require("node-fetch")
const gql = require("graphql-tag")
const WebSocket = require("ws")

const POST_NODE_TYPE = `Post`
const AUTHOR_NODE_TYPE = `Author` 

const client = new ApolloClient({
  link: split(
    ({ query }) => {
      const definition = getMainDefinition(query)
      return (
        definition.kind === "OperationDefinition" &&
        definition.operation === "subscription"
      )
    },
    new WebSocketLink({
      uri: `ws://gatsby-source-plugin-api.glitch.me/`,
      options: {
        reconnect: true,
      },
      webSocketImpl: WebSocket,
    }),
    new HttpLink({
      uri: `https://gatsby-source-plugin-api.glitch.me/`,
      fetch,
    })
  ),
  cache: new InMemoryCache(),
})

exports.sourceNodes = async ({
  actions,
  createContentDigest,
  createNodeId,
  getNodesByType,
}) => {
  const { createNode } = actions

  const { data } = await client.query({
    query: gql`
      query {
        posts {
          id
          description
          slug
          imgUrl
          imgAlt
          author {
            id
            name
          }
        }
        authors {
          id
          name
        }
      }
    `,
  })

  // Recurse through data and create Gatsby nodes.
  data.posts.forEach(post =>
    createNode({
      ...post,
      id: createNodeId(`${POST_NODE_TYPE}-${post.id}`),
      parent: null,
      children: [],
      internal: {
        type: POST_NODE_TYPE,
        content: JSON.stringify(post),
        contentDigest: createContentDigest(post),
      },
    })
  )
  data.authors.forEach(author =>
    createNode({ 
      ...author,
      id: createNodeId(`${AUTHOR_NODE_TYPE}-${author.id}`),
      parent: null,
      children: [],
      internal: {
        type: AUTHOR_NODE_TYPE,
        content: JSON.stringify(author),
        contentDigest: createContentDigest(author),
      },
    })
  )

  return
}

First, we add another constant representing the type name of the Author nodes. This should remain consistent across the site, just like the type name of the Post nodes.

Second, we include a recursive handler for author nodes that runs through each individual author retrieved from the API and generates a new Gatsby node for each one.

We can test this again by issuing the following query in GraphiQL after running gatsby develop on our example site and navigating to https://localhost:8000/___graphql:

{
  allPost {
    edges {
      node {
        id
        description
        imgUrl
      }
    }
  }
  allAuthor {
    edges {
      node {
        id
        name
      }
    }
  }
}

The result of our GraphQL query is illustrated in Figure 9-2.

Figure 9-2. The result of our GraphQL query after introducing handling for Author nodes in addition to Post nodes

Optimizing remote images and creating remote File nodes

A common requirement of Gatsby sites is to retrieve image URLs from an external resource before optimizing those remote images for use in Gatsby by means of the gatsby-image component. In our scenario, as you can see in Figure 9-2, each post in the upstream API is accompanied by an imgUrl field containing an Unsplash image URL. With source plugins, we can preemptively retrieve the images we need and optimize them to prevent any impact on performance due to fetching remote images.

In order for us to be able to optimize images based on their remote URLs, we need to add File nodes to Gatsby to represent each of the remote images. Then, we need to install the required plugins to both find the images and provide needed data for the gatsby-image component. Let’s start by installing one source plugin we know we’ll need from the get-go into our custom source plugin:

$ npm install gatsby-source-filesystem

Note that our in-progress source plugin has gatsby-source-filesystem as a dependency. It’s perfectly appropriate to have plugins, even source plugins, depend on one another in hierarchically complex ways. Next, we need to implement another Gatsby API, onCreateNode. For each node created in Gatsby, we want to check if it was a post and, if so, create a file based on the imgUrl field therein.

To do this, we need to first add the Gatsby helper we need to create the File node, namely createRemoteFileNode, within our dependency imports at the top of the file:

const { ApolloClient } = require("apollo-client")
const { InMemoryCache } = require("apollo-cache-inmemory")
const { split } = require("apollo-link")
const { HttpLink } = require("apollo-link-http")
const { WebSocketLink } = require("apollo-link-ws")
const { getMainDefinition } = require("apollo-utilities")
const fetch = require("node-fetch")
const gql = require("graphql-tag")
const WebSocket = require("ws")
const { createRemoteFileNode } = require(`gatsby-source-filesystem`)

Then we’ll export a new onCreateNode function at the bottom of the file and invoke the createRemoteFileNode helper inside for each creation of a Post node:

exports.onCreateNode = async ({
  node, // i.e. the just-created node
  actions: { createNode },
  createNodeId,
  getCache,
}) => {
  if (node.internal.type === POST_NODE_TYPE) { 
    const fileNode = await createRemoteFileNode({ 
      // The remote image URL for which to generate a node.
      url: node.imgUrl,
      parentNodeId: node.id,
      createNode,
      createNodeId,
      getCache,
    })

    if (fileNode) {
      node.remoteImage___NODE = fileNode.id 
    }
  }
}

Each time a node is created (i.e., when we call createNode), we check to see if the node is a Post.

If it is, then we create a remote node, which returns a fileNode.

Finally, we define the just-created File node’s id value as a field named remoteImage__NODE.

In Gatsby, this last step establishes a clear relationship between this field and the File node, thus allowing File node fields to be queried via this relationship from the Post node. In Gatsby parlance, this is known as inference. If we were to exclude the ___NODE suffix, which establishes the relationship with the Post nodes, we would solely be able to retrieve the identifier of remoteImage through the following query:

{
  allPost {
    edges {
      node {
        remoteImage
        # Returns a UUID.
      }
    }
  }
}

But that’s not very helpful when we need information from the File node to which the identifier found in the Post node refers. Adding the ___NODE suffix to establish the needed relationship across different node types enables us to query the related node’s internals as well:

{
  allPost {
    edges {
      node {
        remoteImage {
          id
          relativePath
        }
      }
    }
  }
}

The final state of our source plugin’s gatsby-node.js file is as follows:

const { ApolloClient } = require("apollo-client")
const { InMemoryCache } = require("apollo-cache-inmemory")
const { split } = require("apollo-link")
const { HttpLink } = require("apollo-link-http")
const { WebSocketLink } = require("apollo-link-ws")
const { getMainDefinition } = require("apollo-utilities")
const fetch = require("node-fetch")
const gql = require("graphql-tag")
const WebSocket = require("ws")
const { createRemoteFileNode } = require(`gatsby-source-filesystem`)

const POST_NODE_TYPE = `Post`
const AUTHOR_NODE_TYPE = `Author`

const client = new ApolloClient({
  link: split(
    ({ query }) => {
      const definition = getMainDefinition(query)
      return (
        definition.kind === "OperationDefinition" &&
        definition.operation === "subscription"
      )
    },
    new WebSocketLink({
      uri: `ws://gatsby-source-plugin-api.glitch.me/`,
      options: {
        reconnect: true,
      },
      webSocketImpl: WebSocket,
    }),
    new HttpLink({
      uri: `https://gatsby-source-plugin-api.glitch.me/`,
      fetch,
    })
  ),
  cache: new InMemoryCache(),
})

exports.sourceNodes = async ({
  actions,
  createContentDigest,
  createNodeId,
  getNodesByType,
}) => {
  const { createNode } = actions

  const { data } = await client.query({
    query: gql`
      query {
        posts {
          id
          description
          slug
          imgUrl
          imgAlt
          author {
            id
            name
          }
        }
        authors {
          id
          name
        }
      }
    `,
  })

  // Recurse through data and create Gatsby nodes.
  data.posts.forEach(post =>
    createNode({
      ...post,
      id: createNodeId(`${POST_NODE_TYPE}-${post.id}`),
      parent: null,
      children: [],
      internal: {
        type: POST_NODE_TYPE,
        content: JSON.stringify(post),
        contentDigest: createContentDigest(post),
      },
    })
  )

  data.authors.forEach(author =>
    createNode({
      ...author,
      id: createNodeId(`${AUTHOR_NODE_TYPE}-${author.id}`),
      parent: null,
      children: [],
      internal: {
        type: AUTHOR_NODE_TYPE,
        content: JSON.stringify(author),
        contentDigest: createContentDigest(author),
      },
    })
  )

  return
}

exports.onCreateNode = async ({
  node, // i.e. the just-created node
  actions: { createNode },
  createNodeId,
  getCache,
}) => {
  if (node.internal.type === POST_NODE_TYPE) {
    const fileNode = await createRemoteFileNode({
      // The remote image URL for which to generate a node.
      url: node.imgUrl,
      parentNodeId: node.id,
      createNode,
      createNodeId,
      getCache,
    })
    if (fileNode) {
      node.remoteImage___NODE = fileNode.id
    }
  }
}

We now have local image files generated from the remote image URLs, and we’ve established the association between our images and posts. But we still haven’t done anything with the transformer plugins we need to make these images available for use with the gatsby-image component.

Note

We cover schema customization in Gatsby briefly later in this chapter and at length in Chapter 13. If you’d like an early look at Gatsby’s schema customization APIs, you can also consult the Gatsby documentation on the subject.

Transforming File nodes with Sharp plugins

As we saw in our examination of the gatsby-image component in Chapter 7, Sharp plugins are what make the optimization of Gatsby v2 images possible during build-time compilation. We can leverage those same plugins to apply optimizations to the images we now have available in our filesystem. Because these transformations occur at build time, we need to install Sharp dependencies in the example site (i.e., within the gtdg-ch9-example-site directory), unless we plan to package them with the source plugin. Execute the following command to install the required dependencies:

$ npm install gatsby-plugin-sharp gatsby-transformer-sharp

Then, add both plugins to the Gatsby configuration file:

module.exports = {
  plugins: [
    {
      resolve: require.resolve(`../gtdg-ch9-source-plugin`),
    },
    `gatsby-plugin-sharp`,
    `gatsby-transformer-sharp`,
  ],
}

Now that we’ve installed the Sharp plugins, we know that they’ll execute after the source plugin has populated our GraphQL API. In the process, the Sharp plugins will transform each File node into an optimized image set and incorporate fields for the optimized images within the childImageSharp field. The gatsby-transformer-sharp plugin seeks out File nodes with appropriate image extensions (like .jpg and .png), creates the optimized versions, and generates the GraphQL fields on our behalf.

Now, when we start up the development server, we’ll be able to access those optimized images through GraphiQL with the following query, whose response is illustrated in Figure 9-3:

{
  allPost {
    edges {
      node {
        remoteImage {
          childImageSharp {
            id
          }
        }
      }
    }
  }
}
Figure 9-3. The response for the preceding query, showing the individual Sharp-optimized images now available as part of each Post node

Establishing Foreign Key Relationships

We’ve successfully established relationships between two nodes that have interrelated information, namely our Post and File nodes, through the ___NODE suffix in our gatsby-node.js file. But we also need to draw similar referential relationships between Post and Author nodes so that we can generate author pages listing the blog posts they’ve contributed. In order to link those two nodes together, we need to establish a foreign key relationship.

The quickest way to establish such a relationship is to customize the GraphQL schema to account for the association between the Post and Author types. Through an implementation of the createSchemaCustomization API, you can define how a node’s data structures look and, in the process, link a node to other nodes to establish a relationship.

Add the following createSchemaCustomization function to the bottom of the source plugin’s gatsby-node.js file:

exports.createSchemaCustomization = ({ actions }) => {
  const { createTypes } = actions
  createTypes(`
    type Post implements Node {
      id: ID!
      slug: String!
      description: String!
      imgUrl: String!
      imgAlt: String!
      # Create relationships between Post and File nodes 
      # for optimized images.
      remoteImage: File @link 
      # Create relationships between Post and Author nodes.
      author: Author @link(from: "author.name" by: "name") 
    }
    type Author implements Node {
      id: ID!
      name: String!
    }`
  )
}

We ask Gatsby to find a remoteImage field within Post nodes and link that field to a File node using the identifier in the File node.

We create an Author node type by instructing Gatsby to link together author.name on the Post node and name on an arbitrary Author node in the Author type. As you can see, we can use this name field or any arbitrary field to link these node types together, as opposed to an id.

Because we’ve now established the relationship between Post nodes and File nodes at the schema level rather than upon node creation, which is more brittle, we can remove the ___NODE suffix from the gatsby-node.js file and rely instead on this schema customization. Here is the complete version of our source plugin’s gatsby-node.js file:

const { ApolloClient } = require("apollo-client")
const { InMemoryCache } = require("apollo-cache-inmemory")
const { split } = require("apollo-link")
const { HttpLink } = require("apollo-link-http")
const { WebSocketLink } = require("apollo-link-ws")
const { getMainDefinition } = require("apollo-utilities")
const fetch = require("node-fetch")
const gql = require("graphql-tag")
const WebSocket = require("ws")
const { createRemoteFileNode } = require(`gatsby-source-filesystem`)

const POST_NODE_TYPE = `Post`
const AUTHOR_NODE_TYPE = `Author`

const client = new ApolloClient({
  link: split(
    ({ query }) => {
      const definition = getMainDefinition(query)
      return (
        definition.kind === "OperationDefinition" &&
        definition.operation === "subscription"
      )
    },
    new WebSocketLink({
      uri: `ws://gatsby-source-plugin-api.glitch.me/`,
      options: {
        reconnect: true,
      },
      webSocketImpl: WebSocket,
    }),
    new HttpLink({
      uri: `https://gatsby-source-plugin-api.glitch.me/`,
      fetch,
    })
  ),
  cache: new InMemoryCache(),
})

exports.sourceNodes = async ({
  actions,
  createContentDigest,
  createNodeId,
  getNodesByType,
}) => {
  const { createNode } = actions

  const { data } = await client.query({
    query: gql`
      query {
        posts {
          id
          description
          slug
          imgUrl
          imgAlt
          author {
            id
            name
          }
        }
        authors {
          id
          name
        }
      }
    `,
  })

  // Recurse through data and create Gatsby nodes.
  data.posts.forEach(post =>
    createNode({
      ...post,
      id: createNodeId(`${POST_NODE_TYPE}-${post.id}`),
      parent: null,
      children: [],
      internal: {
        type: POST_NODE_TYPE,
        content: JSON.stringify(post),
        contentDigest: createContentDigest(post),
      },
    })
  )

  data.authors.forEach(author =>
    createNode({
      ...author,
      id: createNodeId(`${AUTHOR_NODE_TYPE}-${author.id}`),
      parent: null,
      children: [],
      internal: {
        type: AUTHOR_NODE_TYPE,
        content: JSON.stringify(author),
        contentDigest: createContentDigest(author),
      },
    })
  )

  return
}

exports.onCreateNode = async ({
  node, // i.e. the just-created node
  actions: { createNode },
  createNodeId,
  getCache,
}) => {
  if (node.internal.type === POST_NODE_TYPE) {
    const fileNode = await createRemoteFileNode({
      // The remote image URL for which to generate a node.
      url: node.imgUrl,
      parentNodeId: node.id,
      createNode,
      createNodeId,
      getCache,
    })
    if (fileNode) {
      node.remoteImage = fileNode.id 
    }
  }
}

exports.createSchemaCustomization = ({ actions }) => { 
  const { createTypes } = actions
  createTypes(`
    type Post implements Node {
      id: ID!
      slug: String!
      description: String!
      imgUrl: String!
      imgAlt: String!
      # Create relationships between Post and File nodes 
      # for optimized images.
      remoteImage: File @link
      # Create relationships between Post and Author nodes.
      author: Author @link(from: "author.name" by: "name")
    }
    type Author implements Node {
      id: ID!
      name: String!
    }`
  )
}

Remove the aforementioned suffix.

Add schema customization.

Now, thanks to these foreign key relationships, we can issue the following query in GraphiQL to get a post’s author and a post’s remote image as part of the allPost query as opposed to in separate queries:

{
  allPost {
    edges {
      node {
        id
        author {
          name
        }
        remoteImage {
          id
        }
      }
    }
  }
}

The result of this query is shown in Figure 9-4.

Figure 9-4. The result of the preceding query in GraphiQL, demonstrating the ability to query related types in a single allPost query

Now, you can use these GraphQL queries in conjunction with page and component queries in order to populate your pages with the necessary information, including not only data from the external GraphQL API but also optimized versions of the remote Unsplash images. A typical page query for a blog’s home page might look something like this:

{
  allPost {
    edges {
      node {
        id
        slug
        description
        imgAlt
        author {
          id
          name
        }
        remoteImage {
          id
          childImageSharp {
            id
            fluid {
              ...GatsbyImageSharpFluid
            }
          }
        }
      }
    }
  }
}

Using Plugin Options to Allow Customization

Earlier in this chapter, I introduced the concept of plugin options to allow plugin users to customize the plugin configuration according to their needs. Source plugins also need to be configured, including information such as the arbitrary URL at which an external API or database is located.

Consider a scenario where our source plugin requires some additional option (say, optionC). To account for this, we can add an option to gatsby-config.js to pass the value in to the source plugin:

module.exports = {
  plugins: [
    {
      resolve: require.resolve(`../gtdg-ch9-source-plugin`),
      options: {
        optionC: true,
      },
    },
    `gatsby-plugin-sharp`,
    `gatsby-transformer-sharp`,
  ],
}

Within our source plugin, to handle the plugin option appropriately, we can either introduce an options schema with validation or use it within the files we use to hook into Gatsby APIs, like our implementation of the sourceNodes API in gatsby-node.js:

exports.sourceNodes = async ({
  actions,
  createContentDigest,
  createNodeId,
  getNodesByType
}, pluginOptions) => {
  const { createNode, touchNode, deleteNode } = actions
  console.log(pluginOptions.optionC) // true
}

Fortunately, plugin options work the same way across the board, whether you’re building a generic, source, or transformer plugin.

Note

Enabling GraphQL subscriptions on your Apollo client is well outside the scope of this book, but the Gatsby documentation provides an exhaustive tutorial for proactively updating data with subscriptions as part of a longer walkthrough that covers the same material as this source plugin overview.

Creating Transformer Plugins

Transformer plugins comprise the other class of plugins that operate on Gatsby’s data layer. The most logical way to think about the relationship between source plugins and transformer plugins is as follows: if source plugins populate nodes with external data, then transformer plugins transform that data into new output nodes or new node fields in the same node.

Many Gatsby sites require both source and transformer plugins to function properly, as it’s often the case that the data we receive from an external source isn’t ready for use within the Gatsby GraphQL API. Due to this loose coupling between source and transformer plugins, it’s eminently possible to create arbitrary but highly complex data transformation pipelines in the Gatsby context.

Tip

For a comprehensive walkthrough of an end-to-end transformer plugin implementation process, consult the Gatsby documentation’s how-to guide on creating a Remark transformer plugin.

Reviewing an Example: gatsby-transformer-yaml

Just like other types of plugins, transformer plugins are typical NPM packages, containing a package.json file enumerating dependencies as well as a gatsby-node.js file where we can implement Gatsby Node APIs. One example of a transformer plugin that is common in the Gatsby community is gatsby-transformer-yaml, which accepts YAML nodes (e.g., a .yml file of media type text/yaml) and outputs JavaScript objects that represent the YAML data structure.

Let’s work through a very rudimentary example of transformer plugins using the YAML-to-JavaScript example use case. To avoid the complexity of source plugins entirely, we’ll use gatsby-source-filesystem to account for our untransformed YAML files as File nodes rather than an external service. Each YAML file looks like the following:

# src/data/sample.yml
- name: Toni Morrison
  description: Author of The Bluest Eye, Song of Solomon, and Beloved
- name: Alice Walker
  description: Author of The Color Purple, Meridian, and >>>
The Third Life of George Copeland

Given a collection of YAML files that resemble this one in structure, how can we transform these into JavaScript objects that Gatsby can understand and manipulate?

Ensuring Needed Data Is Sourced

Because transformer plugins often work in tandem with source plugins, let’s first make sure our gatsby-source-filesystem plugin is pulling appropriately from the local filesystem. If all of our YAML files are located in src/data/ within our Gatsby site, we can execute the following to install gatsby-source-filesystem:

$ npm install gatsby-source-filesystem

In our Gatsby configuration file, we identify the relevant directory containing the YAML files to the source plugin:

module.exports = {
  plugins: [
    {
      resolve: `gatsby-source-filesystem`,
      options: {
        path: `./src/data/`,
      },
    },
  ],
}

As we know from our previous experiences with the gatsby-source-filesystem plugin, each of these files will then be exposed in the GraphQL API as File nodes and therefore can be queried as follows:

{
  allFile {
    edges {
      node {
        internal {
          type
          mediaType
          description
          owner
        }
      }
    }
  }
}

Once the sourcing process is complete, we now have a collection of File nodes that represent our YAML files. But Gatsby still cannot work with the data contained inside.

Transforming Nodes

Just like with source plugins, we need to leverage Gatsby’s Node APIs to conduct the needed data transformation within gatsby-node.js. We’ll make use of the onCreateNode API, which is a lifecycle API traversed upon each File node’s creation, to conduct the transformation for us and make the information available to GraphQL.

To start, we need to install several dependencies, including js-yaml and lodash:

$ npm install js-yaml lodash

To keep things simple, we’ll write the plugin directly in the site’s logic before extracting it out as a plugin later. Within the site’s gatsby-node.js file, we’ll add the following:

const jsYaml = require(`js-yaml`) 

exports.onCreateNode = async ({ node, loadNodeContent }) => { 
  function transformObject(object, id, type) { 
    const yamlNode = {
      ...object,
      id,
      children: [],
      parent: null,
      internal: {
        contentDigest: createContentDigest(object),
        type,
      },
    }
    createNode(yamlNode)
  }

  // Only handle nodes of mediaType `text/yaml`.
  if (node.internal.mediaType !== `text/yaml`) {
    return
  }
  const content = await loadNodeContent(node)
  const parsedContent = jsYaml.load(content) 
}

Import the js-yaml library as a dependency. This will perform the conversion of YAML to JavaScript objects on our behalf.

Implement the onCreateNode API to detect whether a File node is a YAML file.

This is a transformation function that will serve as a helper function to parse the YAML files into Gatsby nodes that match their structure, creating a new yamlNode with each recursion.

If the File node is a YAML file, invoke js-yaml.load() to perform the transformation.

Establishing the Transformer Relationship

Next, we need to create the relationship between the parent File node and the child nodes (transformed YAML content) by using the createParentChildLink function. If we don’t do this, Gatsby has no clue which transformed YAML nodes are associated with which File nodes sourced from the filesystem. The transformer relationship will center on the parent File node’s id value, which we add to each yamlNode to forge the connection:

const jsYaml = require(`js-yaml`)

exports.onCreateNode = async ({ node, loadNodeContent }) => {
  function transformObject(object, id, type) {
    const yamlNode = {
      ...object,
      id,
      children: [],
      parent: node.id, 
      internal: {
        contentDigest: createContentDigest(object),
        type,
      },
    }
    createNode(yamlNode)
    createParentChildLink({ 
      parent: node,
      child: yamlNode,
    })
  }

  // Only handle nodes of mediaType `text/yaml`.
  if (node.internal.mediaType !== `text/yaml`) {
    return
  }
  const content = await loadNodeContent(node)
  const parsedContent = jsYaml.load(content)
}

Identify the parent of each transformed YAML object as the File node it was transformed from.

Establish a link between the parent File node and the child transformed YAML node.

Note

For other examples of transformation relationships, consult the source code for the gatsby-transformer-remark plugin and the gatsby-transformer-sharp plugin.

Creating New Nodes from Derived Data and Querying

Our final step is to generate new nodes based on the transformed data by iterating through each transformed YAML object within the onCreateNode function and invoking our transformation helper function where needed. We’ll modify our gatsby-node.js file to look like the following:

const jsYaml = require(`js-yaml`)
const _ = require(`lodash`) 

exports.onCreateNode = async ({
  node,
  actions, 
  loadNodeContent,
  createNodeId,
  createContentDigest,
}) => {
  function transformObject(object, id, type) {
    const yamlNode = {
      ...object,
      id,
      children: [],
      parent: node.id,
      internal: {
        contentDigest: createContentDigest(object),
        type,
      },
    }
    createNode(yamlNode)
    createParentChildLink({
      parent: node,
      child: yamlNode,
    })
  }

  const { createNode, createParentChildLink } = actions 

  // Only handle nodes of mediaType `text/yaml`.
  if (node.internal.mediaType !== `text/yaml`) {
    return
  }
  const content = await loadNodeContent(node)
  const parsedContent = jsYaml.load(content)

  parsedContent.forEach( (object, i) => { 
    transformObject(
      object,
      object.id ? object.id : createNodeId(`${node.id} [${i}] >>> YAML`),
      _.upperFirst(_.camelCase(`${node.name} Yaml`))
    )
  })
}

Import lodash as a dependency for its casing helper functions.

Pull additional elements that we’ll need later from the onCreateNode API, including actions, createNodeId, and createContentDigest.

Extract the createNode and createParentChildLink functions from actions.

Iterate through our transformed YAML objects to populate new nodes with them.

Now, we can query for our YAML directly within the Gatsby GraphQL API as follows:

{
  allSampleYaml {
    edges {
      node {
        id
        name
        description
      }
    }
  }
}
Tip

Transformations from one format to another can be time-consuming and computationally expensive. To avoid having to repeat the entire process for each build and avoid some of the latency, we can employ Gatsby’s global cache. For more information, consult the Gatsby documentation page about using the Gatsby cache in transformer plugins.

Publishing and Maintaining Plugins

Once you’ve created a plugin, whatever its type, it’s time to consider releasing it to the Gatsby community so others can benefit from it. Of course, there is nothing stopping you from keeping your plugin private and using it locally, and many organizations leveraging Gatsby do just that with proprietary, closed-source plugins. However, given that Gatsby is an open source framework, it’s in the best interests of the community and ecosystem to consider publishing and maintaining your plugin.

Note

The NPM documentation contains a guide to contributing packages to the NPM registry, and the Gatsby documentation provides guides for required files in plugins and a README template for contributing plugins to the community.

Submitting Plugins to the Gatsby Plugin Library

One of the easiest ways to ensure your plugin reaches a wider audience is to submit it to the Gatsby Plugin Library, a public resource for Gatsby users seeking plugins that satisfy different requirements. Follow these steps to ensure your plugin is included:

  1. Publish a package to the NPM registry.

  2. Include any required files within your plugin repository—usually package.json and a Gatsby API file such as gatsby-node.js.

  3. Add a keywords field to your plugin’s package.json file, containing at minimum gatsby and gatsby-plugin. If your plugin is a theme (see the next chapter), also add gatsby-theme.

  4. Add documentation for your plugin, including at minimum a README file.

Once Gatsby’s official Algolia account reindexes all Gatsby plugins meant to be represented in the Plugin Library, you’ll be able to find your plugin there and share the link with your team or others.

Warning

If referring to an image for your plugin, be sure to use an absolute URL rather than a relative URL, as the information is used in various locations.

Maintaining Plugins

Maintaining a plugin for the long haul can be difficult work. Open source maintenance is often a thankless job that is poorly compensated, if at all. Nonetheless, long-term maintenance is a core responsibility of plugin authors. Once the development work is complete, it’s time to consider how to handle issues like security vulnerabilities, semantic versioning, and dependency updates that don’t break Gatsby sites.

Handling plugin patches and improvements

Gatsby, like many other JavaScript ecosystems, recommends semantic versioning to manage your releases. Though a full overview of semantic versioning is well outside the scope of this book, it’s best to adhere to some best practices. The first public release of a plugin is often assigned version 0.1.0 or 1.0.0, the latter of which is generally reserved for the first stable release of a package.

When bugs are fixed or minor issues resolved, that generally results in a patch release (from 0.1.0 to 0.1.1 or 1.0.0 to 1.0.1). If you’ve added or modified features in the plugin, that is broadly considered grounds for a minor release (from 0.1.0 to 0.2.0 or 1.0.0 to 1.1.0). Finally, major releases are reserved for changes that break the API or disrupt backward compatibility.

Tip

For more information about semantic versioning, consult Semantic Versioning 2.0.0.

Writing a README and documentation

The README file in your plugin repository should also be very clear about major version releases so your plugin users don’t get confused. If you lack a separate documentation site, the README is your primary opportunity to give plugin users the rationales for changes to your plugin, working examples, and an account of how major version releases evolve the project.

Gatsby recommends performing spring cleaning of your plugin repository every so often to clear out older versions that are no longer needed. In the vast majority of cases, only the last few versions are required.

Managing dependency versions

The Gatsby documentation recommends two distinct tools for verifying and managing dependency versions. In JavaScript ecosystems, dependency management can be particularly difficult, but projects like Version Lens for Visual Studio Code and the npm-check-updates command-line interface offer useful introspection and automated management capabilities.

While Version Lens provides a tool to update dependencies directly from within your package.json file in your development tool, namely by displaying the latest version number available for a given package, using npm-check-updates offers a more active means of determining if any dependencies are obsolete. To use npm-check-updates, install the package and then execute the ncu command:

$ npm install -g npm-check-updates
$ ncu -u

Remember that the primary motivation for releasing a plugin to the community is to benefit those users in the community who need it. As such, it may also be a good idea to provide user support and example codebases for potential users. Gatsby, for instance, uses Discord as its primary conduit for community support. The more resources you have available for your users, the more likely they are to adopt your plugin and perhaps even contribute to it on their own time.

Note

For more information about dependency version management, consult the Version Lens documentation and npm-check-updates documentation, respectively.

Conclusion

Plugins are the primary way to extend functionality for any Gatsby site, while starters offer foundations on which other Gatsby developers can build. Generic, source, and transformer plugins serve as essential entry points for Gatsby developers looking to customize or add functionality. In addition, plugin options and options schemas undergird the configuration system in Gatsby, allowing for arbitrary settings to be enabled and disabled on a per-plugin basis.

Though source plugins are responsible for retrieving data and transformer plugins for converting data, the two classes of plugins share much more in common than it may seem. Plugin options work precisely the same way across both, as do the release and maintenance processes. This brings us to our next subject, Gatsby themes, which are a special type of plugin with some unusual quirks and features.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
44.220.62.183