We started to get organized in the previous chapter by changing the architecture and adding checks for coding standards and best practices. In this chapter, we’ll take this a little further by splitting the code into multiple files and adding tools to ease the process of development. We’ll use Webpack to help us split front-end code into component-based files, inject code into the browser incrementally, and refresh the browser automatically on front-end code changes.
Some of you may find this chapter not worth your time because it is not making any progress on the real features of the application, and/or because it discusses nothing about the technologies that make up the stack. That’s a perfectly valid thought if you are not too concerned about all these, and instead rely on someone else to give you a template of sorts that predefines the directory structure as well as has configuration for the build tools such as Webpack. This can let you focus on the MERN stack alone, without having to deal with all the tooling. In that case, you have the following options:
Download the code from the book’s GitHub repository (https://github.com/vasansr/pro-mern-stack-2) as of the end of this chapter and use that as your starting point for your project.
Use the starter-kit create-react-app (https://github.com/facebook/create-react-app) to start your new React app and add code for your application. But note that create-react-app deals only with the React part of the MERN stack; you will have to deal with the APIs and MongoDB yourself.
Use mern.io (http://mern.io) to create the entire application’s directory structure, which includes the entire MERN stack.
But if you are an architect or just setting up the project for your team, it is important to understand how tools help developer productivity and how you can have finer control over the whole build and deployment process as well. In which case, I encourage you not to skip this chapter, even if you use one of these scaffolding tools, so that you learn about what exactly happens under the hood.
Back-End Modules
You saw all this while in api/server.js how to include modules in a Node.js file. After installing the module, we used the built-in function require() to include it. There are various standards for modularization in JavaScript, of which Node.js has implemented a slight variation of the CommonJS standard. In this system, there are essentially two key elements to interact with the module system: require and exports.
The require() element is a function that can be used to import symbols from another module. The parameter passed to require() is the ID of the module. In Node's implementation, the ID is the name of the module. For packages installed using npm, this is the same as the name of the package and the same as the sub-directory inside the node_modules directory where the package’s files are installed. For modules within the same application, the ID is the path of the file that needs to be imported.
For example, to import symbols from a file called other.js in the same directory as api/server.js, the ID that needs to be passed to require() is the path of this file, that is, './other.js', like this:
const other = require('./other.js');
Now, whatever is exported by other.js will be available in the other variable. This is controlled by the other element we talked about: exports. The main symbol that a file or module exports must be set in a global variable called module.exports within that file, and that is the one that will be returned by the function call to require(). If there are multiple symbols, they can be all set as properties in an object, and we can access those by dereferencing the object or by using the destructuring assignment.
To start, let’s separate the function GraphQLDate() from the main server.js file and create a new file called graphql_date.js for this. Apart from the entire function itself, we also need the following in the new file:
1.
require() statements to import GraphQLScalarType and Kind from the other packages.
2.
The variable module.exports to be set to the function, so that it can be used after importing the file.
The contents of this file are shown in Listing 8-1, with changes from the original in api/server.js highlighted in bold.
const { GraphQLScalarType } = require('graphql');
const { Kind } = require('graphql/language');
const GraphQLDate = new GraphQLScalarType({
...
});
module.exports = GraphQLDate;
Listing 8-1
api/graphql_date.js: Function GraphQLDate() in a New File
Now, in the file api/server.js, we can import the symbol GraphQLDate like this:
...
const GraphQLDate = require('graphql_date.js');
...
As you can see, whatever was assigned to module.exports is the value that a call to require() returns. Now, this variable, GraphQLDate can be seamlessly used in the resolver, as before. But we won’t be making this change in server.js just yet, because we’ll be making more changes to this file.
The next set of functions that we can separate out are the functions relating to the about message. Though we used an anonymous function for the resolver about, let’s now create a named function so that it can be exported from a different file. Let’s create a new file that exports the two functions getMessage() and setMessage() called about.js in the API directory. The contents of this file are quite straightforward, as listed in Listing 8-2. But instead of exporting just one function as we did in graphql_date.js, let’s import both the setMessage and getMessage as two properties in an object.
let aboutMessage = 'Issue Tracker API v1.0';
function setMessage(_, { message }) {
...
}
function getMessage() {
return aboutMessage;
}
module.exports = { getMessage, setMessage };
Listing 8-2
api/about.js: Separated About Message Functionality to New File
Now, we can import the about object from this file and dereference about.getMessage and about.setMessage when we want to use them in the resolver, like this:
...
const about = require('about.js');
...
const resolvers = {
Query: {
about: about.getMessage,
...
},
Mutation: {
setAboutMessage: about.setMessage,
...
},
...
};
This change could be in server.js, but we’ll be separating all this into one file that deals with Apollo Server, the schema, and the resolvers. Let’s create that file now and call it api/api_handler.js. Let’s move the construction of the resolvers object and the creation of the Apollo server into this file. As for the actual resolver implementations, we’ll import them from three other files—graphql_date.js, about.js, and issue.js.
As for the export from this file, let’s export a function that will do what applyMiddleware() did as part of server.js. We can call this function installHandler() and just call applyMiddleware() within this function.
The full contents of this new file are shown in Listing 8-3, with changes compared to the original code in server.js highlighted.
api/api_handler.js: New File to Separate the Apollo Server Construction
We have not created issue.js yet, which is needed for importing the issue-related resolvers. But before that, let’s separate the database connection creation and a function to get the connection handler into a new file. The issue.js file will need this DB connection, etc.
Let’s call the file with all database-related code db.js and place it in the API directory. Let’s move the functions connectToDb() and getNextSequence() into this file, as also the global variable db, which stores the result of the connection. Let’s export the two functions as they are. As for the global connection variable, let’s expose it via a getter function called getDb(). The global variable url can also now be moved into the function connectDb() itself.
The contents of this file are shown in Listing 8-4, with changes from the original in server.js highlighted in bold.
api/db.js: Database Related Functions Separated Out
Now, we are ready to separate the functions related to the Issue object. Let’s create a file called issue.js under the API directory and move the issue related to this file. In addition, we’ll have to import the functions getDb() and getNextSequence() from db.js. to use them. Then, instead of using the global variable db directly, we’ll have to use the return value of getDb(). As for exports, we can export the functions issueList and issueAdd, but now that they are within the module, their names can be simplified to just list and add. The contents of this new file are shown in Listing 8-5.
Finally, we can make changes to the fileapi/server.js to use all this. After all the code has been moved out to individual files, what’s left is only the instantiation of the application, applying the Apollo Server middleware and then starting the server. The contents of this entire file are listed in Listing 8-6. The removed code is not explicitly shown. New code is highlighted in bold.
console.log(`API server started on port ${port}`);
});
} catch (err) {
console.log('ERROR:', err);
}
}());
Listing 8-6
api/server.js: Changes After Moving Out Code To Other Files
Now, the application is ready to be tested. You can do that via the Playground as well as using the Issue Tracker application UI to ensure that things are working as they were before the modularization of the API server code.
Front-End Modules and Webpack
In this section, we’ll deal with the front-end, or the UI code, which is all in one big file, called App.jsx. Traditionally, the approach to using split client-side JavaScript code was to use multiple files and include them all (or whichever are required) using <script> tags in the main HTML file or index.html. This is less than ideal because the dependency management is done by the developer, by maintaining a certain order of files in the HTML file. Further, when the number of files becomes large, this becomes unmanageable.
Tools such as Webpack and Browserify provide alternatives. Using these tools, dependencies can be defined using statements equivalent to require() that we used in Node.js. The tools then automatically determines not just the application’s own dependent modules, but also third-party library dependencies. Then, they put together these individual files into one or just a few bundles of pure JavaScript that has all the required code that can be included in the HTML file.
The only downside is that this requires a build step. But then, the application already has a build step to transform JSX and ES2015 into plain JavaScript. It’s not much of a change to let the build step also create a bundle based on multiple files. Both Webpack and Browserify are good tools and can be used to achieve the goals. But I chose Webpack, because it is simpler to get all that we want done, which includes separate bundles for third-party libraries and our own modules. It has a single pipeline to transform, bundle, and watch for changes and generate new bundles as fast as possible.
If you choose Browserify instead, you will need other task runners such as gulp or grunt to automate watching and adding multiple transforms. This is because Browserify does only one thing: bundle. In order to combine bundle and transform (using Babel) and watch for file changes, you need something that puts all of them together, and gulp is one such utility. In comparison, Webpack (with a little help from loaders, which we’ll explore soon) can not only bundle, but can also do many more things such as transforming and watching for changes to files. You don’t need additional task runners to use Webpack.
Note that Webpack can also handle other static assets such as CSS files. It can even split the bundles such that they can be loaded asynchronously. We will not be exercising those aspects of Webpack; instead, we’ll focus on the goal of being able to modularize the client-side code, which is mainly JavaScript at this point in time.
To get used to what Webpack really does, let’s use Webpack from the command line just like we did for the JSX transform using the Babel command linel. Let’s first install Webpack, which comes as a package and a command line interface to run it.
$ cd ui
$ npm install --save-dev webpack@4 webpack-cli@3
We are using the option --save-dev because the UI server in production has no need for Webpack. It is only during the build process that we need Webpack, and all the other tools we will be installing in the rest of the chapter. To ensure we can run Webpack using the command line, let’s check the version that was installed:
$ npx webpack --version
This should print out the version like 4.23.1. Now, let’s “pack” the App.js file and create a bundle called app.bundle.js. This can be done simply by running Webpack on the App.js file and specifying an output option, which is app.bundle.js, both under the public directory.
The 'mode' option has not been set, webpack will fallback to 'production' for this value. Set 'mode' option to 'development' or 'production' to enable defaults for each environment.
You can also set it to 'none' to disable any default behavior. Learn more: https://webpack.js.org/concepts/mode/
To get rid of the warning message, let’s supply the mode as development in the command line:
$ npx webpack public/App.js --output public/app.bundle.js --mode development
The differences between the two modes are the various things Webpack does automatically, such as removing the names of modules, minifying, etc. It’s good to have all these optimizations when building a bundle for deploying in production, but these may hamper debugging and efficient development process.
The resulting file app.bundle.js is hardly interesting and is not very different from App.js itself. Note also that we didn’t run it against the React file App.jsx, because Webpack cannot handle JSX natively. All it did in this case was minify App.js. We did it just to make sure we’ve installed Webpack correctly and are able to run it and create output. To actually let Webpack figure out dependencies and put together multiple files, let’s split the single file App.jsx into two, by taking out the function graphQLFetch and placing it in a separate file.
We could, just as in the back-end code, use the require way of importing other files. But you will notice that most front-end code examples on the Internet use the ES2015-style modules using the import keyword. This is a newer and more readable way of importing. Even Node.js has support for the import statement, but as of version 10 of Node.js, it is experimental. If not, it could have been used for the back-end code as well. Using import makes it mandatory to use Webpack. So, let’s use the ES2015-style import for the front-end code only.
To import another file, the import keyword is used, followed by the element or variable being imported (which would have been the variable assigned to the result of require()), followed by the keyword from, and then the identifier of the file or module. For example, to import graphQLFetch from the file graphQLfetch.js, this is what needs to be done:
...
import graphQLFetch from './graphQLFetch.js';
...
Using the new ES2015 style to export a function is as simple as prefixing the keyword export before the definition of whatever is being exported. Further, the keyword default can be added after export if a single function is being exported, and it can be the result of the import statement directly (or a top-level export). So, let’s create a new file ui/src/graphQLFetch.js with the contents of the same function copied over from App.jsx. We’ll also need the implementation of jsonDateReviver along with the function The contents of this file are shown in Listing 8-7, with export default added to the definition of the function.
const dateRegex = new RegExp('^\d\d\d\d-\d\d-\d\d');
function jsonDateReviver(key, value) {
if (dateRegex.test(value)) return new Date(value);
return value;
}
export default async function graphQLFetch(query, variables = {}) {
...
}
Listing 8-7
ui/src/graphQLFetch.js: New File with Exported Function graphQLFetch
Now, let’s remove the same set of lines from ui/src/App.jsx and replace them with an import statement. This change is shown in Listing 8-8.
...
/* eslint "react/no-multi-comp": "off" */
/* eslint "no-alert": "off" */
import graphQLFetch from './graphQLFetch.js';
const dateRegex = new RegExp('^\d\d\d\d-\d\d-\d\d');
function jsonDateReviver(key, value) {
if (dateRegex.test(value)) return new Date(value);
return value;
}
class IssueFilter extends React.Component {
...
}
async function graphQLFetch(query, variables = {}) {
...
}
...
Listing 8-8
ui/src/App.jsx: Replace graphQLFetch with an Import
At this point, ESLint will show an error to the effect that extensions (.js) in an import statement are unexpected, because this extension can be automatically detected. But it turns out that the import statement is good only to detect .js file extensions, and we’ll soon be importing .jsx files as well. Further, in the back-end code we used the extensions in require() statements. Let’s make an exception to this ESLint rule and always include extensions in import statements, except, of course, for packages installed via npm. This change to the .eslintrc file in the ui/src directory is shown in Listing 8-9.
ui/src/.eslintrc: Exception for Including Extensions in Application Modules
If you are running npm run watch, you’ll find that both App.js and graphQLFetch.js have been created after the Babel transformation in the public directory. If not, you can run npm run compile in the ui directory. Now, let’s run the Webpack command again and see what happens.
$ npx webpack public/App.js --output public/app.bundle.js --mode development
This should result in output like this:
Hash: 4207ff5d100f44fbf80e
Version: webpack 4.23.1
Time: 112ms
Built at: 10/25/2018 10:21:06 PM
Asset Size Chunks Chunk Names
app.bundle.js 16.5 KiB main [emitted] main
Entrypoint main = app.bundle.js
[./public/App.js] 9.07 KiB {main} [built]
[./public/graphQLFetch.js] 2.8 KiB {main} [built]
As you can see in the output, the packing process has included both App.js and graphQLFetch.js. Webpack has automatically figured out that App.js depends on graphQLFetch.js due to the import statement and has included it in the bundle. Now, we’ll need to replace App.js in index.html with app.bundle.js because the new bundle is the one that contains all the required code. This change is shown in Listing 8-10.
...
<script src="/env.js"></script>
<script src="/App.js/app.bundle.js"></script>
</body>
...
Listing 8-10
ui/public/index.html: Replace App.js with app.bundle.js
If you test the application at this point, you should find it working just as before. For good measure, you could also check in the Network tab of the Developer Console in the browser that it is indeed app.bundle.js that is being fetched from the server.
So, now that you know how to use multiple files in the front-end code, we could possibly create many more files like graphQLFetch.js for convenience and modularization. But the process was not simple, as we had to manually Babel transform the files first, and then put them together in a bundle using Webpack. And any manual step can be error prone: one can easily forget the transform and end up bundling an older version of the transformed file.
Transform and Bundle
The good news is that Webpack is capable of combining these two steps, obviating the need for intermediate files. But it can’t do that on its own; it needs some helpers called loaders. All transforms and file types other than pure JavaScript require loaders in Webpack. These are separate packages. To be able to run Babel transforms, we’ll need the Babel loader.
Let’s install this now.
$ cd ui
$ npm install --save-dev babel-loader@8
It’s a bit cumbersome to use this loader in the command line of Webpack. To make the configuration and options easier, Webpack can be supplied configuration files instead. The default file that it looks for is called webpack.config.js. Webpack loads this file as a module using Node.js require(), so we can treat this file as a regular JavaScript, which includes a module.exports variable that exports the properties that specify the transformation and bundling process. Let’s start building this file under the ui directory, with one property: mode. Let’s default it to development as we did in the command line previously.
...
module.exports = {
mode: development,
}
...
The entry property specifies the file that is the starting point from which all dependencies can be determined. In the Issue Tracker application, this file is App.jsx under the src directory. Let’s add this next.
...
entry: './src/App.jsx',
...
The output property needs to be an object with the filename and path as two properties. The path has to be an absolute path. The recommended way to supply an absolute path is by constructing it using the path module and the path.resolve function.
...
const path = require('path');
module.exports = {
...
output: {
filename: 'app.bundle.js',
path: path.resolve(__dirname, 'public'),
},
...
Loaders are specified under the property module, which contains a series of rules as an array. Each rule at the minimum has a test, which is a regex to match files and a use, which specifies the loader to use on finding a match. We’ll use a regex that matches both .jsx and .js files and the Babel loader to run transforms when a file matches this regex, like this:
...
{
test: /.jsx?$/,
use: 'babel-loader',
},
...
The complete file, ui/webpack.config.js, is shown in Listing 8-11.
const path = require('path');
module.exports = {
mode: 'development',
entry: './src/App.jsx',
output: {
filename: 'app.bundle.js',
path: path.resolve(__dirname, 'public'),
},
module: {
rules: [
{
test: /.jsx?$/,
use: 'babel-loader',
},
],
},
};
Listing 8-11
ui/webpack.config.js: Webpack Configuration
Note that we did not have to supply any further options to the Babel loader, since all Webpack did was use the existing Babel transformer. And this used the existing configuration from .babelrc in the src directory.
At this point, you can quickly run the Webpack command line without any parameters and see that the file app.bundle.js is created, without creating any intermediate files. You may have to delete the intermediate files App.js and graphQLFetch.js in the public directory to ensure that this is happening. The command line for doing this is as follows:
$ npx webpack
This may take a bit of time. Also, just like the --watch option for Babel, Webpack too comes with a --watch option, which incrementally builds the bundle, transforming only the changed files. Let’s try this:
$ npx webpack --watch
This command will not exit. Now, if you change one of the files, say graphQLFetch.js, you will see the following output on the console:
Hash: 3fc38bc043fafe268e06
Version: webpack 4.23.1
Time: 53ms
Built at: 10/25/2018 11:09:49 PM
Asset Size Chunks Chunk Names
app.bundle.js 16.6 KiB main [emitted] main
Entrypoint main = app.bundle.js
[./src/graphQLFetch.js] 2.71 KiB {main} [built]
+ 1 hidden module
Note the last line in the output: +1 hidden module. What this really means is that App.jsx was not transformed when only graphQLFetch.js was changed. This is a good time to change the npm scripts for compile and watch to use the Webpack command instead of the Babel command.
Webpack has two modes, production and development, which change the kind of optimizations that are added during transformation. Let’s assume that during development we’ll always use the watch script and to build a bundle for deployment, we’ll use the mode as production. The command line parameters override what is specified in the configuration file, so we can set the mode accordingly in the npm scripts.
The changes needed to do this are shown in Listing 8-12.
"watch": "babel src --out-dir public --watch --verbose"
"watch": "webpack --watch"
},
...
Listing 8-12
ui/package.json: Changes to npm Scripts to Use Webpack Instead of Babel
Now, we are ready to split the App.jsx file into many files. It is recommended that each React component be placed in its own file, especially if the component is a stateful one. Stateless components can be combined with other components when convenient and when they belong together.
So, let’s separate the component IssueList from App.jsx. Then, let’s also separate the first level of components in the hierarchy—IssueFilter, IssueTable, and IssueAdd—into their own files. In each of these, we will export the main component. App.jsx will import IssueList.jsx, which in turn will import the other three components. IssueList.jsx will also need to import graphQLFetch.js since it makes the Ajax calls.
Let’s also move or copy the ESLint exceptions to the appropriate new files. All files will have an exception for declaring React as global; IssueFilter will also have the exception for stateless components.
The new file IssueList.jsx is described in Listing 8-13.
/* globals React */
/* eslint "react/jsx-no-undef": "off" */
import IssueFilter from './IssueFilter.jsx';
import IssueTable from './IssueTable.jsx';
import IssueAdd from './IssueAdd.jsx';
import graphQLFetch from './graphQLFetch.js';
export default class IssueList extends React.Component {
...
}
Listing 8-13
ui/src/IssueList.jsx: New File for the IssueList Component
The new IssueTable.jsx file is shown in Listing 8-14. Note that this contains two stateless components, of which only IssueTable is exported.
/* globals React */
function IssueRow({ issue }) {
...
}
export default function IssueTable({ issues }) {
...
}
Listing 8-14
ui/src/IssueTable.jsx: New File for the IssueTable Component
The new IssueAdd.jsx file is shown in Listing 8-15.
/* globals React PropTypes */
export default class IssueAdd extends React.Component {
...
}
Listing 8-15
ui/src/IssueAdd.jsx: New File for the IssueAdd Component
The new IssueFilter.jsx file is shown in Listing 8-16.
export default class IssueFilter extends React.Component {
...
}
Listing 8-16
ui/src/IssueFilter.jsx: New File for the IssueFilter Component
Finally, the main class App.jsx will have very little code, just an instantiation of the IssueList component and mounting it in the contents <div>, and the necessary comment lines to declare React and ReactDOM as globals for ESLint. This file is shown in its entirety (deleted lines are not shown for the sake of brevity) in Listing 8-17.
ui/src/App.jsx: Main File with Most Code Moved Out
If you run npm run watch from the ui directory, you will find that all the files are being transformed and bundled into app.bundle.js. If you now test the application, it should work just as before.
Exercise: Transform and Bundle
1.
Save any JSX file with only a spacing change, while running npm run watch. Does Webpack rebuild the bundle? Why not?
2.
Was it necessary to separate the mounting of the component (in App.jsx) and the component itself (IssueList) into different files? Hint: Think about what other pages we’ll need in the future.
Until now, to keep things simple, we included third-party libraries as JavaScript directly from a CDN. Although this works great most times, we have to rely on the CDN services to be up and running to support our application. Further, there are many libraries that need to be included, and these too have dependencies among them.
In this section, we’ll use Webpack to create a bundle that includes these libraries. If you remember, I discussed that npm is used not only for server-side libraries, but also client-side ones. What’s more, Webpack understands this and can deal with client-side libraries installed via npm.
Let’s first use npm to install the client-side libraries that we have used until now. This is the same list as the list of <script>s in index.html.
$ cd ui
$ npm install react@16 react-dom@16
$ npm install prop-types@15
$ npm install whatwg-fetch@3
$ npm install babel-polyfill@6
Next, to use these installed libraries, let’s import them in all the client-side files where they are needed, just like we imported the application’s files after splitting App.jsx. All the files with React components will need to import React. App.jsx will need to import ReactDOM in addition. The polyfills—babel-polyfill and whatwg-fetch—can be imported anywhere, since these will be installed in the global namespace. Let’s do this in App.jsx, the entry point. This, and the other components’ changes, are shown in Listings 8-18 to 8-22.
/* globals React ReactDOM */
import 'babel-polyfill';
import 'whatwg-fetch';
import React from 'react';
import ReactDOM from 'react-dom';
import IssueList from './IssueList.jsx';
...
Listing 8-18
App.jsx: Changes for Importing Third-Party Libraries
-/* globals React */
-/* eslint "react/jsx-no-undef": "off" */
import React from 'react';
import IssueFilter from './IssueFilter.jsx';
...
Listing 8-19
IssueList.jsx: Changes for Importing Third-Party Libraries
export default class IssueFilter extends React.Component {
...
Listing 8-20
IssueFilter.jsx: Changes for Importing Third-Party Libraries
-/* globals React */
import React from 'react';
function IssueRow(props) {
...
Listing 8-21
IssueTable.jsx: Changes for Importing Third-Party Libraries
/* globals React PropTypes */
import React from 'react';
import PropTypes from 'prop-types';
export default class IssueAdd extends React.Component {
...
Listing 8-22
IssueAdd.jsx: Changes for Importing Third-Party Libraries
If you have npm run watch already running, you will notice in its output that the number of hidden modules has shot up from a few to a few hundred and that the size of app.bundle.js has increased from a few KBs to more than 1MB. The new output of the Webpack bundling will now look like this:
The fact that the bundle includes all of the libraries is a minor problem. The libraries will not change often, but the application code will, especially during the development and testing. Even when the application code undergoes a small change, the entire bundle is rebuilt, and therefore, a client will have to fetch the (now big) bundle from the server. We’re not taking advantage of the fact that the browser can cache scripts when they are not changed. This not only affects the development process, but even in production, users will not have the optimum experience.
A better alternative is to have two bundles, one for the application code and another for all the libraries. It turns out we can easily do this in Webpack using an optimization called splitChunks. To use this optimization and automatically name the different bundles it creates, we need to specify a variable in the filename. Let’s use a named entry point and the name of the bundle as a variable in the filename in the Webpack configuration for the UI like this:
...
entry: './src/App.jsx',
entry: { app: './src/App.jsx' },
output: {
filename: 'app.bundle.js',
filename: '[name].bundle.js',
path: path.resolve(__dirname, 'public'),
},
...
Next, let’s save some time by excluding libraries from transformation: they will already be transformed in the distributions that are provided. To do this, we need to exclude all files under node_modules in the Babel loader.
...
test: /.jsx?$/,
exclude: /node_modules/,
use: 'babel-loader',
...
Finally, let’s enable the optimization splitChunks. This plugin does what we want out of the box, that is, it separates everything under node_modules into a different bundle. All we need to do is to say that we need all as the value to the property chunks. Also, for a convenient name of the bundle, let’s give a name in the configuration, like this:
...
splitChunks: {
name: 'vendor',
chunks: 'all',
},
...
The complete set of changes to webpack.config.js under the ui directory is shown in Listing 8-23.
module.exports = {
mode: 'development',
entry: './src/App.jsx',
entry: { app: './src/App.jsx' },
output: {
filename: 'app.bundle.js',
filename: '[name].bundle.js',
path: path.resolve(__dirname, 'public'),
},
module: {
rules: [
{
test: /.jsx?$/,
exclude: /node_modules/,
use: 'babel-loader',
},
],
},
optimization: {
splitChunks: {
name: 'vendor',
chunks: 'all',
},
},
};
Listing 8-23
ui/webpack.config.js: Changes for Separate Vendor Bundle
Now, if you restart npm run watch, it should spew out two bundles—app.bundle.js and vendor.bundle.js—as can be seen in the sample output of this command:
Now that we have all the third-party libraries included in the bundle, we can remove the loading of these libraries from the CDN. Instead, we can include the new script vendor.bundle.js. The changes are all in index.html, as shown in Listing 8-24.
ui/public/index.html: Removal of Libraries Included Directly from CDN
If you now test the application (after starting the API and the UI servers, of course), you will find that the application works just as before. Further, a quick look at the Network tab in the Developer Console will show that the libraries are no longer being fetched from the CDN; instead, the new script vendor.bundle.js is being fetched from the UI server.
If you make a small change to any of the JSX files and refresh the browser, you will find that fetching vendor.bundle.js returns a “304 Not Modified” response, but the application’s bundle, app.bundle.js, is indeed being fetched. Given the sizes of the vendor.bundle.js file, it is a great saving of both time and bandwidth.
Hot Module Replacement
The watch mode of Webpack works well for client-side code, but there is a potential pitfall with this approach. You must keep an eye on the console that is running the command npm run watch to ensure that bundling is complete, before you refresh your browser to see the effect of your changes. If you press refresh a little too soon, you will end up with the previous version of your client-side code, scratch your head wondering why your changes didn’t work, and then spend time debugging.
Further, at the moment, we need an extra console for running npm run watch in the UI directory to detect changes and recompile the files. To solve these issues, Webpack has a powerful feature called Hot Module Replacement (HMR). This changes modules in the browser while the application is running, removing the need for a refresh altogether. Further, if there is any application state, that will also be retained, for example, if you were in the middle of typing in some text box, that will be retained because there is no page refresh. Most importantly, it saves time by updating only what is changed, and it removes the need for switching windows and pressing the refresh button.
There are two ways to implement HMR using Webpack. The first involves a new server called webpack-dev-server, which can be installed and run from the command line. It reads the contents of webpack.config.js and starts a server that serves the compiled files. This is the preferred method for applications without a dedicated UI server. But since we already have a UI server, it’s better to modify this slightly to do what the webpack-dev-server would do: compile, watch for changes, and implement HMR.
HMR has two middleware packages that can be installed within the Express application, called webpack-dev-middleware and webpack-hot-middleware. Let’s install these packages:
$ cd ui
$ npm install --save-dev webpack-dev-middleware@3
$ npm install --save-dev webpack-hot-middleware@2
We’ll use these modules within the Express server for the UI, but only when explicitly enabled because we don’t want to do this in production. We’ll have to import these modules and mount them in the Express app as middleware. But these modules need special configuration, different from the defaults in webpack.config.js. These are:
They need additional entry points (other than App.jsx) so that Webpack can build the client-side code necessary for this extra functionality into the bundle.
A plugin needs to be installed that generates incremental updates rather than entire bundles.
Rather than create a new configuration file for this, let’s instead modify the configuration on the fly, when HMR is enabled. Since the configuration itself is a Node.js module, this can be easily done. But we do need one change in the configuration, one that does not affect the original configuration. The entry point needs to be changed to an array so that a new entry point can be pushed easily. This change is shown in Listing 8-25.
...
entry: { app: './src/App.jsx' },
entry: { app: ['./src/App.jsx'] },
...
Listing 8-25
ui/webpack.config.js: Change Entry to an Array
Now, let’s add an option for the Express server to enable HMR. Let’s use an environment variable called ENABLE_HMR and default it to true, as long as it is not a production deployment. This gives developers a chance to switch it off in case they prefer the webpack --watch way of doing things.
if (enableHMR && (process.env.NODE_ENV !== 'production')) {
console.log('Adding dev middleware, enabling HMR');
...
}
...
To enable HMR, the first thing we’ll do is import the modules for Webpack and the two new modules we just installed. We’ll also have to let ESLint know that we have a special case where we are installing development dependencies conditionally, so a couple of checks can be disabled.
Next, let’s import the configuration file. This is also just a require() call, as the configuration is nothing but a Node.js module:
...
const config = require('./webpack.config.js');
...
In the config, let’s add a new entry point for Webpack that will install a listener for changes to the UI code and fetch the new modules when they change.
Finally, let’s create a Webpack compiler from this config and create the dev middleware (which does the actual compilation of the code using the config and sends out the bundles) and the hot middleware (which incrementally sends the new modules to the browser).
...
const compiler = webpack(config);
app.use(devMiddleware(compiler));
app.use(hotMiddleware(compiler));
...
Note that the dev and hot middleware have to be installed before the static middleware. Otherwise, in case the bundles exist in the public directory (because npm run compile was executed some time back), the static module will find them and send them as a response, even before the dev and hot middleware get a chance.
The changes to the uiserver.js file are shown in Listing 8-26.
ui/uiserver.js: Changes for Hot Module Replacement Middleware
At this point, the following are different ways that the UI server can be started:
npm run compile + npm run start: In production mode (the NODE_ENV variable is defined to production), starting of the server requires that npm run compile has been run and app.bundle.js and vendor.bundle.js have been generated and are available in the public directory.
npm run start: In development mode (NODE_ENV is either not defined or set to development), this starts the server with HMR enabled by default. Any changes to the source files will be hot replaced in the browser instantly.
npm run watch + npm run start, ENABLE_HMR=false: In development or production mode, these need to be run in two consoles. The watch command looks for changes and regenerates the JavaScript bundles, and the start command runs the server. Without ENABLE_HMR, the bundles will be served from the public directory, as generated by the watch command.
Let’s add these as comments before the scripts in package.json in the UI. Since a JSON file cannot have comments like JavaScript, we’ll just use properties prefixed with # to achieve this. The changes to ui/package.json are shown in Listing 8-27.
...
"scripts": {
"#start": "UI server. HMR is enabled in dev mode.",
"#compile": "Generates JS bundles for production. Use with start.",
"compile": "webpack --mode production",
"#watch": "Compile, and recompile on any changes.",
"watch": "webpack --watch"
},
...
Listing 8-27
ui/package.json: Comments to Define Each Script
Now, if you run npm start in the UI as well as npm start in the API server, you’ll be able to test the application. If you are running npm run watch, you may stop it now. The application should work as before. You will also see the following in the browser’s Developer Console, assuring you that HMR has indeed been activated:
[HMR] connected
But when a file changes, say IssueFilter.jsx, you will see a warning in the browser’s console to this effect:
[HMR] bundle rebuilding
HMR] bundle rebuilt in 102ms
[HMR] Checking for updates on the server...
Ignored an update to unaccepted module ./src/IssueFilter.jsx -> ./src/IssueList.jsx -> ./src/App.jsx -> 0
[HMR] The following modules couldn't be hot updated: (Full reload needed)
This is usually because the modules which have changed (and their parents) do not know how to hot reload themselves. See https://webpack.js.org/concepts/hot-module-replacement/ for more details.
[HMR] - ./src/IssueFilter.jsx
This means that although the module was rebuilt and received in the browser, it couldn’t be accepted. In order to accept the changes to a module, its parent needs to accept it using the HotModuleReplacementPlugin’s accept() method. The plugin’s interface is exposed via the module.hot property. Let’s accept all changes unconditionally right at the top of the hierarchy of modules, App.jsx. The change for this is shown in Listing 8-28.
Now, if you change the contents of say, IssueFilter.jsx, you will see in the Developer Console that not just this module, but all modules that include this and upwards in the chain of inclusions, are updated: IssueList.jsx and then App.jsx. One effect of this is that the App.jsx module is loaded again (an equivalent of import is executed) by the HMR plugin. This has the effect of running the entire code in the contents of this file, which includes the following:
Thus, the IssueList component is constructed again and rendered, and almost everything gets refreshed. This can potentially lose local state. For example, if you had typed something in the text boxes for Owner and Title in the IssueAdd component, the text will be lost when you change IssueFilter.jsx.
To avoid this, we should ideally look for changes in every module and mount the component again, but preserve the local state. React does not have methods that make this possible, and even if it did, it would be tedious to do this in every component. To solve these problems, the react-hot-loader package was created. At compile time, it replaces a component’s methods with proxies, which then call the real methods such as render(). Then, when a component’s code is changed, it automatically refers to the new methods without having to remount the component.
This could prove to be useful in applications where local state is indeed important to preserve across refreshes. But for the Issue Tracker application, let’s not implement react-hot-loader, instead, let’s settle for the entire component hierarchy reloading when some code changes. It does not take that much time, in any case, and saves the complexity of installing and using react-hot-loader.
Exercise: Hot Module Replacement
1.
How can you tell that the browser is not being fully refreshed when a module’s code is changed? Use the Network section of your browser’s developer tools and watch what goes on.
Answers are available at the end of the chapter.
Debugging
The unpleasant thing about compiling files is that the original source code gets lost, and if you have to put breakpoints in the debugger, it’s close to impossible, because the new code is hardly like the original. Creating a bundle of all the source files makes it worse, because you don’t even know where to start.
Fortunately, Webpack solves this problem by its ability to give you source maps, things that contain your original source code as you typed it in. The source maps also connect the line numbers in the transformed code to your original code. Browsers’ development tools understand source maps and correlate the two, so that breakpoints in the original source code are converted breakpoints in the transformed code.
Webpack configuration can specify what kind of source maps can be created along with the compiled bundles. A single configuration parameter called devtool does the job. The kind of source maps that can be produced varies, but the most accurate (and the slowest) is generated by the value source-map. For this application, because the UI code is small enough, this is not discernably slow, so let’s use it as the value for devtool. The changes to webpack.config.js in the UI directory are shown in Listing 8-29.
...
optimization: {
...
},
devtool: 'source-map'
};
...
Listing 8-29
ui/webpack.config.js: Enable Source Map
If you are using the HMR-enabled UI server, you should see the following output in the console that is running the UI server:
As you can see, apart from the package bundles, there are accompanying maps with the extension.map. Now, when you look at the browser’s Developer Console, you will be able to see the original source code and be able to place breakpoints in it. A sample of this in the Chrome browser is shown in Figure 8-1.
Figure 8-1
Breakpoints in the original source code using source maps
You will find the sources in other browsers in a roughly similar manner, but not exactly the same. You may have to look around a little to locate them. For example in Safari, the sources can be seen under Sources -> app.bundle.js -> “” -> src.
If you are using the Chrome or Firefox browser, you will also see a message in the console asking you to install the React Development Tools add-on. You can find installation instructions for these browsers at https://reactjs.org/blog/2015/09/02/new-react-developer-tools.html. This add-on provides the ability to see the React components in a hierarchical manner like the DOM inspector. For example, in the Chrome browser, you’ll find a React tab in the developer tools. Figure 8-2 shows a screenshot of this add-on.
Figure 8-2
React Developer Tools in the Chrome browser
Note
The React Developer Tools has a compatibility problem with React version 16.6.0 at the time of writing this book. If you do face an issue (there will be an error in the console like Uncaught TypeError: Cannot read property 'displayName' of null), you may have to downgrade the version of React to 16.5.2.
DefinePlugin: Build Configuration
You may not be comfortable with the mechanism that we used for injecting the environment variables in the front-end: a generated script like env.js. For one, this is less efficient than generating a bundle that already has this variable replaced wherever it needs to be replaced. The other is that a global variable is normally frowned upon, since it can clash with global variables from other scripts or packages.
Fortunately, there is an option. We will not be using this mechanism for injecting environment variables, but I have discussed it here so that it gives you an option to try out and adopt if convenient.
To replace variables at build time, Webpack’s DefinePlugin plugin comes in handy. As part of webpack.config.js, the following can be added to define a predefined string with the value like this:
Now, within the code for App.jsx, instead of hard-coding this value, the __UI_API_ENDPOINT__ string can be used like this (note the absence of quotes; it is provided by the variable itself):
Although this approach works quite well, it has the downside of having to create different bundles or builds for different environments. It also means that once deployed, a change to the server configuration, for example, cannot be done without making another build. For these reasons, I have chosen to stick with the runtime environment injection via env.js for the Issue Tracker application.
Production Optimization
Although Webpack does all that’s necessary, such as minifying the output JavaScript when the mode is specified as production, there are two things that need special attention from the developer.
The first thing to be concerned about is the bundle size. At the end of this chapter, the third-party libraries are not many, and the size of the vendor bundle is around 200KB in production mode. This is not big at all. But as we add more features, we’ll be using more libraries and the bundle size is bound to increase. As we progress along in the next few chapters, you will soon find that when compiling for production, Webpack starts showing a warning that the bundle size for vendor.bundle.js is too big and that this can affect performance. Further, there will also be a warning that the combined size for all assets required for the entry point app is too large.
The remedy to these issues depends on the kind of application. For applications that are frequently used by users such as the Issue Tracker application, the bundle size is not of much concern because it will be cached by the user’s browser. Except for the first time, the bundles will not be fetched unless they have changed. Since we have separated the application bundle from the libraries, we’ve more or less ensured that most of the JavaScript code, which is part of the vendor bundle, does not change and hence need not be fetched frequently. Thus, the Webpack warning can be ignored.
But for applications where there are many infrequent users, most of them visiting the web application for the first time, or after a long time, the browser cache will have no effect. To optimize the page load time for such applications, it’s important to not just split the bundles into smaller pieces, but also to load the bundles only when required using a strategy called lazy loading. The actual steps to split and load the code to improve performance depends on the way the application is used. For example, it would be pointless to postpone loading of the React libraries upfront because without this, any page’s contents will not be shown. But in later chapters you will find that this is not true, when the pages are constructed using server rendering and React can indeed be lazy loaded.
For the Issue Tracker application, we’ll assume that it’s a frequently used application and therefore the browser cache will work great for us. If your project’s needs are different, you will find the Webpack documentation on code splitting (https://webpack.js.org/guides/code-splitting/) and lazy loading (https://webpack.js.org/guides/lazy-loading/) useful.
The other thing to be concerned about is browser caching, especially when you don’t want it to cache the JavaScript bundle. This happens when the application code has changed and the version in the user’s browser’s cache is the wrong one. Most modern browsers handle this quite well, by checking with the server if the bundle has changed. But older browsers, especially Internet Explorer, aggressively cache script files. The only way around this is to change the name of the script file if the contents have changed.
This is addressed in Webpack by using content hashes as part of the bundle’s name, as described in the guide for caching in the Webpack documentation at https://webpack.js.org/guides/caching/. Note that since the script names are generated, you will also need to generate index.html itself to contain the generated script names. This too is facilitated by a plugin called HTMLWebpackPlugin.
Continuing in the spirit of coding hygiene in the previous chapter, we modularized the code in this chapter. Since JavaScript was not originally designed for modularity, we needed the tool Webpack to put together and generate a few bundles from a handful of small JavaScript files and React components.
We removed the dependency on the CDN for runtime libraries such as React and the polyfills. Again, Webpack helped resolve dependencies and create bundles for these. You also saw how Webpack’s HMR helped us increase productivity by efficiently replacing modules in the browser. You then learned about source maps that help in debugging.
In the next chapter, we’ll come back to adding features. We’ll explore an important concept of client-side routing that will allow us to show different components or pages and navigate between them in a seamless manner, even though the application will continue to be single page application (SPA) practically.
Answers to Exercises
Exercise: Transform and Bundle
1.
No, Webpack does not rebuild if you save a file with just an extra space. This is because the preprocessing or loader stage produces a normalized JavaScript, which is no different from the original. A rebundling is triggered only if the normalized script is different.
2.
As of now, we have only one page to display, the Issue List. Going forward, we’ll have other pages to render, for example, a page to edit the issue, maybe another to list all users, yet another to show one’s profile information, and so on. The App.jsx file will then need to mount different components based on user interaction. Thus, it’s convenient to keep the application separate from each top-level component that may be loaded.
3.
Not using the default keyword has the effect of exporting the class as a property (rather than itself) of the object that’s exported. It is equivalent to doing this after defining the exportable element:
export { IssueList };
In the import statement, you would have to do this:
import { IssueList } from './IssueList.jsx';
Note the destructuring assignment around the LHS. This allows multiple elements to be exported from a single file, each element you want from the import being separated by a comma. When only a single element is being exported, it’s easiest to use the default keyword.
Exercise: Hot Module Replacement
1.
There are many logs in the browser’s console that tell you that HMR is being invoked. Further, if you look at the network requests, you will find that for a browser refresh, requests are made to all of the assets. Take a look at the sizes of these assets. Typically, vendor.bundle.js is not fetched again when client-side code changes (it would return a 304 response), but app.bundle.js will be reloaded.
But when HMR succeeds, you will see that the entire assets are not fetched; instead, incremental files, much smaller than app.bundle.js, are being transferred.
