Now that we’ve learned what the MySQL Document Store is and how to use it via the MySQL Shell, we can explore a more complex example that demonstrates the three forms of data storage described: a pure relational database solution, a hybrid solution where we use one or more JSON fields using the SQL features of the X DevAPI, and a pure document store solution that uses the X DevAPI exclusively (a NoSQL solution). Therefore, we will see the application implemented in three separate implementations.
However, we must first understand how the sample application is designed and how it works. After all, the best examples should be something the reader can use in their own environment. Thus, the example must be complex enough and complete enough to be meaningful.
To continue the understandability of code in the previous chapters, we will be using Python for the application because Python is very easy to learn and the code reads with a level of clarity better than other languages. But don’t worry if you prefer another language. You can easily rewrite the code in this chapter into any of the languages with connectors that support the X DevAPI.
The user interface on the other hand complicates things a bit. We can mitigate that by using a user interface design that is familiar. For this, we will use a web application. It is unfortunate that writing a web application in pure Python is tedious and requires more knowledge of how web application works than what one can expect in a work of this size.
To overcome that challenge, we will use one of the popular Python web application frameworks. In this case, we will use Flask complete with a primer, tutorial, and walk-through of the user interface code. As you will see, Flask is also easy to learn with only a moderate number of nuances and concepts to learn. Flask was originally developed by Armin Ronacher and has proven to be one of the easiest and most stable web platforms for Python.
In Chapter 9, we will complete the application adding the database access methods described previously.
Getting Started
If you want to follow along and implement the sample projects, you will need a few things installed on your computer to get going. This section will help you prepare your computer with the tools needed: what you need to install and how to configure your environment. We will also see a short primer on the user interface tools. Let’s begin with a more detailed description of the application.
Library Application
The example application in this chapter is a rather simple application designed to demonstrate concepts. It is complete in that it supports the create, read, update, and delete (CRUD) operations on data. Error handling and the user interface components have less sophistication to keep the focus on the interaction with data. That said we will see how to implement a robust and nice looking web interface in Python using Flask.
The data for the application is a simple book database. We will be storing basic information about books such as the ISBN, title, publisher, and so forth. We also will have a notes section so we can keep notes on the books. I used something similar to this for many of my research papers and even some more advanced projects. The concept of operations was to record the bibliography information for each book along with notes about the content so that later it could be used to create a list of references. For example, if a book contained information pertinent to a topic in the paper, I would add a note indicating the subject and list page numbers and other important information. The information in the notes varied based on what I was recording, so all that was required was a search in a simple text field.
Unlike the application I used for research that permitted storing information about books, magazines, articles, blogs, and so forth, the application for this chapter has been simplified to store only books. This keeps the project small enough to be discussed without unnecessary detail. The focus for the chapter is to examine the benefits of migrating to a document store, not how best to implement a media reference application.
Thus, the basic operations will be to store and retrieve information about books, authors, and publishers. The user interface is designed to present a list of all the books in the database with the option to edit any book in the list. The default view is books but the first versions of the application (1 and 2) will allow you to view lists of authors and publishers. Users will also be permitted to create new books (authors, and publishers), edit, and delete books.
Each version of the application will behave slightly differently as we see how changing the way the data is stored and retrieved affects application design. A more detailed explanation of each project is included in later sections that discuss the project versions.
Now, let’s look at how to setup our computers to run the sample application projects.
Setup Your Environment
The changes to your environment are not difficult nor are they lengthy. We will be installing Flask and a few extensions, which are needed for the application user interface. Flask is one of several web libraries you can use with Python. These web libraries make developing web applications with Python much easier than using raw HTML code and writing your own handlers and code for the requests. Plus, Flask is not difficult to learn.
List of Libraries Required
Library | Description | Documentation |
---|---|---|
Flask | Python Web API | |
Flask-Script | Scripting support for Flask | |
Flask-Bootstrap | User interface improvements and enhancements | |
Flask-WTF | WTForms integration | |
WTForms | Forms validation and rendering |
Note
Depending on how your system is configured, you may see additional or fewer components installed for the components installed in this section.
Of course, you should already have Python installed on your system. If not, be sure to download and install the latest version of either the 2.X or 3.X editions. The example code in this chapter was tested with Python 2.7.10 and Python 3.6.0.
To install the libraries, we can use the Python package manager, pip, to install the libraries from the command line. The pip utility is included in most Python distributions, but if you need to install it, you can see the installation documentation at https://pip.pypa.io/en/latest/installing/ .
If you need to install pip on Windows, you will need to download an installer, get-pip.py ( https://pip.pypa.io/en/stable/installing/#installing-with-get-pip-py ), and then add the path to the installed directory to the PATH environment variable. There are several articles that document this process in more detail. You can google for “installing pip on Windows 10” and find several including https://matthewhorne.me/how-to-install-python-and-pip-on-windows-10/ , which is among the most accurate.
Note
If you have multiple versions of Python installed on your system, the pip command will install into whichever Python version environment is the default. To use pip to install to a specific version, use pipN where N is the version. For example, pip3 installs packages in the Python 3 environment.
The pip command is very handy because it makes installing registered Python packages—those packages registered in the Python Package Index, abbreviated as PyPI1 (https://pypi.python.org/pypi)—very easy. The pip command will download, unpack, and install using a single command. Let’s discover how to install each of the packages we need.
Caution
Some systems may require running pip with elevated privileges such as sudo (Linux, macOS), or in a command window run as an administrator user (Windows 10). You will know if you need elevated privileges if the install fails to copy files due to permission issues.
Installing Flask
Listing 8-1 demonstrates how to install Flask using the command, pip install flask. Note that the command downloads the necessary components, extracts them, and then runs the setup for each. In this case, we see Flask is composed of several components including Werkzeug, MarkupSafe, and Jinja2. We will learn more about some of these in the “Flask Primer” section.
Installing Flask
Installing Flask-Script
Listing 8-2 demonstrates how to install Flask-Script using the command, pip install flask-script. Note that in this case, we see the installation checking for prerequisites and their versions.
Installing Flask-Script
Installing Flask-Bootstrap
Listing 8-3 demonstrates how to install Flask-Bootstrap using the command, pip install flask-bootstrap. Once again, we see the installation checking for prerequisites and their versions as well as installation of dependent components.
Installing Flask-Bootstrap
Installing Flask-WTF
Listing 8-4 demonstrates how to install Flask-WTF using the command, pip install flask-wtf.
Installing Flask-WTF
Installing WTForms
The following demonstrates how to install WTForms using the command, pip install WTforms. In this case, the installation is simple because we only need the one package.
Using Python Virtual Environments
One of the nice things about working with Python is you can use a virtual environment to try things out. A virtual environment is a local (think private) installation of Python, which you can install packages and make changes to the Python environment without affecting the global Python installation on your system. So, for example, if you used a virtual environment to install Flask, it is only available to that virtual environment – it doesn’t affect any other virtual environment or the global Python installation.
To use a virtual environment, you must have the virtualenv application installed. Not all systems have this and indeed it isn’t supported on all platforms (but is on many). To install virtual environment on Linux, use the command, sudo apt-get install python-virtualenv. To install virtual environment on macOS, use the command, sudo easy_install virtualenv. To install virtual environment on Windows 10, you must download ez_setup.py (part of setuptools) from https://github.com/pypa/setuptools . Once downloaded, open a command window with administrative privileges then enter the command, python ez_setup.py to install easy_install then enter the command, easy_install virtualenv to install virtual environment.
To create and use a virtual environment, issue the command, virtualenv project1. This creates a folder name project1 with the virtual environment files that keep track of all the changes made when in that environment. To activate the environment, use the source ./project1/bin/activate command. Note that we are invoking a script in the new virtual environment folder. This will change your prompt to indicate you’re using a virtual environment. To deactivate the environment, use the deactivate command while the virtual environment is active. This will return your Python environment back to the global defaults. The following demonstrates these commands on macOS.
Removing a virtual environment is simply done by deleting the environment folder (after deactivating it):
Some recommend always using a virtual environment when experimenting with new things in Python, and for some things such as untrusted or untried libraries or libraries that conflict with existing installed libraries, which is a good practice. However, for mainstream items such as Flask and its supporting libraries, it isn't needed. If you want to use a virtual environment for the proceeding projects, feel free to do so. Just remember to activate it before issuing any Python commands and deactivate it when you’re finished.
To learn more about virtual environments, see https://virtualenv.pypa.io/en/stable/ .
You should also have the MySQL Connector/Python 8.0.5 or later database connector installed. If you do not, download it from https://dev.mysql.com/downloads/connector/python/ and install it. If you have multiple versions of Python installed, be sure to install it in all Python environments you want to use. Otherwise, you may see an error like the following when starting the code.
Pip also can be used to install MySQL Connector/Python. The following shows how to use PIP to install the connector.
If you installed MySQL Connector/Python manually or from source, you also may need to install Protobuf. You can use pip to install it as shown in the following.
Now that our computer is setup, let’s take a crash course on Flask and its associated extensions.
Flask Primer
Flask is one of several web application libraries (sometimes called frameworks or application programming interfaces—APIs) for use with Python. Flask is unique among the choices in that it is small and, once you are familiar with how it works, easy to use. That is, once you write the initialization code, most of your work with Flask will be limited to creating web pages, redirecting responses, and writing your feature code.
Flask is considered a micro framework because it is small and lightweight, and it doesn’t force you into a box writing code specifically to interact with the framework. It provides everything you need leaving the choice of what to use in your code up to you.
Flask is made of two major components that provide the basic functionality: a Web Server Gateway Interface (WSGI) that handles all the work hosting web pages; and a template library for easier web page development that reduces the need to learn HTML, removes repetitive constructs, and provides a scripting capability for HTML code. The WSGI component is named Werkzeug, which loosely translated from German means, “work stuff” ( http://werkzeug.pocoo.org/ ). The template component is named Jinja2 and is modelled after Django ( http://jinja.pocoo.org/docs/2.10/ ). Both were developed and maintained by the originators of Flask. Finally, both components are installed when you install Flask.
Flask is also an extensible library allowing other developers to create additions (extensions) to the basic library to add functionality. We saw how to install some of the extensions available for Flask in the previous section. We will be using the scripting, bootstrap, and WTForms extensions in this chapter. Having the ability to pick and choose the extensions you want means you can keep your application as small as necessary adding only what you need.
One of the components that you may consider “missing” from flask is the ability to interact with other services such as database systems. This was a purposeful design and functionality like this can be achieved through extensions. In fact, there are several database extensions available for Flask including those that allow you to work with MySQL. However, because we want to use the X DevAPI, we must use the Oracle-provided connector, MySQL Connector/Python. This is not only possible, it also illustrates the freedom you have when using Flask; we aren’t limited to certain functionality as database server access, we can use any other Python library we want or require.2
Tip
If you’re curious about the MySQL support for Flask, see http://flask-mysql.readthedocs.io/en/latest/ .
Flask, together with the extensions described previously, provides all the wiring and plumbing you need to make a Web application in Python. It removes almost all the burdens required to write web applications such as interpreting client response packets, routing, HTML form handling, and more. If you’ve ever written a web application in Python, you will appreciate the ability to create robust web pages without the complexity of writing HTML and style sheets. Once you’re familiar with how to use Flask, it will allow you to focus on the code for your application rather than spending a lot of time writing the user interface.
Now, let’s get started learning Flask! If you take your time and try the sample application, your first Flask application will work on the first try. The hardest part of learning Flask is already past—installing Flask and its extensions. The rest is learning the concepts of writing applications in Flask. Before we do that, let’s learn more about the terminology in Flask as well as how to setup the base code we will use to initialize the application instance that we will be using in this chapter.
Tip
If you want to explore Flask further, you should consider reading the online documentation, user guide, and examples at http://flask.pocoo.org/docs/0.12/ .
Terminology
Flask is designed to make a lot of the tedium of writing web applications easier. In Flask parlance, a web page is rendered using two parts of your code: a view, which is defined in the HTML file(s) and a route, which processes the requests from a client. Recall, we can see one of two requests: a GET request that requests loading of a web page (read from the client’s perspective), and a POST request that sends data from the client via the web page to the server (write from the client’s perspective). Both requests are handled in Flask using functions you define.
These functions then render the web page to send back to the client to satisfy the request. Flask calls the functions view functions (or views for short). The way Flask knows which method to call is using decorators that identify the URL path (called a route in Flask). You can decorate a function with one or more routes making it possible to provide multiple ways to reach the view. The decorator used is @app.route(<path>). The following shows an example of multiple routes for a view function.
Note that there are multiple decorators. The first is book, which allows us to use a URL such as localhost:5000/book, which causes Flask to route execution to the book() function. The second is book/<isbn_selected>, which demonstrates how to use variables to pass information to the view. In this case, if the user (the application) uses the URL localhost:5000/book/978-1-4842-1294-3, which Flask places the value, 978-1-4842-1294-3, in the isbn_selected variable. In this way, we can pass information dynamically to our views.
Note also that the routes specify the methods allowed for each route. In this application, we can have a GET or POST for either route. If you leave these off the decorator, the default is GET only making the web page read only.
Finally, note that at the end of the function we return with a call to the render_template() function (imported from the flask module) that tells flask to return (refresh) the web page with data we’ve acquired or assigned. The web page, book.html, although part of the view is called a form in Flask. It is this concept that we will use to retrieve information from the database and send it to the user. We can return a simple HTML string (or an entire file) or what is called a form. Because we are using the Flask-WTF and WTForms extensions, we can return a template rendered as a form class. We will discuss forms, form classes, and other routes and views for the chapter project in a later section. As you will see, templates are another powerful feature making it easy to create web pages.
What’s a Decorator?
In Python, we can specify special handling parameters by using decorators. Decorators are simply a way to change the behavior of functions. For example, you can use decorators to add stronger type checking, define macros, and invoke functions before and after execution. Decorators in Flask for routing are some of the best examples of using decorators correctly. To learn more about decorators, see https://www.python.org/dev/peps/pep-0318 .
Flask builds a list of all the routes in the application making it easy for the application to route execution to the correct function when requested. But, what happens when a route is requested but it doesn’t exist in the application? By default, you will get a generic error message like “Not Found. The requested URL was not found on the server.” We will see how to add our own custom error handling routes in a later section.
Now that we know more about the terminology used in Flask and how it is structured to work with web pages, let’s look at how a typical Flask application with the extensions we need is constructed.
Initialization and the Application Instance
Flask and its extensions provide the entry point for your web application. Instead of writing all that onerous code yourself, Flask does it for you! The Flask extensions we will be using in this chapter include Flask-Script, Flask-Bootstrap, Flask-WTF, and WTForms. The following sections briefly describe each.
Flask-Script
Flask-Script enables scripting in Flask applications by adding a command-line parser (manifested as manager) that you can use to link to functions you’ve written. This is enabled by decorating the function with @manager.command. The best way to understand what this does for us is through an example.
The following is a basic, raw Flask application that does nothing. It’s not even a “hello, world” example because nothing is shown and there are no web pages hosted—it’s just the raw Flask application.
Note the app.run() call. This is called the server startup and is executed when we load the script using the Python interpreter. When we run this code, all we see is the default message from Flask as shown in the following. Note that we don’t have any way to see help as there are no such options. We also see that the code launches using defaults for the web server (which we can change in code if we desire). For example, we can change the port that the server is listening.
With Flask-Script , we add not only a help option but options to control the server. The following code shows how easy it is to add the statements to enable Flask-Script. The new statements are highlighted in bold.
When this code is run, we can see there are additional options available. Note that the documentation string (immediately following the method definition) is shown as the help text for the command added.
Note that we see the command line arguments (commands) we added, hello_world, but we also see two new ones supplied by Flask-Script; shell and runserver. You must choose one of these commands when launching the server. The shell command allows you to use the code in a Python interpreter or similar tool and the runserver executes the code starting the web server.
Not only can we get help about the commands and options, Flask-Script also provides more control over the server from the command line. In fact, we can see all the options for each command by appending the --help option as shown in the following.
Note here that we can control all manner of things about the server including the port, host, and even how it executes.
Finally, we can execute the method we’ve decorated as a command-line option as shown in the following.
Thus, Flask-Script provides some very powerful features with only a few lines of code. You’ve got to love that!
Flask-Bootstrap
Flask-Bootstrap was originally developed by Twitter for making uniform, nice-looking web clients. It is fortunate that they’ve made it a Flask extension so that everyone can take advantage of its features. Flask-Bootstrap is a framework on its own and provides even more command-line control as well as user interface components for clean, attractive web pages. It also is compatible with the newest web browsers.
The framework does its magic behind the scenes as a client library of cascading style sheets (CSS) and scripts that are invoked from the HTML templates (commonly referred to as either HTML files or template files) in Flask. We will learn more about templates in a later section. Because it is client-side, we won’t see much by initializing it in the main application. Regardless, the following shows how to add Flask-bootstrap to our application code. Here, we see we have a skeleton with Flask-Script and Flask-Bootstrap initialized and configured.
WTForms
WTForms is a component we need to support the Flask-WTF extension. It provides much of the functionality that the Flask-WTF component provides (because the Flask-WTF component is a Flask-specific wrapper for WTForms). Therefore, we need only install it as a prerequisite for Flask-WTF and we will discuss it in the context of Flask-WTF.
Note
Some package installations of Flask-WTF may include WTForms.
Flask-WTF
The Flask-WTF extension is an interesting component providing several very useful additions: most notable for our purposes integration with WTForms (a framework agnostic component) that permits the creation of form classes, and additional web security in the form of cross-site request forgery (CSRF) protection. These two features allow you to take your web application to a higher level of sophistication.
Form Classes
Form classes provide a hierarchy of classes that make defining web pages more logical. With Flask-WTF, you can define your form using two pieces of code; a special class derived from FormForm class (imported from the Flask framework) that you use to define fields using one or more additional classes that provide programmatic access to data, and an HTML file (or template) for rendering the web page. In this way, we see an abstraction layer (form classes) over the HTML files. We will see more about the HTML files in the next section.
DataRequired: Determines if input field is empty
Email: Ensures the field follows email ID conventions
IPAddress: Validates IP addresses
Length: Ensures length of text is in given range
NumberRange: Ensures text is numeric and within given range
URL: Validates URLs
To form classes, we must import the class and any field classes we want to use in the preamble of the application. The following shows an example of importing the form class and form field classes. In this example, we also import some validators that we will use for validating the data automatically.
To define a form class, we must derive a new class from FlaskForm. From there, we can construct the class however we want but it is intended to allow you to define the fields. The FlaskForm parent class includes all the necessary code that Flask needs to instantiate and use the form class.
Let’s look at a simple example. The following shows the form class for the author web page. The author table, which we will link to this code via the view function, contains three fields; an auto increment field (authorid), the first name of the author (firstname), and the last name of the author (lastname). Because the author id field is not something users need to see, we make that field a hidden field and the other fields derivatives of the TextField() class. Note how these were defined in the listing with names (labels) as the first parameter.
Note also that we defined an array of validators in the form of function calls imported from the WTForms component for the fields. In each case, we used strings for the messages to make the code easier to read and more uniform. These strings include the following.
We use the Required() validator that indicates the field must have a value. We augment the default error message with the name of the field to make it easier for the user to understand. We also use a Length() validator function that defines the minimal and maximum length of the field data. Once again, we augment the default error message. Validators are applied only on POST operations (when a submit event has occurred).
Next, we see there are two SubmitField() instances: one for a create (add) button, and another for a delete button. As you may surmise, in HTML parlance, these fields are rendered as <input> fields with a type of “submit”.
Finally, to use a form class we instantiate the class in a view function. The following shows a stub for the author view function. Note that we instantiate the form class named AuthorForm() and assign it to a variable named form, which is passed to the render_template() function.
WTForms Field Classes
Field Class | Description |
---|---|
BooleanField | A checkbox with True and False values |
DateField | Accepts date values |
DateTimeField | Accepts datetime values |
DecimalField | Accepts decimal values |
FileField | File upload field |
FloatField | Accepts a floating-point value |
HiddenField | Hidden text field |
IntegerField | Accepts integer values |
PasswordField | A password (masked) text field |
RadioField | A list of radio buttons |
SelectField | A dropdown list (choose one) |
SelectMultipleField | A dropdown list of choices (choose one or more) |
StringField | Accepts simple text |
SubmitField | Form submit button |
TextAreaField | Multiline text field |
CSRF Protection
CSRF Protection is a technique that permits developers to sign web pages with an encrypted key that makes it more difficult for hackers to spoof a GET or POST request. This is accomplished by first placing a special key in the application code and then referencing the key in each of our HTML files. The following shows an example of the preamble of an application. Note that all we need to do is assign the SECRET_KEY index of the app.config array with a phrase. This should be a phrase that is not easily guessed.
To activate the CSRF in our web pages, we merely add the form.csrf_token to the HTML file. This is a special hidden field that Flask uses to validate the requests. We will see more about where to place this in a later section. But first, let’s see a cool feature of Flask called flash.
Message Flashing
There are many cool features in Flask. The creators and the creators of the Flask extensions seem to have thought of everything—even error messaging. Consider a typical web application. How do you communicate errors to the user? Do you redirect to a new page,3 issue a popup,4 or perhaps display the error on the page? Flask has a solution for this called message flashing.
We will see a mechanism to build flash messaging into all our web pages in the next section.
HTML Files and Templates
Let’s review our tour so far. We have discovered how to initialize an application with the various components, learned how Flask uses routes via the decorators to create a set of URLs for the application, these routes are directed to a view function, which instantiates the form class. The next piece of the puzzle is how to link the HTML web page to the form class.
Recall, this is done via the render_template() function where we pass in the name of a HTML file for processing. The reason template is in the name is because we can use the Jinja2 template component to make writing web pages easier. More specific, the HTML file contains both HTML tags and Jinja2 template constructs.
Note
All HTML files (templates) must be stored in the templates folder in the same location as the main application code. For example, if your code is in a file named my-flask-app.py, there should be a templates folder in the same folder as my-flask-app.py. If you place them anywhere else, Flask won’t be able to find the HTML files.
Templates together with form classes are where the user interface is designed. In short, templates are used to contain presentation logic and HTML files are used to contain the presentation data. These topics are likely to be the areas where some may need to spend some time experimenting with how to use them. The following sections give you a brief overview of Jinja2 templates and how to use them in our HTML files through demonstration of working examples. See the online Flask documentation noted for more details.
Jinja2 Templates Overview
Jinja2 templates , (or templates), are used to contain any presentation logic like looping through data arrays, making decisions on what to display, and even formatting and presentation settings. If you are familiar with other web development environments, you may have seen this encapsulated in scripts or enabled through embedded scripting such as JavaScript.
Recall we rendered our web pages in our main code. This function tells Flask to read the file specified and convert the template constructs (render them) into HTML. That is, Flask will expand and compile the template constructs into HTML that the web server can present to the client.
There are several template constructs you can use to control the flow of execution, loops, and even comments. Whenever you want to use a template construct (think scripting language), you enclose it with {% %} prefix and suffix. This enables the Flask framework to recognize the construct as a template operation rather than HTML.
However, it is not unusual and quite normal to see the template constructs intermixed with HTML tags. In fact, that is exactly how you should do it. After all, the files you will create are named .html. They just happen to contain template constructs. Does that mean you can only use templates when working with Flask? No, certainly not. If you want, you can render a pure HTML file!
At first, looking at templates can be quite daunting. But it isn’t that difficult. Just look at all the lines with the {% and %} as the “code” portions.5 You also may see comments in the form of {# #} prefix and suffix.
Caution
All template constructs require a space after the {% and before the %}.
If you look at the template, you will see the constructs and tags and formatted using indentation of two spaces. Indentation and whitespace in general doesn’t matter outside the tags and constructs. However, most developers will use some form of indentation to make the file easier to read. In fact, most coding guidelines require indentation.
One of the cool features of templates beyond the constructs (think code) is the ability to create a hierarchy of templates. This allows you to create a “base” template that your other templates can use. For example, you can create a boilerplate of template constructs and HTML tags so that all your web pages look the same.
Recall from our look at Flask-Bootstrap, bootstrap provides several nice formatting features. One of those features is creating a pleasant looking navigation bar. Naturally, we would want this to appear on all our web pages. We can do this by defining it in the base template and extending it in our other template (HTML) files. Let’s look at a base template for the library application. Listing 8-5 shows the base template for the library application. Line numbers have been added for ease of discussion.
Sample Base Template
Wow, there is a lot going on here! Note the first line. This tells us that we’re inheriting (extending) another template named bootstrap/base.html template. This is provided for you free when you install Flask-Bootstrap and it is this template that contains support for the bootstrap navigation bar feature. This is a very common method of building a set of HTML files for a Flask application as we will see later in this section.
Let’s start our tour with a bird’s eye view. Note that there are two “blocks” designated with {% block <> %} and {% endblock %} (lines 2, 3, 28, 30, 38, and 40). These are logical sections where we can apply formatting to the tags and constructs inside the block. In coding terms, this would be like a code block. The first block defines the title for the page. In this case, MyLibrary, which is the executable name for the library application.
The last block in lines 30–39 defines the template construct and HTML tags for the flash messages. Let’s take a deeper look at this code (repeated here for convenience).
Here, we see another <div> tag that contains a button. This is the button we use to dismiss the flash message. Note that this tag is placed inside a for loop as designated with {% for ... %} and ended with {% endfor %}. In this case, we are looping over the messages returned from the get_flashed_messages() function, which is collected by the flash() function in our application code. This tells us several things: we can use loops in our templates, the template allows the display of multiple images (which we saw earlier), and templates can call functions! This is an example of the power of templates.
Note
Templates are not required to be formatted in any manner. That is, whitespace doesn’t do anything outside the HTML tags or template constructs.
Finally, note the variable we defined in the for loop in line 32. This variable, message, is defined local to the block in which it appears (in this case, the for loop), and can be referenced at any point by enclosing it in {{ }}. For example, we see in line 35 we use {{ message }} inside the <div> tag, which means this text will appear on the client rendered in place by Flask. The use of variables will become more important when we discuss how to build user interfaces with templates.
Template Language Constructs
The Jinja2 template features are many and a complete discussion of all features is beyond the scope of this book. However, it is handy to have a quick reference for the major constructs of Jinja2. The following present some of the commonly used constructs including some that we discovered in the last section (for completeness). Each is presented with a short example of how the construct would appear in a template. Feel free to refer to this section when exploring the library application later in the chapter or when writing your own Flask applications.
Comments
You can embed your own comments in your templates. You may want to do this to ensure you sufficient explain what you are doing and as a reminder in case you reuse the code later.6 The following is an example of using comments in templates. Recall, comments begin with {# and end with #} and can span multiple lines.
Include
If your template files grow and you find there are portions that are reusable such as a <div> tag, you can save the tag and template constructs in a separate file and include it in other templates using the {% include %} construct. The {% include %} construct takes as a parameter the name of the file you want to include. As with templates, these must reside in the templates folder. In this way, we avoid repetition and the hassle and error-prone task of maintaining repetitive code.
Macros
Another form of reducing repetitive code is to create a macro for use in your templates (think functions). In this case, we use the {% macro … %} and {% endmacro %} constructs to define a macro that we can call (use) later in our code. The following shows an example of defining a simple macro and later using it inside a loop. Note how we pass variables to the macro for operating on the data.
Import
One of the best ways to use macros is to place them in a separate code file therefore further enhancing reusability. To use a macro from a separate file, we use the {% import … %} construct supplying the name of the file from which to import. The following shows an example of importing the macro defined previously in a separate file. As with the include, this file must be in the templates folder. Note we can use an alias and refer to the macros using dot notation.
Extend (Inherit)
We can use a hierarchy of templates by inheriting (extending) them. We saw this earlier when we examined a base template. In this case, we use the {% extend … %} construct supplying the name of the template we want to extend. The following shows an example from the base template previously.
Blocks
Blocks are used to isolate execution and scope (for variables). We use blocks whenever we want to isolate a set of template constructs (think a code block). The {% block … %} construct is used with the {% endblock %} construct to define the block. The constructs allow you to name the block. The following shows an example.
Loops
Loops are a way to execute the same block multiple times. We do this with the {% for <variable> in <data_array> %} construct. In this case, the loop will iterate over the array replacing the value in <variable> with the value in each index of the array. This construct is great for looping through an array to create a table, show a list of data, and similar presentation activities. The following shows a for loop using in constructing a table. Note that we use two for loops: one to loop over the columns in an array named columns, and another to loop over the rows in an array named rows.
You may be wondering at this point how the data in columns and rows gets to the template. Recall the render_template() function. If you want to pass data to the template, you simply list it in the parameters when you render the template. In this case, we would pass the columns and rows as follows. In this case, row_data and col_data are variables defined in the view function and passed to the rows and columns variables in the template through assignment. Cool, eh?
Conditionals
Conditionals or “if” statements (called tests in the Jinja2 documentation) allow you to make decisions in your template. We use the {% if <condition> %} construct, which is concluded with the {% endif %} construct. If you want an “else”, you can use the {% else %} construct. Further, you may chain conditions with the {% elif <condition> %}. You typically use variables or form elements in the conditions and can use the common comparators (for a list of tests, see http://jinja.pocoo.org/docs/2.10/templates/#builtin-tests ).
For example, you may want to change the label of a submit field depending on some event. You may want to define one submit button for adding or updating data. That is, when the web page is used to add a new data item, the text should read “Add” but when you update the data using the same web page, we want the text to read “Update”. This is one of the keys to reusing the template for both GET and POST requests (read and write). The following shows an example of a conditional used in this manner.
There are two conditions in this example. The first demonstrates how to check the text of a label on the form. Note that here we reference the element on the form with form.create_button, which is the name of the field class we defined in the form class, which was instantiated prior to rendering the template (we will see how to do this in a later section). The form variable is passed to the template in the render_template("book.html", form=form) call. In this case, we only display the new_note field and its label if the button text was set to “Update.”
The second example shows a simple test that if the delete_button on the form is active (not hidden or deleted), we display it. This is an example of how to display optional submit fields.
Variables and Variable Filters
Variables are a way to save data values for later processing. The most common use of variables is referencing data passed to the template from the view function (via the render_template() function). We can also use variables in our templates to save data such as counters, for loop data values, and more. Recall we reference a variable by enclosing it in curly braces {{ variable }} or in the case of the for loop, it is defined in the for loop construct. Note that when referenced inside HTML tags, the spaces inside the construct are ignored.
Variable Filters
Filter | Description |
---|---|
Capitalize | Converts the first character of the text to uppercase |
Lower | Converts the text to lowercase characters |
Safe | Renders the text without escaping special characters |
Striptags | Removes HTML tags from text |
Title | Capitalizes each word in the string |
Trim | Removes leading and trailing whitespace |
Upper | Converts the text to uppercase |
Tip
For a more in-depth look at Jinja2 template constructs, see http://jinja.pocoo.org/ .
Now that we have an overview of how templates work and have defined a base template for the library application, let’s look at how to use the base template to form the HTML files for our web pages. As you will see, it involves three concepts we’ve been discussing and will bring the discussion to a conclusion of how Flask works when building web pages and sending them to the client. We will look at getting data from the client in a later section.
HTML Files Using Templates
Now we are ready to see how to manifest the field classes we defined in our form classes. Let’s begin the discussion with a walkthrough of how to present data for the publisher data in the library application. We begin with the form class and the field classes defined to the view function, which renders the template and finally the template itself.
Recall, that the form class is where we define one or more form fields. We will use these field class instances to access the data in our view functions and in the template. Listing 8-6 show the form class (without database access).
Publisher Form Class (No Database Code)
Note that the form class creates three fields: one for the publisher name (name), one for the city in which the publisher is based (city), and another for the publisher URL (url). We also see two submit fields (buttons): one for creating new publisher data (create_button), and one for deleting publisher data (del_button). We also have a hidden field for the publisher id.
We pass the form data to the template when it is rendered after instantiating it in the view function. Listing 8-7 shows the view function for the publisher data. Here, we instantiate the publisher form class first then pass it to the template.
Publisher View Function
Note here that we see the routes we’ve defined for the view. Note also that we have set the methods for requests to include both GET and POST. We can check to see if the request is a POST (submission of data). It is in this condition that we can retrieve data from the form class instance and save it to the database. We’ll look at that a bit more when we add database capabilities.
Finally, note that we instantiate an instance of the publisher form class (form) and later pass that as a parameter to the render_template("publisher.html", form=form) call. In this case, we now render the publisher.html template stored in the templates folder.
Ok, now we have our form class and view function. The focus now is what happens when we render the HTML template file. Listing 8-8 shows the HTML file (template) for the publisher data.
Publisher HTML File
Note that the template begins with extending (inheriting) the base.html template file that we discussed earlier. We see a block defining the title and another block defining the page content. In that block, we see how to define the fields on the page referencing the field class instances from the form class instance (form). Indeed, note that we reference the label of the field as well as the data. The label is defined when you declare that the field class and the data is where the values are stored. When we want to populate a form (GET) we set the data element to the value and when we want to read the data (POST), we reference the data element.
Note that we also added the CSRF token for security, rendered the hidden fields with the form.hidden_tag() function, and included the submit fields conditionally by including the delete submit field (del_button).
Whew! That’s how Flask works to present a web page. Once you’re used to it, it is a nifty way to separate several layers of functionality and make it easy to get data from the user or present it to the user.
Now, let’s look at how to build custom error handlers into our application and later how to redirect control in our application to the correct view functions.
Error Handlers
Recall I mentioned it was possible to create your own error handling mechanisms for errors in your application. There are two such error mechanisms you should consider making: one for the 404 (not found) error, and another for 500 (application errors). To define each, we first make a view function decorated with @app.errorhandler(num), a view function, and an HTML file. Let’s look at each example.
Not Found (404) Errors
To handle 404 (not found) errors, we create a view function with the special error handler routing function, which renders the HTML file. Flask will automatically direct all not found error conditions to this view. The following shows the view function for the 404 not found error handler. As you can see, it is simple.
The associated error handler HTML code is in the file named 404.html as shown in the following. Note that we inherit it from the base.html file so the resulting web page looks the same as any other in the application complete with the menu from the bootstrap component. Note that we also can define the text for the error message and a title. Feel free to embellish your own error handlers to make things more interesting for your users.7
Application (500) Errors
To handle 500 (application) errors, follow the same pattern as before. The following is the error handler for the application errors.
The associated error handler HTML code is in the file named 500.html as shown in the following. Note that we inherit it from the base.html file so the resulting web page looks the same as any other in the application complete with the menu from the bootstrap component.
Creating these basic error handlers is highly recommended for all Flask applications. You may find the application error handler most helpful when developing your application. You can even augment the code to provide debug information to be displayed in the web page.
Redirects
At this point, you may be wondering how a Flask application can programmatically direct execution from one view to another. The answer is another simple construct in Flask: redirects. We use the redirect() function (imported from the flask module) with a URL to redirect control to another view. For example, suppose you had a list form and, depending on which button the user clicks (submitting the form via POST), you want to display a different web page. The following demonstrates how to use the redirect() function to do this.
Here, we see there are three redirects after a POST request. In each case, we are using one of the routes defined in our application to tell Flask to call the associated view function. In this way, we can create a menu or a series of submit fields to allow the user to move from one page to another.
The redirect() function requires a valid route and for most cases, it is simply the text you supplied in the decorator. However, if you need to form a complex URL path, you can use the url_for() function to validate the route before you redirect. The function also helps avoid broken links if you reorganize or change your routes. For example, you can use redirect(url_for(“author”)) to validate the route and form a URL for it.
Additional Features
Application and request context : There are variables you can use to capture application context such as session, global, request, and more. For more information, see http://flask.pocoo.org/docs/0.12/appcontext/ .
Cookies : You can work with cookies if you require. For more information, see http://flask.pocoo.org/docs/0.12/quickstart/#cookies .
Flask-Moment—Localization of dates and times: If you need to work with localization of date and time, see the Flask-Moment extension at https://github.com/miguelgrinberg/Flask-Moment .
Flask Review: Sample Application
Now that we’ve had a brief primer on Flask, let’s see how all of this works. In this section, we review what we have learned in the form of a basic layout for a typical Flask web application. We will be using this as a guide for writing the library application later in this chapter. Don’t worry too much about executing this code as it doesn’t do much and is intended as a jumpstart for the chapter project. However, it does demonstrate how all the parts we’ve learned are pieced together to get the Flask web application running with no forms defined.
Listing 8-9 shows the sample application layout for the library application. Take a moment to read it through. You should find all the topics we’ve discussed thus far with placeholders for field classes, form classes, and view functions.
Sample Flask Application Template
Note that there is one thing in this template we haven’t talked about yet—utility functions. These are your own functions to support your application. One function you may want to consider including in all your Flask applications is a function to loop through the errors on a form and display them in a flash message. Recall flash messages are displayed as popup boxes on the web page. The following presents the utility function for clarity. Note that we use a for loop to loop through the errors array of the form instance flashing each message. This permits you to display multiple messages on the web page.
Feel free to use this template when creating your own Flask applications. We will also be using it in the next section to define the user interface for the library application.
Tip
For more information about Flask and how to use it and its associated packages, the following book is an excellent reference on the topic: Flask Web Development: Developing Web Applications with Python 2nd ed., Miguel Grinberg, (O'Reilly Media, 2018).
Now that we’ve setup the Flask environment and discovered Flask and its extensions, let’s look at the user interface common to the three versions of the application.
Library Application User Interface Design
Now that we know a lot more about Flask and how to build Flask applications, let’s look at the library application user interface. As you may surmise, we build the database access as a separate set of classes but the user interface can be built nearly completely without it. Examining the user interface separate from the database access mechanisms makes it easier to focus on each part. We will discuss the database access mechanisms in the next section.
The user interface for the library application is the same code for all three versions of the application with some modifications to the code to adapt to the different database mechanisms. In particular, we have the full interface as presented here in version 1 (relational database), a reduced user interface for version 2 (hybrid: relational database with JSON), and version 3 will be more concise (document store). Therefore, we will write the form classes, view functions, and templates for all the web pages hosted in the user interface.
However, before we embark on writing the form classes, view functions, and templates for the application, we need to create a few directories.
Preparing the Directory Structure
Before we embark on implementing the three versions of the library application, we need to make a few folders (directories). Recall from the Flask Primer, we need folders to contain the .html files (form templates). We also place the code for interfacing with MySQL in a folder named database. Finally, we need a separate folder for each version of the application. Listing 8-10 shows the folder structure you will need. You can name the version folders however you wish, but the database and templates folder must be named as shown. Note we also have a folder named “base” that will contain the base user interface design without the database folder as discussed in the next section.
Directory Structure
User Interface Features
The library application will host three types of data: books, authors, and publishers linking them to form a library of books. The default view of the application will be a list of the books, which presents all the books in the form of an abbreviated bibliography. There will also be views for all the authors and all the publishers. Users will also be able to view the data for a specific book, author, or publisher allowing them to update or delete data items. Thus, the library application demonstrates the basic create, read, update, and delete (CRUD) operations for data.
Note that on the form there is a select (dropdown) field. This field is populated with the names of the publishers in the database. Likewise, there is a multiselect field that allows users to select one or more authors in the database. As you will see when we discuss the database design, this layout is somewhat forced on us when using relational data. We will populate these lists in the view function. Note also we see both the Add and Delete submit fields (buttons). Recall, we will disable the Delete button in the template—it would not normally be enabled when adding a new data item.
Note that the form is very short. It shows only the two fields along with the Add and Delete button, which will be controlled in the template.
Now that we have had a look at the basic user interface, let’s look at how to build the form classes for the three form classes for the detail views and a single form class for presenting the list, which uses inheritance and a bit of template constructs to share the form class and template among all three list views. Cool, eh?
Form Classes
The form classes for the library application require three form classes. There will be one for each of the author, publisher, and book views along with a form class for the reusable list view. Let’s begin with the simplest form class (author) and work our way to the more complex (book).
Author Form Class
The author form is very simple and requires only three fields: one to store the primary key for the row using a HiddenField field class, one for the first name, and one for the last name. Both name fields use a TextField field class. Validation for the name fields set both to required (hint: they’re defined as NOT NULL in the database table) along with minimal and maximum length checks. We also need two SubmitField field classes: one for Add and another for Delete. Recall, we will control programmatically the delete button in the template. Listing 8-11 shows the AuthorForm form class.
AuthorForm Class
Publisher Form Class
The publisher form also is very simple and requires only four fields: one to store the primary key for the row using a HiddenField field class, one for the publisher name, one for the publisher city of origin, and another for the URL for the publisher. All three visible fields use a TextField field class. Validation for the name and city fields set both to required (hint: they’re defined as NOT NULL in the database table) along with minimal and maximum length checks. The URL field as no validators because it is an optional field for the data (it can be NULL in the database table). We also see the two SubmitFields() for the Add and Delete buttons. Listing 8-12 shows the PublisherForm form class.
PublisherForm Class
Book Form Class
Field Classes for the Book Form Class
Field Name | Field Class | Validation |
---|---|---|
ISBN | TextField() | Required(), Length() |
Title | TextField() | Required() |
Year | IntegerField() | Required() |
Edition | IntegerField() | None |
Language | TextField() | Required(), Length() |
Publisher | NewSelectField() | Required() |
Authors | NewSelectMultipleField() | Required() |
create_button | SubmitField() | N/A |
del_button | SubmitField() | N/A |
new_note | TextAreaField() | None |
Before we discuss the form class for the book detail view, there is a small issue that needs to be overcome. There is a commonly known catch when using the SelectField() and SelectMultipleField() field classes. The prevalidation code can present some unusual results when validated if there is no default selected or if you set the default programmatically. To overcome these limitations, we can create our own derived field class and override the prevalidation code. Listing 8-13 shows the code used to create custom versions of these field classes to overcome the limitation. You will need to place these in your code if you want to use either of these field classes.
Creating Custom Field Classes
Note that in each case, we override the pre_validate() and process_formdata() methods allowing us to ignore prevalidation and make it easier to update the values. Now let’s look at the code for the book form class. Listing 8-14 shows the code for the BookForm form class. Note that we use the new field classes for the authors and publisher fields. We also see the two SubmitFields() for the Add and Delete buttons.
BookForm Class
List Form Class
To save duplicate code, we will create a simple form class that we can use to create a simple list of rows in the form of an HTML table. The code for this is very simple because all the presentation code will be in the template file. The following is the code for the ListForm form class.
Now that we’ve seen all the form classes for the library application, we now examine the associated view functions.
View Functions
View functions are where and how Flask applications direct execution. Together with the routes we define, we can build our application without loops or polling. Let’s dive in starting with the simplest view function. We will see the view functions with the routes defined (via decorators). The basic code for the author, publisher, and book view functions are the same and need no additional discussion. The only differences are the routes and the population or the select and multiselect fields in the book view function. Each function is show in Listing 8-15 (named author), Listing 8-16 (named publisher), and Listing 8-17 (named book).
Author View Function
Publisher View Function
Book View Function
The list view function is more complicated. Recall, we want to create a list that we can reuse. Thus, we will need to be able to create an HTML table with a list of column names and the rows we want to display. We can pass the columns and rows using parameters in the render_template() function. We also want to define the size of the columns. We can do this by passing HTML code to the template. In this case, we define them as HTML tags for the column names and in the template, use the safe filter to display it without translation.
We also want to create a link for each row that contains the primary key for the row, which we will pass as the first data item in each row. For authors and publishers, it is the auto increment primary key. For books, it is the ISBN. Thus, ISBN will be listed twice in the row. To determine which data we want, we use a variable in the list route. For example, if we want books, our URL would be localhost:5000/list/book. Cool.
Finally, because this view function is a default view, the routes are simple: the default (index) and list. Listing 8-18 shows the complete code for the list view function named simple_list. Take some time to read through it so you understand the code.
List View Function
Now that we’ve seen the code for the form classes and view functions, let’s look at the remaining piece of the puzzle: templates.
Templates
Templates are where we place all our HTML and template constructs for building our web pages (in the context of a database application, a data view or simply view8). The templates are presented with short descriptions and are provided for reference so that you can see all the parts together. Because we have four view functions, we will create four template files all of which will use the base template explained earlier. Recall, the base template defines the bootstrap navigation bar and a for loop for displaying an array of flash messages.
Note
Remember, template files are in the templates folder and named XXX.html by convention.
Author Template
The author template creates a form for viewing, editing, and creating author data. As such, we give the page a legend, host the hidden field (for the auto increment primary key), and place the label and form field on the form for each of the fields. We keep it simple by listing the fields vertically (but you can use any format you want). We also set the size of the fields using the form fields default function. For example, to set the size for the first name field to 75 characters, we use form.firstname(size=75). Finally, we see the logic to turn on the delete button if it is defined (we will see how to disable it later). Listing 8-19 shows the completed template for the author data (named author.html).
Author Template (author.html)
Publisher Template
The publisher template creates a form for viewing, editing, and creating publisher data. As such, we give the page a legend, host the hidden field (for the auto increment primary key), and place the label and form field on the form for each of the fields. We keep it simple by listing the fields vertically (but you can use any format you want). We also set the size of the fields. Finally, we see the logic to turn on the delete button if it is defined (we will see how to disable it later). Listing 8-20 shows the completed template for the publisher data (named publisher.html).
Publisher Template (publisher.html)
Book Template
The book template is a bit more complex. We start out with a legend and the hidden tag, which stores the ISBN for the current data, then build the form listing the labels and the fields vertically setting the size of the fields as we go. So far, this is similar to how we built the author and publisher templates.
Things get a bit more interesting when we try to set the field size for the select fields. In this case, we need to use the style parameter passing in the width parameter using units in pixels. This is one of the very few nuances of Flask templates that can be a bit tricky because the size parameter doesn’t apply to the select fields (but now you know how to get around it). As with the previous templates, we see the logic to turn on the delete button if it is defined (we will see how to disable it later).
After that, we see some additional logic for working with notes. The notes feature allows users to add notes to the book after it has been created. Thus, we need to both show any existing notes and provide a means to add a new note, but only when the page is used for update operations. You can see how this is done near the bottom of the file.
Listing 8-21 shows the completed template for the publisher data (named book.html). Take some time to read through the file until you are confident you understand how it works.
Book Template (book.html)
List Template
Despite the rather complex view function for the list feature, the template for the list view is rather straightforward. We simply add the New button (submit field) at the top, provide a legend, and then format the table using the columns array from the view function. We then build the HTML table using the rows also provided from the view function. Listing 8-22 shows the completed template for the list data (named list.html).
List Template (list.html)
Other Templates
Recall, there are three other templates that we’ve seen previously that we will be using: the 404 and 500 error handlers (404.html, 500.html) as described in the section, “Error Handlers” and the base template (base.html) as shown in Listing 8-5.
Application Code
Now, let’s put these concepts together in the application code completing the basic layout presented previously. Listing 8-23 shows the application code for the library application. Because this is the base version of the library application, we name the file mylibrary_base.py. We can use it as the basis for the three versions of the library application (named mylibrary_v1.py, mylibrary_v2.py, and mylibrary_v3.py).
The listing is presented for completeness without additional discussion. The portions of the code previously discussed are marked with [...] placeholders to avoid duplication. Rather, the listing is provided as reference for later sections that discuss the three versions. Feel free to read through the code to ensure you understand all the parts of the code and refer to it when reading the sections on the different versions.
Base MyLibrary Application Code
Now that we have a firm foundation in Flask and how the user interface is designed, we are ready to begin writing the code for each of the versions of the application starting with the relational database version.
Summary
Building MySQL applications using a good framework for not only the database access, but more important for the user interface. Deciding on a language and platform to use can sometimes become a science project or even an academic exercise or perhaps a mandate that cannot be overridden. Presenting concepts such as the document store with examples can be even more complicated as you must choose a language and framework that is easy to use and easy to understand. Perhaps even more challenging, choosing an application that illustrates the concepts in a meaningful manner.9
In this book, the choice for these technologies is Python, the Flask framework, and of course the MySQL Connector/Python database connector and the X DevAPI. Python is easy to read and anyone—even those who don’t write a lot of code—can understand it. Plus, it is a very powerful language. However, user interfaces in Python are limited to command-line (terminal) output unless you use a user interface framework. Once again, choosing one can be a challenge. However, frameworks for web applications are just the ticket to help build a decent looking example that readers can use as the basis for their own experiments and applications.
In this chapter, we learned a new web application library for Python named Flask. We also saw how Flask is built as an extensible framework that is easily augmented with components to make your application more robust. We also covered an introduction to the user interface for the library application building on the foundations of what we learned about Flask.
In the next chapter, I look at three versions of the application: a pure relational database solution that uses the old protocols, a relational database with JSON (hybrid) that uses the X DevAPI and SQL statements, and a pure document store version. Each version presents foundations on how to build the application using a different database access mechanism. As you will see, there is a profound transition from the old to the new.