Chapter 2: Project Configuration

Source code is considered the meat and bones, or the framing of a home, in any software. In this chapter, we will build a project that contains the files where the source code lives. We will discuss several tools that will come in handy when developers are working directly with their source code. When working with Django, while any tool can be used to edit the source code, some tools are more productive than others. In this chapter, we will explore some of the countless tools that exist and discuss why an Integrated Development Environment (IDE) might also be used.

We'll also learn about the importance of working with the Django settings.py file(s) of a project. Of course, software also requires a database to store and retrieve data entered and created by its users, and we will install a local and remote database for each environment of a project. We will go over the various database types that are available and then focus on the most popular type to use for the examples throughout this book. It is not required to use the same database type that we will be using, but you might encounter slight variations if you use a different one; proceed with caution. Reading this chapter is crucial before going through the rest of the book.

In this chapter, we will cover how to do the following:

• Choosing developmental tools
• Starting a project
• Creating a virtual environment
• Project configuration
• Using basic database settings
• Preparing PostgreSQL for Heroku

Technical requirements

To work with the code in this chapter, the following tools will need to be installed on your local machine:

• Python version 3.9 – used as the underlying programming language for the project
• Django version 4.0 – used as the backend framework of the project
• pip package manager – used to manage third-party Python/Django packages

Next, you will need a way to edit the code that we will be writing in this chapter and throughout the rest of this book. The first section of this chapter will provide you with several development tool choices from text editors to IDEs. The same IDE will be used to demonstrate a few actions but it is not necessary to use the same tool. You are welcome to use any IDE you like or use no IDE at all, using the terminal or command-line window instead. A database will also be required, and the third section of this chapter will provide several options to choose from. Any database type will work with your project and configuration examples will be provided, but only PostgreSQL will be used to work with our project and Heroku.

This book will focus on the concepts of Django and enterprise development, instead of guiding you through how to use Git operations. For a crash course on how to use Git operations, you can watch the video found here: https://www.packtpub.com/product/git-and-github-the-complete-git-and-github-course-video/9781800204003.

All of the code created in this chapter can be found in the GitHub repository for this book: https://github.com/PacktPublishing/Becoming-an-Enterprise-Django-Developer. The code used throughout this chapter will relate to every file in the core of the project. In every chapter, starting from Chapter 3, Models, Relations, and Inheritance, code related to a particular chapter will reside in its own Django app folder. Refer to the subsection of this chapter titled Creating a Django app to learn more about what that means.

Check out the following video to see the Code in Action: https://bit.ly/3NqNuFG

Choosing development tools

Configuring our project refers to how we will structure and arrange the files that make up an application. This also refers to how we share those files within a collaborative team. Some tools create files that can be shared among your team members, such as preconfigured settings related to development and debugging features. These files are sometimes referred to as configuration or solution files. This means you can preconfigure a set of development tools to help get your team up to speed quickly. Making sure that all members use similar tools makes debugging and looking at code that is not written by you much easier. This consistency also makes verbal and written communication among members of your team more efficient when synchronizing workflows.

While there are benefits to sharing project configuration files, it is also not necessary for everyone on your team to be using the same tools. In fact, it's even possible to create many different configuration files for a variety of development tools all included in the same repository, giving your developers a multitude of preconfigured project files to choose from. Even if we provide configuration files in a repository, a developer can still use a basic text editor with the terminal or command-line window to make edits and run the project locally. Let's compare several of these text editor tools and discuss some of the benefits that they provide. Then, we will discuss what IDEs are and how they differ from text editors.

Text editors

A text editor is quite simply exactly what it sounds like, a means to edit text, or in our case, source code. Text editors are sometimes referred to as lightweight IDEs, given the number of features that some of these popular editors come packed with.

Likely, what are considered the three most popular text editors today are the following:

• Atom
• Notepad++
• Sublime Text

Atom

Atom is intended to be a fully editable and customizable text editor. The tool's official website uses the term hackable when describing their software. This editor allows you to install packages that help enhance your code-writing abilities. It has smart code auto-completion and code styling to let you type less and see more. This tool has a built-in file browser that allows you to find and replace code in the folders and files that you choose to search through. Atom also works on all three major operating systems, Windows, Mac, and Linux. It is free to use and also open source, allowing you to modify the editor's source code itself. To download Atom and start using it today, visit https://atom.io/.

Notepad++

Notepad++ is another free and open source editor that comes with a stylish code syntax highlighter and provides code auto-completion suggestions. It was built to work with and run a vast number of different programming languages. It even lets you write macros and run custom scripts directly from the editor. Like Atom, it also has the capability of installing plugins. It's really lightweight and can be a nice editor to set as the default for your operating system. To download Notepad++, visit https://notepad-plus-plus.org/downloads/.

Sublime Text

Sublime Text is another popular choice. This editor allows you to create project files and includes features to build a wide variety of programming languages. One cool feature is how it will use your Graphical Processing Unit (GPU) to improve the performance of the editor's UI while you work. It uses what they call context-aware auto-completion, meaning the editor will intelligently offer code completion suggestions based on what code exists on the page, regardless of whether you are in a file that is of another language. For example, you can edit code contained in a <script type="text/javascript"></script> that lives inside of a .html document to be styled and displayed differently from the HTML found in that same file.

Another application in the Sublime family is called Sublime Merge, which is a simple way to merge code and perform Git operations. My favorite feature is using it to view history states and commit logs. Sublime Text is free to evaluate for a certain amount of time but eventually, it will prompt you to purchase a license after the trial period is over. To get started with Sublime Text, visit https://www.sublimetext.com/.

There are dozens, if not hundreds, of other text editors to choose from that were not mentioned in this chapter. Don't limit yourself to only the popular choices provided and feel free to explore many other tools. The number of features and capabilities that some text editors provide can sometimes be referred to as a lightweight IDE; let's discuss what an IDE is next.

Integrated development environments

An IDE is what we call software that combines many different programming tools into one single desktop application that a developer can use to build other applications. It is a development tool that your team can use and benefit from to stay productive. IDEs primarily consist of a way to view and edit source code, automate local build processes, and provide ways to help you debug and analyze your code. The tool contains ways to style and format code, show you errors as you type, and provide code completion suggestions to reduce keystrokes as you work. These environments also provide ways to search other code files contained in your project and push or deploy your code to external repositories. For example, the Sourcetree app that was mentioned in Chapter 1, Undertaking a Colossal Project, would no longer be needed if you are using an IDE that has Git features built into it.

The same applies to a text editor choice that also has Git features included. Similar to some of the text editors with fancy features, IDEs will create common configuration files that can be shared with your team. Some of these files we don't want to share, such as files that store breakpoints and other local debugging settings specific to that developer and local instance only. However, sharing configuration settings with your team allows members to get their IDEs up and running on their machines much easier and quicker, with all of the tools ready or almost ready to use.

A simple text editor is really all you need. A project could start out this way and then your team could introduce the use of an IDE later on. Sometimes, when using an IDE, you can structure your project files in subdirectories that would otherwise not exist. If you already know your team will need the productivity tools from the get-go, you can start out with an IDE and create a project through the IDE itself, letting the IDE structure your files in a way that is natural to that IDE. It's sometimes easier this way because some IDEs may configure your file structures slightly differently than what would otherwise be created by using just the Django startproject command from a terminal or command-line window.

Let's discuss some of the popular IDE choices that developers use with Django today and then settle upon one of these choices to demonstrate the concept of using an IDE. This chapter will demonstrate working with Django management commands both in the IDE and the terminal to showcase the benefits of using one versus the other. For every action demonstrated using the IDE in this chapter, the command-line-driven command equivalent of that action will also be provided. This will also allow you to bypass the IDE altogether if you wish to do so. Moving forward, all future chapters will only provide the standard command-line-driven Django management commands and continuing to use the IDE will be optional.

PyDev with Eclipse

PyDev is actually a plugin for the Eclipse IDE but it can also be used as a standalone IDE. PyDev can be downloaded and installed directly by itself because it will come preinstalled with LiClipse, a lightweight version of the Eclipse IDE. The Eclipse IDE is a fully integrated development experience. It allows the debugging of code in a variety of ways. Profiling is just one of those ways, which is a tool that helps the developer understand the timing of events, memory usage, disk usage, or any other diagnostics. The term CPU Profiler is often used to discuss tools that can help you find out what specific process is bogging down your system. It will tell you things such as how long it was hanging for and give you an idea as to how to fix it. All of the bells and whistles that come with all of the text editors I mentioned previously also come with Eclipse. PyDev has a vast library of packages to choose from for many different languages that exist today. Eclipse and PyDev both work on all standard operating systems today, such as Windows, Mac, and Linux. To download Eclipse, visit https://www.eclipseclp.org/download.html.

Eclipse and PyDev are both free to use and are both open source licenses, allowing you to modify the IDE software itself. The Eclipse IDE is a bit more difficult to install than other desktop applications today. Its installers require a good deal of reading before you know how to get started with installing it. Eclipse also requires Java to be installed and running on your machine. Java is a high-level programming language and a computing platform that is usually not required to work with Django and Python unless you are working with Eclipse. For this reason, we will not be using this IDE in this chapter. To download PyDev, visit https://www.pydev.org/download.html.

PyCharm

PyCharm is likely the most popular choice among Python and Django developers today. It is easy to use and much easier to install than PyDev by using a simple executable file available for Windows, Mac, or Linux machines. It comes with a free community version as well as a paid professional version. The paid version offers many more features and provides more specialized scientific and web development tools, such as advanced database integration and database development, directly from within the IDE, as well as other debugging and profiling tools. The free community version is enough for most Django developers to get by on. This version allows developers to work with Django project integrations and run virtual environments while connecting to remote Git repositories. To download and install PyCharm, visit https://www.jetbrains.com/pycharm/download/.

While this is likely the most popular IDE among Django developers, Visual Studio is likely the most popular IDE among developers of any language or framework. For this reason, we will use Visual Studio to demonstrate the examples throughout this chapter.

Visual Studio

Visual Studio has been a very popular tool of choice among many .NET, C#, C++, and other developers for over 20 years now. It's a very robust IDE that comes packed with all of the tools that you can think of and then more. It comes in many different varieties and flavors, and for years it was only available for a price. Then, around 2013, Microsoft started offering the Visual Studio Community Edition for free to the public. A drawback for some was that Visual Studio was only available on Windows platforms until the year 2017, when it became available on Mac platforms. Currently, Linux is not supported. However, Visual Studio Code, a lightweight IDE, is available on all three platforms, Windows, Mac, and Linux.

Visual Studio Code has likely become the most popular tool of all time among any type of developer. Both Visual Studio and Visual Studio Code support using Python and Django. Upon installing either of these tools, you will need to select related Python/Django packages that will be included in your installation or install them separately. For those of you working on a Linux machine, or if you just don't want to use an IDE, refer to the management commands provided after the IDE demonstration of each action involved in creating a project. If you are on a Windows or Mac system and wish to follow along using an IDE, download the Visual Studio 2019 – Community Edition installer found here: https://visualstudio.microsoft.com/downloads/. Make sure, during your installation, that you select any Python development extensions that it offers, as shown in the following screenshot:

Figure 2.1 – Visual Studio – Python development extension

You might find other useful tools under the Individual components tab of this interface. Include any other tools that you would like to include before proceeding with your installation. Now that we have Visual Studio 2019 – Community Edition installed, let's build a solution file for a project that can be shared with other developers in a repository.

Note

While Visual Studio 2019 is the latest product available from Microsoft, Visual Studio 2022 will be released about the same time this book will be published. If you are using the newer version of Visual Studio, you should be able to perform all of the same actions as depicted in this chapter. The screenshots of the Visual Studio IDE may not appear exactly the same and some code adjustments might also be necessary.

Starting a project

There are two ways to start a project and this chapter will allow you to choose which method you want to follow. We encourage you to use the IDE as becoming proficient with using this tool in your team will be beneficial in the long run. However, if your team is using an IDE other than Visual Studio or you are only using a text editor to work with your code, the command-line equivalent of performing each step is also provided to allow anyone to work through this chapter. All other chapters in this book will focus on code only, which can be used with or without an IDE.

Using the IDE

Open the Visual Studio IDE and select Create New Project. When you are directed to the next screen, search for the django keyword and in the list of results, select Blank Django Web Project, as depicted here:

Figure 2.2 – Visual Studio – Create a new project

On the next screen, enter becoming_a_django_entdev for Project Name. This will auto-populate your Solution Name field as you type. For the Location option, select the folder on your drive where you placed your local Git repository from Chapter 1, Undertaking a Colossal Project, as your location. At the bottom, make sure the checkbox labeled Place solution and project in the same directory is checked as is done in the following screenshot. Enabling this checkbox will place a solution file in the same folder as the manage.py file, known as the root of a project. It will make using things such as the terminal that is provided within the IDE just a little easier:

Figure 2.3 – Visual Studio – Creating a Django project

Note

When using the Visual Studio IDE to create a project, the files found in the /becoming_a_django_entdev/becoming_a_django_entdev/ folder, such as the settings.py and urls.py files, are automatically generated using Django 2.1.2. These files will still work when used with later versions of Django. Additionally, when we get to the Creating a virtual environment section of this chapter, we will actually be installing the Django 4.0 package, which is the version used throughout this book. Even though version 2.1.2 was used to initially create some of the project files, the project will always use version 4.0 and run successfully. When using the terminal or command-line window to create a project, this scenario will not occur. Later, in Chapter 9, Django Testing, you will learn how to verify what version is actually installed and being used.

Using the command line

The Django terminal or command-line window equivalent of starting a Django project is the startproject command. There are two ways to create a project using this command. The first method is to create your project using the version of Django that was installed on your machine globally and then build your virtual environment next.

The other way is to create your virtual environment first and then activate your environment, install the version of Django that you desire, and then build your project using the version of Django that was installed in your virtual environment. What the IDE did for us was create a project first using a version of Django that the IDE provides, and then when we created the virtual environment, the version of Django was updated to the version specified in the requirements.txt file.

When a package gets updated in the virtual environment, the old version gets uninstalled and the new version gets installed fresh. For this exercise, we will uninstall any version of Django that may exist globally, then install the latest version available when this book was written. Then, we will create the virtual environment using the command line in the next section, following along as close as we can to the examples provided using the IDE.

Follow these steps to create your project files:

1. Open your terminal or command-line window and navigate to your local repository folder created in Chapter 1, Undertaking a Colossal Project. Make sure you are not in a virtual environment at this time. Then, execute the following commands to uninstall any existing versions of Django and then install the proper version of Django that we will be using globally on your machine:

   PS C:ProjectsPacktRepo> pip uninstall django

   PS C:ProjectsPacktRepo> pip install django==4.0

2. Execute the Django command that will create a project and all of the core files necessary to work with Django, based on Django version 4.0. We will name this project the same as in the IDE example: becoming_a_django_entdev. The startproject command creates a manage.py, wsgi.py, and asgi.py file and several other boilerplate files that serve as the foundation of all Django projects. The becoming_a_django_entdev option is the name of the project and the folder that a project will be placed in. Execute the following command to create the project:

   PS C:ProjectsPacktRepo> python -m django startproject becoming_a_django_entdev

The preceding is a friendly command that has been tested and proven to work on a Windows machine. Traditionally, developers would use the following command to start a project; however, this command does not work on a Windows operating system:

PS C:ProjectsPacktRepo> django-admin startproject becoming_a_django_entdev

Next, let's create and configure a virtual environment, which is needed to work with any third-party packages included in a project.

Creating a virtual environment

We should not have a virtual environment for our project at this time. If you do have one, go ahead and disregard it and create a new one for this next exercise. Whether you created a project using the Visual Studio IDE or by using the Django commands from a terminal or command-line window in the previous exercises, the file structure in your repository should look like the following tree structure:

├── .git

├── readme.md

├── requirements.txt

├── becoming_a_django_entdev

│ ├── .vs

│ ├── becoming_a_django_entdev.sln

│ ├── db.sqlite3

│ ├── manage.py

│ ├── obj

│ ├── requirements.txt

│ ├── staticfiles

│ └── becoming_a_django_entdev

│ ├── __init__.py

│ ├── asgi.py

│ ├── settings.py

│ ├── urls.py

│ └── wsgi.py

We now have two requirements.txt files and two folders called /becoming_a_django_entdev/becoming_a_django_entdev/, where various files reside. We will leave the folder structure the way it is now and configure additional settings later to allow Heroku to work with this folder structure. We are doing this because creating a project through the IDE will only produce this result, and note that other IDEs or even lightweight IDE text editors may create folder structures that even vary from the preceding tree. If you decide to use the Django commands to create a new project, you do have the option to specify an additional option to prevent creating your project in a subdirectory. This would put the manage.py file in the same directory as the root of a repository and result in only one requirements.txt file. Doing that would require using the following command example:

PS C:ProjectsPacktRepo> python -m django startproject becoming_a_django_entdev ./

Here, we add the additional./ option at the end of the preceding command, which states to put the project in the folder that we are located in now. Without this option, an additional subfolder would have been created by default, naming that folder the same as the name provided for the project name. However, doing this would result in no solution file and now no way to run the project through the Visual Studio IDE. Since the objective of this chapter is to demonstrate the use of an IDE, this is the reason we are keeping the preceding tree structure. There are alternative options to doing this, which would lead you down the path in Visual Studio of creating a new project using the from existing code option. This would allow you to create a project using the terminal or command-line window, structuring the folders how you desire and then creating your IDE solution files afterward. Due to the complexity involved in doing that, we will not discuss how to do that. Instead, we can write one line of code in the first requirements.txt file in the root of our repository to work with this configuration.

We will discuss doing that next.

Configuring the requirements.txt file(s)

In the same folder that the manage.py file is located in, Visual Studio created for us a requirements.txt file. In Chapter 1, Undertaking a Colossal Project, we already created a blank requirements.txt file just so that we can satisfy the needs of Heroku during deployment to each environment that we created. If the requirements.txt file in the root of the repository does not exist, Heroku will fail to deploy. Heroku needs that copy in order to identify a project as a Python project; that's just how their test scripts have been written to work. This is why we now have two requirements.txt files to work with. The other file, nested one folder down in /becoming_a_django_entdev/, is needed to enable all of the features and services that the Visual Studio IDE offers us. This requirements.txt file is where we will write all of our required dependencies.

In the requirements.txt file found in the root of your local repository, which is the same folder that your .git folder is located in, add the following code:

# ././requirements.txt in Root Directory

# Path to Visual Studio requirements.txt below

-r becoming_a_django_entdev/requirements.txt

The preceding code declares the path to the requirements.txt file nested in what we will call the root of our project from now on, /becoming_a_django_entdev/. Open the requirements.txt file in the root of your project, which is also the same folder that the manage.py file resides in, and then add the following items:

# requirements.txt

django~=4.0

django-extensions

django-heroku

dj-database-url

gunicorn

python-dotenv

pydotplus

psycopg2

psycopg2-binary==2.8.6

whitenoise

secret-key-generator

Make sure each item is located on its own line. Any packages that are required for your project will always be put in this file. You can get even more modular and add other imports to these files, for what is called cascading requirements files. This is done the same way, using -r becoming_a_django_entdev/requirements.txt to specify the path of the file.

In the preceding example, the ~= operator is used and indicates greater than or equal to the version number specified. For example, it will provide Django in the version 4.0 range, meaning it will go to 4.0.9 if that exists but it will not provide 4.1 or higher. It will only provide the highest version of the 4.0.X range. The == operator would mean to include the package that matches the exact version number specified. Not denoting any version at all means it will grab the latest version that exists for that package at the time of installation, re-installation, or when an upgrade is performed. Anything you would find in the PyPI package library can be placed in this file, requiring that it be installed in your virtual environment.

The django package is the most important as it is the basis of the framework we are using. The django-heroku package is a package created by Heroku that includes a series of other packages that Heroku itself depends on. Those Heroku dependencies will be automatically installed for you when the parent package is installed. The other packages listed previously will be used in the next sections to help us configure a project properly for how we will be using Django throughout the rest of this book.

Now that we have defined a number of packages to install into a virtual environment, let's create our virtual environment using the Visual Studio IDE.

Using the IDE

For those of you who wish to use the command line instead, skip ahead to the subsection titled Using the command line.

Within your Visual Studio IDE, navigate to the Solution Explorer section of your IDE, right-click where it says Python Environments, and then select Add Environment. The window that opens is shown here:

Figure 2.4 – Visual Studio – Add environment

When the window pops up, enter virtual_env as the name of your virtual environment and choose your base interpreter. This name can be anything you choose but name it virtual_env to be consistent with the examples throughout this book. Python version 3.9 was chosen as the base interpreter at the time of creating this exercise. The location is very important, where it asks whether you want to Install packages from a file (optional); this option refers to the location of your requirements.txt file that lives in the same folder as your manage.py file. Changing this location may break some of the features of using your IDE, such as operations that work with your virtual environment, and lead you to experience unresolved import issues. Unresolved imports may allow your project to still run, but code highlighting and formatting will often break. You should now see your virtual environment in your Solution Explorer and your Python Environments window within your IDE, as depicted here:

Figure 2.5 – Visual Studio – virtual environment a success

If you did want to install packages later, for instance, if someone added a package to the requirements.txt file 2 months from now, a developer could right-click on the virtual environment in the Solution Explorer and then select Install from requirements.txt. Visual Studio will then update any versions and install any missing packages. Visual Studio and pip do not usually remove unused packages; you may have to manually uninstall a package if you are experiencing conflicts.

Running/activating the project

By now, everything we did should allow us to run a project locally because Visual Studio did a lot of the heavy lifting for us. If you decided not to use the IDE, you cannot currently run your project and would have to work through most of this chapter before achieving this result. In either scenario, you will not currently be able to run your project on Heroku. Using the Play/Run Web Server button found at the top of the IDE, select your desired browser, if it provides a dropdown of choices, and then press play, as depicted in the following screenshot:

Figure 2.6 – Visual Studio – run project

When play is pressed, a command window will open where you can watch any console messages being printed to the screen and a new tab in the browser chosen will also open. The address in the address bar will point to http://localhost:#####/ and your project should now be running successfully. The port number will usually be a random number. Visual Studio uses a port that it determines is currently unused on your machine and that is also not a default or reserved port, such as 8000. This is a built-in feature of Visual Studio for people who run multiple projects side by side, as I often find myself doing.

If you do not see this button, you can add it to your toolbar by right-clicking a blank space in the toolbar area of Visual Studio. In the dropdown that appears, select Standard from the list of choices. If you select Python from this list, you can include tools that work with your virtual environment directly from your toolbar.

Along with the command-line window that opens up, the browser you selected will also open with a new tab pointing to http://127.0.0.1:#####/, where ##### is the random port that Visual Studio used. Here, you will see the iconic landing page for a successful Django installation, as depicted in the following screenshot:

Figure 2.7 – Django installation success

To use the standard port 8000 with the play button in Visual Studio, pointing to http://localhost:8000/, follow the steps in the next subsection.

Manually setting a port

We can specify the port number very easily in Visual Studio to control which port each project is using. On the Project tab at the top of your IDE, select becoming_a_django_entdev Properties from the dropdown that appears. On the tab that opens in your editor, click on the Debug tab inside and specify the port number in the field titled Port Number.

Next, we'll see how to create a virtual environment using the command line.

Using the command line

For many of you who prefer the terminal or command-line window, a popular module used to create virtual environments that is available for Windows, Mac, and Linux is called venv.

Follow these steps to create your virtual environment:

1. Open up your terminal or command-line window and navigate to the local Git repository that we created in Chapter 1, Undertaking a Colossal Project.
2. Navigate into your first folder called becoming_a_django_entdev. This folder should already exist and be populated with files created when using the command line or IDE to start a project earlier in this chapter:

   PS C:ProjectsPacktRepo> cd becoming_a_django_entdev

3. In this directory, the same directory that the manage.py file lives in, run the following command to create a virtual environment called virtual_env:

   PS C:ProjectsPacktRepoecoming_a_django_entdev> python -m venv virtual_env

The venv module should come standard with all Python installations, but if you have problems running this command on any of the three major platforms, visit the documentation found here to help you debug the issue: https://docs.python.org/3/library/venv.html.

4. For Windows, activate your virtual environment:

   PS C:ProjectsPacktRepoecoming_a_django_entdev> virtual_env/Scripts/activate

Mac and Linux users should jump to the Activating the virtual environment subsection to learn how to activate a virtual environment on those platforms.

5. Next, install the packages defined in the requirements.txt file found in the root of a project, where the manage.py file lives, by running the following command:

   PS C:ProjectsPacktRepoecoming_a_django_entdev> pip install -r requirements.txt

An alternative method to creating a virtual environment on Windows, Mac, and Linux is using virtualenv, as shown:

PS C:ProjectsPacktRepoecoming_a_django_entdev> pip install virtualenv

PS C:ProjectsPacktRepoecoming_a_django_entdev> virtualenv virtual_env

Now that we have created a virtual environment for our project, let's activate that virtual environment and run the project next.

Activating the virtual environment

Instead of using the IDE, we can activate a virtual environment and run a project directly from the command line. If you already activated your virtual environment in the previous subsection, you can skip this subsection. The following are examples that show how to activate your virtual environment for each major platform (Windows, Mac, and Linux).

Follow these steps to activate your virtual environment:

1. For Windows users, navigate into the root of your project, where the manage.py file is located, and then activate your virtual environment by using the following command:

   PS C:ProjectsPacktRepoecoming_a_django_entdev> virtual_env/Scripts/activate

2. Mac and Linux users will need to run the following command instead:

   PS C:ProjectsPacktRepoecoming_a_django_entdev> source virtual_env/bin/activate

If successful, you will now see the following prompt in your terminal, waiting for the next command to be executed from within your virtual environment:

(virtual_env) PS C:ProjectsPacktRepoecoming_a_django_entdev>

You can only execute standard Django management commands when your virtual environment is activated.

Note

From now on, the preceding example will be depicted as the following in order to save space, remove clutter, and prevent confusion when providing terminal or command-line examples from within an active virtual environment:

(virtual_env) PS >

Now that the virtual environment is activated, let's run our project.

Running the project

If you decided to create your project using the command line and not the Visual Studio IDE, you will not currently be able to run your project. This is because Visual Studio created a local SQLite database, made any necessary migrations, and migrated them automatically for every package that we included in the requirements.txt file. For reference, the command to run your project is shown in the following code snippet. You will have to work through the exercises found in the PostgreSQL section of this chapter to configure your database before successfully executing the following command. You can come back to this section after having done that.

Making sure you are still in the same folder as the manage.py file and that your virtual environment is active, execute the runserver command shown here:

(virtual_env) PS > python manage.py runserver

If this was successful, you will then see the following information printed in your terminal. Whenever you load a page and your project is running, you will see all messages that get printed to the screen in this terminal:

Watching for file changes with StatReloader

Performing system checks...

System check identified no issues (0 silenced).

Django version 2.2.24, using settings 'becoming_a_django_entdev.settings'

Starting development server at http://127.0.0.1:8000/

Quit the server with CTRL-BREAK.

Manually setting a port

Manually specifying a port number is easy using the terminal or command-line window. However, this must be done every time we run the project, unlike in the IDE where we set it in the configuration of the project within the IDE. The following example adds a port number option to the same runserver command used in the previous section, stating to use port 8000 to run the project:

(virtual_env) PS > python manage.py runserver 8000

Project configuration

All projects need configuring in some way to work with all of the packages and hosts that are involved. We will take the project that we just created and configure files such as settings.py, .env, procfile, and .gitignore. Take the settings.py file—this file or files will store all of the global constants that are used throughout the code of your project. When packages are used, they usually provide a way to customize the behavior of that package from within the main settings file. Other files, such as .env and procfile, will be used to prevent deployment issues when working with Heroku as the host. Currently, we would not be able to deploy successfully to the Heroku environment as we have created the project files. Work through the following configuration sections before attempting a successful deployment.

Django settings.py file

In the settings.py file that was automatically generated for us when we created a project, we need to add settings specific to our project.

Follow these steps to configure your project settings:

1. At the top of the settings.py file, add import django_heroku just below the first two imports that exist. Also, add import dotenv and dj_database_url. These two packages will be used to establish a database connection. The top of your settings.py file should look as in the following example:

   # /becoming_a_django_entdev/settings.py

   import os

   import posixpath

   import django_heroku

   import dj_database_url

   import dotenv

Note, that some systems may display from pathlib import Path at the top of their files instead of import os and import posixpath.

At the bottom of the settings.py file, add the following code:

# /becoming_a_django_entdev/settings.py

...

django_heroku.settings(locals())

This statement will import Heroku-specific settings from the django_heroku package.

2. In order for your DNS to work properly, you will need to tell Django that you are allowing a host to gain access to this site. This is a built-in security feature of Django that is intended to block common HTTP Host Header Attacks. We also need to add the URLs of each of the Heroku apps to make sure those databases will function properly. Your ALLOWED_HOSTS should look similar to the following example:

   # /becoming_a_django_entdev/settings.py

   ...

   ALLOWED_HOSTS = [

       'your-domain.com',

       'www.your-domain.com',

       'dev.your-domain.com',

       'staging.your-domain.com',

       'becoming-an-entdev.herokuapp.com',

       'mighty-sea-09431.herokuapp.com',

       'pure-atoll-19670.herokuapp.com',

   ]

Some options that are available include a wildcard such as the asterisk character (*), which will allow anything. However, this is not considered a best practice and is extremely insecure. If a domain starts with a single period, like the second entry of the following example, it will also act as a wildcard, allowing all subdomains of the corresponding parent domain. Use these wildcard options with caution if you decide to use them at all:

# /becoming_a_django_entdev/settings.py

...

ALLOWED_HOSTS = [

    '*',

    '.your-domain.com',

]

3. Starting with Django 3.2, we have to add a variable to the settings file named DEFAULT_AUTO_FIELD. Prior to this version of Django, this was not necessary. This setting tells Django how to process and handle all primary keys of objects. Without this setting, we would otherwise have to add a field called id = models.AutoField(primary_key=True) to every model class that we created. Since this is a daunting task, we can avoid it altogether by using the following example, placed anywhere in our settings.py file:

   # /becoming_a_django_entdev/settings.py

   ...

   DEFAULT_AUTO_FIELD = 'django.db.models.AutoField'

For now, let's just leave the default DATABASES = {...} setting as it is. In the Preparing PostgreSQL for Heroku section of this chapter, we will discuss changing this setting for our particular use case. Other examples will be provided for all the supported database types, providing a quick reference when working on a project that does not follow along with every example in this book.

Let's create our environment files next.

Creating an environment file

Heroku will use an environment file called .env to store environment-related variables that are used when running a project in each environment. We use this to tell Heroku things such as what database is being used for each environment or if debug should be turned on/off. Heroku recommends that we use SQLite locally for a Django/PostgreSQL setup but not in their environment apps. It can be configured to use PostgreSQL locally, which we will demonstrate near the end of this chapter. Even if you are using SQLite3 locally, you still need to install the PostgreSQL software suite on your machine in order for the drivers to work with all of your remote connections. We will need to walk through the rest of configuring a project before we can discuss setting up PostgreSQL for use locally.

Local variables

To create your local variables, follow these steps:

1. Run the following command from your project's root directory, where your manage.py file exists:

   (virtual_env) PS > echo 'DATABASE_URL=sqlite:///db.sqlite3' > .env

The echo statement is used to create a new file called .env with a single line of content, that content being DATABASE_URL=sqlite:///db.sqlite3.

Windows users might experience an error message such as UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte when trying to run the preceding command. There is an encoding issue that often arises on Windows machines that triggers this kind of error. To overcome this, open your .env file in Notepad++, go to the Encoding dropdown, select UTF-8 from the list of choices, and then save. If that fails, which is possible, just delete the file and recreate it from scratch using Notepad++ to create the file, using the proper encoding.

2. Whichever way this file was created, make sure the following line of code is in your local .env file:

   # /becoming_a_django_entdev/.env

   DATABASE_URL = sqlite:///db.sqlite3

The code we placed in this file points to the location of your SQLite3 database file called db.spqlite3. If you didn't use the Visual Studio IDE to create your project, this file will not exist yet and your project will not currently run.

3. We can also add other variables to this file, and then access those variables using the python-dotenv package in the settings.py file. To access these variables, we have to load the .env file in the settings.py file using the following example:

   # /becoming_a_django_entdev/settings.py

   ...

   import dotenv

   BASE_DIR = os.path.dirname(

       os.path.dirname(os.path.abspath(__file__))

   )

   dotenv_file = os.path.join(BASE_DIR, ".env")

   if os.path.isfile(dotenv_file):

       dotenv.load_dotenv(dotenv_file)

   SECRET_KEY = os.getenv('SECRET_KEY')

Place the highlighted code somewhere near the top of the settings.py file, just below the existing BASE_DIR variable. In this example, we also replaced the string value of the SECRET_KEY variable with os.getenv('SECRET_KEY'), with os.getenv() we can access any variable found in the .env file.

4. We need to add the string value of the SECRET_KEY variable that was in the settings.py file to the .env file in order for our project to work locally. Write your variable using the following example, without quotation marks for any string values found in this file:

   # /becoming_a_django_entdev/.env

   ...

   SECRET_KEY = my_randomly_generated_key

That's it, our local variables are now configured. Let's configure our remote variables next.

Remote variables

Remote variables are the same local .env variables that we created, now with values that pertain to each remote environment. Since we will be ignoring the .env file in our repository, which we will discuss in the subsection titled Creating a .gitignore file, we need to manually create the SECRET_KEY variable inside of the .env file for each Heroku environment that exists. Since Heroku already created these .env files for us, we will just use this as an opportunity to add the variables that are needed. These steps can also be used at any time during the SDLC to add variables as they are needed.

To add your remote variables, use the Heroku Command-Line Interface (CLI) to log into your Heroku account and replace the app name provided in the following code block with your Heroku app name:

(virtual_env) PS > heroku login

(virtual_env) PS > heroku config:add SECRET_KEY=my_randomly_generated_key --app becoming-an-entdev

(virtual_env) PS > heroku config:add SECRET_KEY=my_randomly_generated_key --app mighty-sea-09431

(virtual_env) PS > heroku config:add SECRET_KEY=my_randomly_generated_key --app pure-atoll-19670

In the preceding code, one example is provided for each of the three environments that exist. Execute them one at a time for each of your environments.

It is best to provide a different SECRET_KEY value for each environment. In Chapter 3, Models, Relations, and Inheritance, after we discuss working with the Django shell in the subsection titled Generating a SECRET_KEY variable, we will explore how to generate a SECRET_KEY variable in a safer way. Adding that SECRET_KEY variable to each Heroku environment will still be done as in this section.

Next, let's create a Procfile, short for process file.

Creating a Procfile

A Procfile is used to store additional project-related settings. Heroku uses the concept of containerization to host all of their apps for all of their clients. This is why we need to create a file with no file extension named procfile located in the root of a repository, where the .git folder is located. This file tells Heroku where the rest of your project's files live, specifically, the location of the wsgi.py or asgi.py file inside of the procfile file, using standard Python path syntax. The wsgi.py file is commonly found in the project_name folder of any Django project, upon creation; that folder is found in the same folder that the manage.py file resides in. If you wanted to use an asynchronous server gateway interface, you would specify the location of the asgi.py file instead.

Hypothetically speaking, if the manage.py file and the Django project folder lived in the same folder as the root of a repository, we would include the following path inside of that procfile:

# ./Procfile in Root of Repository

web: gunicorn becoming_a_django_entdev.wsgi

Our project files are nested one level deep in the /becoming_a_django_entdev/ folder, as was done using the IDE or when using the startproject command to create a project earlier in this chapter. If we tried to add this directory using standard Python path syntax, we would have problems during deployment to Heroku. Instead, in one line of code, tell Heroku to change directory first and then execute the preceding command, using the following code example:

# ./Procfile in Root of Repository

web: sh -c 'cd ./becoming_a_django_entdev/ && exec gunicorn becoming_a_django_entdev.wsgi --log-file -'

Please use the second example. Only use the first example if you have structured your project to live in the root of your repository and are likely not using Heroku as your host. Heroku will also fail to deploy without the --log-file – parameter shown previously. With the log parameter, you may read deployment errors from within your Heroku dashboard for each app.

Next, we need to control how the static files are managed when Django is used with Heroku.

Django static files

Traditionally, we do not need to modify the settings files to allow Django to work with static files in a project. In our case, we will need to add to the settings that exist in order for Heroku to work with them. Heroku uses a package called whitenoise, installed in the requirements.txt file, to work with your static files. These are files such as a .css, .js, image, or font files that are found in any Django app's static folder.

Follow these steps to configure your project to work with the whitenoise package:

1. Add the following line to the MIDDLEWARE setting found in your settings.py file. Add it below any item that already exists in that list:

   # /becoming_a_django_entdev/settings.py

   ...

   MIDDLEWARE = [

       ...,

       'whitenoise.middleware.WhiteNoiseMiddleware',

   ]

2. Heroku will also need us to add a variable called STATICFILES_STORAGE. Add this variable to your settings.py file just above your STATIC_URL and STATIC_ROOT variables, as shown:

   # /becoming_a_django_entdev/settings.py

   ...

   STATICFILES_STORAGE = 'whitenoise.storage.CompressedManifestStaticFilesStorage'

   STATIC_URL = '/staticfiles/'

   STATIC_ROOT = posixpath.join(

       *(BASE_DIR.split(os.path.sep) + ['staticfiles'])

   )

3. Change the value of your STATIC_URL and STATIC_ROOT variables to equal what is shown in the preceding code, if your value has not already changed.

Our static files should now be wired up properly. At this point in time, we have configured everything locally in order to successfully deploy to any of our remote Heroku environments.

Note

In Django, when a project is run locally, with DEBUG = False, static files will fail to load even if they appear in your remote environments. The reason why this happens is that the server is usually configured to handle how it serves up your static files versus Django controlling that, similar to how Heroku uses the whitenoise package to serve up its static files. The whitenoise package is also used to serve up static files locally when DEBUG is set to False.

Let's wire up our media files next.

Django media files

Similar to the static files, media file paths need to be configured in order to work with the whitenoise package and Heroku. Media files are considered anything that the user uploads to your system, such as an image, audio file, or another document. The two variables, MEDIA_URL and MEDIA_ROOT, are defined in Django by default as empty strings; we just need to set their values to point to the media folder that we want to place them in. In addition to these settings, additional steps may also be necessary to work with Heroku and media files. Please refer to the detailed Python guide found here to learn more: https://devcenter.heroku.com/articles/s3-upload-python.

To define your media-related variables, in your settings.py file, just below your STATIC_URL and STATIC_ROOT variables, include the following two variables:

# /becoming_a_django_entdev/settings.py

…

MEDIA_URL = '/media/'

MEDIA_ROOT = posixpath.join(

    *(BASE_DIR.split(os.path.sep) ['media'])

)

That's it, this is all that we need to configure in our settings.py file to wire up static and media files. In Chapter 4, URLs, Views, and Templates, in the subsection titled Function – static(), we will need to configure additional URL patterns before these are considered fully integrated into our project.

Before we attempt to deploy, we need to create the .gitignore file, which was mentioned earlier, as well as working through the Preparing PostgreSQL for Heroku section of this chapter.

Creating a .gitignore file

The last file that we want to create before we deploy is a .gitignore file so that we can share only what we want and leave out things like all of those bulky files that are now seen in the virtual_env and .vs folders. This file will be used to ensure we don't accidentally push any unwanted code to our remote repositories. Once a file has been pushed to a repository, it will always be tracked until it has been deleted; so, we want to make sure we do this first. It's a good idea to also ignore the .suo files created by Visual Studio. These files contain user-specific information, such as breakpoint and debugger watches. You will also want to ignore any build, bin, and log files that do not need to be shared with other developers. This is also where we will define a pattern to ignore the .env file, which defines the environment-specific variables. We will be creating the .env files in development, staging, and production soon when we get to the PostgreSQL subsection of this chapter, which will be needed to successfully deploy.

Create a file called .gitignore in the root of your repository, where your .git folder resides. Then, add the items shown here. They will serve as your ignore patterns:

# ./.gitignore in Root of Repository

# Keep Rule

!gitkeep.txt

# Django #

db.sqlite3

*.log

*.pyc

__pycache__

# Media – User Generated Content #

media/

# Environments #

.env

virtual_env/

# Visual Studio and Visual Studio Code #

*.suo

*.pyproj

*.pyproj.user

*.history

.vs/

obj/

# Heroku

staticfiles/

These are just a few examples that relate to our specific Heroku/Django and Visual Studio configuration. Your .gitignore file can contain many more ignore patterns. In the source code provided with this book, many other ignore patterns have been provided as examples, broken down into categories that you are welcome to use if you need to.

Ignore patterns accept wildcards such as the asterisk (*) that is used in the preceding examples. They also accept an exclude operator denoted by the exclamation mark (!) symbol. The!gitkeep.txt pattern is commonly used in situations where an empty folder is needed to live in a repository, such as the media folder. Git will automatically ignore a folder if nothing exists inside of it; if we place a gitkeep.txt file inside that media folder, we can get around this dilemma when ignoring everything within that folder. The media folder is used for user-generated content but we do not want to track files that get placed inside of it. Some of these empty folders are actually needed to prevent errors during runtime, such as when uploading an image to the system and the media folder does not exist yet. Sometimes, this can prevent a fresh clone of a repository from running properly for the first time altogether, depending on the existence of that folder.

You may now push your code to development, staging, or production as you see fit. Just be aware of what remote environment and branch you are pushing to or pulling from, when you perform your Git operations. When this is done, you can visit your Heroku dashboard as was discussed in Chapter 1, Undertaking a Colossal Project, to see whether a deployment was successful, and if not, read the logs to see why it failed.

Creating a Django app

In Django, the entire set of files from the folder where your manage.py file is located is considered your project. Your project can contain many apps, where an app in Django is considered a process that does something within your project, such as logging, trivia, or record keeping, to name a few processes. They can also be something simple, such as a special form field or an event listener that comes from a PyPI package, where that particular package is essentially considered a Django app that we are installing.

Apps are where we write models, views, test cases, forms, admin classes, HTML templates, and static files pertinent to that app. Apps are also where the bulk of the code in a project will exist. They are also designed to be modular where an app can be shared in many projects if we want to. Next, we will create an app for ourselves and call it chapter_2. Every upcoming chapter will follow the same app naming convention. We will attempt to organize apps by the content of each chapter in this book. Some chapters may involve every file of the project, which is the case for this chapter of this book. This is because, in this chapter, we are working with the global files of a project. Choose the route you would like to use to create your app, whether that is using the IDE or the command-line-driven approach.

Using the IDE

To create an app easily, right-click the name of your project in the Solution Explorer, called becoming_a_django_entdev, and then click Add | Django App…. When prompted to do so, enter the name chapter_2, as shown here:

Figure 2.8 – Visual Studio – adding app

In Visual Studio, you can select any folder or subfolder within your project to create a Django app in. Just right-click the folder you want to create that app in instead of the one described previously if you wish to structure your project in a different way.

Note

Visual Studio will install apps using the version of Django that was installed or updated in the virtual environment. There is no special version 2.1.2 use case like before when we created a project through the IDE; the version of the app files to be installed will be Django 4.0. Also note that you can create an app using the IDE in any directory found in your project root. You are not limited to installing them in the directory used in this exercise.

Let's create a Django app using the command line next.

Using the command line

To create a Django app using the command line, you first need to create the folder structure for that new app. Here, we want it to mimic exactly how the IDE created its folder structure in the previous exercise, to make sure each approach produces the same results and works with the rest of the book. From the same folder that the manage.py file lives in, make sure your virtual environment is activated and run the following create folder command. Then, execute the traditional startapp command shown after, to create the app called chapter_2:

(virtual_env) PS > mkdir becoming_a_django_entdev/chapter_2

(virtual_env) PS > python manage.py startapp chapter_2 becoming_a_django_entdev/chapter_2

In the preceding examples, we first created the chapter_2 folder with the mkdir command and then executed the startapp command. We also provided a parameter that designates the folder that the app will be installed into, the same folder that Visual Studio placed it in for us. Again, you are not limited to installing apps in this directory— adjust where needed throughout this book if you created your folder structure in a different way.

Activating a new Django app

Once a Django app has been created, it will not automatically work in your project until you actually include it as an installed application within Django.

To activate your app, follow these steps:

1. All Django apps must be included in the INSTALLED_APPS list. Add your chapter app, as shown:

   # /becoming_a_django_entdev/settings.py

   ...

   INSTALLED_APPS = [

       ...

   'django_extensions',

     'becoming_a_django_entdev.chapter_2',

   ]

   DEFAULT_AUTO_FIELD'= 'django.db.models.AutoField'

2. Sometimes, it is necessary to tell Django where to look for your app in your directory tree. In the apps.py file of the app you just created, you can specify the location of your app using standard Python path syntax. Modify the value of the name = variable in the following example:

   # /becoming_a_django_entdev/chapter_2/apps.py

   from django.apps import AppConfig

   class chapter_2Config(AppConfig):

       name= 'becoming_a_django_entdev.chapter_2'

Note that you will have to do this for all future chapter apps that you create.

The project's file structure should now look like the following tree, with the highlighted items being the additions that we have made since we began configuring the virtual environment:

├── .git

├── .gitignore

├── procfile

├── requirements.txt

├── readme.md

├── becoming_a_django_entdev

│ ├── .vs

│ ├── becoming_a_django_entdev.sln

│ ├── db.sqlite3

│ ├── manage.py

│ ├── media

│ ├── obj

│ ├── requirements.txt

│ ├── virtual_env

│ ├── staticfiles

│ └── becoming_a_django_entdev

│ ├── chapter_2

│ ├── __init__.py

│ ├── asgi.py

│ ├── settings.py

│ ├── urls.py

│ └── wsgi.py

Next, let's discuss the different types of databases supported by Django.

Using basic database settings

A website by itself is far from useful without a database to talk to; this is why the Visual Studio IDE comes with a lightweight and portable SQLite3 database. Visual Studio will create a file called db.sqlite3 in the same folder as your manage.py file whenever the startproject command is executed. If you created your project using the terminal or command-line window, then you will not have a SQLite database that is used in the following configuration examples, and if you attempt to run your project without this database, it will fail. This is one of the five standard database types that Django directly supports. Database types other than the five types that Django directly supports can also be used. We will also provide an example of how to configure other types of databases, such as the Microsoft SQL Server database. Types that are not any of the five standard types will require using a different engine than the engines that Django provides, which means you will need to find a PyPI package to install or write your own for the database type that you are supporting.

The five standard Django database types are as follows:

• SQLite
• MySQL
• MariaDB
• Oracle
• PostgreSQL

The following examples will help you to configure an existing database of your choice. The last example, PostgreSQL, will be the type used when moving forward with the examples of this chapter. You may use any type for the remaining chapters of this book. There is more to using a database, such as creating tables and performing queries, that will be discussed in the chapters to come. These examples relate to settings written just to establish a working connection to your database, whether that connection is local or remote.

SQLite

SQLite is a Relational Database Management System (RDBMS) based on the C language. It is extremely lightweight and portable and is sometimes referred to as the on-disk method of choice when it comes to database management. This is the method of choice to get a proof-of-concept project up and running in a very short amount of time. This database type can even be shared on drives and repositories for quick transportation and portability of the project and its data. This is not recommended if security is as important to you as it should be.

There are a lot of problems with using this database type in real-world applications and in combination with Heroku like we are doing to host the app. You may find that you can get it to work in a remote Heroku environment momentarily. However, each deployment of your app throughout its life cycle will result in the complete loss of data. For this reason, we will have to deviate from the standard database that comes with Django and rely on a more robust database system for our environment instead.

A standard SQLite3 database configuration will look like the following example:

# /becoming_a_django_entdev/settings.py

...

DATABASES = {

    'default': {

        'ENGINE': 'django.db.backends.sqlite3',

        'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),

    }

}

Add additional default parameters as needed.

MySQL

MySQL is a more robust SQL RDBMS. It is open source and, just like SQLite, it is compatible with Windows, Mac, and Linux systems. It is made to work as a client-server model, meaning the software is installed on the client's machine in order to perform request operations that the server side will listen to and respond to. MySQL has become one of the most popular and most used database types of all time, especially if you consider all the other database choices that were built using a fork of MySQL.

Follow these steps to configure your MySQL database connection:

1. A standard connection to a MySQL database would look like the following example:

   # /becoming_a_django_entdev/settings.py

   ...

   DATABASES = {

       'default': {

           'ENGINE': 'django.db.backends.mysql',

           'NAME': 'database_name',

           'USER': 'database_user',

           'PASSWORD': 'database_password',

           'HOST': 'localhost',

           'PORT': '3306',

       }

   }

2. In order for this database type to work, you also need to install the following package and include it in your requirements.txt file:

   # requirements.txt

   ...

   Mysqlclient

Adjust your default parameters as needed for your project.

MariaDB

MariaDB is actually a fork of MySQL; it is a community-developed version that evolved over time into what it is today. Technical support is available for both the MariaDB and MySQL database types. With both being open source, there is also a lot of information and resources available online for free. MariaDB is not used as much as MySQL but it is still very popular. Since MariaDB is essentially just another MySQL installation, the ENGINE in the DATABASES configuration of your settings.py file will be the same as the example in the previous subsection, titled MySQL. Both will also require that you install the same mysqlclient package and include it in your requirements.txt file.

Oracle

Oracle Database is a multimodel, object-relational database management system created by Oracle. It's primarily used for Online Transaction Processing (OLTP) and data warehousing. This database is a bit more complicated in terms of its use and capabilities compared to other web development databases. It is intended for enterprise grid computing, which is a method of grouping a series of network computers to act as one larger virtual super-computer. Oracle has developed a database specifically to be used in network cluster configurations such as these. It's considered one of the most complex and robust database systems that exist today and is beyond the scope of this book. Much of the code in this book can still be used with this database type but some alterations may be necessary.

Follow these steps to configure your Oracle database connection:

1. A typical Oracle database connection can be established using the following example:

   # /becoming_a_django_entdev/settings.py

   ...

   DATABASES = {

       'default': {

           'ENGINE': 'django.db.backends.oracle',

           'NAME': 'xe',

           'USER': 'database_user',

           'PASSWORD': 'database_password,

           'HOST': 'database_host',

           'PORT': 'database_port_#',

       }

   }

2. The Oracle database type also requires installing the following package and including it in the requirements.txt file:

   # requirements.txt

   ...

   cx_Oracle

Oracle also requires following additional setup and configuration steps and they provide directions on their website here: https://cx-oracle.readthedocs.io/en/latest/user_guide/installation.html.

SQL Server

This is the one database type that is not one of the five standard types supported by Django that we will provide an example of. Developed by Microsoft, the Microsoft SQL Server database type is also a fork of MySQL and is widely used in Microsoft enterprise database systems today. Many systems today rely on this database type, which is often used in combination with Azure-hosted apps.

Follow these steps to configure your Microsoft SQL Server database connection:

1. A Microsoft SQL Server database type can be established using the following connection settings:

   # /becoming_a_django_entdev/settings.py

   ...

   DATABASES = {

       'default': {

           'ENGINE': 'sql_server.pyodbc',

           'NAME': 'database_name',

           'USER': 'database_user',

           'PASSWORD': 'database_password',

           'HOST': 'database_host',

           'PORT': '1433',

           'OPTIONS': {

               'driver': 'ODBC Driver 17 for SQL Server',

           },

       }

   }

2. This database type also requires installing the following packages and including them in your requirements.txt file. Depending on your setup, you may only need one or two of the following three packages; however, adding all three will not hurt either:

   # requirements.txt

   ...

   pyodbc

   django-pyodbc

   django-pyodbc-azure

Next, we will be installing the type used throughout the rest of this book, PostgreSQL.

PostgreSQL

PostgreSQL has become the go-to database of choice for most Django developers when there is no specific reason to use any of the other choices. It is considered an RDBMS, which is a common tool for storing object-oriented data in tables that relate to other tables, known as objects. Its primary features includes ANSI SQL Compliance and extensibility, meaning it is a database type built with the idea of people being able to build onto it. It will run on every major operating system, such as Windows, Mac, and Linux, making it versatile.

Standard PostgreSQL settings.py

If you are working with a normal Django project outside of Heroku, configure your PostgreSQL database connection.

Add the following settings to your settings.py file:

# /becoming_a_django_entdev/settings.py

…

DATABASES = {

 '  'defa'lt': {

     '  'ENG'NE': 'django.db.backends.postgre'ql',

     '  'N'ME': 'database_n'me',

     '  'H'ST': 'localh'st',

     '  'U'ER': 'database_u'er',

     '  'PASSW'RD': 'database_passw'rd',

     '  'P'RT': '5'32',

    }

}

Since we are working with Heroku, we cannot use the preceding settings even though we are still using PostgreSQL for each remote Heroku environment. The next section will provide us with the settings and tools specific to a Heroku environment, as well as providing the rest of the instructions on how to use PostgreSQL locally.

Preparing PostgreSQL for Heroku

This section is dedicated to actually configuring each of your environments, whether using an IDE or command-line window or terminal. To install and use PostgreSQL, we will need to install it locally and then again in each of our remote environments.

Installing PostgreSQL

This section will guide you through installing the PostgreSQL software needed on your local and remote machines.

Local installation

To use PostgreSQL, we will need to install a suite of software and drivers on each of our machines. In the Heroku dashboards, we will need to include add-ons to get PostgreSQL installed in those environments. To install it on your development machine, you can choose to download the installer for your platform directly from the publisher's website, found here: https://www.postgresql.org/download/. During installation, make note of the port that you set, the password that you create, and whether it asks you to add anything to your environment PATH variables, do so! It is always easier to check/enable these options during installation versus manually configuring them later.

During installation, the installer will ask whether you want to install any development tools, such as database command-line tools or a database management tool. Select PgAdmin during the installation of your PostgreSQL drivers; we will be using this tool to demonstrate several examples in this chapter. The PgAdmin management tool is used to access and view the database table structures and the data within them. PgAdmin is also compatible with every major operating system, and it can be downloaded and installed separately from the PostgreSQL installation; it can be found on their website here: https://www.pgadmin.org/download/.

In the .env file found in the local repository that we created earlier, replace DATABASE_URL=sqlite:///db.sqlite3 with the following value:

# .env

DATABASE_URL=postgres://postgres:your_password@localhost:5432/local_postgresql

Replace the your_password placeholder with the password that you entered during your installation of PostgreSQL and PgAdmin. Replace any other placeholders as needed. The username is usually postgres by default, but this can be changed to something else during installation. We will need to work with a database management tool, PgAdmin, to create a local database before we can actually run the project now. Even if you used the Visual Studio IDE earlier to create your project, it will now be broken.

There is a file called example.env provided with the code of this book that you can use as a reference to make sure your file consists of the proper settings for our software stack.

Remote installation – Heroku dashboard

In Heroku, we have to add PostgreSQL one at a time to all three of our remote environments. There are two ways to do this: through the Heroku dashboard or the Heroku CLI. We will demonstrate both ways. Navigate to the Heroku dashboard, found here: https://dashboard.heroku.com/apps. We should see all three environments, also known as Heroku apps. Click on each app and navigate to the Resources tab, scroll down that page, and under Add-ons, type Heroku Postgres into the search field provided. Next, it will ask you what plan to choose; you can use the free plan called Hobby-Dev, which is what will be used for the demonstrations in this book. Later, you can decide whether upgrading and paying for a new plan is what you need. For more technical needs, these plans include support for increased traffic, the number of connections at one time, the amount of RAM, encryption measures, added security, and so much more. Visit their plans page found here to learn more about what they offer: https://elements.heroku.com/addons/heroku-postgresql.

If successful, you will see that Heroku automatically created a DATABASE_URL configuration variable for us. This variable can be found in your Heroku dashboard by going to the Settings tab, then scrolling down on the page, under Config Vars, and it can be found after clicking Reveal Config Vars. The following screenshot also shows us that the same SECRET_KEY variable that we created using the CLI earlier in this chapter can also be found here. Any other sensitive environment variables can be created the same way and found here. Clicking the Add button eliminates the need to use the CLI to create an environment variable operation:

Figure 2.9 – Heroku – DATABASE_URL

Let's walk through installing PostgreSQL with the Heroku CLI instead of the Heroku dashboard next.

Remote installation – Heroku CLI

Using the Heroku CLI, we first need to log into our account in the terminal or command-line window. The following examples show how to install PostgreSQL on each of our remote environments, starting with production:

(virtual_env) PS > heroku login

(virtual_env) PS > heroku addons:create heroku-postgresql:hobby-dev

(virtual_env) PS > heroku addons:create heroku-postgresql:hobby-dev --app mighty-sea-09431

(virtual_env) PS > heroku addons:create heroku-postgresql:hobby-dev --app pure-atoll-19670

We have to specify the randomly generated app name that Heroku gave us when we created these environments. If we need to specify the version of PostgreSQL that we want to use, we can do that with the following example. Without specifying a version number, the latest version available will be used:

(virtual_env) PS > heroku addons:create heroku-postgresql:hobby-dev --version=10

Instead of going to your Heroku dashboard to view these variables, executing the following commands will also display a list of the variables that exist for each environment. Replace the app name for each of the following examples with your Heroku app name:

(virtual_env) PS > heroku config --app becoming_an_entdev

(virtual_env) PS > heroku config --app mighty-sea-09431

(virtual_env) PS > heroku config --app pure-atoll-19670

Using the PgAdmin tool

PgAdmin is the database management tool that we are going to use to manage all local and remote databases, as well as creating the local database that we need to connect to. This tool should have been installed earlier in the Installing PostgreSQL subsection. If it has not been installed, please revisit that subsection for more information.

Creating a local database

Within the Browser tab, you should see an existing server connection, such as PostgreSQL 13. Other version numbers may be depicted if you installed another version and it's also possible that your system's installation did not create anything in this list. If so, follow the steps to establish a remote database connection in order to create your local server. For those of you for whom this already exists, right-click where it says Databases and select Create | Database.... When the next window opens, give it a database name of your choice and ensure the encoding is set to UTF-8, and the owner/user should be set to postgres. The database name I have given to my local database is local_postgresql when referenced in code examples later.

Figure 2.10 – PgAdmin – creating a local database

Next, we will connect to a remote database.

Connecting to a remote database

For each remote database, we need to add a new server connection inside of the PgAdmin tool.

Follow these steps to connect to a remote database:

1. Start by gathering some information from your Heroku account first. In your Heroku dashboard, under each Heroku app, navigate to the Resources tab and, under Add-ons, click on the opens in a new tab icon, as depicted in the following screenshot:

Figure 2.11 – Heroku – opening a database in a new tab

2. In the new browser tab that opens, navigate to the Settings tab and make note of the information that is provided. The highlighted fields in the following screenshot are the important pieces that we will need to establish a connection inside the PgAdmin tool. Make note of them:

Figure 2.12 – Heroku – database information

3. Open PgAdmin and from the top navigation bar of this app, under the Object tab, select Create | Server. In the window that opens, under the General tab, enter any name of your choice. This is to name your server connections for local reference only. Name them so that you know which environment is which, as shown in the following screenshot:

Figure 2.13 – PgAdmin – Create - Server

4. This task only creates your server connection inside the PgAdmin tool; it is not creating the database server itself. Using the information that was found previously in Figure 2.12, under the Connection tab, fill in the corresponding fields, as highlighted in the following screenshot:

Figure 2.14 – PgAdmin – creating a server connection

5. Replace the value in the field labeled Maintenance database with the database name provided by Heroku; do the same for Username.
6. If successful, you will now see your new server connection in the Browser panel of the PgAdmin app. The Browser panel is often found on the left-hand side of the PgAdmin program. Since we are using the free plan called Hobby-Dev, we will see dozens and dozens of other databases in this list. All but one will be grayed out, just like in the following screenshot:

Figure 2.15 – PgAdmin – viewing tables

The one database that is colored is your database name, displayed in Figure 2.12. Expand your database and you should be able to gain access; attempting to open any other database will result in permission denied. You can avoid a shared plan like this by purchasing a dedicated database hosting plan that Heroku does offer, which is much more secure.

Now, push your file changes to your Git development repository and you should see a successful deployment to your Heroku app.

Next, we will adjust our environment connection settings for use with PostgreSQL locally.

Environment connection settings

For our Heroku configuration, we need to define a default database using the following example, in your settings.py file, instead of using a standard PostgreSQL connection setting.

To configure PostgreSQL for use with Heroku and local use, follow these steps:

1. Change your DATABASES variable to use the code shown here:

   # /becoming_a_django_entdev/settings.py

   ...

   DATABASES = {

       'default': dj_database_url.config(

           conn_max_age=600

       )

   }

You can also increase the length of time allowed for established database connections to exist by adjusting the value of conn_max_age. The default value is 600 seconds, which equates to 10 minutes. Since the dj_database_url module will try to log into Heroku using a Secure Sockets Layer (SSL) connection, we will have issues running the project locally when connected to a remote database. To avoid those issues, we will have to add the following code to the bottom of the settings.py file.

2. Place the following code under the django_heroku.settings(locals()) line that we placed at the bottom of this file earlier; it should look like the following example:

   # /becoming_a_django_entdev/settings.py

   ...

   django_heroku.settings(locals())

   options = DATABASES['default'].get('OPTIONS', {})

   options.pop('sslmode', None)

That's it. Let's build our initial table structures locally next.

Building initial table structures

Next, we need to create the table structures that relate to the models within a project and any third-party packages that we installed. These operations can be performed using the Visual Studio IDE or through a terminal or command-line window. Choose your method from the following.

Using the IDE

In your IDE, you need to perform three actions: make migrations, migrate, and create a superuser, in that order. Within the Visual Studio IDE, go to your Solution Explorer and right-click on your project name. In the menu that pops up, under Python, select Django Make Migrations and then Django Migrate…, as shown in the following screenshot. Then, select the Django Create Superuser operation:

Figure 2.16 – Visual Studio – manage.py commands

Next, we will use the command-line-driven approach.

Commands – makemigrations and migrate

If you are not using an IDE or decide to run your project outside of the IDE, these commands are for you. Make sure you have activated your virtual environment and you are in the same directory as your manage.py file. Using the terminal or command-line window, create your table structures in your local database by using the following two examples:

(virtual_env) PS > python manage.py makemigrations

(virtual_env) PS > python manage.py migrate

Since we have not created any models yet, these two commands will only create the default User Auth and other Django management models that come as standard with all Django installations.

Command – createsuperuser

The first time we create database tables, we are required to create a superuser so that we can access the admin panel successfully. Additional users can be created from within the Django admin panel itself or by executing the command again, as shown in the following code snippet. Run this command now to create a superuser:

(virtual_env) PS > python manage.py createsuperuser

Next, when you are prompted to do so, enter the username, email address, and password that it will ask you to provide. Remember this information as you will need it to access the Django admin site, introduced in Chapter 6, Exploring the Django Admin Site.

Remote data migrations

Performing remote data migrations means running the same migration commands that we executed for a local database except on each remote database.

To run the migration commands manually for each of the remote environments that we created, we need to activate the Bash shell for each one first. Follow these steps to do that:

1. Log into your Heroku account and then execute each of the following commands, one at a time, for each environment that exists. Note that after each shell that is started, you should follow these steps before starting the next shell:

   (virtual_env) PS > heroku login

   (virtual_env) PS > heroku run bash --app becoming-an-entdev

   (virtual_env) PS > heroku run bash --app mighty-sea-09431

   (virtual_env) PS > heroku run bash --app pure-atoll-19670

2. For each environment that you activated a Bash shell for, once it has finished loading, you will see your command line now starts with the dollar sign character ($). You will also notice that no matter what directory you were in before, it will put you in the same directory that your .git folder lives in now. Navigate to where your manage.py file is located using the cd command and then run the following migration commands just like we did for a local database:

   $ cd becoming_a_django_entdev

   $ python manage.py makemigrations

   $ python manage.py migrate

3. Type exit and wait for it to escape each shell and proceed to the next environment.

Keep in mind that all these commands do is migrate your table structures; they don't actually migrate data found within those tables.

Next, we will practice using the push/pull operations provided by Heroku to migrate data and table structures for us, eliminating the need to run these commands from within the Bash shell.

Heroku database push/pull operations

Heroku has built-in functionality to push and pull data to and from the database indicated. These commands will allow us to merge data and table structures. With this, we can run the makemigrations and migrate commands locally, populate our database with sample data, and then push it to our other environments, eliminating the need to run these commands individually on each remote environment. You should be very cautious of performing these tasks as it is entirely possible to override or experience a complete loss of data when the wrong operation is executed. The good news is that whenever any of these commands are executed, Heroku will automatically generate a data backup of the current state of your database before any action is performed, allowing you to reverse any action that has been done.

Follow these steps to perform your operations:

1. To visibly see the changes that are happening, navigate to the Overview tab of your database dashboard in Heroku. Here, we can see a numeric count of the tables that exist for the remote database we are viewing. We can watch this number to see how a database changes as we execute each push or pull operation that includes adding or removing tables.
2. Using the information that we gathered from the page in Figure 2.12, execute the following command to push your database, including a parameter to indicate which environment we are pushing to:

   (virtual_env) PS > heroku login

   (virtual_env) PS > PGUSER=postgres PGPASSWORD=password heroku pg:push local_postgresql postgresql-tetrahedral-62413 --app mighty-sea-09431

Make sure you use the username and password that were specified when creating your local PostgreSQL database. Replace postgresql-tetrahedral-62413 with the contents of label number 6 in Figure 2.12. This is the database name that Heroku generated when we created our database in that environment. The database name can also be found at the top of your database dashboard. Number 7 of that same image is your app name, which is declared using the --app parameter. local_postgresql is the name I used for my local database.

You should now be able to see a total of 10 tables in the Overview tab of your database dashboard. These 10 tables are the standard Auth and Django management tables that were created when we first ran the migration commands locally.

Note for Windows users

Windows users might have difficulty executing the preceding command. Make sure the path to your bin folder is included in your Windows environment variables settings. That folder is usually found at C:Program FilesPostgreSQL##in on a Windows 10 installation, where ## is the numeric version number of PostgreSQL that is installed on your machine. This will get the heroku pg: command and the psql command to work from within your CLI. A restart of your CLI or operating system is usually recommended after making changes to your environment variables. The following commands will set variables that are used during the time that your CLI terminal window is open. Running them together on one line in Windows will usually give you errors. When these variables are set and your window remains open, you will be able to run your push or pull commands as you need them:

(virtual_env) PS > heroku login

(virtual_env) PS > set PGHOST=localhost

(virtual_env) PS > set PGUSER=postgres

(virtual_env) PS > set PGPASSWORD=password

(virtual_env) PS > set PGPORT=5432

(virtual_env) PS > heroku pg:push local_postgresql postgresql-tetrahedral-62413 --app mighty-sea-09431

If we wanted to pull data instead of push data, that command is just as simple as changing push to pull, as depicted in the following example:

(virtual_env) PS > heroku pg:pull local_postgresql postgresql-tetrahedral-62413 --app mighty-sea-09431

For a complete guide on how to use Heroku command-line operations pertaining to PostgreSQL, view their knowledge base article here: https://devcenter.heroku.com/articles/heroku-postgresql.

Summary

By now, we have done a lot of work but we still haven't really begun building any Django apps for a project yet. All of the work done up until now can be thought of as the preliminary work necessary before handing over a project to the development team. Two methods of creating a project have been provided: one method using a tool to help streamline production, called an IDE, and another method using commands in a terminal or command-line window. We are tracking a solution file in the repository so that we can share it within a team but we are not tracking personal settings and debug files that are automatically created when running a project. Developers who are not using an IDE can still work with the code base even when sharing project configuration files with those who are using an IDE. After we did that, we configured Django to work with the host provider Heroku on both the project level and the database level. Finally, we activated tools that allow developers to view and edit data inside of a local or remote database.

You can now hand this solution over to your team and begin delegating tasks to each member within that team. In the next chapter, we will start creating models that build tables inside a database. These models can be thought of as the elements that construct the rest of the chapters after Chapter 3, Models, Relations, and Inheritance, in this book.