First of all, we'll need to create some configuration templates for the different services used on the server. These will be used by both deployment procedures: staging and production.
- In your Django project, create a deployment directory and inside of it create an ansible_templates directory.
- Create a Jinja template file for time zone configuration:
{# deployment/ansible_templates/timezone.j2 #}
{{ timezone }}
- Create a Jinja template file for Nginx domain configuration before setting the SSL certificates:
{# deployment/ansible_templates/nginx-pre.j2 #}
server{
listen 80;
server_name {{ domain_name }};
location /.well-known/acme-challenge {
root /var/www/letsencrypt;
try_files $uri $uri/ =404;
}
location / {
root /var/www/letsencrypt;
}
}
- Create a Jinja template file at deployment/ansible_templates/nginx.j2 for our Nginx domain configuration, including the SSL certificates. For this file, copy the content from https://raw.githubusercontent.com/PacktPublishing/Django-3-Web-Development-Cookbook-Fourth-Edition/master/ch12/myproject_virtualenv/src/django-myproject/deployment-nginx/ansible_templates/nginx.j2.
- Create a template for the Gunicorn service configuration:
# deployment/ansible_templates/gunicorn.j2
[Unit]
Description=Gunicorn daemon for myproject website
After=network.target
[Service]
PIDFile=/run/gunicorn/pid
Type=simple
User={{ user_username }}
Group=www-data
RuntimeDirectory=gunicorn
WorkingDirectory={{ python_path }}
ExecStart={{ project_root }}/env/bin/gunicorn --pid /run/gunicorn/pid --log-file={{ project_root }}/logs/gunicorn.log --workers {{ ansible_processor_count | int }} --bind 127.0.0.1:8000 {{ project_name }}.wsgi:application --env DJANGO_SETTINGS_MODULE={{ django_settings }} --max-requests 1000
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s TERM $MAINPID
PrivateTmp=true
[Install]
WantedBy=multi-user.target
- Create a template for the PostgreSQL configuration file at deployment/ansible_templates/postgresql.j2 with content from https://github.com/postgres/postgres/blob/REL_10_STABLE/src/backend/utils/misc/postgresql.conf.sample. Later you can tweak the configuration in this file.
- Create a template for the PostgreSQL permissions configuration file (currently, it is very permissive, but you can tweak it later according to your needs):
{# deployment/ansible_templates/pg_hba.j2 #}
# TYPE DATABASE USER CIDR-ADDRESS METHOD
local all all ident
host all all ::0/0 md5
host all all 0.0.0.0/32 md5
host {{ db_name }} {{ db_user }} 127.0.0.1/32 md5
- Create a template for the Postfix email server configuration:
{# deployment/ansible_templates/postfix.j2 #}
# See /usr/share/postfix/main.cf.dist for a commented, more
# complete version
# Debian specific: Specifying a file name will cause the first
# line of that file to be used as the name. The Debian default
# is /etc/mailname.
# myorigin = /etc/mailname
smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
biff = no
# appending .domain is the MUA's job.
append_dot_mydomain = no
# Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h
readme_directory = no
# TLS parameters
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_use_tls=yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
# See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc
# package for information on enabling SSL
# in the smtp client.
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
myhostname = {{ domain_name }}
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
mydestination = $myhostname, localhost, localhost.localdomain, ,
localhost
relayhost =
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
inet_protocols = all
virtual_alias_domains = {{ domain_name }}
virtual_alias_maps = hash:/etc/postfix/virtual
- Create a template for the email forwarding configuration:
{# deployment/ansible_templates/virtual.j2 #}
# /etc/postfix/virtual
hello@{{ domain_name }} [email protected]
@{{ domain_name }} [email protected]
- Create a template for the memcached configuration:
{# deployment/ansible_templates/memcached.j2 #}
# memcached default config file
# 2003 - Jay Bonci <jaybonci@debian.org>
# This configuration file is read by the start-memcached script
# provided as part of the Debian GNU/Linux distribution.
# Run memcached as a daemon. This command is implied, and is not
# needed for the daemon to run. See the README.Debian
# that comes with this package for more information.
-d
# Log memcached's output to /var/log/memcached
logfile /var/log/memcached.log
# Be verbose
# -v
# Be even more verbose (print client commands as well)
# -vv
# Use 1/16 of server RAM for memcached
-m {{ (ansible_memtotal_mb * 0.0625) | int }}
# Default connection port is 11211
-p 11211
# Run the daemon as root. The start-memcached will default to
# running as root if no -u command is present
# in this config file
-u memcache
# Specify which IP address to listen on. The default is to
# listen on all IP addresses
# This parameter is one of the only security measures that
# memcached has, so make sure it's listening
# on a firewalled interface.
-l 127.0.0.1
# Limit the number of simultaneous incoming connections. The
# daemon default is 1024
# -c 1024
# Lock down all paged memory. Consult with the README and homepage
# before you do this
# -k
# Return error when memory is exhausted (rather than
# removing items)
# -M
# Maximize core file limit
# -r
- Finally, create a Jinja template for the secrets.json file:
{# deployment/ansible_templates/secrets.json.j2 #}
{
"DJANGO_SECRET_KEY": "{{ django_secret_key }}",
"DATABASE_ENGINE": "django.contrib.gis.db.backends.postgis",
"DATABASE_NAME": "{{ db_name }}",
"DATABASE_USER": "{{ db_user }}",
"DATABASE_PASSWORD": "{{ db_password }}",
"EMAIL_HOST": "{{ email_host }}",
"EMAIL_PORT": "{{ email_port }}",
"EMAIL_HOST_USER": "{{ email_host_user }}",
"EMAIL_HOST_PASSWORD": "{{ email_host_password }}"
}
Now let's work on the Vagrant and Ansible scripts specific to the staging environment:
- In the .gitignore file, add the following lines to ignore some Vagrant- and Ansible-specific files:
# .gitignore
# Secrets
secrets.json
secrets.yml
# Vagrant / Ansible
.vagrant
*.retry
- Create the deployment/staging and deployment/staging/ansible directories.
- In the deployment/staging/ansible directory, create a Vagrantfile file with the following script to set up a virtual machine with Ubuntu 18 and run the Ansible script in it:
# deployment/staging/ansible/Vagrantfile
VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
config.vm.box = "bento/ubuntu-18.04"
config.vm.box_version = "201912.14.0"
config.vm.box_check_update = false
config.ssh.insert_key=false
config.vm.provider "virtualbox" do |v|
v.memory = 512
v.cpus = 1
v.name = "myproject"
end
config.vm.network "private_network", ip: "192.168.50.5"
config.vm.provision "ansible" do |ansible|
ansible.limit = "all"
ansible.playbook = "setup.yml"
ansible.inventory_path = "./hosts/vagrant"
ansible.host_key_checking = false
ansible.verbose = "vv"
ansible.extra_vars = { ansible_python_interpreter:
"/usr/bin/python3" }
end
end
- Create a hosts directory with a vagrant file containing the following content:
# deployment/staging/ansible/hosts/vagrant
[servers]
192.168.50.5
- Create a vars.yml file there with the variables that will be used in the installation scripts and Jinja templates for configurations:
# deployment/staging/ansible/vars.yml
---
# a unix path-friendly name (IE, no spaces or special characters)
project_name: myproject
user_username: "{{ project_name }}"
# the base path to install to. You should not need to change this.
install_root: /home
project_root: "{{ install_root }}/{{ project_name }}"
# the python module path to your project's wsgi file
wsgi_module: myproject.wsgi
# any directories that need to be added to the PYTHONPATH.
python_path: "{{ project_root }}/src/{{ project_name }}"
# the git repository URL for the project
project_repo: [email protected]:archatas/django-myproject.git
# The value of your django project's STATIC_ROOT settings.
static_root: "{{ python_path }}/static"
media_root: "{{ python_path }}/media"
locale: en_US.UTF-8
timezone: Europe/Berlin
domain_name: myproject.192.168.50.5.xip.io
django_settings: myproject.settings.staging
letsencrypt_email: ""
- We'll also need a secrets.yml file containing secret values, such as passwords and authentication keys. First, create a sample_secrets.yml file that will have no sensitive information, but only the variable names, and then copy it to secrets.yml and fill in the secrets. The former file will be under version control whereas the latter will be ignored:
# deployment/staging/ansible/sample_secrets.yml
# Django Secret Key
django_secret_key: "change-this-to-50-characters-long-random-string"
# PostgreSQL database settings
db_name: "myproject"
db_user: "myproject"
db_password: "change-this-to-a-secret-password"
db_host: "localhost"
db_port: "5432"
# Email SMTP settings
email_host: "localhost"
email_port: "25"
email_host_user: ""
email_host_password: ""
# a private key that has access to the repository URL
ssh_github_key: ~/.ssh/id_rsa_github
- Now create an Ansible script (a playbook) at deployment/staging/ansible/setup.yml for installing all the dependencies and configuring services. Copy the content for this file from https://raw.githubusercontent.com/PacktPublishing/Django-3-Web-Development-Cookbook-Fourth-Edition/master/ch12/myproject_virtualenv/src/django-myproject/deployment-nginx/staging/ansible/setup.yml.
- Then create another Ansible script at deployment/staging/ansible/deploy.yml for dealing with the Django project. Copy the content for this file from https://raw.githubusercontent.com/PacktPublishing/Django-3-Web-Development-Cookbook-Fourth-Edition/master/ch12/myproject_virtualenv/src/django-myproject/deployment-nginx/staging/ansible/deploy.yml.
- And create a bash script that you can execute to start the deployment:
# deployment/staging/ansible/setup_on_virtualbox.sh
#!/usr/bin/env bash
echo "=== Setting up the local staging server ==="
date
cd "$(dirname "$0")"
vagrant up --provision
- Add execution permissions for the bash script and run it:
$ chmod +x setup_on_virtualbox.sh
$ ./setup_on_virtualbox.sh
- If the script fails with errors, it's likely that the virtual machine needs to be rebooted for the changes to take effect. You can do that by connecting to the virtual machine via ssh, changing to the root user, and then rebooting as follows:
$ vagrant ssh
Welcome to Ubuntu 18.04.3 LTS (GNU/Linux 4.15.0-72-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
System information as of Wed Jan 15 04:44:42 CET 2020
System load: 0.21 Processes: 126
Usage of /: 4.0% of 61.80GB Users logged in: 1
Memory usage: 35% IP address for eth0: 10.0.2.15
Swap usage: 4% IP address for eth1: 192.168.50.5
0 packages can be updated.
0 updates are security updates.
*** System restart required ***
This system is built by the Bento project by Chef Software
More information can be found at https://github.com/chef/bento
Last login: Wed Jan 15 04:43:32 2020 from 192.168.50.1
vagrant@myproject:~$ sudo su
root@myproject:/home/vagrant#
reboot
Connection to 127.0.0.1 closed by remote host.
Connection to 127.0.0.1 closed.
- To browse the Django project directories, ssh to the virtual machine and change the user to myproject as follows:
$ vagrant ssh
Welcome to Ubuntu 18.04.3 LTS (GNU/Linux 4.15.0-74-generic x86_64)
# …
vagrant@myproject:~$ sudo su - myproject
(env) myproject@myproject:~$ pwd
/home/myproject
(env) myproject@myproject:~$ ls
commands db_backups logs public_html src env