I’m hoping by now you can see Ansible is an easy way to remotely control applications and system configurations. But running each command in the command line is not the most efficient way of deploying these changes to our environments. This is where we can introduce playbooks. Ansible playbooks can contain a play or multiple plays and help you automate system configurations in a reusable set of tasks which will step through the implementation of your system into its desired state.
We start by taking a look at YAML syntax and how we use it in our playbooks.
We then move on to converting some of the command-line modules we learned in our last chapter into an Ansible playbook which will be the start of our LAMP stack project.
Our focus then moves to the specifics of our working playbooks and how they function.
We will then take our project further by creating a database and integrating Python in our environments all by using playbooks.
We finally take a look at how we can make our playbooks more reusable especially when things get more complex using the import and include functions.
From our last chapter, we mentioned that each playbook includes a set of plays, which map a group of hosts to a specific role. As you will see in the following example, a host will be mapped to a web server role. The roles are represented by a list of tasks. These specific tasks act like a recipe, using our Ansible modules to then our system to a specific state. Before we start to work more on our playbooks, we will first gain a further understanding on how YAML syntax works, so we can be more proficient in implementing our playbooks later in the chapter.
Ansible and YAML Syntax
Before we move into using playbooks in Ansible, we need to take a quick moment to discuss YAML syntax. As we discussed in the first chapter of this book, our basic Ansible playbook used YAML syntax to output a simple message to the screen. The goal of YAML is to be easily readable by humans. If you’ve used similar configuration file formats like XML or JSON, you will notice YAML is also a lot easier to read compared to these other formats.
YAML File Names End with .yml – All our playbooks will use the .yml (or) .yaml file name extension, and this lets everyone know it’s a YAML format file.
Syntax Starts with Three Dashes – The first thing you’ll notice is our plays in playbooks all start with three dashes (---). This will allow applications reading the file to know it is a YAML format file.
Whitespace Denotes Structure – Indentation and whitespace in our YAML file denote the structure of the file. If your indentation is out of place, this could mean your configuration or playbooks are not being read correctly. Make sure you never use a tab character as indentation.
A Comment Uses the Hash Symbol – You’ll most likely want to use comments through your playbook. All comments are preceded with the hash (#) symbol.
List Members Use a Leading Hyphen – When displaying a list of items, all items in the list will begin at the same indentation level and start with a hyphen (-). Here is an example list of operating systems, using two spaces and then a hyphen before each of the values:
---Operating Systems:- Ubuntu- Debian- Red Hat- Centos- Windows- MacDictionaries Are Represented As a “Key: Value” – We simply add values to a dictionary by indenting our values after the dictionary name and then adding a key-value pair on each line. Here is an example of a dictionary in YAML where we have two dictionaries, one named vince and the second named neil. Both then have three key-value pairs added in each with keys specified as full_name, job, and comment:
---vince:full_name: Vincent Sestojob: devops engineercomment: Ansible is Fun!neil:full_name: Neil Diamondjob: Singer and Songwritercomment: Good Lord!Span Multiple Lines with a Pipe – When needing to add multiple lines of data for one entry, you can use the pipe (|) symbol to add multiple lines and will include the new line in the YAML file, while using the less than (>) symbol will ignore any new lines. In the following example, we are setting environment variables to be used on a system and are using the pipe symbol to have all new line characters present:
env: |-DB_HOST= "database.host.com"DB_DATABASE= "testdb"DB_USERNAME= "dbadmin"SUDO= "root"JAVA_HOME= "/usr/lib/jvm/java-11-oracle"HOME= "/home/ubuntu"USER= "ubuntu"MAIL= "/var/mail/ubuntu"
YAML is an acronym for “YAML Ain’t Markup Language” or “Yet Another Markup Language” and is being widely used across different languages due to the ease of reading and the fact it is being widely supported by different programming languages. We have only touched on the basics of YAML, and this should be enough for you to continue with the next chapter, but for a more in-depth discussion on using YAML, feel free to look through the following Wikipedia documentation:
Command-Line Modules to Ansible Playbooks
The goal of this book is to get you working as quickly as possible, so we’ll stop the talk about syntax for the time being and get you started with a playbook which we can easily relate back to the work we did in the first chapter. You’ll remember we installed Apache2, and then we checked if the application was running.
By using the preceding modules, we can now construct a playbook which will run through these tasks. This is a great example to start with because it will take us from the modules we were running on the command line to a running playbook.
- 1.
Start by creating a new directory where we can create our playbook. Run the following command to create the directory test_playbooks and will also move into the directory:
mkdir test_playbooks; cd test_playbooks - 2.
Create a new hosts file in the directory we are now working in. Open the hosts file with your text editor and make sure there is an entry for a group [webserver] as we have in the following and feel free to add an IP address for a remote server if you are comfortable performing this installation on a remote server:
1 [mylaptop]2 localhost34 [webserver]5 127.0.0.1 - 3.
We can now start creating our playbook. Run the following command to create a new file called webserver-playbook.yml in the current directory you are working in:
touch webserver-playbook.yml - 4.
As we discussed earlier, we need to start our file with three dashes (---) to make sure our file is recognized as YAML file format. We will also set up the host it will deploy to; in this exercise, it will be the webserver:
1 ---2 - hosts: webserver
Our example is Ansible is configured to run as the root user and does not need to run the sudo command before it installs and makes changes to our system. If you are deploying your playbook to a host where you need to become the root user to deploy, you will also need to use the “become: sudo” command after the hosts entry in our playbook.
- 5.
Our playbook will next set up our first task to install Apache2 onto our system. As you can see in the following text, we define our plays using the word tasks, with all the modules we then need as part of our playbook listed below tasks. Enter the following lines into your playbook where we use the name entry in line 4 to provide a clear description of the task we are performing. The -name entry is not mandatory, and if you don’t use this as part of your task, the module name will be output instead. Line five is the same as our apt command-line module where we specify the name of the package and the state which defines the application version; in this case, we want to install apache2 and the version as latest:
3 tasks:4 - name: ensure apache is installed and up to date5 apt: name=apache2 state=latest - 6.
In a real-world example, we would want to have a preconfigured Apache2 configuration file available to install onto our new system. In this exercise, we are simply going to use the default configuration file Apache2 uses as an example. Run the following command to obtain a copy of a default Apache2 configuration file and place it in our current working directory:
- 7.
We can now use the copy module to add the configuration file to our webserver-playbook.yml file. You will also notice we have set up a notify action. This will ensure if the configuration file ever changes, Apache2 will be restarted. The value listed in line 9 specifies the name entry of the handler we will set up shortly:
6 - name: write the apache config file7 copy: src=000-default.conf dest=/etc/apache2/sites-available/000-default.conf8 notify:9 - restart apache2 - 8.
Our next task in our playbook will be to make sure the Apache2 service is running. Add the following two lines to your playbook which will use the service module to make sure Apache2 is running and enable it to start up when the system is booted:
10 - name: apache is running (and enable it at boot)11 service: name=apache2 state=started enabled=yes - 9.
In lines 8 and 9 of our playbook, we specified a notify action to make sure Apache2 is restarted if there are any changes to the configuration. For this to work correctly, we need to set up the handlers that will respond to the “restart apache” action. Enter the following final three lines that will set up a new section outside our tasks, called handlers. Here, we specify the name and the modules to be run, when “restart apache2” is called on our playbook:
- 10.
Save the playbook, and it’s time to now run the playbook on our system. From the command line, in the directory where you created the webserver-playbook.yml file, run the following ansible-playbook command:
ansible-playbook -i hosts webserver-playbook.ymlPLAY [webserver] ***************************************************TASK [Gathering Facts] *********************************************ok: [localhost]TASK [ensure apache is installed and up to date] *******************changed: [localhost]TASK [write the apache config file] ********************************changed: [localhost]TASK [apache is running and enable it at boot] *********************ok: [localhost]PLAY RECAP *********************************************************localhost: ok=5 changed=0 unreachable=0 failed=0skipped=0 rescued=0 ignored=0
The Apache2 Ubuntu Default Page Should Be Visible from Your Browser
Ansible Playbook Verbose Output
One -v argument adds the default debug output, while adding more values like -vv or -vvv or -vvvv will increase the amount of data being output and the detail of what is being run by Ansible. By expanding the output, we get to see the specific commands being run by the modules, and as you’ll see later on, it will provide you with a good basis to troubleshoot problems with your playbooks.
As you can see, the more v arguments we add to our command, the more data we get. When debugging a playbook, Ansible users will limit their output to three values (-vvv) as this generally gives enough information while not overwhelming the user with too much data.
Ansible Syntax in Finer Detail
Before we move on to our next exercise, we should go over some of the finer points in using and creating Ansible playbooks and recap what we have done in the previous section of this chapter.
Hosts and Remote Users
All is not the only value we can list for -hosts. We can also list the asterisk (*) for all; we can also specify the hostname as the host. We can list multiple hosts, separated with a colon, like localhost:127.0.0.1, or multiple groups in the same manner, such as webserver:dbserver. We can also exclude a group of hosts with an exclamation mark, such as webserver:!dbserver.
We could also set the order as inventory, which is the default order in our inventory file, and reverse_inventory, which is the reverse of the inventory order. Sorted will deploy to our hosts in alphabetical order and reverse_sorted in reverse alphabetical order, and shuffle will deploy in a random order to our hosts.
We set up our hosts in Chapter 1 to make sure we can both ssh and run commands as root. If you are wanting to run the playbook and have the playbook ask for a password, you can also run it with the --ask-become-pass (or) -k command-line argument as we did in Chapter 1.
Tasks
Always make sure your whitespace is consistent as this may cause problems when your playbook is run.
Notify
We used notify in our playbook earlier in this chapter to allow our playbook to respond to changes which may be occurring in our environment. Specifically, we wanted to tell Apache2 to restart if the configuration file changes. We can set up a notify action at the end of a block of tasks that will only be triggered once. We could have numerous changes or tasks that could trigger a change, but Ansible will make sure to only perform this once.
Now that we have clarified some of the finer points of our playbook, we can move on further with our project and create a new playbook that creates a database for our LAMP stack.
Adding More Playbook Functionality with Variables and Loops
We have a nice web server up and running, but in the following exercise, we are going to add some more functionality into our playbook. As we discussed in the first chapter, the project we are currently setting up is a web server environment. We have gone about halfway so far by setting up our Apache2 server, so we can now continue to add further functionality by adding a database.
It’s time for us to add this into our playbook, and we will cover off some more functionality of Ansible playbooks as we go. In the following exercise as part of creating the database server, we will define variables to be used within the playbook, and we will create loops using the with_items function to iterate over multiple values. We will introduce some new modules specific to MySQL, specifically mysql_user to create our root user and mysql_db to create databases on our server.
- 1.
Make sure you are in the test_playbook directory we created earlier and the web server we implemented earlier in this chapter is also running.
- 2.We need to make a decision first on where the database will live. Will it be on the same server as the web server or on a different host? In a production server environment, they should be separated, but for ease, we will install it on the same host our web server resides on. To do this, we will make another entry in our hosts file, so open the file with your text editor and add in an entry for our database server. The hosts file should now look similar to the following one, but if you are wanting to add the database to the same server, feel free to add another entry with the localhost IP address:1 [mylaptop]2 localhost34 [webserver]5 127.0.0.167 [mysql]8 <database_server_ip_address>
- 3.
Create a new playbook file called dbserver-playbook.yml in your working directory. Use the following command to create the file from the command line:
touch dbserver-playbook.yml - 4.
Open the new playbook with your text editor, and we can start to fill in the details for our new playbook. We can start with the first two lines of the playbook, adding the name of the host we are wanting to deploy our configuration management to; in this instance, it will be the mysql entry of our hosts file:
1 ---2 - hosts: mysql - 5.We will now do something new. We are going to add in a variable into our playbook. Just as we created a section called tasks to list all our tasks, we do the same thing but call this section vars. Add the following code to create a variable named mysql_root_password and assign the value of password to it:34 vars:5 mysql_root_password: password
Note In our playbook code earlier, we are creating a password for our database. This is not the best way to store passwords as it stores them in plain text. Ansible provides a way for us to store our passwords in a safe manner, and we will have devoted a large section in a following chapter to storing passwords in this manner.
- 6.
We can now start listing the tasks to create our database server. Our first task installs our database and supporting applications. To do this, we create our first loop in our playbook:
6 tasks:7 - name: install mysql and python-myslqdb8 apt: name={{ item }} update_cache=yes cache_valid_time=3600 state=present9 with_items:10 - python3-mysqldb11 - mysql-serverAs usual, we start the task with a descriptive name. As you will see in line 8 though, we use the apt module and provide {{ item }} as the name argument. This then refers to line 9 that uses the with_items, providing a list of all the applications needed to be installed as part of this task. The apt module will loop through and install all of the applications in the with_items list.
- 7.Next, add the following code to make sure MySQL is running. We use the shell module to run the “service mysql start” command and then ensure the service remains running on startup:12 - name: start up the mysql service13 shell: "service mysql start"14 - name: ensure mysql is enabled to run on startup15 service: name=mysql state=started enabled=true
- 8.Our final task will now update the mysql root passwords and grant privileges using the mysql_user module. For ease of reading, we have added all of the arguments for the module in a list below the mysql_user module from lines 16 to 22, which sets up all of the configuration items for the database user:16 - name: update mysql root password for all root accounts17 mysql_user:18 name: root19 host: "{{ item }}"20 password: "{{ mysql_root_password }}"21 login_user: root22 login_password: "{{ mysql_root_password }}"23 check_implicit_admin: yes24 priv: "*.*:ALL,GRANT"25 with_items:26 - "{{ ansible_hostname }}"27 - 127.0.0.128 - ::129 - localhost
We have set up a large task here which is creating the root mysql user on our database host using our password specified earlier in the playbook as a variable. It loops through the possible hostnames for our local environment to perform this change on.
- 9.
We can now use the mysql_db Ansible module to create a new test database in our new database server installation. Add the following code to create the database named testdb and assign it as the root user to this database:
30 - name: create a new database31 mysql_db: name=testdb state=present login_user=root login_password="{{ mysql_root_password }}" - 10.
If you’ve worked with databases before, you might know you can create sql script that will allow you to provision the tables needed and import specific data, instead of having to perform the changes manually. Our playbook can do this by first using the copy module to copy the sql file into a directory on the host in line 31 and then using the mysql_db module again to import the sql file:
32 - name: add sample data to database33 copy: src=dump.sql dest=/tmp/dump.sql34 - name: insert sample data into database35 mysql_db: name=testdb state=import target=/tmp/dump.sql login_user=root login_password="{{ mysql_root_password }}" - 11.
Save the playbook, but before we run the playbook to deploy the new database server, we need to create the sql script that is run by the preceding code. Create the file named dump.sql in your current working directory:
touch dump.sql - 12.
Open the dump.sql file with your text editor and add the following lines of code into the file. The following code will create a table named test and then add in some default data to the table we created:
1 CREATE TABLE IF NOT EXISTS test (2 message varchar(255) NOT NULL3 ) ENGINE=MyISAM DEFAULT CHARSET=utf8;4 INSERT INTO test(message) VALUES('Ansible To Do List');5 INSERT INTO test(message) VALUES('Get ready');6 INSERT INTO test(message) VALUES('Ansible is fun') - 13.
If all our tasks have been set up correctly, we can now run our playbook. Run the following ansible-playbook command to see if your hard work has paid off:
ansible-playbook -i hosts dbserver-playbook.yml
In a matter of minutes, we've been able to demonstrate how you can use a playbook to install and run the appropriate applications to create a database server on our system. We then configured the application to set up users and databases and should now have a full functioning and accessible database that has data preloaded into a preconfigured table.
Plugging In Our LAMP Stack
So far, we have our Linux, Apache2, and MySQL all set up for our LAMP stack. The last part to set up is of course the “P” which stands for either PHP or Python. We’ve chosen Python in this example, and we just need to make some small changes to our Apache2 web server to ensure it’ll play nicely with Python scripts.
YAML File Names End with .yml – All our playbooks will use the .yml file name extension, and this lets everyone know it’s a YAML format file.
We need to update the 000-default.conf file. Using Ansible, we will be able to use the file module to install it onto our environment.
We will need to make some further changes to the Apache2 configurations in the webservice-playbopok.yml to ensure it recognizes Python and can run Python scripts. The apache2_module module will allow us to do that.
We will need to use the pip3 Ansible module to install the pymysql Python module.
Finally, we need to create and install the index.py file, which in this instance will connect to our database and extract some data to then display on a simple web page.
- 1.
You should be back in your test_playbooks directory, and for the following changes to the web server to work, you will need to make sure the database from the previous exercises is working and accessible by the web server.
- 2.
From our working directory, we will first need to make some minor changes to the 000-default.conf configuration file in our Apache2 installation.
- 3.Open the 000-default.conf file with your text editor. After the first line that will specify the VirtualHost for port 80, add in the following five lines of configuration into your file. In the following code, we have also included the first line specifying the VirtualHost which will be left unchanged:1 <VirtualHost *:80>23 <Directory /var/www/test>4 Options +ExecCGI5 DirectoryIndex index.py6 </Directory>7 AddHandler cgi-script .py
The configuration is setting up a new directory called /var/www/test which will run our Python code.
- 4.
Move further down in the 000-default.conf file and you will see a DocumentRoot entry; you will need to amend it to the new directory we have specified at the start of the configuration:
19 DocumentRoot /var/www/testWith the current version of Apache2, this should be around line 19 but may differ for your configuration file.
Our web server playbook already uses the 000-default.conf file as part of the installation, so we won’t need to add it to the playbook.
- 5.
We now need to make some further changes to Apache2 to get it to work nicely with Python, so open the webservice-playbook.yml file with your text editor to add some more functionality to it.
- 6.The following addition we will make will turn off multiprocessor mode and activate cgi. Make some space for the extra configuration after the last set of tasks at around line 12 and before the handler section of the playbook and add the following output:12 - name: disable pmp_event on apache13 shell: "a2dismod mpm_event"14 notify:15 - restart apache216 - name: enable cgi on apache17 shell: "a2enmod mpm_prefork cgi"18 notify:19 - restart apache2
Notice both tasks also use the notify handler to let Apache2 know it requires a restart after making these changes.
Note At the time of writing this book, there were some issues with the apache2_module. This module would usually give you enough functionality to make the preceding changes, but due to issues with the module not working correctly, we decided to use the shell commands instead.
- 7.We will add in three more tasks into our playbook before we have our handler section. Add in the following code which will install the pymysql Python module using the pip3 Ansible module at line 20. We will then create the DocumentRoot directory of /var/www/test in line 22, and the last task will add the new index script into this directory in line 24. Make note of the mode we need to set for the index.py file as it needs to be executable:20 - name: install pymysql module for index to use21 pip: name=pymysql executable=pip322 - name: add in a test directory23 file: path=/var/www/test/ state=directory24 - name: add in your index file25 copy: src=index.py dest=/var/www/test/index.py mode=75526 notify:27 - restart apache2
- 8.Your handler section should still be in place after the tasks you have included as part of this exercise. Your webserver-playbook.yml file should look like the following code, with all tasks running from lines blah to blah, with the handler section finishing up the playbook:1 ---2 - hosts: webserver3 tasks:4 - name: ensure apache is installed and up to date5 apt: name=apache2 state=latest6 - name: write the apache config file7 copy: src=000-default.conf dest=/etc/apache2/sites-available/000-default.conf8 notify:9 - restart apache210 - name: apache is running and enable it at boot11 service: name=apache2 state=started enabled=yes12 - name: disable pmp_event on apache13 shell: "a2dismod mpm_event"14 notify:15 - restart apache216 - name: enable cgi on apache17 shell: "a2enmod mpm_prefork cgi"18 notify:19 - restart apache220 - name: install pymysql module for index to use21 pip: name=pymysql executable=pip322 - name: add in a test directory23 file: path=/var/www/test/ state=directory24 - name: add in your index file25 copy: src=index.py dest=/var/www/test/index.py mode=75526 notify:27 - restart apache228 handlers:29 - name: restart apache230 service: name=apache2 state=restarted31
- 9.
We now need to create the index.py file that we are now referring to in our playbook, so create a file called index.py in your current working directory where your playbooks are located:
touch index.py - 10.Open the new index.py file with your text editor and add the following code. We are not going to go too much in depth as to what this script does, but to quickly explain, we import our Python modules, connect to the database, and then print out all of the data in the new tables we created:1 #!/usr/bin/python323 import pymysql45 # Print necessary headers.6 print("Content-Type: text/html")7 print()89 # Connect to the database.10 conn = pymysql.connect(11 db='testdb',12 user='root',13 passwd='password',14 host='localhost')15 c = conn.cursor()1617 # Print the contents of the table.18 c.execute("SELECT * FROM test;")19 for i in c:20 print(i)
- 11.Save the changes you have made to the the index.py file, and just before we run out playbook again, let’s make sure we have all our files. In our current working directory, we should have the following list of files:
- a.
000-default.conf – The Apache2 configuration file to be deployed and now allowing the ability for the application to interact with Python and the database we have created
- b.
dbserver-playbook.yml – The playbook that manages configurations of our database server
- c.
dump.sql – The SQL file that creates our default database and table and adds in additional data
- d.
hosts – The lists hosts Ansible is using to deploy to
- e.
index.py – Our web server index file to display content from the database
- f.
webserver-playbook.yml – The Ansible playbook that manages configuration of our web server
- a.
- 12.
Our webserver-playbook.yml is ready to be deployed. We have set this up to run and install our web server from a new installation, or it will update an established installation. Run the following ansible-playbook command to hopefully have all our configuration complete:
ansible-playbook -i hosts webserver-playbook.yml
The New Web Page Visible from Your Browser
It is really not very interesting, but you need to remember, in a small amount of time, you have been able to set up code to create and provision your database server, create the database and user, and add default data and tables to the database. We then set up a web server with Python3 support that can interact with our database and connect and extract data and serve to a web page. Although we have finished working on our LAMP stack for this part of the chapter, continue on as the following section will discuss how you can start to reuse your code and playbooks.
Organizing Larger Playbooks with Include and Import
At this time, you’re probably seeing benefit of using a playbook but wondering how they work when the deployments and environments you’re creating and supporting become more complex. The way our current playbooks stand, as our environment continues to grow, our playbooks would also continue to grow. But there are a few things we can do to organize things better and ensure our playbooks remain readable and don’t get overly complex. In the next few pages, we will introduce you to include and import.
Both include and import would be familiar to you if you have some background in programming. In Ansible, they work in a similar way and allow us to start to make our playbooks more reusable and modular. Tasks are separated out from the main playbook and are then referred to with an import or include statement in the multiple playbooks or multiple times in the same playbook.
Both include and import do similar things, but import statements are preprocessed at the time the playbook is parsed by Ansible, whereas the include statements are processed as they are encountered during the execution of the playbook.
Please make note if you are using or working with an older version of Ansible. Prior to version 2.4, only include statements were available. If you use an import in these earlier versions, you will be met with an error when running your Ansible playbook.
- 1.
Log back into your work environment and move into the directory you’ve been working in to create the database and web server.
- 2.
Use the following touch command to create the playbook called new-playbook.yml:
touch new-playbook.yml - 3.Open the new playbook with your text editor and add in the following code. As you can see, there is not too much involved, but by using the import_playbook commands, we can import both of our previous playbooks into this one:1 ---2 - import_playbook: webserver-playbook.yml3 - import_playbook: dbserver-playbook.yml
- 4.
That’s all there is to it. We've created a new playbook and reused the playbooks we created earlier in the chapter. Save the file and you can now run it with the following command:
ansible-playbook -i hosts new-playbook.yml
If all goes well, you should see a very similar output to what you saw when you ran each of your playbooks separately with the imported playbooks being deployed in the order they were specified in the new-playbook.yml file.
Instead of importing a playbook, you could create a list of common tasks which are used as part of your deployment. Your playbook running the common tasks would then use include or import tasks to use them.
As long as our notify statement is using the correct task name of “restart apache2,” the handler will work correctly.
Although import and include statements are important parts of using Ansible within playbooks, the use of these statements is limited with more users opting to use roles to organize their playbook structure. Our focus will now move to roles in the next chapter, and we will not have much more exposure to import and include statements.
Summary
I hope you are finding this book interesting and challenging. We’ve done a lot of work in this chapter and covered a lot of ground. We started by taking a look at YAML and how we can set up our Ansible playbooks and then moved onto taking our commandline modules into our first real playbook. We then developed our skills and our LAMP stack further with a MYSQL database and then Python support, all the while discussing what the different syntax of the playbooks mean.
Finally, we take a look at how we organize our playbooks and think about how we can make them more manageable and reusable. This is the theme we take forward into our next chapter as we introduce roles to organize our playbooks into a universal structure. At the same time, we will take this further by putting more features into our LAMP stack.