Installing Cobbler

Let’s start by installing Cobbler on your host. To run Cobbler, you need to install the EPEL repositories.

$ sudo yum install –y epel-release

Then we need to install Cobbler.

$ sudo yum install –y cobbler

This will install some additional YUM utilities and the createrepo package , which assist in repository management. We’ve also installed some additional packages Cobbler uses: the DHCP daemon, a TFTP server, and the Apache web server. You may already have these packages installed, in which case YUM will skip them.

Once everything is installed, we need to enable cobblerd, the daemon process, at boot and start it.

$ sudo systemctl enable cobblerd httpd
$ sudo systemctl start cobblerd
$ sudo systemctl start httpd

Cobbler requires access to the Apache server to be started. Also, we need to ensure that the cobblerd service can access the httpd server port. SELinux by default will prevent this, so we need to issue the following:

$ sudo setsebool -P httpd_can_network_connect true

Cobbler has some specific SELinux settings , and you can view them with the following command:

$ sudo getsebool -a|grep cobbler
cobbler_anon_write --> off
cobbler_can_network_connect --> off
cobbler_use_cifs --> off
cobbler_use_nfs --> off
httpd_can_network_connect_cobbler --> off
httpd_serve_cobbler_files --> off

We will enable the following SELinux Booleans:

$ sudo setsebool -P httpd_serve_cobbler_files on
$ sudo setsebool -P httpd_can_network_connect_cobbler  on

Configuring Cobbler

After you’ve installed the required packages, you need to configure Cobbler. Cobbler comes with a handy check function that tells you what needs to be done to configure it. To see what needs to be done, run the following:

The following are potential configuration items that you may want to change:

1 : The 'server' field in /etc/cobbler/settings must ... by all machines that will use it.
2 : For PXE to be functional, the 'next_server' field ... the IP of the boot server on the PXE network.
3 : SELinux is enabled. Please review the following ... https://github.com/cobbler/cobbler/wiki/Selinux
4 : change 'disable' to 'no' in /etc/xinetd.d/tftp
5 : some network boot-loaders are missing from /var/lib/cobbler/loaders, ...is the easiest way to resolve these requirements.
6 : debmirror package is not installed, it will be required to manage debian deployments and repositories
7 : ksvalidator was not found, install pykickstart
8 : The default password used by the sample templates ...  'your-password-here'" to generate new one
9 : fencing tools were not found, and are required to use ... cman or fence-agents to use them

Restart cobblerd and then run 'cobbler sync' to apply changes.

You can see there are a few things you need to do to get Cobbler running. Let’s work through each of these issues.

First, you configure the /etc/cobbler/settings file . You need to update two fields in this file: server and next_server. You need to replace the existing values (usually 127.0.0.1) with the IP address of your host so a PXE-booted host can find your Cobbler host. In our case, we specify the following:

server 192.168.0.1
next_server 192.168.0.1

To update Cobbler’s configuration, you then run this:

$ sudo cobbler sync

You also need to configure a DHCP server (like the one we introduced in Chapter 10). You have two choices here: you can get Cobbler to manage your existing DHCP server, or you can tell your existing DHCP server to point to Cobbler.

After you have run cobbler sync and rerun cobbler check, you will notice the list of outstanding things to check has been reduced. We are going to now install the Cobbler loaders and debmirror binary.

$ sudo cobbler get-loaders

For debmirror, you need to download the file from Debian, untar it, and copy it to a common location (alternative, you could use FPM, like we showed you in Chapter 9, to create a package to do this for you in a repeatable way!).

We need at least these Perl modules installed:

$ sudo yum install -y perl-LockFile-Simple perl-IO-Zlib perl-Digest-MD5 perl-Net-INET6Glue perl-LWP-Protocol-https

Next we will download and install the debmirror package , untar it, and place it in the /usr/local/bin directory.

$ curl -s http://archive.ubuntu.com/ubuntu/pool/universe/d/debmirror/debmirror_2.25ubuntu2.tar.xz -o debmirror_2.25.tar.xz
$ tar xf debmirror_2.25.tar.xz && sudo cp debmirror-2.25ubuntu2/debmirror /usr/local/bin/

To test we have everything installed correctly for debmirror, run debmirror --help and make sure you don’t get any Perl module errors.

Lastly, we are going to change the default root password that gets placed on the hosts. First you can create a secure SHA-512 password using python3with the following:

python3 -c 'import crypt; print(crypt.crypt("yourpasswordhere", crypt.mksalt(crypt.METHOD_SHA512)))'
$6$KnsQG.tEetSCSmid$HpqUNyEk1UPkt9Dc9MPcwPY...guKOGdUeNXoA7.ugUBGGaDIk8RY8FRYVOwzmsM.u01                              

Then you need to update the default_password_crypted: setting in the /etc/cobbler/settings file. Remember to run cobbler sync after each change.

Now when we run $ sudo cobbler check, the list contains only three items, which we don’t need to address.

Cobbler Managing Your DHCP

If you want to enable Cobbler to manage your DHCP server, then you need to enable another option in the /etc/cobbler/settings file.

manage_dhcp: 1

You also need to update a template file that Cobbler will use to configure your DHCP server, /etc/cobbler/dhcp.template. Listing 19-1 shows an example of this file.

Listing 19-1. The /etc/cobbler/dhcp.template File

# ******************************************************************
# Cobbler managed dhcpd.conf file
#
# generated from cobbler dhcp.conf template ($date)
# Do NOT make changes to /etc/dhcpd.conf. Instead, make your changes
# in /etc/cobbler/dhcp.template, as /etc/dhcpd.conf will be
# overwritten.
#
# ******************************************************************

ddns-update-style interim;

allow booting;
allow bootp;

ignore client-updates;
set vendorclass = option vendor-class-identifier;

option pxe-system-type code 93 = unsigned integer 16;

key dynamic-update-key {
        algorithm hmac-sha256;
        secret "RZqM/JutbhgHiBR8ICG0LDyN+9c1LpNU83ycuU9LPaY=";
}

zone 0.168.192.in-addr.arpa. {
    key dynamic-update-key;
    primary 192.168.0.1;
}

zone example.com. {
    key dynamic-update-key;
   primary 192.168.0.1;
}

subnet 192.168.0.0 netmask                                                                                                                 255.255.255.0 {
    option routers 192.168.0.254;
    option domain-name "example.com";
    option domain-name-servers 192.168.0.1;
    option broadcast-address 192.168.0.255;

    next-server $next_server;
    filename "/pxelinux.0";
    group "static" {
       use-host-decl-names on;
        host au-mel-rhel-1 {
             hardware ethernet 00:16:3E:15:3C:C2;
           fixed-address au-mel-rhel-1.example.com;
       }
    }
    pool {
        range 192.168.0.101 192.168.0.150;
        deny unknown clients;
   }
   pool {
      range 192.168.0.151 192.168.0.200;
      allow unknown clients;
     default-lease-time 7200;
      max-lease-time 21600;
   }
}

If you have an existing DHCP server with a configuration, you should update this template to reflect that configuration. You can see we’ve adjusted the template in Listing 19-1 to reflect the DHCP configuration we used in Chapter 10. We’ve added two settings.

allow booting;
allow bootp;

These two options tell the DHCP server to respond to queries from hosts who request network boots.

The other two important settings to note in Listing 19-1 are next-server and filename configuration options. The next-server option is set to $next_server. This value will be replaced by the IP address we just configured in the next_server option in the /etc/cobbler/settings file. This tells our DHCP server where to route hosts that request a net boot.

The filename option is set to /pxelinux.0, which is the name of the boot file that PXE-booted hosts should look for to start their boot process. We’ll set up this file shortly.

Now, after changing these files, you need to run the following command:

$ sudo cobbler sync

allow booting;
allow bootp;

subnet 192.168.0.0 netmask 255.255.255.0 {
     option routers 192.168.0.254;
     option domain-name "example.com";
     option domain-name-servers 192.168.0.1;
     option broadcast-address 192.168.0.255;
     filename "/pxelinux.0";
     next-server 192.168.0.1;
     group "static" {
        use-host-decl-names on;
         host au-mel-rhel-1 {
             hardware ethernet 00:16:3E:15:3C:C2;
           fixed-address au-mel-rhel-1.example.com;
          }
      }
      pool {
        range 192.168.0.101 192.168.0.150;
        deny unknown clients;
     }
     pool {
      range 192.168.0.151 192.168.0.200;
      allow unknown clients;
      default-lease-time 7200;
      max-lease-time 21600;
    }
}

allow booting;
allow bootp;

next-server 192.168.0.1

disable = yes

disable = no

$ sudo systemctl enable tftp
$ sudo systemctl start tftp

$ sudo firewall-cmd --zone=public --add-service=tftp --permanent
$ sudo firewall-cmd --zone=public --add-service=httpd –permanent
$ sudo firewall-cmd --zone=public --add-port=25150:25151/tcp –permanent

Using Cobbler

Once you’ve configured Cobbler, you can start to make use of it. Cobbler allows you to specify a distribution you’d like to build hosts with, imports that distribution’s files, and then creates a profile. You can then build hosts using this distribution and profile.

We have mounted our ISO files to /mnt/centos and /mnt/ubuntu, respectively. This is done like so:

$ sudo mount –o loop /path/to/downloaded.iso /path/to/mountpoint

Let’s start by creating our first profile using the import command .

$ sudo cobbler import --path=/mnt/centos --name=CentOS7 --arch=x86_64
task started: 2016-12-22_055922_import
task started (id=Media import, time=Thu Dec 22 05:59:22 2016)
Found a candidate signature: breed=redhat, version=rhel6
Found a candidate signature: breed=redhat, version=rhel7
Found a matching signature: breed=redhat, version=rhel7
Adding distros from path /var/www/cobbler/ks_mirror/CentOS7-x86_64:
creating new distro: CentOS7-x86_64
trying symlink: /var/www/cobbler/ks_mirror/CentOS7-x86_64 -> /var/www/cobbler/links/CentOS7-x86_64
creating new profile: CentOS7-x86_64
associating repos
checking for rsync repo(s)
checking for rhn repo(s)
checking for yum repo(s)
starting descent into /var/www/cobbler/ks_mirror/CentOS7-x86_64 for CentOS7-x86_64
processing repo at : /var/www/cobbler/ks_mirror/CentOS7-x86_64
need to process repo/comps: /var/www/cobbler/ks_mirror/CentOS7-x86_64
looking for /var/www/cobbler/ks_mirror/CentOS7-x86_64/repodata/*comps*.xml
Keeping repodata as-is :/var/www/cobbler/ks_mirror/CentOS7-x86_64/repodata
*** TASK COMPLETE ***

We will import our Ubuntu ISO also.

$ sudo cobbler import --path=/mnt/ubuntu --name Ubuntu-16.04 --breed=ubuntu --os-version=xenial

You issue the cobbler command with the import option. The --path option specifies the source of the distribution you want to import—in our case, /mnt/ubuntu and /mnt/centos. The --name is any name you want to give the distribution, and you can add --breed and --os-version to help the import command find the right signature to match your distribution.

You can also sync with online repositories. Here’s an example:

$ sudo cobbler reposync
task started: 2016-12-22_063019_reposync
task started (id=Reposync, time=Thu Dec 22 06:30:19 2016)
hello, reposync
run, reposync, run!
running: /usr/bin/debmirror --nocleanup --verbose --ignore-release-gpg --method=http --host=archive.ubuntu.com --root=/ubuntu --dist=xenial,xenial-updates,xenial-security --section=main,universe /var/www/cobbler/repo_mirror/Ubuntu-16.04-x86_64 --nosource -a amd64

This will sync with online repositories and, in this case, uses the debmirror binary we installed earlier to sync our Ubuntu Xenial release.

Cobbler will run the import process and then return you to the prompt. Depending on the performance of your host (and, if you are importing over the network, the speed of your connection), this may take some time.

You can view the distributions install via this command:

$ sudo cobbler distro list
   CentOS7-x86_64
   Ubuntu-16.04-x86_64

The import will create two profiles as well; we can see them with this command:

$ sudo cobbler profile list
   CentOS7-x86_64
   Ubuntu-16.04-x86_64

After you’ve created your distribution and profile, you can see the full details in Cobbler using the report option, as shown in Listing 19-3.

Listing 19-3. A Cobbler Report

$ sudo cobbler report
distros:
==========
Name                           : CentOS7-x86_64
Architecture                   : x86_64
TFTP Boot Files                : {}
Breed                          : redhat
Comment                        :
Fetchable Files                : {}
Initrd                         : /var/www/cobbler/ks_mirror/CentOS7-x86_64/images/pxeboot/initrd.img
Kernel                         : /var/www/cobbler/ks_mirror/CentOS7-x86_64/images/pxeboot/vmlinuz
Kernel Options                 : {}
Kernel Options (Post Install)  : {}
Kickstart Metadata             : {'tree': 'http://@@http_server@@/cblr/links/CentOS7-x86_64'}
Management Classes             : []
OS Version                     : rhel7
Owners                         : ['admin']
Red Hat Management Key         : <<inherit>>
Red Hat Management Server      : <<inherit>>
Template Files                 : {}

Name                           : Ubuntu-16.04-x86_64
Architecture                   : x86_64
TFTP Boot Files                : {}
Breed                          : ubuntu
Comment                        :
Fetchable Files                : {}
Initrd                         : /var/www/cobbler/ks_mirror/Ubuntu-16.04/install/netboot/ubuntu-installer/amd64/initrd.gz
Kernel                         : /var/www/cobbler/ks_mirror/Ubuntu-16.04/install/netboot/ubuntu-installer/amd64/linux
Kernel Options                 : {}
Kernel Options (Post Install)  : {}
Kickstart Metadata             : {'tree': 'http://@@http_server@@/cblr/links/Ubuntu-16.04-x86_64'}
Management Classes             : []
OS Version                     : xenial
Owners                         : ['admin']
Red Hat Management Key         : <<inherit>>
Red Hat Management Server      : <<inherit>>
Template Files                 : {}

This option displays all the distributions, and their profiles are currently imported into Cobbler.

Listing 19-3 shows our vanilla CentOS7-x86_64 distribution and the profile we created, CentOS7-x86_64. Most of the information in Listing 19-3 isn’t overly important to us; we are going to use these profiles to create a new system shortly.

Building a Host with Cobbler

Now that you’ve added a profile and a distribution, you can boot a host and install your distribution. Choose a host (or virtual machine) you want to build and reboot it. Your host may automatically search for a boot device on your network, but more likely you will need to adjust its BIOS settings to adjust the boot order. To boot from Cobbler, you need to specify that your host boots from the network first.

We have created a VirtualBox host on the same Host-only Adapter interface as our Cobbler server. We created VirtualBox hosts in Chapter 3 and created a basic host with an 8Gb hard drive. In the System settings, we are going to select Network for the boot device.

In Figure 19-1 we are selecting the Network Boot option. We will come back to here and set it to Hard Disk when we are finished. We now start the hosts like we would normally.

Figure 19-1. Setting Network Boot

When your host boots, it will request an IP address from the network and get an answer from your DHCP server, as you can see in Figure 19-2.

Figure 19-2. Requesting DHCP address

Your host will boot to the Cobbler boot menu. You can see an example of this menu in Figure 19-3.

Figure 19-3. The Cobbler menu

From this menu, you can select the profile you’d like to install (e.g., CentOS7-x86_64). If you don’t select a profile to be installed, Cobbler will automatically launch the first item on the menu (local), which continues the boot process on the localhost.

We will select CentOS7-x86_64, which will automatically install CentOS onto our host. If we select Ubuntu-16.04-x86_64, we will install Ubuntu; Figures 19-4 through 19-6 tell that story.

Figure 19-4. Installing CentOS

Figure 19-5. Installing Ubuntu

Figure 19-6. Selecting Boot from Hard Disk

Remember, when we have finished installing our host on VirtualBox, we need power on the host and need to change the boot device and then start the host again.

Figure 19-7. CentOS installed

Figure 19-8. Ubuntu installed

We selected a profile, and then this profile started the installation process using the instructions contained in the associated Kickstart or Preseed file. If you are watching your installation process, you will see the installation screens progress—all without requiring input from you to continue or select options.

Using Cobbler, you can also specify configuration options for particular hosts. You don’t need to do this, but it is useful if you have a specific role in mind for a host and want to specify a particular profile or Kickstart configuration.

To do this, you add hosts to Cobbler, identifying them via their MAC or IP addresses , using the system command.

$ sudo cobbler system add --name=web1.example.com --mac=08:00:27:66:EF:C2
--profile=CentOS7-x86_64 --interface eth1

Here we’ve added a system named web1.example.com with the specified MAC address.

The new host uses the CentOS7-x86_64 profile . So far it is no different from the hosts we have built previously. If a host with the appropriate MAC address connects to our Cobbler host, then Cobbler will use these configuration settings to provision the host.

If you need to change the way you build a host, you can create a new profile. Your new profiles can inherit settings from other profiles, which are regarded as their parents. We are going to create a profile called centos-base that will inherit the distro and other settings from the parent CentOS7-x86_64.

$ sudo cobbler profile add --name centos-base --parent CentOS7-x86_64

This is how we can use different common Kickstart or preseed files for different host groups. Kickstart and Preseed files may have different disk configurations or package lists that are tailored for particular profiles. To add a particular Kickstart or Preseed file, you first copy and modify any existing one to the way you like it and add it to the /var/lib/cobbler/kickstarts directory. Then you can add it to the profile with the --kickstart option.

You can list the configured hosts using the list and report options .

$ sudo cobbler system list
web1.example.com

A full listing of the gateway.example.com system definition can be seen using the report option.

$ sudo cobbler system report –name=web1.example.com

We can also delete a system using the remove command.

$ sudo cobbler system remove --name=web1.example.com

Cobbler Web Interface

Cobbler also has a simple web interface you can use to manage some of its options. It’s pretty simple at this stage, and the command-line interface is much more fully featured, but it is available if you want to implement it. You can find instructions at http://cobbler.github.io/manuals/2.8.0/5_-_Web_Interface.html .

Troubleshooting Cobbler

You can troubleshoot the network boot process by monitoring elements on your host, including your log files, and by using a network monitoring tool like the tcpdump or tshark command.

You can start by monitoring the output of the DHCP process by looking at the /var/log/messages log files. Cobbler also logs to the /var/log/cobbler/cobbler.log file and the files contained in the kicklog and syslog directories also under /var/log/cobbler.

You can also monitor the network traffic passing between your booting host and the boot server. You can use a variety of network monitoring tools for this.

$ sudo tcpdump port tftp

Cobbler has a wiki page available that contains documentation at http://cobbler.github.io/manuals . The documentation includes some useful tips for troubleshooting at http://cobbler.github.io/manuals/2.8.0/7_-_Troubleshooting.html . The Cobbler community also has a mailing list and other support channels that you can see here: http://cobbler.github.io/community.html .

Kickstart

On CentOS, the language used to automatically install your host is called Kickstart . On Ubuntu, it is called Preseed . For simplicity’s sake and because it’s an easier language to use, we’re going to show you how to use Kickstart to automate your installation for both CentOS and Ubuntu. Where something isn’t supported on Ubuntu, we’ll show you how to use Preseed to configure it.

A Kickstart configuration file contains the instructions required to automate the installation process. It’s a simple scripted process for most installation options, but it can be extended to do some complex configuration.

You can find the latest detailed documentation for Kickstart at http://pykickstart.readthedocs.io/en/latest/kickstart-docs.html .

You can find documentation on Preseed and its directives at https://wiki.debian.org/DebianInstaller/Preseed . We’ll work with a few of these directives later in this section.

You’ve already seen how to specify Kickstart files to your provisioning environments, using Cobbler. Let’s start by looking at some of the contents of a simple Kickstart file in Listing 19-4.

Listing 19-4. A Kickstart File

install
# System authorization information
auth --enableshadow  --passalgo sha512
# System bootloader configuration
bootloader --location=mbr
# Partition clearing information
clearpart --all --initlabel
# Use text mode install
text

Listing 19-4 shows a list of configuration directives starting with the install option, which dictates the behavior of the installation process by performing an installation.

You can then see configuration directives with options, for example, auth --enableshadow  --passalgo sha512, which tell Kickstart how to answer particular installation questions. The auth statement has the values --enableshadow and --passalgo sha512 here, which enable shadow passwords and specify that passwords hashes must use the SHA512 password algorithm, respectively.

The option that follows, bootloaderwith a value of --location=mbr, tells Kickstart to install the boot loader into the MBR. Next is the directive clearpart, which clears all partitions on the host and creates default labels for them. The final option, text, specifies we should use text-based installation as opposed to the GUI.

There are too many directives to discuss them individually, so we show you in Table 19-1 the directives that must be specified and some of the other major directives that you may find useful.

You can also find a useful list of the available directives with explanations at http://pykickstart.readthedocs.io/en/latest/kickstart-docs.html#chapter-2-kickstart-commands-in-fedora .

url --url http://192.168.0.1/centos/

url --url=$tree

url --url ftp://jsmith:[email protected]/centos

harddrive --dir=/centos --partition=/installsource

# System keyboard
keyboard us
# System language
lang en_AU
# System timezone
timezone Australia/Melbourne

rootpw --iscrypted $6$D7CxLkSBeC9.k.k3$S8G9s3/Y5LJ4dio....S5GS78p2laxALxaJ.lCN9tzKB1zIpYz38Fs9/

user jsmith --password password

# Firewall configuration
firewall --enabled --http --ssh --smtp
# SELinux configuration
selinux --enabled

# Network information
network --bootproto=static --device=eth0 --gateway=192.168.0.254
--ip=192.168.0.1 --nameserver=192.168.0.1 --netmask=255.255.255.0 --onboot=on

network --bootproto=dhcp --device=eth0 --onboot=on

$ sudo cobbler system edit --name=gateway.example.com --mac=00:0C:29:3B:22:46
--profile=centos-base --interface=eth0 --ip=192.168.0.1 --subnet=255.255.255.0 --                      
gateway=192.168.0.254 --hostname=gateway --bootproto=static

$SNIPPET('network_config')

# Partition clearing information
clearpart --all --initlabel
part /boot --asprimary --bytes-per-inode=4096 --fstype="ext4" --size=150
part / --asprimary --bytes-per-inode=4096 --fstype="ext4" --size=4000
part swap --bytes-per-inode=4096 --fstype="swap" --size=512

part raid.01 --asprimary --bytes-per-inode=4096 --fstype="raid" --grow --ondisk=sda
--size=1
part raid.02 --asprimary --bytes-per-inode=4096 --fstype="raid" --grow --ondisk=sdb
--size=1
part raid.03 --asprimary --bytes-per-inode=4096 --fstype="raid" --grow --ondisk=sdc
--size=1
part raid.04 --asprimary --bytes-per-inode=4096 --fstype="raid" --grow --ondisk=sdd
--size=1
part raid.05 --asprimary --bytes-per-inode=4096 --fstype="raid" --grow --ondisk=sde
--size=1
raid / --bytes-per-inode=4096 --device=md0 --fstype="ext4" --level=5 raid.01 raid.02
raid.03 raid.04 raid.05

part /boot --fstype ext4 --size=150
part swap --size=1024
part pv1 --size=1 --grow
volgroup vg_root pv1
logvol / --vgname=vg_root --size=81920 --name=lv_root

%packages
@ Administration Tools
@ Server Configuration Tools
@ System Tools
@ Text-based Internet
dhcp

%packages
@ kubuntu-desktop
dhcp-client

Pre- and Post-installation

You can run scripts before and after Kickstart installs your host. The prerun scripts run after the Kickstart configuration file has been parsed, but before your host is configured. Any prerun script is specified at the end of the Kickstart file and prefixed with the line %pre.

Each %post and %pre section must have a corresponding %end.

The postrun scripts are triggered after your configuration is complete and your host is installed. They should also be specified at the end of the Kickstart file and prefixed by a %post line. This is the %post section from our sample.ks configuration file:

%post
$SNIPPET('post_install_kernel_options')
$SNIPPET('post_install_network_config')
%end

Here we’ve specified two postrun Cobbler snippets that configure kernel and network options.

This postrun scripting space is useful to run any required setup applications or scripts.

Preseed

Preseed is the Debian installation automation tool. It is more opaque than Kickstart, but it performs the same function of automating an installation.

To provide some context , each d-i line that you see is a call to the Debian installer. The format of the file is the same as instructions passed to the debconf-set-selection command. It takes the following form:

<owner> <question name> <question type> <value>

So something like setting the locale for your system would look something like this:

d-i debian-installer/locale string en

The owner is the debian-installer, the question is debian-installer/locale, the type is string, and the value is en, or English.

Installation Source

Cobbler is selected as the initial installation source via the live-installer question.

d-i live-installer/net-image string http://$http_server/cobbler/links/$distro_name/install/filesystem.squashfs

During the install, you can set up your apt repositories. This points your apt sources to the Ubuntu mirrors.

d-i mirror/country string manual
d-i mirror/http/hostname string archive.ubuntu.com
d-i mirror/http/directory string /ubuntu
d-i mirror/http/proxy string

You can choose the different apt pools with these settings to get to packages available from backports and the like:

#d-i apt-setup/restricted boolean true
#d-i apt-setup/universe boolean true
#d-i apt-setup/backports boolean true

You can just uncomment what pool you would like.

Keyboard, Language, and Time Zone

Setting the keyboard and language can be a time-consuming process but not with the installer. You can select the following in Preseed:

d-i debian-installer/locale string en
d-i debian-installer/country string AU
d-i debian-installer/locale string en_AU.UTF-8
d-i debian-installer/language string en
d-i console-setup/ask_detect boolean false
d-i console-setup/layoutcode string us
d-i console-setup/variantcode string
d-i keyboard-configuration/layoutcode string us
d-i clock-setup/ntp boolean true
d-i clock-setup/ntp-server string ntp.ubuntu.com
d-i time/zone string UTC
d-i clock-setup/utc boolean true

Here we set the locale and country, and we disabled the keyboard prompt to ask for our selection and answered all the questions concerning keyboard layout. Then we enable NTP for our clocks and set their time to UTC.

Managing Users

With Cobbler and Preseed we enable the root user, which Ubuntu doesn’t normally do.

d-i passwd/root-login boolean true
d-i passwd/root-password-crypted password $default_password_crypted
d-i passwd/make-user boolean false

So when you build your hosts, you will need to sign in as root. To keep your familiar setup, you could add a user ubuntu either in the Preseed or in a SNIPPET at the end of the installation.

#d-i passwd/user-fullname string Ubuntu User
#d-i passwd/username string ubuntu

Firewall and Network

You can set up the network in whichever fashion suits you with Preseed, but you cannot do any firewall configurations. You can also add firewall configurations in post-install scripts in Cobbler if that is a requirement.

# IPv4 example
#d-i netcfg/get_ipaddress string 192.168.1.42
#d-i netcfg/get_netmask string 255.255.255.0
#d-i netcfg/get_gateway string 192.168.1.1
#d-i netcfg/get_nameservers string 192.168.1.1
#d-i netcfg/confirm_static boolean true

You can set a static IP or allow for DHCP in your network settings.

Disks and Partitions

Disk partition can be fairly complex in Preseed. Here we are just creating a simple LVM partition setup:

d-i partman-auto/method string lvm
d-i partman-lvm/device_remove_lvm boolean true
d-i partman-lvm/confirm boolean true
d-i partman-auto/choose_recipe select atomic
d-i partman/confirm_write_new_label boolean true
d-i partman/choose_partition select finish
d-i partman/confirm boolean true                                                

Package Management

Using Preseed, you can specify the packages you want to install. For group packages, you can use tasksel (or task select owner) to multiselect a package group—like ubuntu-desktop.

tasksel tasksel/first multiselect ubuntu-desktop

For individual packages, you can just use the following:

d-i pkgsel/include string openssh-server

You can have more than one package selected if you like.

d-i pkgsel/include string openssh-server build-essential

CentOS Installation

With the latest packages from Puppet, all the required packages are installed by default when we install the server. On CentOS, on both servers and clients, you will need to add the Puppet repositories for Red Hat–based machines.

$ sudo yum install -y https://yum.puppetlabs.com/puppetlabs-release-pc1-el-7.noarch.rpm

There are several components that make up the Puppet ecosystem such as Facter, a tool for discovering system facts on nodes. System facts are things like operating system, IP addresses, and any custom facts. Another one is Hiera, a key/value lookup database for declaring Puppet configuration data. Finally, there’s MCollective, an orchestration tool for managing Puppet nodes.

On the master, you install the puppetserver, and this will install facter, hiera, the agent, and other required packages from the Puppet repository.

$ sudo yum install puppetserver

On the Puppet nodes, or clients, we can install the puppet-agent package by itself, and it will contain or require all that it needs to run.

$ sudo yum install –y puppet-agent

This of course will require the installation of the YUM repository, like earlier, first.

Ubuntu Installation

On Ubuntu, we again install the apt repository and then install puppetserver on the master server, which will bring down all the necessary Puppet components, like Facter and Hiera and the agent.

On the server or master, we need to do this:

$ wget https://apt.puppetlabs.com/puppetlabs-release-pc1-xenial.deb -O xenial.deb && sudo dpkg -i xenial.deb
$ sudo apt-get update
$ sudo apt-get install -y puppetserver

On the client, you need the following:
$ sudo apt-get install –y puppet-agent

You will now have all the necessary components installed on your systems.

Configuring the Master

Typically the master’s /etc/puppetlabs/puppet/puppet.conf will look something like this:

[main]
certname = puppetmaster.example.com
server = puppet
environment = production
runinterval = 30m
strict_variables = true

[master]
dns_alt_names = puppetmaster,puppet,puppet.example.com

The [main] section contains the defaults for both the master and agents. Here we determine the certname, which will be the common name specified in the TLS certificate that we will generate on startup. This is related to the dns_alt_names setting that provides alternative DNS names that agents can use to verify the Puppet master. The server = puppet is the name of the Puppet master this Puppet agent will try to connect to. You can see that this matches one of the dns_alt_names.

The Puppet agents can specify the environment that they should use to collect their catalog when they connect to the Puppet master. This is often used to test version control system (VCS) branches of your Puppet code or can be used to multihome your Puppet master for more than one organization.

Do not fall into the mistake of making environments that mirror the roles your hosts may perform. That is, don’t have an environment for development, UAT, staging, and production and assign relevant hosts to those environments. It is easier to treat all your hosts as production and handle the different roles and profiles these hosts may take on in the Puppet manifest itself. It often leads to a horrible divergence of your Puppet code between systems and VCS branches. By all means create an environment to test your Puppet code, but roll that as soon as you can into the production branch. Use alternatives like Hiera and the “roles and profiles” patterns to achieve this. See https://docs.puppet.com/hiera/3.2/ and https://docs.puppet.com/pe/2016.4/r_n_p_full_example.html .

The runinterval is the time between each Puppet run, that is, when the agent will call into the Puppet master for its catalog. strict_variables means that the parse will raise an error when referencing unknown variables.

In the [master] section, we define setting for the Puppet master server. We are not going to set anything here except the dns_alt_names value. Settings that might belong in here are codedir, where Puppet will look for the Puppet code, or the manifest, which we are going to write. However, we are going to take the defaults, which means our codedir will be /etc/puppetlabs/code.

It is in here you will set reporting settings and configuration for using PuppetDB. Using PuppetDB is a great idea as it allows you to do complex catalogs as it collects data from multiple nodes but is outside the scope of this exercise. See here for more details: https://docs.puppet.com/puppetdb/ .

We recommend you create a DNS CNAME for your Puppet host (e.g., puppet.example.com) or add it to your /etc/hosts file.

# /etc/hosts
127.0.0.1 localhost
192.168.0.1 au-mel-ubuntu-1 puppet puppetmaster puppet.example.com puppetmaster.example.com

Starting Puppet Server with the RAL

Here is a neat trick. You can use the Puppet resource command to start your Puppet master server (puppetserver). The Puppet resource command allows you to directly interact with the Puppet Resource Abstraction Layer (RAL) . The RAL is how Puppet interacts and manages the system. With the Puppet resource we will start puppetserver and make it start on boot like this:

sudo /opt/puppetlabs/bin/puppet resource service puppetserver ensure=running enable=true

We have not yet described how Puppet manages resources, and you will get a deeper understanding of what this command is doing shortly, but briefly what this is doing is the following:

• Starting a service (ensure=running)

• Making the necessary changes to start a service at boot (enable=true)

• Using whatever underlying system to start the service (service puppetserver)

In our case, it will use systemctl commands (start and enable) under the hood. You can run this same command on CentOS, Ubuntu, or any other supported system, and it will start the puppetserver process. If you were on a Mac and starting an Apache service, it would use launchctl—it uses whatever is appropriate for the system it is run on.

We can see if it has started using the normal systemctl command, and we can see the logs here:

$ sudo journalctl -u puppetserver -f
-- Logs begin at Tue 2016-12-20 09:24:04 UTC. --
Dec 21 09:25:29 puppetmaster systemd[1]: Starting puppetserver Service...
Dec 21 09:25:29 puppetmaster puppetserver[4877]: OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0
Dec 21 09:26:30 puppetmaster systemd[1]: Started puppetserver Service.

Also, the running server logs can be found in /var/log/puppetlabs.

We can use this for keeping an eye on the tasks in the next section.

Functions

Puppet also has a collection of functions. Functionsare useful commands that can be run on the Puppet master to perform actions. You’ve already seen two functions: template, which we used to create a template configuration file, and include, which we used to specify the classes for our nodes. There are a number of other functions, including the generate function, which calls external commands and returns the result, and the notice function, which logs messages on the master and is useful for testing a configuration.

You can see a full list of functions at https://docs.puppet.com/puppet/latest/function.html .

Reports

Puppet has the ability to report on events that have occurred on your nodes or clients. Combined with PuppetDB, you can get extensive reports about your systems; if you have the enterprise version, you will be able to see these in the Puppet dashboard. You can read more about reporting at https://docs.puppet.com/puppet/latest/reporting_about.html .

External Nodes

As you might imagine, when you begin to have a lot of nodes, your configuration can become quite complex. If it becomes cumbersome to define all your nodes and their configuration in manifests, then you can use a feature known as external nodesto better scale this. External nodes allow you to store your nodes and their configuration in an external source.

The ENC runs as a command on the Puppet master and returns a YAML document describing the manifest for any particular node. It can be from any source, such as a database.

You can read more about external nodes classifiers at https://docs.puppet.com/guides/external_nodes.html .

Documenting Your Configuration

A bane of many system administrators is documentation , both needing to write it and needing to keep it up-to-date. However, Puppet has some suggestions on how to write the documentation for any modules you create and want to publish to the broader community via sites like Puppet Forge. You can read about manifest documentation at https://docs.puppet.com/puppet/latest/modules_documentation.html .

Ansible Installation and Configuration

Ansible is simple to install and is available from either .deb or .rpm packages, Python Pip installation or tar files, and more, depending on your system. We are going to use the Debian package as we are running this on our Xenial server.

Let’s run the aptitude command like so:

$ sudo aptitude install –y ansible

It is also available via yum install. You can also install it via Python Pip, which is a Python package manager. This is available on most operating systems.

$ sudo pip install ansible

Once it’s installed, you can edit the global configuration file, ansible.cfg, that will be in the /etc/ansible directory. When Ansible starts, it looks for the configuration file, first in the environment variable ANSIBLE_CONFIG, then in ansible.cfg in the local directory, then in ∼/.ansible.cfg in the home directory, or lastly the system default in /etc/ansible/ansible.cfg.

When you start out, you don’t need to edit this file. If you have downloaded third-party Ansible modules to a particular location, you can declare that location in in the ansible.cfg file. Other things like SSH options that you want to use as defaults can also go in there.

inventory     = /etc/ansible/hosts
library       = /usr/share/my_modules/
roles_path    = /etc/ansible/roles
log_path      = /var/log/ansible.log
fact_caching  = memory
ssh_args      = -o ControlMaster=auto -o ControlPersist=60s -o ProxyCommand="ssh  -W %h:%p -q jumphost"

Here you can see a small subset of things you may want to change or add to. The inventory is where Ansible expects to see you host list or a program that will dynamically gather your host list. The library is for your own or shared modules. The role_path is where you might install roles. Roles are collections of tasks, variables, templates, files, and handlers that can be used to configure a particular service, like Nginx (see https://galaxy.ansible.com/ for a whole bunch of roles that people have created and shared).

The fact_cachingcan be stored either in memory on your local host or in different shared service like Redis (Redis being an open source key/value store database). This can help to speed up fact collection for multiple users of Ansible.

For the ssh_args the default SSH options are ControlMaster=auto and ControlPersist=60s, which allow for the sharing of multiple sessions over the one connection (meaning we don’t need to connect the target host, execute a task, disconnect, connect again, execute another task, and so on). The option we have added here is how you can run your commands via a jump host, so any target hosts will be accessed via this SSH proxy server.

There is no ansible service to start. However, you may want to automate the process of running playbooks on your hosts; that is where Ansible Tower ( https://www.ansible.com/tower ) comes in. It is a commercial automation and job scheduling tool provided by Red Hat.

You can also, of course, automate your Ansible playbooks by using other continuous delivery (CD) solutions like executing runs as part of a build step in tools like Jenkins ( https://jenkins.io/ ).

Using the ansible Command

Ansible is great for running ad hoc command across multiple hosts. Any of the modules (documented at http://docs.ansible.com/ansible/modules_by_category.html ) can be used to execute ad hoc commands via the ansible command. If we bundle up the ad hoc tasks into a list of tasks, that is called a playbook, which is invoked with the ansible-playbook command .

Let’s look first at the ansible command to run a simple task. At the basic level, Ansible needs to know three things.

• How to find the host to target

• The host to target

• The module to run and any arguments for that module

While there are many more options to the ansible command, we can apply this to run our first task. We are going to use the setup module, a module that gathers the facts that we can use in our tasks or playbooks.

$ ansible –c local localhost –m setup
localhost | SUCCESS => {
    "ansible_facts": {
        "ansible_all_ipv4_addresses": [
            "192.168.0.61",
            "10.0.2.15"
        ],
...
        "ansible_virtualization_type": "virtualbox",
        "module_setup": true
    },
    "changed": false
}

There is a long list of Ansible facts returned as a JSON string; we have shown only a small portion. The ansible command is using a local connection (-c local) and operating on the target host localhost. We executed on that host the setup module (-m setup).

Let’s now take it one step further; on this local host we will install the Nginx package. To do that, we use the apt or yum module, depending on which host operating system we are targeting.

$ ansible -c local localhost -m apt -a 'name=nginx state=latest update_cache=yes'
localhost | FAILED! => {
    "changed": false,
    "cmd": "apt-get update '&&' apt-get install python-apt -y -q --force-yes",
    "failed": true,
    "msg": "W: chmod 0700 of directory /var/lib/apt/lists/partial failed - SetupAPTPartialDirectory (1: Operation not permitted)
E: Could not open lock file /var/lib/apt/lists/lock - open (13: Permission denied)
E: Unable to lock directory /var/lib/apt/lists/
W: Problem unlinking the file /var/cache/apt/pkgcache.bin - RemoveCaches (13: Permission denied)
W: Problem unlinking the file /var/cache/apt/srcpkgcache.bin - RemoveCaches (13: Permission denied)
E: Could not open lock file /var/lib/dpkg/lock - open (13: Permission denied)
E: Unable to lock the administration directory (/var/lib/dpkg/), are you root?",
    "rc": 100,
    "stderr": "W: chmod 0700 of directory /var/lib/apt/lists/partial failed - SetupAPTPartialDirectory (1: Operation not permitted)
E: Could not open lock file /var/lib/apt/lists/lock - open (13: Permission denied)
E: Unable to lock directory /var/lib/apt/lists/
W: Problem unlinking the file /var/cache/apt/pkgcache.bin - RemoveCaches (13: Permission denied)
W: Problem unlinking the file /var/cache/apt/srcpkgcache.bin - RemoveCaches (13: Permission denied)
E: Could not open lock file /var/lib/dpkg/lock - open (13: Permission denied)
E: Unable to lock the administration directory (/var/lib/dpkg/), are you root?
",
    "stdout": "",
    "stdout_lines": []
}

We have an error! You can see from the output that we have tried to execute the "apt-get update '&&' apt-get install python-apt -y -q --force-yes" commands and we have been denied permission. This should not be a surprise; we don’t let unauthorized users install packages without appropriate sudo privileges. Let’s provide Ansible with the ability to use sudo with the request.

$ ansible -c local localhost --become -m apt -a 'name=nginx state=latest update_cache=yes'
localhost | SUCCESS => {
    "cache_update_time": 1481951319,
    "cache_updated": true,
    "changed": true,
    "stderr": "",
    "stdout": "....”
}

Now we have added the --become argument to the ansible command, and now it will attempt to execute the commands via sudo. The output has again been shortened, but you can see that we have "change": true, which means that the task was executed on the system and the system was changed.

What happens if we run that ansible task again?

$ ansible -c local localhost --become -m apt -a 'name=nginx state=latest update_cache=yes'
localhost | SUCCESS => {
    "cache_update_time": 1481951681,
    "cache_updated": true,
    "changed": false
}

Again, we are successful but this time, because Nginx is already install there was nothing to change, so "changed" is false. That is installing one thing on one host, how do you do that on many hosts?

$ sudo vi /etc/ansible/hosts

somehost.example.com

[all_centos]
gateway.example.com
backup.example.com

[all_ubuntu]
mail.example.com
monitor.example.com

[dbs]
db.example.com

[all_servers:children]
all_centos
all_ubuntu
dbs

[all_ubuntu:vars]
ansible_user=ubuntu

[all_centos:vars]
ansible_user=vagrant

[dbs]
db.example.com ansible_user=vagrant

[remote:vars]
ansible_ssh_common_args: '-o ProxyCommand="ssh  -W %h:%p -q jumphost"'

$ ansible all_servers -m ping
gateway.example.com | SUCCESS => {
    "changed": false,
    "ping": "pong"
}
...
db.example.com | SUCCESS => {
    "changed": false,
    "ping": "pong"
}

mail.example.com | UNREACHABLE! => {
    "changed": false,
    "msg": "ERROR! SSH encountered an unknown error during the connection. We recommend you re-run the command using -vvvv, which will enable SSH debugging output to help diagnose the issue",
    "unreachable": true
}

$ ansible all_servers --become –m service –a "name=sshd state=restarted"

---
- hosts: ahostgroup
  become: true
  gather_facts: true
  vars:
    - akey: avalue
  tasks:
    - name: do something
       module: module=arguments
  handlers:
  - name: a handler
    module: module=arguments

$ vi playbooks/backup.yml
- hosts: backup.example.com
  become: true
  gather_facts: true
  vars:
    url: http://download.bareos.org/bareos/release/latest/{{ ansible_distribution }}_{{ ansible_distribution_major_version }}
    bareos_db_packages: bareos-database-mysql
    sql_import: '/usr/lib/bareos/scripts/ddl/creates/mysql.sql'

        "ansible_distribution": "CentOS",
        "ansible_distribution_major_version": "7",

    url: http://download.bareos.org/bareos/release/latest/CentOS/7

/etc/bareos/bareos-dir.conf

{{ /etc/bareos/bareos-dir.conf | basename }}

$ ansible -c local localhost -m debug -a "msg={{ '/etc/bareos/bareos-dir.conf' |basename }}"
localhost | SUCCESS => {
    "msg": "bareos-dir.conf"
}

  - name: install mariadb
    apt: name=mariadb-server state=latest
    when: ansible_distribution == "Ubuntu"

our_config: {
    our_url:  ‘https://endpoint.example.com’

}

{% if our_config["our_url"] is not defined -%}
  url: {{ our_url | default('https://www.example.com') }}
{% endif %}

  tasks:
  - name: install epel
    yum: name=epel-release state=latest

- name: optional or description
  module_name: module_arg1=value1.... module_argx=valuex

  - name: add bareos
    get_url:
      url: "{{ url }}/bareos.repo"
      dest: /etc/yum.repos.d/bareos.repo
      mode: 0444

  - name: install pip
    yum: name={{ item }} state=latest update_cache=yes
    with_items:
      - python-pip
      - python-devel

  - name: install mariadb
    yum: name={{ item }} state=latest
    with_items:
      - mariadb-devel
      - mariadb-server
    notify: mariadb_restarted

  - name: install pre-reqs
    pip: name=mysql state=latest

  - name: start db service
    service: name=mariadb enabled=yes state=started

  - name: install bareos
    yum: name={{ item }} state=installed
    with_items:
      - bareos-database-mysql
      - bareos-client
      - bareos-director
      - bareos-storage
      - bareos-storage-glusterfs
      - bareos-bconsole
      - bareos-filedaemon

  - name: create db
    mysql_db: login_user=root name=bareos state=import target={{ sql_import }}

  - name: create db user bareos
    mysql_user: login_user=root name=bareos password={{ backup_database_password }} encrypted=yes priv=bareos.*:ALL state=present

SELECT PASSWORD(‘strongpasswordstring’);
+---------------------------------------------------------+
| password('strongpasswordstring')                                 |
+---------------------------------------------------------+
| *35D93ADBD68F80D63FF0D744BA55CF920B2A45BD |
+---------------------------------------------------------+

backup_database_password: "{{ vault_backup_database_password }}"

vault_backup_database_password: '*35D93ADBD68F80D63FF0D744BA55CF920B2A45BD'

$ ansible-vault encrypt playbooks/group_vars/dbs/vault.yml

  - name: add bareos-dir.conf
    copy: src=files/bareos-dir.conf dest=/etc/bareos/bareos-dir.conf owner=bareos group=bareos mode=0640

  - name: add bareos-sd.conf
    copy: src=files/bareos-sd.conf dest=/etc/bareos/bareos-sd.conf owner=bareos group=bareos mode=0640

  - name: add bareos-fd.conf
    copy: src=files/bareos-fd.conf dest=/etc/bareos/bareos-fd.conf owner=bareos group=bareos mode=0640

  - name: add bconsole.conf
    copy: src=files/bconsole.conf dest=/etc/bareos/bconsole.conf owner=root group=bareos mode=0640

$ vi playbooks/files/bareos-fd.conf
Client {
  Name = bareos-fd
  Description = "Client resource of the Director itself."
  Address = localhost
  Password = "YVcb9Ck0MvIXpZkZCM8wBV1qyEi1FD6kJjHUrk+39xun"          # password for FileDaemon
}

$ vi playbooks/templates/backup_bareos_fd.conf.j2
Client {
  Name = bareos-fd
  Description = "Client resource of the Director itself."
  Address = localhost
  Password = "{{ bareos_fd_dir_password }}"
}

  - name: create backup directory
    file: path=/data/backups/FileStorage state=directory owner=bareos group=bareos mode=0750

  handlers:
  - name: mariadb_restart
    service: name=mariadb state=restarted

$ sudo ansible-playbook --list-hosts -b playbooks/backup.yml
ERROR! Decryption failed

$ sudo ansible-playbook --list-hosts --ask-vault-pass -b playbooks/backup.yml
Vault password:

playbook: playbooks/backup.yml

  play #1 (backup.example.com):         TAGS: []
    pattern: [u'backup.example.com']
    hosts (1):
      backup.example.com

TASK [create db user bareos] ***************************************************
fatal: [backup.example.com]: FAILED! => {"failed": true, "msg": "ERROR! 'backup_database_password' is undefined"}

[dbs]
db.example.com ansible_user=vagrant
backup.example.com

$ sudo ansible-playbook  --ask-vault-pass -b playbooks/backup.yml
Vault password:

PLAY ***************************************************************************

TASK [setup] *******************************************************************
ok: [backup.example.com]

TASK [install epel] ************************************************************
changed: [backup.example.com]

TASK [install pip] *************************************************************
changed: [backup.example.com] => (item=[u'python-pip', u'python-devel'])

TASK [add bareos] **************************************************************
changed: [backup.example.com]

TASK [install mariadb] *********************************************************
changed: [backup.example.com] => (item=[u'mariadb-devel', u'mariadb-server'])

                    TASK                                           [install pre-reqs] ********************************************************
changed: [backup.example.com]

TASK [start db service] ********************************************************
changed: [backup.example.com]

TASK [install bareos] **********************************************************
changed: [backup.example.com] => (item=[u'bareos-client', u'bareos-director', u'bareos-storage', u'bareos-storage-glusterfs', u'bareos-bconsole', u'bareos-filedaemon'])

TASK [create db] ***************************************************************
changed: [backup.example.com]

TASK [create db user bareos] ***************************************************
changed: [backup.example.com]

TASK [install bareos-database-mysql CentOS] ************************************
changed: [backup.example.com]

TASK [add bareos-dir.conf] *****************************************************
changed: [backup.example.com]

TASK [add bareos-sd.conf] ******************************************************
changed: [backup.example.com]

TASK [add bareos-fd.conf] ******************************************************
changed: [backup.example.com]

TASK [add bconsole.conf] *******************************************************
changed: [backup.example.com]

TASK [create backup directory] *************************************************
ok: [backup.example.com]

PLAY RECAP *********************************************************************
backup.example.com         : ok=2   changed=14   unreachable=0    failed=0

Installing Serverspec

In this testing scenario, we have checked out a particular Git repository that will have our configuration management files we want to test. We are going to use Vagrant to help bring up and test how our configurations are applied to our hosts. We use Serverspec to start our Vagrant hosts if they are not started, apply our provisioning instructions, and then test the results of those instructions.

Let’s assume we have a Git repository already created, and we will clone to our local system.

$ git clone [email protected]:/ouruser/ourrepo.git
$ cd ourrepo

In this checkout, we already have a Vagrantfile that contains the following:

$ vi Vagrantfile
Vagrant.configure(2) do |config|
  config.vm.provider "virtualbox" do |vb|
     vb.memory = "1024"
  end

  config.vm.define "ansible" do |xenial|
    xenial.vm.box = "ubuntu/xenial64"
    xenial.vm.hostname = "ansible"
    xenial.vm.provision :shell do |shell|
      shell.path = "bootscript.sh"
    end
    xenial.vm.provision :ansible do |ansible|
      ansible.playbook = 'ansible/playbooks/ansible.yml'
    end
  end
  config.vm.define "puppet" do |centos|
    centos.vm.box = "centos/7"
    centos.vm.hostname = "puppet"
    centos.vm.provision :shell do |shell|
      shell.path = "bootscript.sh"
    end
    centos.vm.provision "puppet"
  end
end

This Vagrantfile will allow us to run up a Xenial Ubuntu host and a CentOS 7 host. The Xenial host will apply an Ansible playbook while the CentOS host will run a Puppet apply. We will create the provisioning files shortly. We will then test that both hosts have the same configuration, which will be a HTTPD server listening on port 80.

Now let’s talk about Serverspec. Serverspec is available as a Ruby gem. You can use another Ruby gem called Bundler ( http://bundler.io/ ) to keep track of your installed gems for a particular application in a file called a Gemfile. We are going to use that to install our required gems as we go along. This also helps us pin particular version of gems to our git commits and helps us track changes to the gem release versions we are using.

$ sudo yum install –y ruby rubygems && gem install bundler --no-ri --no-rdoc

Here we are using a CentOS host and install the ruby and rubygems packages and then install into the local account the gem bundler (without the associated docs and help for a faster install).

With the Bundler gem installed, we will now create a gem file with the following in our repository directory like so:

$ vi Gemfile
source 'https://rubygems.org'

gem 'serverspec'

Here we have added the Serverspec gem to the gemfile. The source statement is where we will be downloading the gem from, which is the public rubygems.org server where people publish their Ruby gems.

Now we can use the bundle command to install the gem locally.

$ bundle install --path vendor/cache
Fetching gem metadata from https://rubygems.org/.......
Fetching version metadata from https://rubygems.org/.
Resolving dependencies...
Installing diff-lcs 1.2.5
Installing multi_json 1.12.1
Installing net-ssh 3.2.0
...

This will install the serverspec gem and any serverspec gem requirements. We are now ready to initialize serverspec for our currently checked-out repository, which is the equivalent of running a setup script. We do that with the serverspec-init command.

$ serverspec-init
Select OS type:

  1) UN*X
  2) Windows

Select number: 1

We are asked what type of OS we are going to test; here we choose 1) UN*X.

Select a backend type:

  1) SSH
  2) Exec (local)

Select number: 1

We can now choose how we are going to access the host: either run serverspec commands locally or use SSH to sign into a host and run commands from inside that host. We are going to use SSH, so choose 1.

Vagrant instance y/n: y
Auto-configure Vagrant from Vagrantfile? y/n: y
0) ansible
1) puppet
Choose a VM from the Vagrantfile: 0
 + spec/
 + spec/ansible/
 + spec/ansible/sample_spec.rb
 + spec/spec_helper.rb

Serverspec has detected the Vagrantfile and now wants to know whether we want to configure one of those hosts automatically for us. Choosing either puppet or ansible is fine here, so we will use the copy command to add the other.

We have now set up Serverspec . There is a spec directory that has been created, and Serverspec is managed via the spec/spec_helper.rb file. Serverspec will look for a directory in the spec directory that matches a host declared in the Vagrantfile and run any tests it finds in those directories that end in *_spec.rb. So, we can now copy the spec/ansible directory to spec/puppet, and now both hosts will be tested.

$ cp –r spec/ansible spec/puppet

If we take a look at the sample_spec.rb file, it will show us our Serverspec tests.

require 'spec_helper'

describe package('httpd'), :if => os[:family] == 'redhat' do
  it { should be_installed }
end

describe package('apache2'), :if => os[:family] == 'ubuntu' do
  it { should be_installed }
end

describe service('httpd'), :if => os[:family] == 'redhat' do
  it { should be_enabled }
  it { should be_running }
end

describe service('apache2'), :if => os[:family] == 'ubuntu' do
  it { should be_enabled }
  it { should be_running }
end

describe service('org.apache.httpd'), :if => os[:family] == 'darwin' do
  it { should be_enabled }
  it { should be_running }
end

describe port(80) do
  it { should be_listening }
end

The _spec.rb file should be easily read, which is one of the design goals of RSpec to make testing clear. The first line is Ruby; the require is similar to a Python import statement and is just making the spec_helper available to us.

The next lines read like this. We want to describe a package called httpd. If we are on a redhat family host, that package should be installed. Now you can read the others, and they are similar and describe what we expect to find when we run Serverspec. Serverspec will take these plain descriptions and handle how it will validate our tests.

We will remove the second last test describing a Darwin family operating system (Mac OS), but the rest of the tests suit our purpose well.

Running Tests

Let’s start by running our tests; we can then see the work we need to do to get our tests to pass. We do this using some tools that came when we installed Serverspec.

$ rake spec:ansible

Package "apache2"
  should be installed (FAILED - 1)

Service "apache2"
  should be enabled (FAILED - 2)
  should be running (FAILED - 3)

Port "80"
  should be listening (FAILED - 4)

To run our tests, we are going to execute what’s called a rake task. rake is a Ruby version of the make utility; you can see more about it here: https://github.com/ruby/rake . The task we are going to run is called spec:ansible, and that will fire up the Ansible Vagrant host and then run the Serverspec tests.

You can see why the tests failed in the Failures: section.

Failures:

  1) Package "httpd" should be installed
     On host `puppet'
     Failure/Error: it { should be_installed }
       expected Package "httpd" to be installed
       sudo -p 'Password: ' /bin/sh -c rpm -q httpd
       package httpd is not installed

     # ./spec/puppet/sample_spec.rb:4:in `block (2 levels) in <top (required)>'

You can see that Serverspec tried to run the rpm –q httpd command but could not find httpd installed. This is as expected as we haven’t installed it yet. We are now going to write the Ansible code to install it and have it provisioned on our Vagrant host.

$ vi ansible/playbooks/ansible.yml
---
- hosts: all
  gather_facts: true
  become: true
  tasks:
  - name: install apache2
     apt: name=apache2 state=latest

We will do the same for Puppet now too.

$ vi manifests/site.pp
class httpd {

  package {'httpd': ensure => 'latest' }

}

include httpd

We are going to provision our hosts now using the vagrant provision command. Let’s run the Serverspec test again.

$ rake spec
Package "apache2"
  should be installed

Service "apache2"
  should be enabled
  should be running

Port "80"
  should be listening

Finished in 0.06987 seconds (files took 11.03 seconds to load)
4 examples, 0 failures
...
Package "httpd"
  should be installed

Service "httpd"
  should be enabled (FAILED - 1)
  should be running (FAILED - 2)

Port "80"
  should be listening (FAILED - 3)

Running rake spec will run the tests over any of the hosts in our spec/ folder. Our tests for the Ansible host is already all green. That is because on Ubuntu when you install service packages they can be started automatically. With CentOS, they won’t do this until they are told. Let’s make the CentOS host green.

$ vi manifests/site.pp
class httpd {

  package {'httpd': ensure => 'latest' }
  service { ‘httpd’: enable => true, ensure => true }

}

include httpd

Running the provision and the rake spec again, and we can now see we are green in both Ansible and Puppet.

Package "apache2"
  should be installed

Service "apache2"
  should be enabled
  should be running

Port "80"
  should be listening

Finished in 0.06992 seconds (files took 8.41 seconds to load)
4 examples, 0 failures

-----
Package "httpd"
  should be installed

Service "httpd"
  should be enabled
  should be running

Port "80"
  should be listening

Finished in 0.11897 seconds (files took 7.98 seconds to load)
4 examples, 0 failures

There are many more tests that you can run, and these were very basic, but we now get the idea how helpful this can be when you are making lots of changes to systems. You should now hook these tests up to your Jenkins or CI testing infrastructure and get it to run prior to any commit to your master VCS branch.

There is a tutorial for Serverspec here: http://serverspec.org/tutorial.html .

You can see more resource types to test here: http://serverspec.org/resource_types.html .