When automating tasks among many servers, information from one node may affect the configuration of another node or nodes. For example, if you configure DNS servers using Puppet, then you can have Puppet tell the rest of your nodes where all the DNS servers are located. This sharing of information is called catalog storage and searching in Puppet.
Catalog storage and searching was previously known as storeconfigs and enabled using the storeconfig
option in puppet.conf
. Storeconfigs was able to use SQLite, MySQL, and PostgreSQL; it is now deprecated in favor of
PuppetDB.
The current method of supporting exported resources is PuppetDB, which uses Java and PostgreSQL and can support hundreds to thousands of nodes with a single PuppetDB instance. Most scaling issues with PuppetDB can be solved by beefing up the PostgreSQL server, either adding a faster disk or more CPU, depending on the bottleneck.
We will begin our discussion of exported resources by configuring PuppetDB. We will then discuss exported resource concepts and some example usage.
The easy way to configure PuppetDB is to use the puppetdb
Puppet module on Puppet Forge at https://forge.puppetlabs.com/puppetlabs/puppetdb. We will install PuppetDB using the module first to show how quickly you can deploy PuppetDB. In the subsequent section, we'll configure PuppetDB manually to show how all the components fit together.
The steps to install and use PuppetDB that we will outline are as follows:
puppetdb
module on Puppet master (stand
).puppetlabs-repo
and Puppet on PuppetDB host.puppetdb
module onto PuppetDB host.We will start with a vanilla EL6 machine and install PuppetDB using the puppetdb
module. In Chapter 4, Public Modules, we used a Puppetfile
in combination with librarian-puppet
or r10k to download modules. We used the puppetdb
module since it was a good example of dependencies; we will rely on PuppetDB being available to our catalog worker for this example. If you do not already have PuppetDB downloaded, do it using one of those methods or simply use puppet module install puppetlabs-puppetdb
as shown in the following screenshot:
After installing the puppetdb
module, we need to install the puppetlabs
repo on our PuppetDB machine and install Puppet using the following command:
[root@puppetdb ~]# yum -y install http://yum.puppetlabs.com/puppetlabs-release-pc1-el-7.noarch.rpm puppetlabs-release-pc1-el-7.noarch.rpm | 5.1 kB 00:00:00 Examining /var/tmp/yum-root-dfBZAN/puppetlabs-release-pc1-el-7.noarch.rpm: puppetlabs-release-pc1-1.0.0-1.el6.noarch Marking /var/tmp/yum-root-dfBZAN/puppetlabs-release-pc1-el-7.noarch.rpm as an update to puppetlabs-release-pc1-0.9.2-1.el7.noarch Resolving Dependencies --> Running transaction check ... root@puppetdb ~]# yum -y install puppetdb Resolving Dependencies --> Running transaction check ---> Package puppetdb.noarch 0:3.2.0-1.el7 will be installed -->Processing Dependency: net-tools for package: puppetdb-3.2.0-1.el7.noarch --> Processing Dependency: java-1.8.0-openjdk-headless for package: puppetdb-3.2.0-1.el7.noarch --> Running transaction check ---> Package java-1.8.0-openjdk-headless.x86_64 1:1.8.0.65-2.b17.el7_1 will be installed -->Processing Dependency: jpackage-utils for package: 1:java-1.8.0-openjdk-headless-1.8.0.65-2.b17.el7_1.x86_64 ...
Our next step is to deploy PuppetDB on the PuppetDB machine using Puppet. We'll create a wrapper class to install and configure PuppetDB on our master, as shown in the following code (in the next chapter this will become a profile). Wrapper classes, or profiles, are classes that bundle lower-level classes (building blocks) into higher-level classes.
classpdb { # puppetdb class class { 'puppetdb::server': } class { 'puppetdb::database::postgresql': listen_addresses => '*' } }
At this point, the PuppetDB server also needs network ports opened in iptables; the two ports are 5432
(postgresql
) and 8081
(puppetdb
). Using our knowledge of the firewall
module, we could do this with the following snippet included in our pdb
class:
firewall {'5432 postgresql': action => 'accept', proto => 'tcp', dport => '5432', } firewall {'8081 puppetdb': action => 'accept', proto => 'tcp', dport => '8081', }
We then apply this pdb
class to our PuppetDB machine. For this example, I used the hiera_include
method and the following puppetdb.yaml
file:
--- classes: pdb
Now we run Puppet agent on PuppetDB to have PuppetDB installed (running Puppet agent creates the SSL keys for our PuppetDB server as well; remember to sign those on the master).
Back on our workers, we need to tell Puppet to use PuppetDB; we can do this by defining a puppet::master
class that configures Puppet and applying it to our workers:
class puppet::master { class {'puppetdb::master::config': puppetdb_server => 'puppetdb.example.com', puppet_service_name => 'httpd', } }
Now we configure our stand.yaml
file to include the previous class as follows:
--- classes: puppet::master
The Puppet master will need to be able to resolve puppetdb.example.com
, either through DNS or static entries in /etc/hosts
. Now run Puppet on our Puppet master to have puppetserver
configured to use PuppetDB. The master will attempt to communicate with the PuppetDB machine over port 8081
. You'll need the firewall (iptables) rules to allow this access at this point.
Now we can test that PuppetDB is operating by using the puppet node status
command as follows:
[root@stand ~]# puppet node status puppetdb.example.com Currently active Last catalog: 2015-11-27T10:43:42.243Z Last facts: 2015-11-27T10:43:26.539Z
18.188.152.157