Search in book...
Toggle Font Controls
Create new playlist

Name your new playlist

Playlist description (optional)
Sign In

Email address

Password

Forgot Password?

or

Continue with Facebook

Continue with Google
Sign Up

Full Name

Email address

Confirm Email Address

Password

or

Continue with Facebook

Continue with Google

Additional Hiera backends

The possibility of creating and adding different backends where we can store data is one of the strong points of Hiera, as it allows feeding Puppet with data from any possible source.

This allows integrations with existing tools and gives more options to provide data in a safe and controlled way, for example, a custom web frontend or a CMDB.

Let's review some of the most interesting backends that exist.

Hiera-file

Hiera-file (https://github.com/adrienthebo/hiera-file) has been conceived by Adrien Thebo to manage a kind of data that previously couldn't be stored in a sane way in Hiera—that is, plain files.

To install it, just clone the previous Git repository in our modulepath or use its gem as follows:

gem install hiera-file

We configure it by specifying a datadir path where our data files are placed:

---
:backends:
  - file
:hierarchy:
  - "fqdn/%{fqdn}"
  - "role/%{role}"
  - "common"
:file:
  :datadir: /etc/puppetlabs/code/data

Here, the key used for Hiera lookups is actually the name of a file present in .d subdirectories inside our datadir according to our hierarchy.

For example, consider the following Puppet code:

file { '/etc/ssh/sshd_config':
  ensure  => present,
  content => hiera('sshd_config'),
}

Given the previous hierarchy (first file found is returned), the code will create a /etc/ssh/sshd_config file with the content taken from files searched in these places:

/etc/puppetlabs/code/data/fqdn/<fqdn>.d/sshd_config
/etc/puppetlabs/code/data/role/<role>.d/sshd_config
/etc/puppetlabs/code/data/common.d/sshd_config

In the preceding examples, <fqdn> and <role> have to be substituted with the actual FQDN and role of our nodes.

If we want to provide an ERB template using hiera-file we can use this syntax:

file { '/etc/ssh/sshd_config':
  ensure  => present,
  content => inline_template(hiera('sshd_config.erb')),
}

This will look for an erb template and parse it from:

/etc/puppetlabs/code/data/fqdn/<fqdn>.d/sshd_config.erb
/etc/puppetlabs/code/data/role/<role>.d/sshd_config.erb
/etc/puppetlabs/code/data/common.d/sshd_config.erb

Hiera-file is quite simple to use and very powerful because it allows us to move to Hiera what is generally placed in (site) modules: plain files with which we manage and configure our applications.

Encryption plugins

Being Puppet code and data generally versioned with a SCM and distributed accordingly, it has always been an issue to decide how and where to store reserved data such as passwords, private keys, and credentials. They were generally values assigned to variables either in clear text or as MD5/SHA hashes, but the possibility to expose them has always been a concern for Puppeteers, and various more or less imaginative solutions have been tried (sometimes, the solution has been to just ignore the problem and have no solution).

Hiera-gpg

One of these solutions is the hiera-gpg (https://github.com/crayfishx/hiera-gpg) plugin, kept in this edition for historical reasons, as it can still be used in old Puppet versions even if it's now deprecated.

Install hiera-gpg via its gem (we also need to have the gpg executable in our PATH, gcc and the Ruby development libraries package (ruby-devel)):

gem install hiera-gpg

A sample hiera.yaml code is as follows:

---
:backends:
  - gpg
:hierarchy:
  - %{env}
  - common
:gpg:
  :datadir: /etc/puppetlabs/code/gpgdata
#  :key_dir: /etc/puppet/gpg

The key_dir is where our gpg keys are looked for, if we don't specify it, they are looked by default in ~/.gnupg, so, on our Puppet Master, this would be the .gnupg directory in the home of the puppet user.

First of all, create a GPG key, with the following command:

gpg –-gen-key

We will be asked for the kind of key, its size and duration (default settings are acceptable), a name, an e-mail, and a passphrase (even if gpg will complain, do not specify a passphrase as hiera-gpg doesn't support them).

Once the key is created, we can show it using the following command (eventually move the content of our ~/.gnupg to the configured key_dir):

gpg --list-key
/root/.gnupg/pubring.gpg
------------------------
pub   2048R/C96EECCF 2013-12-08
uid                  Puppet Master (Puppet) <[email protected]>
sub   2048R/0AFB6B1F 2013-12-08

Now we can encrypt files, move into our gpg datadir, and create normal YAML files containing our secrets, for example:

---
mysql::root_password: 'V3rys3cr3T!'

Note that this is a temporary file that we will probably want to delete, because we'll use its encrypted version, which has to be created with the following command:

gpg --encrypt -o common.gpg -r C96EECCF common.yaml

The -r argument expects our key ID (as seen via gpg –list-key), and -o expects the output file, which must have the same name/path of our data source with a .gpg suffix.

Then we can finally use hiera to get the key's value from the encrypted files:

hiera mysql::root_password -d

DEBUG: <datetime>: Hiera YAML backend starting
DEBUG: <datetime>: Looking up mysql::root_password in YAML backend
DEBUG: <datetime>: Looking for data source common
DEBUG: <datetime>: [gpg_backend]: Loaded gpg_backend
DEBUG: <datetime>: [gpg_backend]: Lookup called, key mysql::root_password resolution type is priority
DEBUG: <datetime>: [gpg_backend]: GNUPGHOME is /root/.gnupg
DEBUG: <datetime>: [gpg_backend]: loaded cipher: /etc/puppet/gpgdata/common.gpg
DEBUG: <datetime>: [gpg_backend]: result is a String ctx #<GPGME::Ctx:0x7fb6aaa2f810> txt ---
mysql::root_password: 'V3rys3cr3T!'
DEBUG: <datetime>: [gpg_backend]: GPG decrypt returned valid data
DEBUG: <datetime>: [gpg_backend]: Data contains valid YAML
DEBUG: <datetime>: [gpg_backend]: Key mysql::root_password found in YAML document, Passing answer to hiera
DEBUG: <datetime>: [gpg_backend]: Assigning answer variable

Now we can delete the cleartext common.yaml file and safely commit in our repository the encrypted GPG file and use our public key for further edits.

When we need to edit our file, we can decrypt it with the following command:

gpg -o common.yaml -d common.gpg

Note

Note that we'll need the gpg private key to decrypt a file; this is needed on the Puppet Master and we need it on any system where we intend to edit these files.

Hiera-gpg is a neat solution to manage sensitive data but it has some drawbacks, the most relevant one is that we have to work with full files and we don't have a clear control on who makes changes to, unless we distribute the gpg private key to each member of our team.

Hiera-eyaml

Hiera-eyaml is currently the most recommended plugin to manage secure data with hiera. It was created as an evolution of hiera-gpg, and has some improvements over the older plugin:

It only encrypts the values, and does it individually, so files can be easily reviewed and compared
It includes a tool to edit and view the files, so they can be used almost as easily as nonencrypted data files
It also manages nonencrypted data, so it can be used as the only solution for YAML files
It uses basic asymmetric encryption that reduces dependencies, but it also includes a pluggable framework that allows the addition of other encryption backends.

Let's see how hiera-eyaml works, as it is more used and maintained than hiera-gpg.

We install gem:

gem install hiera-eyaml

We edit the hiera.yaml to configure it:

---
:backends:
  - eyaml
:hierarchy:
  - "nodes/%{fqdn}"
  - "env/%{env}"
  - common
:eyaml:
  :datadir: /etc/puppet/code/hieradata
  :pkcs7_private_key: /etc/puppet/keys/private_key.pkcs7.pem
  :pkcs7_public_key:  /etc/puppet/keys/public_key.pkcs7.pem

Now, at our disposal is the powerful eyaml command, which makes the whole experience pretty easy and straightforward. We can use it to create our keys, encrypt and decrypt files or single strings, and directly edit on-the-fly files with encrypted values:

First, let's create our keys using the following command:
```
eyaml createkeys:
```
They are placed in the ./keys directory; make sure that the user under which the Puppet Master runs (usually puppet) has read access to the private key.

Now we can generate the encrypted value of any hiera key with:

 eyaml encrypt -l 'mysql::root_password' -s 'V3ryS3cr3T!'

Write a space before the command so it's not stored in bash history. This will print, on stdout, both the plain encrypted string and a block of configuration that we can directly copy in our .eyaml files:
```
vi /etc/puppet/hieradata/common.eyaml
---
mysql::root_password: >
    ENC[PKCS7,MIIBeQYJKoZIhvcNAQcDoIIBajCCAWYCAQAxggEhMII  […]
    +oefgBBdAJ60kXMMh/RHpaXQYX3T]
```

Note that the value is in this format: ENC[PKCS7,Encrypted_Value].

Luckily, in a similar fashion, we have to generate the keys only once, since great things happen when we have to change our encrypted values in our eyaml files. We can directly edit them with the following command:

eyaml edit /etc/puppet/hieradata/common.eyaml

Our editor will open the file and we will see the decrypted values, as it will decrypt them on the fly, so that we can edit our secrets in clear text and save the file again (of course, we can do this only on a machine where we have access to the private key). The decrypted values will appear in the editor between brackets and prefixed by DEC and the encryption backend used, PKCS7 by default:

mysql::root_password: DEC(1)::PKCS7[V3ryS3cr3T!]!

This makes the management and maintenance of our secrets particularly easy. To view the decrypted content of a eyaml file, we can use:

eyaml decrypt -f /etc/puppet/hieradata/common.eyaml

Since hiera-eyaml manages both clear text and encrypted values, we can use it as our only backend if we want to work only on YAML files.

Hiera-http, hiera-mysql

Hiera-http (https://github.com/crayfishx/hiera-http) and hiera-mysql (https://github.com/crayfishx/hiera-mysql) are other powerful Hiera backends written by Craig Dunn; they perfectly interpret Hiera's modular and extendable design and allow us to retrieve our data either via a REST interface or via MySQL queries on a database.

A quick view of how they could be configured can give an idea of how they can fit different cases. To configure hiera-http in our hiera.yaml, place something like this:

:backends:
  - http
:http:
  :host: 127.0.0.1
  :port: 5984
  :output: json
  :failure: graceful
  :paths:
    - /configuration/%{fqdn}
    - /configuration/%{env}
    - /configuration/common

To configure hiera-mysql, run the following code:

---
:backends:
  - mysql
:mysql:
  :host: localhost
  :user: root
  :pass: examplepassword
  :database: config
  :query: SELECT val FROM configdata WHERE var='%{key}' AND environment='%{env}'

We will not examine them deeper and leave the implementation and usage details to the official documentation. Just consider how easy and intuitive the syntax to configure them and what powerful possibilities they open. They let users manage Puppet data from, for example, a web interface, without touching Puppet code or even logging in to a server and working with a SCM such as Git.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.

Table of Contents for Additional Hiera backends

Create new playlist

Sign In

Sign Up