Puppet often needs to know your secrets; for example, passwords, private keys, and other credentials need to be configured on the node, and Puppet must have access to this information. The problem is how to make sure that no-one else does. If you are checking this data into a Git repo, it will be available to anybody who has access to the repo, and if it's a public GitHub repo, everybody in the world can see it.
Clearly, it's essential to be able to encrypt secret data in such a way that Puppet can decrypt it on individual nodes where it's needed, but it's indecipherable to anybody who does not have the key. The popular GnuPG encryption tool is a good choice for this. It lets you encrypt data using a public key which can be distributed widely, but only someone with the corresponding private key can decrypt the information.
Hiera has a pluggable backend system which allows it to support various different ways of storing data. One such backend is called hiera-eyaml-gpg, which allows Hiera to use a GnuPG-encrypted data store. Rather than encrypting a whole data file, hiera-eyaml-gpg lets you mix encrypted and plaintext data in the same YAML file. That way, even someone who doesn't have the private key can still edit and update the plaintext values in Hiera data files, although the encrypted data values will be unreadable to them.
First, we'll need to install GnuPG and create a key pair for use with Hiera. The following instructions will help you do this:
- Run the following command:
sudo apt-get install gnupg rng-tools - Once GnuPG is installed, run the following command to generate a new key pair:
gpg --gen-key - When prompted, select the RSA and RSA key type:
Please select what kind of key you want: (1) RSA and RSA (default) (2) DSA and Elgamal (3) DSA (sign only) (4) RSA (sign only) Your selection? 1 - Select a 2,048 bit key size:
RSA keys may be between 1024 and 4096 bits long. What keysize do you want? (2048) 2048 - Enter
0for the key expiry time:Key is valid for? (0) 0 Key does not expire at all Is this correct? (y/N) y
- When prompted for a real name, email address, and comment for the key, enter whatever is appropriate for your site:
Real name: Puppet Email address: [email protected] Comment: You selected this USER-ID: "Puppet <[email protected]>" Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? o
- When prompted for a passphrase, just hit Enter (the key can't have a passphrase, because Puppet won't be able to supply it).
It may take a few moments to generate the key, but once this is complete, GnuPG will print out the key fingerprint and details (yours will look different):
pub 2048R/40486112 2016-09-30
Key fingerprint = 6758 6CEE D221 7AA0 8369 FF3A FEC1 0055 4048 6112
uid Puppet <[email protected]>
sub 2048R/472954EB 2016-09-30This key is now stored in your GnuPG keyring, and Hiera will be able to use it to encrypt and decrypt your secret data on this node. We'll see later in the chapter how to distribute this key to other nodes managed by Puppet.
A Hiera source using GPG-encrypted data needs a couple of extra parameters. Here's the relevant section from the example hiera.yaml file:
- name: "Secret data (encrypted)"
lookup_key: eyaml_lookup_key
path: "secret.eyaml"
options:
gpg_gnupghome: '/home/ubuntu/.gnupg'As with normal data sources, we a have name and a path to the data file, but we also need to specify the lookup_key function, which in this case is eyaml_lookup_key, and set options['gpg_gnupghome'] to point to the GnuPG directory, where the decryption key lives.
You're now ready to add some secret data to your Hiera store.
- Create a new empty Hiera data file with the following commands:
cd /etc/puppetlabs/code/environments/pbg sudo touch data/secret.eyaml
- Run the following command to edit the data file using the
eyamleditor (which automatically encrypts the data for you when you save it). Instead of[email protected], use the email address that you entered when you created your GPG key.sudo /opt/puppetlabs/puppet/bin/eyaml edit --gpg-always-trust [email protected] data/secret.eyaml - If the system prompts you to select your default editor, choose the editor you prefer. If you're familiar with Vim, I recommend you choose that, but otherwise, you will probably find
nanothe easiest option. (You should learn Vim, but that's a subject for another book.) - Your selected editor will be started with the following text already inserted in the file:
#| This is eyaml edit mode. This text (lines starting with #| at the top of the #| file) will be removed when you save and exit. #| - To edit encrypted values, change the content of the DEC(<num>)::PKCS7[]! #| block (or DEC(<num>)::GPG[]!). #| WARNING: DO NOT change the number in the parentheses. #| - To add a new encrypted value copy and paste a new block from the #| appropriate example below. Note that: #| * the text to encrypt goes in the square brackets #| * ensure you include the exclamation mark when you copy and paste #| * you must not include a number when adding a new block #| e.g. DEC::PKCS7[]! -or- DEC::GPG[]!
- Enter the following text below the commented message, exactly as shown, including the beginning three hyphens:
--- test_secret: DEC::GPG[This is a test secret]!
- Save the file and exit the editor.
- Run the following command to test that Puppet can read and decrypt your secret:
sudo puppet lookup --environment pbg test_secret --- This is a test secret
To prove to yourself that the secret data is actually encrypted, run the following command to see what it looks like in the data file on disk:
cat data/secret.eyaml
---
test_secret: ENC[GPG,hQEMA4+8DyxHKVTrAQf/QQPL4zD2kkU7T+FhaEdptu68RAw2m2KAXGujjnQPXoONrbh1QjtzZiJBlhqOP+7JwvzejED0NXNMkmWTGfCrOBvQlZS0U9Vrgsyq5mACPHyeLqFbdeOjNEIR7gLP99aykAmbO2mRqfXvns+cZgaTUEPXOPyipY5Q6w6/KeBEvekTIZ6ME9Oketj+1/zyDz4qWH+0nLwdD9L279d7hnokpts2tp+gpCUc0/qKsTXpdTRPE2R0kg9Bl84OP3fFlTSTgcT+pS8Dfa1/ZzALfHmULcC3hckG9ZSR+0cd6MyJzucwiJCreIfR/cDfqpsENNM6PNkTAHEHrAqPrSDXilg1KtJSAfZ9rS8KtRyhoSsk+XyrxIRH/S1Qg1dgFb8VqJzWjFl6GBJZemy7z+xjoWHyznbABVwp0KXNGgn/0idxfhz1mTo2/49POFiVF4MBo/6/EEU4cw==]Of course, the actual ciphertext will be different for you, since you're using a different encryption key. The point is, though, the message is completely scrambled. GnuPG's encryption algorithms are extremely strong; even using every computer on Earth simultaneously, it would take (on average) many times the current age of the Universe to unscramble data encrypted with a 2,048-bit key. (Or, to put it a different way, the chances of decrypting the data within a reasonable amount of time are many billions to one.)
When you reference a Hiera key such as test_secret in your manifest, what happens next? Hiera consults its list of data sources configured in hiera.yaml. The first source in the hierarchy is secret.eyaml, which contains the key we're interested in (test_secret). Here's the value:
ENC[GPG,hQEMA4 … EEU4cw==]
The ENC tells Hiera that this is an encrypted value, and the GPG identifies which type of encryption is being used (hiera-eyaml supports several encryption methods, of which GPG is one). Hiera calls the GPG subsystem to process the encrypted data, and GPG searches the keyring to find the appropriate decryption key. Assuming it finds the key, GPG decrypts the data and passes the result back to Hiera, which returns it to Puppet, and the result is the plaintext:
This is a test secret
The beauty of the system is that all of this complexity is hidden from you; all you have to do is call the function lookup('test_secret', String) in your manifest, and you get the answer.
If the secret data is stored in encrypted form, you might be wondering how to edit it when you want to change the secret value. Fortunately, there's a way to do this. Recall that when you first entered the secret data, you used the following command:
sudo /opt/puppetlabs/puppet/bin/eyaml edit --gpg-always-trust [email protected] data/secret.eyaml
If you run the same command again, you'll find that you're looking at your original plaintext (along with some explanatory comments):
--- test_secret: DEC(1)::GPG[This is a test secret]!
You can edit the This is a test secret string (make sure to leave everything else exactly as it is, including the DEC::GPG[]! delimiters). When you save the file and close the editor, the data will be re-encrypted using your key, if it has changed.
Don't remove the (1) in parentheses after DEC; it tells Hiera that this is an existing secret, not a new one. As you add more secrets to this file, they will be identified with increasing numbers.
For convenience of editing, I suggest you make a shell script, called something like /usr/local/bin/eyaml_edit, which runs the eyaml edit command. There's an example on your Vagrant box, at /examples/eyaml_edit.sh, which you can copy to /usr/local/bin and edit (as before, substitute the gpg-recipients email address with the one associated with your GPG key):
#!/bin/bash /opt/puppetlabs/puppet/bin/eyaml edit --gpg-always-trust [email protected] /etc/puppetlabs/code/environments/pbg/data/secret.eyaml
Now, whenever you need to edit your secret data, you can simply run the following command:
sudo eyaml_edit
To add a new secret, add a line like this:
new_secret: DEC::GPG[Somebody wake up Hicks]!
When you save and quit the editor, the newly-encrypted secret will be stored in the data file.
Now that your Puppet manifests use encrypted Hiera data, you'll need to make sure that each node running Puppet has a copy of the decryption key. Export the key to a text file using the following command (use your key's email address, of course):
sudo sh -c 'gpg --export-secret-key -a [email protected] >key.txt'
Copy the key.txt file to any nodes which need the key, and run the following command to import it:
sudo gpg --import key.txt sudo rm key.txt
Make sure that you delete all copies of the text file once you have imported the key.
Note
Important note
Because all Puppet nodes have a copy of the decryption key, this method only protects your secret data from someone who does not have access to the nodes. It is still considerably better than putting secret data in your manifests in plaintext, but it has the disadvantage that someone with access to a node can decrypt, modify, and re-encrypt the secret data. For improved security you should use a secrets management system where the node does not have the key, and Puppet has read-only access to secrets. Some options here include Vault, from Hashicorp, and Summon, from Conjur.