Some information in data bags can be safely stored in the Chef server in plain text, but under some circumstances, sensitive information might be safer if encrypted. Companies might not like production API keys, private keys, or similar sensitive content to be stored in plain text on the Chef server or on third-party services, such as GitHub. We'll see how to encrypt and decrypt data in the command line and from inside a Chef recipe.
To step through this recipe, you will need:
Our goal is to create a configuration file containing our AWS credentials for the us-east-1
region, and it's not acceptable that you store the credentials in clear text on the Chef server. We'd like to use a data bag, as it can be encrypted:
aws
to store the credentials for the us-east-1
region:$ mkdir data_bags/aws
$ knife data bag create aws Created data_bag[aws]
aws
data bag folder, create a sample us-east-1.json
file containing the credentials:{ "id": "us-east-1", "aws_access_key": "AKIAJWTIBGE3NFDB4HOB", "aws_secret_key": "h77/xZt/5NUafuE+q5Mte2RhGcjY4zbJ3V0cTnAc" }
This is the standard procedure for a normal data bag. If we upload it now as is, it won't be encrypted.
The solution to use an encrypted data bag is to send it encrypted from our workstation. The encryption is done through a shared secret, the secret being either a file or a string. Let's use the string s3cr3t
as an encryption key (weak). To simply send the encrypted version of the data bag, let's use the encryption feature of the knife
command:
$ knife data bag from file --encrypt --secret s3cr3t aws us-east-1.json Updated data_bag_item[aws::us-east-1]
If we request the data without providing a decryption key, we'll get the encrypted data from the Chef server:
$ knife data bag show aws us-east-1 WARNING: Encrypted data bag detected, but no secret provided for decoding. Displaying encrypted data. aws_access_key: cipher: aes-256-cbc encrypted_data: RwbfsWgKk16sSCkMD38tXKGHmT1AHFGHRm/7fyzppye7wSS0kk19Zml0VuhQ XxxI iv: iRRgrKfz6Ou2qdpYLkUA+w== version: 1 aws_secret_key: cipher: aes-256-cbc encrypted_data: uSppKMYrRbEYn/njDYo3CIGC5tY+pptN1Z7LiARtNIU/zsllBNdSVENC1XwX QksifE6g00sdcHTGlHlVU0WJ0Q== iv: ppjeAJcegZ9Yyn9rXgHRBQ== version: 1 id: us-east-1
It looks like we got what we wanted: data is stored encrypted on the Chef server!
As it may not be a secure move to store unencrypted data bags on version control systems, such as Git, we can ask for a JSON-formatted encrypted version, such as the following, and redirect the output to a JSON file for storage purposes:
$ knife data bag show aws us-east-1 -Fj { "id": "us-east-1", "aws_access_key": { "encrypted_data": "RwbfsWgKk16sSCkMD38tXKGHmT1AHFGHRm/7fyzppye7wSS0kk19Zml0VuhQ XxxI ", "iv": "iRRgrKfz6Ou2qdpYLkUA+w== ", "version": 1, "cipher": "aes-256-cbc" }, "aws_secret_key": { "encrypted_data": "uSppKMYrRbEYn/njDYo3CIGC5tY+pptN1Z7LiARtNIU/zsllBNdSVENC1XwX QksifE6g00sdcHTGlHlVU0WJ0Q== ", "iv": "ppjeAJcegZ9Yyn9rXgHRBQ== ", "version": 1, "cipher": "aes-256-cbc" } }
This might be the content you'd like to store on Git!
To access unencrypted data from the knife CLI, the process is as easy as encrypting data—pass the shared secret as an argument:
$ knife data bag show aws us-east-1 --secret s3cr3t Encrypted data bag detected, decrypting with provided secret. aws_access_key: AKIAJWTIBGE3NFDB4HOB aws_secret_key: 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 id: us-east-1
Now we have access to our data but in an unencrypted form.
Now that the data is safely stored on the Chef server, how do we access it from inside a Chef recipe? Let's say our objective is to create a file named /etc/aws/credentials
that will contain the unencrypted value from the encrypted version on the Chef server. The final file should look like this:
[region_name] aws_access_key_id = the_access_key aws_secret_access_key = the_secret_key
aws
inside the mysite
cookbook:$ chef generate recipe aws
/etc/aws
folder using the directory
resource:directory "/etc/aws" do owner 'root' group 'root' mode '0755' action :create end
templates/aws.erb
ERB template file for our destination, namely /etc/aws/credentials
:[<%= @aws_region %>] aws_access_key_id = <%= @aws_access_key %> aws_secret_access_key = <%= @aws_secret_key %>
We see the template is expecting the aws_region
, aws_access_key
, and aws_secret_key
variables. Let's write the code to inject these values to the aws.rb
recipe. To begin with, let's access our encrypted data bag item us-east-1
from the aws
data bag, using the inline shared secret s3cr3t
:
aws = Chef::EncryptedDataBagItem.load("aws", "us-east-1", 's3cr3t')
/etc/aws/credentials
file:template "/etc/aws/credentials" do source 'aws.erb' owner 'root' group 'root' mode '0600' variables( aws_region: aws['id'], aws_access_key: aws['aws_access_key'], aws_secret_key: aws['aws_secret_key'] ) end
Here we are! The Chef server is now safely storing encrypted data. For added security, it's better to not hardcode the shared key—use the key file that is sent separately (but this creates an added layer of complexity in the deployment system).
While using Puppet, it is a good practice to store the credentials and site information in Hiera, as we saw in Chapter 6, Fundamentals of Managing Servers with Chef and Puppet. Using hiera-eyaml
, it is possible to encrypt sensitive data. Using our previous LAMP setup with Vagrant, let's try to encrypt the root password for MySQL.
We need to install a new backend for Hiera. We have not discussed a lot about Hiera yet, and it's time to do so. Hiera is used to store data out of manifests, and is based on a hierarchy to look up data. A default configuration is provided with the Puppet server installation and is located at /etc/puppetlabs/puppet/hiera.yaml
:
--- :backends: - yaml :hierarchy: - "nodes/%{::trusted.certname}" - common :yaml: :datadir:
Here, a yaml
backend is defined, allowing us to use yaml
files in the hieradata
directory of our environments. Then, a hierarchy is defined. Puppet will first try to look up data in a yaml
file with the name matching the name of the client certificate (that is, the FQDN node) and located under the nodes
subdirectory. If no data is found, Puppet will try to look up a common.yaml
file.
With hiera-eyaml
, we need to declare a new backend to look up data in encrypted files. This backend is eyaml
, and by default, we will look for files with the .eyaml
extension. This backend relies on a key pair to read data, so we need to generate these keys.
Fortunately, a Puppet module, named puppet/hiera
, exists to handle all of this for us. So we just need to add it to Puppetfile
with its dependencies (do not forget to run r10k puppetfile install
):
mod 'puppetlabs/inifile' mod 'puppet/hiera' mod 'puppetlabs/puppetserver_gem'
With this module, it is now very easy to prepare the Puppet server using the following:
node 'puppet.pomes.pro' { # Create a service resource for the puppetserver # This is needed by the hiera module, in order # to restart the server once hiera-eyaml is installed service {'puppetserver': ensure => running, } # Configure hiera class { 'hiera': hierarchy => [ 'nodes/%{::trusted.certname}', ], eyaml => true, manage_package => true, provider => 'puppetserver_gem', master_service => 'puppetserver', } }
This piece of code will:
puppet/hiera
module (see parameter master_service
)eyaml
backend in the Puppet serverPrivate and public keys will be respectively placed in /etc/puppetlabs/puppet/keys/private_key.pkcs7.pem
and /etc/puppetlabs/puppet/keys/public_key.pkcs7.pem
.
To prepare the workstation, follow these steps:
eyaml
. Let's install it using the following:$ sudo puppet resource package hiera-eyaml provider=puppet_gem
keys
folder under $HOME
:$ ls ~/keys/ private_key.pkcs7.pem public_key.pkcs7.pem
$ chmod 500 keys $ chmod 400 keys/private_key.pkcs7.pem
eyaml
configuration file, located in ~/.eyaml/config.yaml
, with this content (do not forget to adjust the path of your $HOME
directory):--- pkcs7_public_key: "/Users/me/keys/public_key.pkcs7.pem" pkcs7_private_key: "/Users/me/keys/private_key.pkcs7.pem"
From the command line, eyaml
can encrypt values. Here is a session example:
$ eyaml encrypt -s 'super_secure_password' [hiera-eyaml-core] Loaded config from /Users/me/.eyaml/config.yaml string: ENC[PKCS7,MIIBiQYJKoZIhvcNAQcDoIIBejCCAXYCAQAxggEhMIIBHQIBADAFMAACAQEwDQYJKoZIhvcNAQEBBQAEggEALjJ2a9uZ04lk2V5xKqEd0n3BtA4OLe1B6rA2iVruJRKxWJdevuGvJ55DDedRwBMZmqbvSMO1cgMUyPbfEy54i3SXw4x3LEuxc1R31ILoOspBgzU4OLuepCotuhBASA/pI/xu40y66AZAcCQ4CtD9SZJYjiWNtUA91rcARy/xYQGK39QievxT2eq5De89qIn2w/5fIRIkJBRyNqnwyYCWKcKSRwaiLbimpwmarOP+dxGHEFRrD/FiM4NfoV1WNNVr1UkPEFuNrWBzwBpvyZUnMbGHN676Rg5vq9sS6aWI6zPxTrJyLtssZm1f4GsfhmE+anFmuxrcWtEH6C82wKMOoTBMBgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBC3MhSP09yUw8XTj0XdlG1VgCCDCGhqIFdUmORYKlq0Pn5CE/cDZKTO+bhHxdBw5amAGQ==] OR block: > ENC[PKCS7,MIIBiQYJKoZIhvcNAQcDoIIBejCCAXYCAQAxggEhMIIBHQIBADAFMAACAQEw DQYJKoZIhvcNAQEBBQAEggEALjJ2a9uZ04lk2V5xKqEd0n3BtA4OLe1B6rA2 iVruJRKxWJdevuGvJ55DDedRwBMZmqbvSMO1cgMUyPbfEy54i3SXw4x3LEux c1R31ILoOspBgzU4OLuepCotuhBASA/pI/xu40y66AZAcCQ4CtD9SZJYjiWN tUA91rcARy/xYQGK39QievxT2eq5De89qIn2w/5fIRIkJBRyNqnwyYCWKcKS RwaiLbimpwmarOP+dxGHEFRrD/FiM4NfoV1WNNVr1UkPEFuNrWBzwBpvyZUn MbGHN676Rg5vq9sS6aWI6zPxTrJyLtssZm1f4GsfhmE+anFmuxrcWtEH6C82 wKMOoTBMBgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBC3MhSP09yUw8XTj0Xd lG1VgCCDCGhqIFdUmORYKlq0Pn5CE/cDZKTO+bhHxdBw5amAGQ==]
Since the eyaml
backend is looking for files with the .eyaml
extension, we just need to create a hieradata/nodes/web.pomes.pro.eyaml
file with the following content:
--- root_password: > ENC[PKCS7,MIIBiQYJKoZIhvcNAQcDoIIBejCCAXYCAQAxggEhMIIBHQIBADAFMAACAQEw DQYJKoZIhvcNAQEBBQAEggEALjJ2a9uZ04lk2V5xKqEd0n3BtA4OLe1B6rA2 iVruJRKxWJdevuGvJ55DDedRwBMZmqbvSMO1cgMUyPbfEy54i3SXw4x3LEux c1R31ILoOspBgzU4OLuepCotuhBASA/pI/xu40y66AZAcCQ4CtD9SZJYjiWN tUA91rcARy/xYQGK39QievxT2eq5De89qIn2w/5fIRIkJBRyNqnwyYCWKcKS RwaiLbimpwmarOP+dxGHEFRrD/FiM4NfoV1WNNVr1UkPEFuNrWBzwBpvyZUn MbGHN676Rg5vq9sS6aWI6zPxTrJyLtssZm1f4GsfhmE+anFmuxrcWtEH6C82 wKMOoTBMBgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBC3MhSP09yUw8XTj0Xd lG1VgCCDCGhqIFdUmORYKlq0Pn5CE/cDZKTO+bhHxdBw5amAGQ==]
However, eyaml
has a very handy feature—the edit
mode—allowing us to create and edit encrypted values in plain text, based on the keys stored in the $HOME
directory:
$ eyaml edit hieradata/nodes/web.pomes.pro.eyaml
This command will launch an editor, and we just need to enter the following content:
--- root_password: > DEC::PKCS7[super_secure_password]!
While saving, eyaml
will write the file with the encrypted content for root_password
. If needed, we can edit the file again and all the encrypted values will be automatically decrypted.
As our last step, we need to modify the main manifest to do a Hiera lookup in order to get the password:
node 'web.pomes.pro' { ... $pass=hiera('root_password'); ... class { 'mysql::server': root_password => $pass; } ... }
Now the root password is encrypted in Hiera, and only people having the keys can recover it.
hiera-eyaml
GitHub repository with its documentation at https://github.com/TomPoulton/hiera-eyamlpuppet-hiera
GitHub repository with its documentation at https://github.com/voxpupuli/puppet-hiera3.22.77.63