In Chapter 5, Variables, expressions, and facts, we learned how to use Puppet's each function to iterate over an array or hash, creating resources as we go. Let's apply this technique to some Hiera data. In our first example, we'll create some user resources from a Hiera array.

Run the following command:

sudo puppet apply --environment pbg /examples/hiera_users.pp
Notice: /Stage[main]/Main/User[katy]/ensure: created
Notice: /Stage[main]/Main/User[lark]/ensure: created
Notice: /Stage[main]/Main/User[bridget]/ensure: created
Notice: /Stage[main]/Main/User[hsing-hui]/ensure: created
Notice: /Stage[main]/Main/User[charles]/ensure: created

Here's the data we're using (from the /etc/puppetlabs/code/environments/pbg/data/common.yaml file):

users:
  - 'katy'
  - 'lark'
  - 'bridget'
  - 'hsing-hui'
  - 'charles'

And here's the code which reads it and creates the corresponding user instances (hiera_users.pp):

lookup('users', Array[String]).each | String $username | {
  user { $username:
    ensure => present,
  }
}

Combining Hiera data with resource iteration is a powerful idea. This short manifest could manage all the users in your infrastructure, without you ever having to edit the Puppet code to make changes. To add new users, you need only edit the Hiera data.

Of course, real life is never quite as simple as a programming language example. If you were really managing users with Hiera data in this way, you'd need to include more data than just their names: you'd need to be able to manage shells, UIDs, and so on, and you'd also need to be able to remove the users if necessary. To do that, we will need to add some structure to the Hiera data.

Run the following command:

sudo puppet apply --environment pbg /examples/hiera_users2.pp
Notice: Compiled catalog for ubuntu-xenial in environment pbg in 0.05 seconds
Notice: /Stage[main]/Main/User[katy]/uid: uid changed 1001 to 1900
Notice: /Stage[main]/Main/User[katy]/shell: shell changed '' to '/bin/bash'
Notice: /Stage[main]/Main/User[lark]/uid: uid changed 1002 to 1901
Notice: /Stage[main]/Main/User[lark]/shell: shell changed '' to '/bin/sh'
Notice: /Stage[main]/Main/User[bridget]/uid: uid changed 1003 to 1902
Notice: /Stage[main]/Main/User[bridget]/shell: shell changed '' to '/bin/bash'
Notice: /Stage[main]/Main/User[hsing-hui]/uid: uid changed 1004 to 1903
Notice: /Stage[main]/Main/User[hsing-hui]/shell: shell changed '' to '/bin/sh'
Notice: /Stage[main]/Main/User[charles]/uid: uid changed 1005 to 1904
Notice: /Stage[main]/Main/User[charles]/shell: shell changed '' to '/bin/bash'
Notice: Applied catalog in 0.17 seconds

The first difference from the previous example is that instead of the data being a simple array, it's a hash of hashes:

users2:
  'katy':
    ensure: present
    uid: 1900
    shell: '/bin/bash'
  'lark':
    ensure: present
    uid: 1901
    shell: '/bin/sh'
  'bridget':
    ensure: present
    uid: 1902
    shell: '/bin/bash'
  'hsing-hui':
    ensure: present
    uid: 1903
    shell: '/bin/sh'
  'charles':
    ensure: present
    uid: 1904
    shell: '/bin/bash'

Here's the code which processes that data (hiera_users2.pp):

lookup('users2', Hash, 'hash').each | String $username, Hash $attrs | {
  user { $username:
    * => $attrs,
  }
}

Each of the keys in the users2 hash is a username, and each value is a hash of user attributes such as uid and shell.

When we call each on this hash, we specify two parameters to the loop instead of one:

| String $username, Hash $attrs |

As we saw in Chapter 5, Variables, expressions, and facts, when iterating over a hash, these two parameters receive the hash key and its value, respectively.

Inside the loop, we create a user resource for each element of the hash:

user { $username:
  * => $attrs,
}

You may recall from the previous chapter that the * operator (the attribute splat operator) tells Puppet to treat $attrs as a hash of attribute-value pairs. So the first time round the loop, with user katy, Puppet will create a user resource equivalent to the following manifest:

user { 'katy':
  ensure => present,
  uid    => 1900,
  shell  => '/bin/bash',
}

Every time we go round the loop with the next element of users, Puppet will create another user resource with the specified attributes.

The previous example makes it easy to manage users across your network without having to edit Puppet code: if you want to remove a user, for example, you would simply change her ensure attribute in the Hiera data to absent. Although each of the users happens to have the same set of attributes specified, this isn't essential; you could add any attribute supported by the Puppet user resource to any user in the data. Also, if there's an attribute whose value is always the same for all users, you need not list it in the Hiera data for every user. You can add it as a literal attribute value of the user resource inside the loop, and thus every user will have it.

This makes it easier to add and update users on a routine basis, but there are other advantages too: for example, you could write a simple web application which allowed HR staff to add or edit users using a browser interface, and it would only need to output a YAML file with the required data. This is much easier and more robust than trying to generate Puppet code automatically. Even better, you could pull user data from an LDAP or Active Directory (AD) server and put it into Hiera YAML format for input into this manifest.

This is a very powerful and flexible technique, and of course you can use it to manage any kind of Puppet resource: files, packages, Apache virtual hosts, MySQL databases—anything you can do with a resource you can do with Hiera data and each. You can also use Hiera's override mechanism to create different sets of resources for different nodes, roles, or operating systems.

However, you shouldn't over-use this technique. Creating resources from Hiera data adds a layer of abstraction which makes it harder to understand the code for anyone trying to read or maintain it. With Hiera, it can also be difficult to work out from inspection exactly what data the node will get in a given set of circumstances. Keep your hierarchy as simple as possible, and reserve the data-driven resources trick for situations where you have a large and variable number of resources which you need to update frequently. In Chapter 11, Orchestrating cloud resources, we'll see how to use the same technique to manage cloud instances, for example.